False positives · the mechanism, vendor-neutral
What an AI detector is actually measuring
Most explanations of AI detectors either oversell them, treat a flag as proof, or dismiss them entirely, treat a flag as noise. Neither is accurate. This page explains what the number is actually counting, so you can judge for yourself how much weight it deserves in a given case.
An AI detector does not read your essay for meaning, intent, or truth. Detector vendors describe scoring how predictable each word is given the words that came before it, a property called perplexity, and how much that predictability varies sentence to sentence, called burstiness. The claim behind the tool is that text generated by a language model tends to sit closer to the statistically likely word more often than a person does, so lower perplexity and lower burstiness are supposed to point toward machine authorship. Whatever the tool is actually catching, it is catching a textual property, not a person, and several entirely human ways of writing land in the same range that property covers: a Stanford study found detectors falsely flagged about 61% of non-native English TOEFL essays as AI-written, for reasons that have nothing to do with who typed the words.
Why this happens
Perplexity, as detector vendors describe it, measures surprise: how unlikely, statistically, each next word is given everything before it, according to a language model's own sense of probable English. A model generating text is expected to choose likely words more consistently than a person does, because that is close to what generation optimizes for. Burstiness measures variation in that surprise across a passage; human writing tends to swing between very predictable stretches and very surprising ones, while generated text is expected to stay closer to a flat middle. The failure mode is treating either number as measuring authorship directly, when at most they measure a textual property that authorship influences but does not determine. A careful, formulaic, or heavily-edited human draft can land in the same statistical range as generated text without a machine ever being involved, and formal ESL writing lands there often enough that a Stanford study found detectors misflagged 61% of TOEFL essays.
Where honest writing and machine writing genuinely overlap
These are real patterns, not coincidences. Each one is something a careful human writer in this situation has good reason to do, and something a model does for entirely different reasons.
- Non-native English formal register
- Writers taught English as a second language, often through formal grammar instruction, tend to use more consistently correct and predictable sentence construction than native speakers who absorbed the language less formally. Liang et al. (2023) found seven detectors flagged 61.3% of non-native TOEFL essays as AI-written, for exactly this reason.
- Technical and scientific writing
- Lab reports, methods sections, and technical documentation are written to a template on purpose, precision over variation, and that same precision reads as low burstiness to a detector trained to associate flatness with generation.
- Heavily edited, multi-draft prose
- A sentence revised five times for clarity usually ends up closer to the most predictable phrasing available, because that is often what clarity means. Careful editing and machine generation can converge on similar statistical territory from opposite directions.
- Legal and policy writing
- Writing bound by convention, contracts, regulations, standardized report formats, uses fixed phrasing on purpose, for consistency and enforceability. That deliberate uniformity scores the same way generated boilerplate does.
What actually helps
Knowing the mechanism changes what you argue, not whether you can win an argument. If you're contesting a flag, the useful claim is specific: 'this detector measures predictability, and my writing is predictable for reasons X and Y, here's evidence of my process', not a blanket claim that detectors are always wrong. If you're evaluating someone else's flagged work, ask what register the writing is in before trusting the score; formal academic prose, technical writing, and writing by someone taught a strict style guide all compress naturally. Treat any single detector's score as one weak signal among several, and check what your own institution's policy actually says a score is worth before assuming it functions as a verdict.
What the flattening looks like
On the left, honest writing with the specifics drained out of it, which is the shape that reads as machine-written. On the right, the same content with the writer's own detail restored. The voice is The Explainer
AI detection tools work by leveraging advanced algorithms to seamlessly analyze text and unlock insights into its origin. It is important to note that these robust systems are not always accurate, and studies show that they can sometimes flag human writing incorrectly, which is a crucial issue in today's fast-paced educational landscape.
AI detection tools analyze text by looking for patterns that human writing and machine writing tend to leave behind. They're useful, but they're not perfect. Research has shown they can flag something a person wrote as AI-generated, which matters a lot in classrooms right now.
Steps worth taking
- Learn the two numbers, not just the verdict Perplexity and burstiness are what most detectors actually output underneath a single pass or fail label. Knowing which one is driving a flag tells you what to argue.
- Name the register, not just the topic 'This is a lab report' or 'I was taught formal ESL grammar' explains a low score in a way a general denial cannot.
- Treat one detector's output as one opinion Different detectors disagree with each other regularly on the same text. A single tool's score is evidence of that tool's opinion, not a settled fact.
- Ask what your institution's policy actually says about detector scores Policies vary in how much weight they give a bare score. Knowing the specific language your own institution has committed to is more useful in a real conversation than assuming it matches some general standard.
Questions
Why do AI detectors think my writing is AI?
They measure predictability and variation in word choice, not authorship. Writing that is naturally consistent, formal ESL prose, technical writing, heavily edited drafts, scores in the same range as machine-generated text for reasons unrelated to who wrote it.
Can human writing be detected as AI?
Yes, regularly. Formal, templated, or heavily revised human writing shares the statistical flatness detectors are built to catch. Liang et al. (2023) found seven detectors flagged 61.3% of non-native English TOEFL essays as AI-written, for exactly this reason.
Why do some AI detectors flag a passage and others don't?
Each tool is trained on different text samples and tuned to different thresholds, so they measure similar properties but draw the line in different places. Disagreement between detectors on the same passage is common and expected, not a sign one of them is broken.
What causes a false positive specifically?
Low variation in word predictability across a passage, most often produced by formal register, strict style conventions, heavy editing, or non-native formal English, all of which compress natural variation without a machine being involved.
Should I trust a detector score at all?
Treat it as one data point about a text property, not a judgment about a person. Different detectors disagree with each other on the same passage regularly, which alone should tell you a single score isn't a verdict. Read your own institution's actual policy on detector scores before deciding how much weight to give one.
underslop edits drafts for voice. It does not check, score, or contest anything, and nothing on this page will change a result you have already been given. What it can do is show you which patterns in a piece of writing read as machine-written, using the same open lint the rest of the site runs on.