Limits
When AI detectors get it wrong
Every detector makes two kinds of mistake. One of them lands on a person who wrote their own work.
01 · Two mistakes
False positives and false negatives
Detectors trade one for the other: a stricter threshold catches more AI text and flags more people. Ask any vendor for both rates.
The public record
What the evidence shows
02 · Who it hurts
Where false positives land
| Writer | Why a detector misreads them |
|---|---|
| Second-language writers | Plainer vocabulary and predictable phrasing look like model output to a detector. Liang et al., Patterns, 2023 |
| Formulaic genres | Lab reports, legal summaries and product copy follow templates, so they score as predictable. |
| Short texts | A few sentences give a detector little to measure; scores swing on small changes. |
| Heavily edited human text | Grammar tools and rewrites smooth the irregular patterns a detector reads as human. |
03 · Evidence, not proof
How much a score is worth
- Level 1A score aloneA number from one tool. Never enough to act on.
- Level 2Score plus passagesThe sentences it flagged, read by a person who knows the writer.
- Level 3Plus a second toolTwo detectors agree, after both passed your own test.
- Level 4Plus the writing processDrafts, version history and a conversation with the writer.
04 · What to do
When a score says AI
Read the flagged passages, not the number
Ask whether they sound like the writer you know. A detector that cannot show passages gives you nothing to check.
Ask for the process
Drafts, notes and version history (Google Docs and Word keep it) say more than any score.
Talk before you decide
Ask the writer to explain their choices. People who wrote a piece can talk about it.
Know your tool's error rate
Run the 20-text test on writing like the one in front of you. If it flags your own human texts, it will flag theirs.
Some institutions have switched AI detection off for these reasons, Vanderbilt University among them in 2023: its guidance.