Advance in Math&AI - November 25 Edition
AI Detectors Are Failing You—And You Should Know Why
Dear friend,
In recent months, countless people have been unfairly accused of AI use while actual AI users operate completely under the radar. This made me curious — and willing to investigate the phenomenon further.
Today I share with you what I investigated. Here’s what I found, in a word: AI detectors are dangerously unreliable, and are inducing us in error.
(Scroll down for the complete data I obtained)
What Are AI Detectors, Anyway?
Several tools claim to identify what percentage of your text was AI-generated. They highlight “suspicious” sections and promise to distinguish human writing from LLM output.
The premise sounds reasonable. The execution? Not so much.
My Test: What Actually Happens
I compared:
100% AI-written text (Claude Haiku 4.5 and Claude Sonnet 4.5)
Professional human writing (opening chapters from Rich Dad Poor Dad and The Eye of the World)
Non-professional human writing (my own math notes—no AI involved)
The detectors I tested:
Pixozone
Scribbr
ZeroGPT
My hypothesis: professional writers would be flagged as human; amateur writers (like me) might look suspicious.
The AI Writing Prompt
Here’s the prompt I used with Claude Sonnet and Claude Haiku (version 4.5):
“Write a creative piece about a character meeting someone at a coffee shop in Mumbai. Channel your inner messy human writer:
- Start with a sensory detail that feels oddly specific
- Use at least one metaphor that’s slightly weird or imperfect
- Vary your rhythm: punchy fragments mixed with long, winding sentences
- Include moments where the narrator gets distracted
- Use unexpected adjectives (”aggressive sunset” not “beautiful sunset”)
- Drop in pop culture references or time period markers
- Let one sentence be slightly awkward or grammatically questionable
- End on something unresolved or ambiguous
- Write like you’re journaling at 2am—messy, with interruptions and trailing thoughts”
Why this prompt matters: I specifically asked Claude to write like a human with flaws, distractions, and imperfect phrasing. This is key to understanding why detectors fail.
Here’s part of What Claude Generated:
There was an uncle at the corner table eating a croissant with a fork and knife.
I watched him for probably thirty seconds, this meticulous dissection of pastry, and I wanted to tell him that’s not how you eat a croissant, you’re supposed to let it fall apart, let the flakes get everywhere, let it be messy, but then again maybe he had the right idea, maybe controlling the chaos was better than—
The door opened.
My entire body knew it was him before my brain caught up, this sudden awareness of my own pulse in places I didn’t know had pulse points—my wrists, my neck, that spot right behind my knee that was suddenly, inexplicably, sweating.
Why This Extract Matters
Notice the Claude text has:
✓ Specific, odd details (sitting at a corner table, 30 seconds)
✓ Imperfect metaphors (”my knee that was suddenly, inexplicably, sweating”)
✓ Awkward phrasing
✓ Stream-of-consciousness distractions
✓ Unresolved ending
It reads human because I asked it to.
This is exactly why detectors fail—they can’t distinguish between AI mimicking human messiness and actual human writing.
🚨 The Results: The Detectors Got It Backwards
What was tested/ How it was flagged
The numbers indicate which percentage of the text (on average) is estimated as being AI for each AI detector.
The clearest evidence:
Scribbr flagged ALL Claude-generated texts as ‘100% Human-written’, while ZeroGPT flagged a Claude-generated text as 0% AI (scroll down for a full breakdown of the data).
Meanwhile, some of my entirely human-written notes were labeled as AI by two of the detectors.
The Takeaway: Don’t Trust AI Detectors
These tools are unreliable, inconsistent, and potentially harmful.
They’re more likely to flag genuine human work than catch actual AI use.
:
:
:
:
Just don’t trust them.
How to Protect Yourself
If you’re a student or professional concerned about AI accusations:
✓ Be transparent. Clearly state which parts are 100% human-produced. For example: “The ideas and research are my own. I used AI to organize and refine the writing.”
✓ Keep your workflow visible. Archive drafts, edits, and earlier versions to show your creative process.
✓ Build a portfolio. Consistent work over time demonstrates your unique voice in ways detectors never can.
Bottom line: Rather than fear false accusations, focus on being honest about your process. Your track record and transparency will matter far more than what any AI detector claims.
Next week
See you next Tuesday for an excursion into the realm of map colouring — a problem that stumped mathematicians for a long time, until a strategic partnership between humans and computer programs yielded a neat proof!
Until then, I wish you a very beautiful week.
With love and curiosity,
Sybille ☕
The Data
