Why short text cannot be reliably detected as AI

The coin-flip intuition

A distributional watermark works by nudging the model to prefer one half of the vocabulary at each step — think of it as a slightly weighted coin. Detection means counting how often the text landed on the weighted side and asking whether that count is surprising.

Flip a slightly biased coin ten times and you learn nothing; the result is consistent with a fair coin. Flip it ten thousand times and the bias is obvious. Confidence grows roughly with the square root of the number of flips, so halving your text does not halve the certainty — it degrades it faster than intuition suggests.

That is the whole reason short text is a dead zone. It is not an implementation weakness someone will patch. It is the arithmetic the method rests on.

Classifiers have the same limit

GPTZero, Turnitin and similar tools score how predictable the word choices are across a passage, plus how much that predictability varies. Both are averages, and an average over three sentences is dominated by whichever sentences you happened to write.

This is why the same detector will call one paragraph of a document machine-written and the next paragraph human, from a single author in a single sitting. The variance is not telling you about the text; it is telling you the sample is too small.

It is also why these tools misfire hardest on the shortest, plainest writing — a concise answer has little room to display the unevenness the classifier is looking for.

Length is not the only thing that shrinks the signal

Low entropy. A watermark needs freedom to choose among near-equivalent words. Code, boilerplate, legal formulae and fixed technical vocabulary offer very little, so even a long passage can carry a weak signal.

Paraphrase and translation. Substantively rewriting changes the token choices being measured, which degrades the mark. Retyping the same words does not, because the words are the thing measured.

Mixed authorship. A document part-written and part-edited dilutes whatever signal exists across text that never carried one.

What follows from this

Treat confident claims about short passages as unfounded, whoever makes them. That includes accusations based on a detector score for a paragraph, and it includes any product promising to certify a short text either way.

It also sets a limit on what anyone can honestly sell. Nobody outside the provider can verify a watermark at all — there is no public detector — and the underlying statistics mean short text would be the weakest case even if there were one. We say the same about our own product: what we can show you is exactly which artifacts are present in a passage, which is a matter of reading the bytes, not of inference.

Questions

How much text is needed before detection means anything?

There is no single published threshold, and it depends on the scheme and how much freedom the text gave the model. The honest general statement is that a few sentences carry very little signal, and confidence builds slowly with length rather than jumping at some word count.

So can a tweet be detected as AI-written?

Not reliably by any statistical method. A passage that short is consistent with both a person and a model, which is precisely why confident verdicts on tweets are not credible.

Does this mean long documents are reliably detectable?

It means the statistics are better, not that a detector is trustworthy. Classifiers remain unreliable in both directions at any length, and watermark verification still requires the provider's key, which no third party has.

Is code harder to watermark than prose?

Yes. Watermarking depends on there being several near-equivalent ways to say the next thing, and code offers far fewer. That is why marking degrades on code and on other fixed-vocabulary writing.