What the model keys on

01 / The finding

Machine prose repeats itself less, and that is the axis the detector reads

Deleting every word and phrase on the “AI vocabulary” list from a machine document costs the detector 0.8 percentage points. Making the same document repeat itself more costs it 33.

The median machine document shares 2.1% of its content words with the sentence beside it; the median human document shares 6.3%. That single property, read through several instruments, is what the detector keys on. A person introduces a term and keeps using it. A model introduces a term and reaches for a synonym, a pronoun, a rephrasing.

The commercial consequence is awkward and belongs on the page rather than in a footnote. Deliberately keyword-dense copy is the condition the detector handles worst. At the shipped operating point it flags 188 of 432 such documents, 43.5%, against 566 of 816 ordinary short-form documents at 69.4% and 883 of 922 long-form documents at 95.8%.

02 / Reading the figures

Three lines of evidence, at three different operating points

This page reconciles measurements taken at three points in the project’s history. The honest way to present them is to label each one rather than to average them into a single narrative, so every figure below carries the point it was taken at, in its prose, in its caption and in a stamp above it.

The two retired points are quoted because the experiments were run there and have not been repeated. A figure taken at 0.984 or at 0.9845 is not a description of the tool as it runs today. Only the last row describes current behaviour.

03 / The property

The property itself, read six ways

On 670 register-matched fresh long-form pairs, eight of the ten strongest interpretable signals measure one behaviour.

Wider vocabulary per unit length, and less overlap between neighbouring sentences. That is the same behaviour observed six ways, and it inverts the popular belief. Machine writing is widely called repetitive; on published long-form prose the opposite is measurably true. What people are noticing when they call it repetitive is structural sameness, parallel paragraph shapes and similar sentence openings, which is not lexical repetition.

04 / The causal test

What actually moves the score

400 machine and 400 human long-form documents were each altered one way at a time and re-scored through the deployed artefact. These figures were measured at the retired 0.984 single threshold, so the absolute rates describe a configuration that no longer runs. The deltas are the point.

The keyword list is not what the model is reading. Removing every one of those words and phrases costs 0.8 points. Two further conditions in the same table make the same case from the other side: truncating to 400 words costs 0.2 points, and reducing repetition further costs 6.8.

Replacing a quarter of content words with the document’s own most frequent content words changes nothing about length, sentence structure or function words. It only makes the text repeat itself more, and it drops detection by 33 points. That is the manipulation aimed directly at the descriptive finding above, and the two lines agree.

The half of the table that is easy to skip

The human column matters as much as the machine one. Under the repetition manipulation human false positives fall from 2.25% to 0.25%, and under truncation to zero. The model goes quiet rather than reckless: the alterations that stop it finding machine text also stop it accusing people.

05 / The cliff

Repetition, tested deliberately rather than observed

432 samples were then generated with repetition as the only variable: two models balanced, lengths held identical to an existing pilot, and one instruction style in two strengths telling the model to repeat an exact keyword phrase and keep a narrow vocabulary. That is ordinary search-optimisation practice rather than an evasion technique. These figures were measured at the retired 0.9845 maximum-only threshold.

Detection is flat at 86–88% all the way down to 0.55, then falls away. The knee is bracketed between 0.55 and 0.46, and not located: the 0.46–0.50 bin holds 33 documents, which is barely above the floor this project applies, and one bin cannot pin a threshold. Below 0.46 the model is effectively blind.

06 / The correction

The correction the source made to itself

An earlier write-up of this work claimed that routine keyword repetition pushes machine text onto the human side. That claim is recorded as overstated, and the correction is worth more than the original claim.

At routine doses it does not happen. The moderate instruction strength, which produced genuinely realistic keyword-optimised copy, came in at 146/209 = 69.9% against a baseline of 582/816 = 71.3%, with confidence intervals overlapping — [63.3, 75.7] against [68.1, 74.3] — and only one of four length bands reaching p < 0.05. The heavy condition reached 31/194 = 16.0%, and getting there took an instruction to repeat a phrase 17 to 25 times in 400 to 600 words, which degraded the output badly enough that 19% of generations came back empty or too short to keep, against 13% in the moderate condition. It is an adversarial condition rather than a sample of ordinary copy.

Detection is unaffected by keyword repetition until type-token ratio falls below roughly 0.55, and collapses below roughly 0.46.

Two things stop that being reassuring. Type-token ratio falls with length on its own, and the baseline 600-word median is already 0.568, just above the knee, so longer commercial copy drifts towards the cliff without anyone trying. At the ratio a real keyword-briefed article occupies the effect is measurable: pooled at 400 and 600 words in the 0.48–0.55 band, 103/136 = 75.7% against a matched-length baseline of 363/415 = 87.5%, z = −3.29, p = 0.001.

07 / The shipped pair

At the operating point that ships today

The whole keyword-repetition arm was re-scored at the shipped pair, so this section — and only this section — describes the tool as it runs now.

The length gradient survives the condition. Within the keyword arm detection rises from 14.0% at 100–199 words to 62.2% at 600–849, and every band sits well below its ordinary-text counterpart. Which model wrote the text makes no material difference: gpt-5.6-sol 88/199 = 44.2% [37.5–51.2], gpt-5.6-luna 100/233 = 42.9% [36.7–49.3].

432 machine documents, flagged at 0.9855 / 0.9763, T = 0.8324, segments-v3, Python onnxruntime 1.29.0, CPU, fp32, measured 30 August 2026. Bands holding fewer than 30 documents get their count and no rate.

08 / Who is at risk

Which human writers the model leans towards

The same probe run from the other direction answers the question for the human population. Correlating the deployed model’s raw margin against each interpretable feature within human documents only, so the result is not simply restating the label, across 4,184 documents.

4,184 human documents. Source: SIGNAL-SCIENCE.md §4.3.

That column describes formal, long-worded, evenly paced prose that signposts itself: academic and corporate writing.

09 / Limits

What this does not prove

Ablation shows sensitivity, not mechanism. A large score shift proves the model responds to the property that was manipulated; it does not prove that property is the model’s reason. Flattening the rhythm is the clearest case. It moves the score furthest at −43.2 points, and it also produces text unlike anything in any training distribution, so the shift may be a distribution-shift artefact rather than evidence about rhythm. The source declines to draw the obvious conclusion from it, and so does this page.

The repetition arm used two models, both from one provider, and was not tested across providers. The descriptive finding underneath it is weakest on Meta’s models, with MATTR at 0.562 and adjacent-sentence cohesion at 0.561, and only moderate on Nvidia’s at 0.633 and 0.694. By register it is weakest on creative writing (0.668) and social posts (0.658).

Every figure on this page is fp32 through Python onnxruntime. The repetition findings have not been confirmed on the browser runtime, and no browser figure is quoted here.

The keyword-repetition corpus is a generated adversarial condition. Its absolute rates describe documents written to an instruction to repeat a phrase, not a sample of published commercial copy, and the two should not be conflated even though the second drifts towards the first as it gets longer.

Two of the three lines of evidence on this page were measured at flag points that no longer ship.

10 / The commercial consequence

What follows commercially

The tool degrades on precisely the kind of copy its commercial users produce on purpose, once that copy is long enough and keyword-focused enough to push type-token ratio below about 0.55. That weakness is published here with its denominator rather than buried, under the same rule that governs every other weakness in this project.

It will also be shared by competitors, because it follows from the signal the whole field relies on rather than from anything specific to this implementation. Any detector reading under-repetition as its principal axis has the same cliff underneath it, whether or not it has been measured.

11 / Provenance

Where every figure came from

Every file named above ships with the open measurement repository, so any figure on this page can be re-run rather than taken on trust — including the two that were taken at retired flag points, which is how a reader can establish for themselves what the shipped pair would do with them.