The verdict is the strongest section, never the average

01 / The finding

Averaging is the intuitive rule, and it is close to useless

The classifier reads 512 tokens at a time. Anything longer is cut into sections and scored section by section, which means something has to turn several numbers into one verdict. That combining step is the single most consequential design decision in the tool, and it is decided by the strongest section rather than the average of them.

On 700 documents that are half machine-written and half human-written, the document mean catches 29. The strongest section catches 638.

02 / Segmentation

Why a document is read in pieces at all

A 1,600-word article does not fit inside a 512-token window, so it is cut. Once a document is several readings rather than one, the combining step exists whether anyone designs it or not, and its consequences are measurable.

The obvious objection to taking the maximum is that a long document gets more chances to trip the flag, so long human documents should be flagged more often for no reason beyond their length. That objection is correct in principle. It was measured, and the rest of this page is what the measurement said.

03 / Length bias

The bias is real, small, and lives where it was expected to

Take every document with at least six sections. Ask what the flag rate would be if the verdict were the maximum over a random subset of j of that document’s own sections, computed exactly over all C(k, j) subsets rather than sampled. Register, source, author and topic are held constant, because it is the same document throughout. Only the number of chances varies.

Each extra section adds 0.14 percentage points to a human document’s chance of being wrongly flagged. The relationship is almost perfectly linear, so going from a two-section document to a nine-section one costs roughly one point. In academic discussion, the register with the worst human false-positive rate, it is +0.74pp per section; in white papers over the same range it is zero.

The right-hand curve is the reason nothing was done about it. The same extra section that costs 0.14pp of human false positives takes AI detection from 81.12% to 91.81%. A section is evidence, not only opportunity, and a correction that removes the opportunity removes the evidence with it.

Two further readings from the same record

Across the whole corpus the human false-positive rate is not monotonic in section count. It peaks at 24/979 = 2.45% for three-section documents and falls to 8/887 = 0.90% at five or more. What dominates is register composition rather than length: human fiction runs at 29/260 = 11.15% and clusters in the three- and four-section buckets, while student essays run at 0/420 and make up a quarter of the two-section bucket.

The pooled per-section mean probability for human documents is also flat or falling with length within every register measured. White papers read 0.409, 0.354, 0.299, 0.176 and 0.208 from one section to five-plus. Longer human writing is not more machine-like section by section. The bias is in the combining step, it is worth 0.14pp per section, and it sits against a register spread of eleven points.

04 / Candidates

The intuitive fix is the worst option available

Correcting the threshold for section count is what a statistician reaches for first. It costs 8.46 points of detection and buys nothing back. Sweeping the correction strength makes the mechanism plain: at γ = 0.5 the five-plus-section bucket loses 15.8 points of detection to save 0.90pp of false positives on 887 documents, and at full Bonferroni the rule collapses to 388/922 = 42.08%. A multiple-comparisons correction assumes the extra tests are noise. Here they are mostly signal.

Read the same table the other way, holding detection fixed at 877/922 and reading off the cost instead. Plain maximum spends 55/4,636 = 1.19% of human false positives, the mean of the top two spends 60/4,636, and the document mean spends 206/4,636 = 4.44%. The averaging rule is expensive at both ends.

05 / The second bar

Two sections agreeing is worth a lower bar

Minimum evidence is the rule that ships, in refitted form. It flags when one section is very confident, or when two sections agree at a lower confidence. Because it is a two-parameter rule fitted on the corpus it was evaluated on, it was cross-validated before it was believed: 200 random split-halves, both rules refitted on half A at a 1.208% false-positive budget and evaluated on half B.

200 random split-halves of the 5,558-document long-form corpus. Both rules refitted on half A at a 1.208% human false-positive budget and evaluated on half B. Source: AGGREGATION-AND-RHYTHM.md §2.

Minimum evidence wins on 178 of 200 splits, ties on 15 and loses on 7. It also spends slightly more false positives out of sample than it was fitted to, which the in-corpus table does not show and which is recorded here because it is the honest half of the result.

The gain lands where short documents live. At the matched budget the two-section bucket goes from 30/37 = 81.08% to 34/37 = 91.89% and the three-section bucket from 178/198 = 89.90% to 183/198 = 92.42%, while five-plus is unchanged at 477 against 478 of 487. The gap between short and long documents narrows from 16.9 points to 6.3.

06 / Refitting

What shipped is not what was fitted

The candidate’s parameters, 0.9845 / 0.9765, were fitted on fp32 section scores because those were the only full-corpus segmented scores that existed. The browser runtime was then measured over the same 5,558 documents and the rule did not hold there. It would have taken browser human false positives from 90/4,636 to 106/4,636, and browser academic discussion from 16/420 = 3.81% to 23/420 = 5.48%. The candidate had been accepted as detection gained at matched false positives. That is true on fp32 and false in the browser, so it is not the trade a browser visitor would have received.

Refitted against both runtimes at once, under the constraint that neither route may finish with more false positives than the rule it replaces, the shipped pair is 0.9855 / 0.9763.

Server rows are fp32 Python onnxruntime; browser rows are int8 onnxruntime-web. The two are different artefacts and their false-positive rates differ materially. Both are printed rather than the flattering one. Source: AGGREGATION-AND-RHYTHM.md §6.

One arithmetic note, because it moves a digit. The stored per-section probabilities carry four decimal places, and the secondary bar is decided by 19 AI and 9 human documents on fp32 that sit close enough to it that rounding matters. At four decimals the shipped rule reads 884 detections; from a full-precision re-score of all 21,093 segments it reads 883. The unrounded run reproduces the canonical 56/4,636 for the prior rule exactly, which is what licenses the 883.

07 / The cost

The cost, published because it was accepted knowingly

Raising the primary bar from 0.9841 to 0.9855 opens a gap, and half-AI documents fall into it.

The 700-document half-AI corpus had to be rebuilt, because the original build script was not saved. On the rebuild, which has a median of three sections rather than five, plain maximum catches 612/700 = 87.43% and the shipped pair catches 604/700 = 86.29%, both fp32. Nine documents lost against one gained, McNemar p = 0.027.

The regression is structural rather than statistical. The secondary arm cannot rescue those documents, because a second-section rule cannot save a half-AI document by definition of what makes it half-AI.

A purely additive rule — primary held at 0.984, secondary added — cannot regress this by construction, and was tested. It fails differently. To stay inside the browser’s false-positive budget the secondary has to rise to 0.9825, at which point the two-section gain disappears completely, back to 30/37 = 81.08%. There is no pair that holds two-section detection, browser false positives and mixed-content detection at once. Every candidate gives up one of them.

The alternative wrongly flags 16 more people in 4,636 and takes browser academic discussion to 5.48%.

08 / Limits

What this does not prove

The half-AI corpus is synthetic. These are spliced halves, not documents a person actually half-wrote. The corpus tests the mechanism that maximum aggregation exists to protect. Its absolute rates must not be quoted as a detection figure for edited text.

Two builds, two numbers. The original 700-document build reads 638/700 for plain maximum; the rebuild reads 612/700 on the same specification. Both are true of their own corpus and neither is a correction of the other. Any comparison must stay inside one build.

Cross-validation is blind to the mixed-content axis. The 5,558-document corpus contains no mixed documents. The shipped pair wins 194 of 200 split-halves on fp32 and 200 of 200 in the browser, and both results are blind to the regression in section 07. That is a limitation of the validation method, and it applies to every future operating point fitted the same way. A mixed-content check belongs inside the fit rather than after it.

The fit did not include WebGPU. The pair was fitted from onnxruntime-web’s WASM provider under headless Node. WebGPU was measured afterwards and holds; that measurement is a separate paper.

Register labels are machine-assigned. Every per-register figure on this page inherits that.

Two figures quoted in the project’s handover notes for this argument — a detection count for averaging on the rebuild, and a pair of mean-against-maximum rates — appear in no measurement record found in this sweep. They are not used anywhere on this page, and they should not be repeated until a measurement file carries them.

Provenance

Every file named above ships with the open measurement repository, so any figure on this page can be re-run rather than taken on trust.