Get in Touch
AI content toolsfrom Opace

Measurement paper · 5,558 documents, 21,093 sections

Two runtimes, one answer

The same trained model runs two ways: a 34 MB quantised file in your browser, and a full-precision file on a server in the EU. One flag point serves both. The two runtimes disagree substantially in the middle of the score range, where nothing is decided, and agree to four or five decimal places at the flag points, where everything is.

On this page
  1. The finding
  2. Two routes, one flag point
  3. Proving the rig first
  4. WASM against WebGPU
  5. Where the providers disagree
  6. fp32 against int8
  7. Why the point cannot be lowered
  8. What this does not prove
Published
30 August 2026
Measured
WebGPU parity 30 August 2026; route reconciliation 29 August 2026
Operating point
The shipped pair, 0.9855 / 0.9763, contract segments-v3
Corpus
5,558 long-form documents: 922 AI across 13 models, 4,636 human. 21,093 sections

01 / The finding

A tool that contradicts itself depending on which route ran is worse than one that is slightly miscalibrated on both

A single flag point serves both routes. That is a constraint rather than a convenience, and it holds for a specific and measurable reason: the two runtimes disagree most where no verdict is taken, and converge where the verdict is decided.

Within 0.01 of either flag point the median disagreement between the browser’s two execution providers is 0.000065 across 1,446 section scores. Across the whole score range it is 0.004572, seventy times larger.

The pooled figure is printed here too, because quoting the flag-point figure alone would be quoting the flattering half. Source: WEBGPU-PARITY.md §3, 21,093 sections under segments-v3.
Verdicts that move between the browser’s two providers16 in 5,558

0.288%. All sixteen sit on the flag point; the largest score movement among them is 0.0048. McNemar exact finds neither side distinguishable from no change.

Browser against server, inside the decision region0.0002

Median document-score disagreement across the 1,274 documents scoring 0.97–1.01. In the middle of the range the same comparison reads 0.2609.

Human documents wrongly flagged, by route45 against 90

Both out of 4,636, at the same flag point. The browser route wrongly flags twice as many people as the server route, and that is a property of the quantised file rather than of the threshold.

Those three numbers are the whole page. The first two say a shared flag point is defensible. The third says the two routes are not equally accurate, and no figure below borrows the other route’s accuracy to cover it.

02 / The two routes

Two routes, one flag point

Three comparisons run on this page, and they must never be mixed inside one table. The browser’s two execution providers run the same model file and differ only in kernels and accumulation order. The browser and the server run different model files, because the browser’s is a quantisation of the server’s. And the accuracy each route reports at the shipped pair is a third thing again, measured separately on each runtime.

Figure 1 The two routes, drawn A diagram, not a measurement: no quantity on it is plotted to scale and nothing here was measured to produce it. It names what differs between the routes — the model file, the precision and the runtime — and what does not: the segmenter, the tokeniser and the flag point, which are shared. Every figure further down the page belongs to one branch of this drawing or to the join at the end of it.
The browser route and the server route meet at one shared flag point Your text shipped segmenter shipped WordPiece tokeniser one route runs Browser route tier3-cycle2-e5small-int8-perchannel.onnx int8 per-channel · 34 MB downloaded once onnxruntime-web, int8 dynamic per-channel WASM, or WebGPU where the device allows it Server route · EU tier3-cycle2-e5small-fp32.onnx fp32, full precision · the parent artefact Python onnxruntime 1.29.0, CPU, fp32 the default route One flag point, both routes 0.9855 strongest section, or 0.9763 second-strongest section
Browser route, int8 Server route, fp32 Shared flag point, 0.9855 / 0.9763
What differs between the two routes and what is shared
StageBrowser routeServer route
Segmentation and tokenisationShared. Same segmenter, same WordPiece tokeniser, contract segments-v3
Model filetier3-cycle2-e5small-int8-perchannel.onnxtier3-cycle2-e5small-fp32.onnx
Precisionint8, dynamic per-channelfp32
Runtimeonnxruntime-web, int8 dynamic per-channel, WASM or WebGPUPython onnxruntime 1.29.0, CPU, fp32
Flag pointShared. A document is flagged when its strongest section reaches 0.9855, or its second-strongest reaches 0.9763

Route composition from research-method.ts, which reads the shipped thresholds.json. Detector tier3-cycle2-e5small-fp32.onnx; browser artefact tier3-cycle2-e5small-int8-perchannel.onnx; both derived from intfloat/e5-small, trained 28 August 2026.

Measurement conditions

Corpus
The 5,558-document long-form corpus of 28 August 2026: 922 AI documents across 13 models, 4,636 human documents, 21,093 sections.
Operating point
0.9855 / 0.9763 · contract segments-v3 · T = 0.8324
Detector
tier3-cycle2-e5small-fp32.onnx, SHA-256 e313ab00de1fffd2…4d2788d
Runtime
Server: Python onnxruntime 1.29.0, CPU, fp32. Browser: onnxruntime-web 1.29.0, int8 per-channel, under WASM (ort-wasm-simd-threaded.asyncify.wasm) and WebGPU (ort.webgpu.bundle.min.mjs).
Measured
30 August 2026 (WebGPU parity); 29 August 2026 (route reconciliation)
Also
WebGPU ran on Chrome for Testing 151.0.7922.34 through Playwright, headed, on a local origin with COOP/COEP. The adapter reports vendor apple, architecture metal-3: real Metal, not the SwiftShader software fallback.

The cost is worth recording, because it is the reason this comparison had not been run before. Putting 21,093 sections through WebGPU took roughly 63 minutes of wall clock, about 5.6 sections a second.

03 / The rig

The measurement rig was proved before any number from it was believed

This matters more than the result. A parity measurement that silently falls back to the runtime it is supposed to be comparing against produces a beautiful null result and means nothing. Four checks were run first, and the fourth is the load-bearing one.

The input was frozen, so the provider is the only variable. Every document was segmented with the shipped segmenter and tokenised with the shipped WordPiece tokeniser, and the resulting token id sequences were written to a flat binary file. Both the WASM control and the WebGPU run consume that same file. Neither segmentation nor tokenisation can contribute to any difference reported below.

The committed browser scores reproduce the published figures. Read straight from the committed full-corpus browser run under the shipped rule: 889 of 922 detected at 90 of 4,636 wrongly flagged, and 877 of 922 at 90 of 4,636 under the prior single-threshold 0.984 rule, which no longer ships. The section count is 21,093, matching the segmentation record.

The scoring pipeline reproduces the committed run bit-exactly. Re-scoring the frozen tokens through WASM under Node returns the committed per-section probabilities with a maximum absolute difference of 0.0 over 190 sections. Not within tolerance. Identical.

The same page under WASM reproduces the Node control to 5.6 × 10⁻¹⁷ — one unit in the last place on 1 of 190 sections, and bit-identical on the other 189. So when the identical page with the provider list set to WebGPU diverges from that control by a median 4.96 × 10⁻⁵, the divergence is a property of the execution provider and not of the measurement rig. A silent fallback is excluded by the adapter probe and by this control, rather than assumed away by one of them.

WebGPU is also bit-deterministic run to run: 300 documents and 1,776 sections re-scored in a fresh browser session came back 1,776 of 1,776 bit-identical. That is what turns “sixteen documents moved” from an anecdote into a measurement.

04 / WASM against WebGPU

The browser’s two execution providers

A visitor with a capable GPU gets WebGPU; everyone else gets WASM. Different kernels, different accumulation order, the same int8 file. The shipped pair was fitted on WASM, so it was unproven for most real visitors until this was run.

Detection and false positives by execution provider at the shipped flag point
Execution providerAI detectedHuman false positives
WASM889/922 = 96.42%90/4,636 = 1.941%
WebGPU885/922 = 95.99%92/4,636 = 1.984%

Both rows are the int8 per-channel browser artefact at the shipped pair 0.9855 / 0.9763, contract segments-v3, over the same 5,558 documents. Neither row describes the server route. Source: WEBGPU-PARITY.md §2.1.

Paired, document by document, the two providers produce 16 verdict changes in 5,558, or 0.288%. Four AI documents lost and none gained; seven human documents newly flagged and five cleared. McNemar exact, two-sided: AI 4 against 0, p = 0.125; human 7 against 5, p = 0.774. Neither movement is distinguishable from no change.

The AI side is one-directional, which is worth saying plainly rather than hiding behind a p-value. All four movements go the same way, and with four discordant pairs that is the smallest count that could ever reach significance — and it does not. All sixteen documents sit on the flag point. The largest score movement among them is 0.0048 and the median is under 0.002. Not one is a document whose score changed meaningfully.

05 / Where they disagree

Where the two providers actually disagree

Figure 2 WASM against WebGPU: median absolute divergence by score band, with the flag points named Rows are score bands, so the flag points are named on the band axis rather than drawn as a vertical rule: the horizontal axis is divergence, not score. Both shipped flag points — the primary 0.9855 and the secondary 0.9763 — fall inside the 0.97–0.99 band, and that band is where the two providers agree most closely, at a median 0.000075 against 0.017520 in the mid-range where nothing is decided. Bands are the band the WASM second-highest section score sits in; n on each row is the number of sections in it, and the right-hand column is that band's worst single section. 21,093 sections in total, same int8 file on both providers, contract segments-v3. Logarithmic axis.
Median absolute divergence between the WASM and WebGPU execution providers, by score band, on a logarithmic axis 0.00001 0.0001 0.001 0.01 0.1 Worst section 0.00–0.50 n = 2,770 sections 0.007869 0.138380 0.50–0.90 n = 1,258 sections 0.017520 0.171313 0.90–0.97 n = 449 sections 0.003433 0.029436 0.97–0.99 · both flag points sit here n = 802 sections 0.000075 0.007346 0.99–1.01 · above both flag points n = 207 sections 0.000017 0.000073
Median |Δ|, WASM against WebGPU, 21,093 sections
Divergence between WASM and WebGPU by score band
WASM second-highest section sits innMedian |Δ|p99Max
0.00–0.50 2,7700.0078690.0790790.138380
0.50–0.90 1,2580.0175200.1018380.171313
0.90–0.97 4490.0034330.0222570.029436
0.97–0.99 — both flag points sit here 8020.0000750.0030660.007346
0.99–1.01 — above both flag points 2070.0000170.0000600.000073

docs/measurements/WEBGPU-PARITY.md §3, measured 30 August 2026. Same model file on both providers; the only variable is the execution provider. No interval is published for these medians, so the counts are printed instead. This is a divergence between two runtimes, not a detection rate, and no operating point applies to any value on the axis.

Restricted to section scores within 0.01 of either flag point, which are the only scores that can change anything, the picture is the same: n = 1,446, median 0.000065, 99th percentile 0.004231, maximum 0.007346. Quoting the pooled figure alone would overstate the risk by roughly a hundredfold, so the pooled figure is given as well: across all 21,093 section scores the median absolute difference is 0.004572 and the maximum is 0.205359.

The premise this comparison started from did not survive it.

The concern going in was that the two providers diverge most between 0.90 and 0.98, which is exactly where the secondary flag point sits. That is true of WASM against fp32, which is where the observation came from. It is false of WASM against WebGPU.

Those two run the same int8 file and differ only in kernels and accumulation order, and that difference is proportionally largest in the mid-range where the logit gap is small, not near saturation where both providers are pinned against the same ceiling. Nine of the sixteen changed verdicts flip on the primary parameter and eight on the secondary, one on both, which is close to proportionate to how crowded the two points are: 137 documents sit within 0.002 of the primary against 40 within 0.002 of the secondary. The primary at 0.9855 is the more exposed of the two, the reverse of what was assumed.

06 / fp32 against int8

The harder comparison: the server’s file against the browser’s

Two things differ at once here, the runtime and the model file. The int8 per-channel artefact is a quantisation of the fp32 parent, and quantisation is not a neutral operation. What follows is therefore not a runtime comparison in the sense the previous section was.

Figure 3 The two routes at the shipped flag point Both routes over the same 5,558 documents at the shipped pair 0.9855 / 0.9763, contract segments-v3. Each series is labelled with the runtime that produced it, and neither route's figure is ever quoted for the other. The browser route catches slightly more machine writing and wrongly flags twice as many people. The percentage axis is logarithmic and starts at 0.5%, because a 1% rate and a 96% rate cannot both be read on a linear one; the counts on every bar are the record. No confidence interval is published at this operating point, so the denominators are printed instead.
Detection and false-positive rates for the server fp32 route and the browser int8 route at the shipped flag point, on a logarithmic percentage axis 0.5% 1% 2% 5% 10% 25% 50% 100% AI documents detected 922 AI documents, 13 models 883/922 (95.8%) 889/922 (96.4%) higher is better Human documents wrongly flagged 4,636 human documents 45/4,636 (0.97%) 90/4,636 (1.94%) browser flags 2× as many
Server route, fp32 — the fp32 runtime our EU server runs Browser route, int8 under WASM — the int8 runtime that runs in your browser
Detection and false positives by route at the shipped flag point, with the register split
RouteMeasured onAI detectedHuman false positives
Server, fp32the whole corpus883/922 (95.8%)45/4,636 (0.97%)
Browser, int8 (WASM)the whole corpus889/922 (96.4%)90/4,636 (1.94%)

services/local-engine/research/corpus-reconciliation-2026-08-29/v3-recommended.txt; reproduced in WEBGPU-PARITY.md §2.1 and AGGREGATION-AND-RHYTHM.md §6. The four values are read on this page from thresholds.json through published-figures.ts, the same file the browser fetches at run time, rather than retyped.

The false-positive rates differ by a factor of two, and both are printed. That is not a parity failure. They are two different artefacts, and a page that published only the flattering one would be misleading about which route the reader is actually using.

The browser route wrongly flags 90 of 4,636 human documents. The server route wrongly flags 45 of the same 4,636, at the same flag point.

Each route reports the accuracy measured on the runtime that produced it. Neither borrows the other’s. If you ran the check in your browser, the browser row is your row.

The gap is not evenly spread across registers, and the two registers where it is widest are published rather than averaged away.

Human false positives by register and route at the shipped flag point
Human registerServer, fp32Browser, int8 (WASM)
Fiction23/260 = 8.85%26/260 = 10.00%
Academic discussion8/420 = 1.90%21/420 = 5.00%

Human false positives by register at the shipped pair 0.9855 / 0.9763. Register labels are machine-assigned. Source: corpus-reconciliation-2026-08-29/v3-recommended.txt.

Verdict disagreement between the two routes at the shipped pair is 54 of 5,558, or 0.97%. Adding WebGPU to the picture does not widen it at all: fp32 against WASM reads 54/5,558, and fp32 against WebGPU reads 54/5,558. The same figure to the document.

Figure 4 fp32 against int8: median absolute divergence by document-score band The same kind of plot as Figure 2, deliberately on its own axes and never on Figure 2's. Two reasons. The values here run two orders of magnitude above Figure 2's, so a shared axis would flatten Figure 2 into a line. And the comparisons are not commensurable: Figure 2 varies the execution provider alone, while this varies the model file as well, because the browser's int8 artefact is a quantisation of the server's fp32 parent. Both flag points fall inside the 0.97–1.01 band, and that band is again where the two agree most closely, at a median 0.0002 against 0.2609 in the mid-range. Bands are the band the browser document score sits in; n is the number of documents. 5,558 documents, measured under segments-v2. Logarithmic axis.
Median absolute divergence between the fp32 server artefact and the int8 browser artefact, by document-score band, on a logarithmic axis 0.0001 0.001 0.01 0.1 1 Worst document 0.00–0.50 n = 1,788 documents 0.1003 0.4521 0.50–0.90 n = 1,604 documents 0.2609 0.7562 0.90–0.97 n = 892 documents 0.0710 0.7480 0.97–1.01 · both flag points sit here n = 1,274 documents 0.0002 0.1760
Median |Δ|, fp32 against int8, 5,558 documents, segments-v2
Divergence between the fp32 and int8 artefacts by document-score band
Browser document score sits innMedian |Δ|p90Max
0.00–0.50 1,7880.10030.25390.4521
0.50–0.90 1,6040.26090.42240.7562
0.90–0.97 8920.07100.23430.7480
0.97–1.01 — both flag points sit here 1,2740.00020.01200.1760

services/local-engine/research/corpus-reconciliation-2026-08-29/browser-fullcurve.txt, measured 29 August 2026 under segments-v2 — an earlier segmentation contract than the segments-v3 the tool ships. No interval is published for these medians; the counts are printed instead. A divergence between two artefacts, not a detection rate.

A median disagreement of 0.26 in the middle of the range is not small. It is also not decisive, because no verdict is taken there. What makes one flag point defensible across both artefacts is the last row, and the last row alone.

07 / The consequence

Below about 0.97 a shared flag point stops existing

The last row of Figure 4 carries an obligation. If the two routes only agree closely above 0.97, then any argument for lowering the flag point — to catch more machine writing — has to answer what happens to route agreement on the way down. It was measured over the same corpus under the earlier single-threshold rule.

Route disagreement by single threshold, under the retired single-threshold rule
Retired single thresholdDocuments where the routes disagreeShare of corpus
0.98448/5,5580.86%
0.98082/5,5581.48%
0.970171/5,5583.08%
0.950369/5,5586.64%
0.8082757/5,55813.62%

Almost every one of those disagreements runs the same way, with the browser flagging where the server clears. That direction is the same one Figure 3 shows at the shipped pair, and it is the reason lowering the point costs route agreement faster than it buys detection.

Measured at a rule that no longer ships

Corpus
The same 5,558-document long-form corpus, scored on both routes.
Retired flag point
0.984, 0.980, 0.970, 0.950 and 0.8082, single-threshold
Detector
tier3-cycle2-e5small-fp32.onnx, SHA-256 e313ab00de1fffd2…4d2788d
Runtime
Server fp32 against browser int8 under WASM.
Measured
29 August 2026, under segments-v2
Also
These are route-disagreement counts under the retired single-threshold rule, which flagged on the strongest section alone. They are not detection rates and they do not describe the rule that ships.

The rule that ships today is 0.9855 / 0.9763. The figures under this stamp answer the same question at a flag point this tool no longer uses, and they are not a description of the tool as it runs now.

08 / Limits

What this does not prove

One GPU, one browser, one model file. Apple Metal through Chromium on macOS. A Windows visitor on D3D12 or an Android visitor on Vulkan runs different kernels again. What is established is that the provider boundary is worth about sixteen documents in 5,558 on this hardware. The mechanism — that the two providers agree to five decimals wherever the logit is saturated — is the part that should generalise. That is a reason to expect other backends to land in the same place, not an assurance that they do.

The two routes are not shown to have equal accuracy. This page claims corpus-scale parity between the browser’s two providers, and agreement between browser and server inside the decision region. It does not claim the fp32 and int8 artefacts are equally accurate, and Figure 3 shows they are not.

The browser has never had a detection-by-length cut. The full-corpus browser threshold curve exists, measured 29 August 2026 under segments-v2. The per-length-band table on which the tool’s length disclosures rest is fp32 only. A visitor pasting an 800-word blog post into the browser route has no band figure of their own.

The mixed-content cost was not re-measured on WebGPU. The 604/700 half-AI figure is fp32 and stays fp32.

The shipped pair was fitted from WASM under headless Node. WebGPU was measured afterwards, and holds. It was not part of the fit.

Register labels are machine-assigned. The fiction and academic-discussion rows above inherit whatever error that classification carries.

The corpus is not fully held out.

Of the 922 AI documents, 268 (29.1%) appear in the cycle-2 dataset and 168 of those in the training split. The human side is effectively clean at 11 of 4,636. For a same-documents comparison of two runtimes this does not matter, because the same weights read the same tokens both times. It would matter for an absolute accuracy claim, and Figure 3 is one, so its rates carry this qualification with them.

Source: corpus-reconciliation-2026-08-29/analysis.txt §2.

The older route-parity record must not be used.

Its entire framing is browser 0.984 against server 0.980, a mismatched pair corrected twice since, and only 3 of its 60 server-side scores came from the live container: the service returned 404 mid-run and the other 57 are local fp32. Its 60-document corpus is also tiny beside 5,558. Its one still-usable qualitative point is that all three of its disagreements lay inside the corridor between the two thresholds.

Provenance

Source file and section for every figure on this page
FigureFileSection
WASM against WebGPU: 889/922 and 90/4,636; 885/922 and 92/4,636; 16 verdict changes; McNemar p = 0.125 and p = 0.774docs/measurements/WEBGPU-PARITY.md§2.1
Divergence by score band; 1,446 sections within 0.01 of a flag point at median 0.000065; pooled median 0.004572docs/measurements/WEBGPU-PARITY.md§3
Rig controls: 0.0 over 190 sections, 5.6 × 10⁻¹⁷ page-against-Node, 1,776/1,776 bit-identicaldocs/measurements/WEBGPU-PARITY.md§1
Both routes at the shipped pair: 883/922 and 45/4,636; 889/922 and 90/4,636; fiction and academic discussioncorpus-reconciliation-2026-08-29/v3-recommended.txt; public/models/local-signals-v1/thresholds.json§2.1, §6
fp32 against int8 divergence by band; route disagreement by single threshold (segments-v2)corpus-reconciliation-2026-08-29/browser-fullcurve.txtwhole file
54/5,558 route disagreement at the shipped pair, fp32 against WASM and against WebGPU alikedocs/measurements/WEBGPU-PARITY.md§2.2
268/922 and 168 train-split overlap; human 11/4,636corpus-reconciliation-2026-08-29/analysis.txt§2

Every file named above ships with the open measurement repository, so any figure on this page can be re-run rather than taken on trust.

Read on

Two runtimes agreeing is one question. What either of them is worth on your own text is the next.