How the detector works

Cobalynx combines multiple independent families of signals.

Statistical language analysis. We measure statistical properties of the text itself — regularities in how words and sentences are chosen and assembled that differ, measurably, between machine-generated and human writing. Human prose carries odd, personal, mildly “wrong” choices; machine text tends to be more uniform in ways these measurements pick up. Several independent statistical signal families are computed and combined, and they are model-agnostic: they don't require knowing which AI wrote the text.

Learned pattern analysis. We also run analysis trained on labeled corpora of human and AI writing from the current model era. This family catches what the statistical analysis misses, and vice versa. The final score fuses the independent families and is calibrated against measured outcomes, so the two kinds of evidence check each other.

What we do not do: we cannot check for provider watermarks (Google's and Anthropic's text watermarks are keyed; only the key-holder can detect them — see our watermark explainer), and we do not ask a chatbot “does this look like AI?” — published evaluations show that approach performs near chance.

File provenance checks

Alongside the text scan, the file check examines uploaded files for provenance — deterministic evidence where it exists, honestly graded where it does not. Every provenance signal is reported in three states — found, definitively not found, or could not be evaluated — never collapsed into a verdict:

  • Images (PNG, JPEG). We cryptographically verify C2PA Content Credentials: verified (signature and content hashes intact against our configured trust anchors, with the signer and any recorded AI-use assertions shown), failed verification (a manifest is present but broken — the file was altered after signing, so its claims are unreliable), or no credentials. The last is the normal case and is not evidence of human authorship — most files carry no credentials, however they were made.
  • Documents (DOCX, PDF). Metadata heuristics only: we read the document's creator, last-modified-by, application and producer fields and flag known AI-tool strings. Metadata is typed by software and trivially edited — these are weak signals that hint at the tool that touched a file, never proof of how its words were written.
  • Pasted text. Pasting strips file-level provenance, so the text scan checks only for invisible Unicode artifacts (zero-width and control characters) — reported as a weak hint of a machine pipeline, and never fed into the score.

What the scores mean

Our headline number is a calibrated probability, not a verdict. A score of 90% means: among texts in our evaluation set that scored like yours, about 9 in 10 were AI-generated. The calibration is derived from measured outcomes, and the detector's underlying ranking quality on the frozen evaluation set is abstention-free AUROC 0.952 on the clean evaluation split, 0.938 when non-native human writing is included.

Every result lands in one of three states:

  • Likely AI — the calibrated probability exceeds our strict threshold, currently operating at a measured false-positive rate of 0.5% (3 of 609; 95% CI 0.2%–1.4%) on our evaluation set.
  • Inconclusive — the signals genuinely disagree, or the text is too short or too edited to score responsibly. We report this state deliberately and often; a detector that never says “I don't know” is lying somewhere.
  • Likely human — statistical evidence points away from AI generation. Note the word likely: we never report “human-verified,” because no technology today can prove human authorship — and text watermarked by a provider can still read “likely human” to every statistical tool on the market.

Known failure modes

We publish these because you cannot interpret a score without them.

  • Short text. Statistical evidence accumulates with length. Below roughly 150–200 words, all detectors degrade sharply; we refuse or heavily caveat short inputs rather than guessing. Our accuracy by length band: at 300–1,000 words we detect 81.6% of clean AI documents (346 of 424 non-abstained) at a 0.6% human false-positive rate (3 of 525); at 1,000+ words, 100.0% (18 of 18) with 0 false positives in 20 human documents; below 300 words the clean-AI evaluation slice is empty by design — we abstain or refuse rather than guess.
  • Heavy paraphrasing and “humanizers.” Adversarial rewriting demonstrably breaks statistical detectors — the RAID benchmark (ACL 2024) showed simple attacks defeat nearly every published detector, and detector-guided paraphrasing is stronger still. Our recall under our adversarial evaluation split: 33.5% of non-abstained adversarial documents (73 of 218; 23.5% counting abstentions as misses). Assume a determined evader can beat us.
  • Unseen registers and new models. Detectors are trained on the model era they know. Genres underrepresented in training (legal boilerplate, poetry, highly technical prose) and brand-new model families shift the distribution. We version every engine change (currently engine v1), and each engine change requires a fresh sanctioned measurement against the frozen evaluation set — so if a new model era degrades us, our published numbers show it.
  • Non-native English speakers. This is the field's most serious documented bias: a 2023 Patterns study found commercial detectors misclassified over half of real TOEFL essays as AI, because simpler L2 vocabulary reads as “predictable.” Our mitigations: we never use raw predictability alone, we set our operating point against development corpora that include L2 writing (the verdict thresholds must pass a per-register false-positive gate that includes non-native text), and we publish our measured false-positive rate on non-native text: 1.7% (2 of 118; 95% CI 0.5%–6.0%). The bias cannot be fully eliminated — which is one more reason scores must never be treated as proof.

Why we publish our numbers

Most detectors advertise a single accuracy figure — “99%!” — with no attack model, no false-positive rate, and no operating point. Independent evaluations routinely measure far lower real-world performance, and the EU's own Code of Practice on AI-content transparency (July 2026) concluded that forensic AI-text detection is “not yet considered reliable enough.” We think the honest response is not to hide that finding but to build around it: publish the evaluation, publish the error rates, and let you see exactly what “likely” means.

So this page shows our current numbers — AUROC 0.952, false-positive rate 0.5% (3 of 609; 95% CI 0.2%–1.4%), false-negative rate 17.6% (78 of 442; 95% CI 14.4%–21.5%) at the default operating point — measured on our documented, reproducible evaluation set, not on cherry-picked demos.

The eval-run disclosure principle. Every figure on this page comes from a versioned, dated evaluation run (engine v1, manifest v5, seed 20260816, August 16, 2026) against a frozen corpus whose composition we document. When we change the model, the thresholds, or the corpus, the numbers on this page change with it — and the changelog says so. If our performance degrades against new models, this page will show it before our marketing does. The full tables, the run ledger and the reproduction command live on the evidence page.

Never sole evidence

A Cobalynx score must never be the sole evidence for punishing anyone — failing a student, firing a writer, rejecting a submission. This is not boilerplate humility; it is the documented consensus. OpenAI shut down its own AI-text classifier in July 2023 “due to its low rate of accuracy.” Vanderbilt and dozens of other universities disabled Turnitin's AI detection over false positives, particularly against non-native speakers. At any realistic false-positive rate, screening thousands of honest documents will flag innocent people.

Use our scores the way they are built to be used: as one probabilistic signal, to be weighed alongside drafts, process evidence, and conversation with the writer. It is the stated policy of this site that Cobalynx output must never be used as sole grounds for academic or employment sanctions — and we mean it.

Sources

  1. Dugan et al., “RAID,” ACL 2024, arxiv.org/abs/2405.07940 (adversarial-attack findings)
  2. Liang et al., “GPT detectors are biased against non-native English writers,” Patterns, 2023, arxiv.org/abs/2304.02819
  3. Krishna et al., “Paraphrasing evades detectors of AI-generated text,” NeurIPS 2023, arxiv.org/abs/2303.13408; adversarial paraphrasing, arxiv.org/abs/2506.07001
  4. OpenAI, classifier retirement notice (Jul 20, 2023), openai.com/index/new-ai-classifier-for-indicating-ai-written-text/
  5. Vanderbilt University, “Why we're disabling Turnitin's AI detector” (Aug 16, 2023)
  6. TechPolicy.Press, “The EU's AI Transparency Code of Practice, Explained” — the Code's reliability finding (accessed Aug 16, 2026)
  7. Jabarian & Imas, NBER Working Paper 34223 (2025) — independent detector audit and cost-based policy framework