Measured, not promised
Every number on this page comes from a benchmark you can rerun from the open-source repo. Two detectors, two benchmarks, and the failure modes we know about.
How detection works
Detection is a fine-tuned ModernBERT classifier (149M parameters) trained on tens of thousands of texts: verified pre-LLM human writing across nine domains, genuine frontier-model output, and a diverse "generic AI" class. It scores how likely a passage is AI, then attributes likely AI text to a named model or flags it as an unknown one. You choose where it runs: on-device (in your browser via WebGPU, so the text never leaves your machine) or cloud (the same model file on our servers, for browsers without WebGPU). Both run the identical model and decision rule.
Earlier versions offered a stylometric heuristic as a second mode. We benchmarked it on the same external set and retired it: it flagged nothing at all, with an AUROC of 0.38 against current-generation AI, meaning it scored modern chatbot text as more human than human text. The code remains in the open-source engine; we just won't sell you a detector that doesn't detect.
Headline results
The decision rule behind these numbers is the same one the site ships: pooled AI probability against a fixed threshold. We calibrated that threshold on the cross-dataset benchmark below, tuning for the false-positive side first, because calling a human writer an AI is the worst mistake a detector can make.
Cross-dataset benchmark
The harder test: raw text from datasets the model never saw in training, with no preprocessing beyond a length filter. Detection rate is the share of each source the detector got right (flagged AI text as AI, passed human text as human). The weak rows are shown too. That is the point of this page.
Method & limitations
- The held-out split is a stratified 80/10/10 with dedup before splitting; the test split is never touched during training or threshold tuning.
- Every cross-dataset source is absent from the training config entirely, including the datasets our "human" and "AI" training text came from.
- Model attribution is calibrated against a generator the classifier never trained on, so text from an unfamiliar AI gets flagged "unknown model" rather than force-named.
- Known limitations: short texts (under ~80 words) carry less signal, so the tool marks those verdicts low-confidence. Casual short-form human writing (wire snippets, quick forum answers) is the main false-positive risk, and 2022-era generators like GPT-3 largely evade detection. Heavily edited or paraphrased AI text can pass. No detector is proof. Treat scores as evidence, not verdicts.
- What's next: v4 trains on exactly these weak spots, adding short casual human text as hard negatives and legacy-generator output to the AI classes.
Training data, evaluation scripts, and the exported model are in the open-source repos: engine · models.