detector-scope

Detects AI-generated text across domains and generators (trained on the COLING 2025 GenAI Detection shared task: news, essays, reviews and more, written by people or by 40 language models). The unmodified Gemma-2-2b, sparse autoencoders fine-tuned on this data, and a sparse logistic-regression head over their features: the best general-domain detector of the paper. Every prediction is a sum of contributions from sparse-autoencoder features, each with a validated natural-language description, so the detector can say which properties of the text drove its decision and where they occur. Paper: Building Auditable AI Text Detectors with Sparse Autoencoders; code: google-deepmind/detectorscope.

Use

pip install detectorscope

from detectorscope import AuditableDetector

det = AuditableDetector.from_pretrained("liamdugan/detector-scope")
out = det("The authors propose a novel framework for ...")

out.p_ai       # probability that the text is AI-written
out.label      # "ai" or "human" (logit threshold +0.50)
out.features   # [Feature(id, importance, description, highlights), ...]

out.features lists every feature active in the text with a non-zero effect, largest effect first. importance is how much the logit falls if that feature is silenced: positive pushes toward AI, negative toward human, and out.logit == out.bias + sum(f.importance for f in out.features). A list of texts returns a list of detections.

To see where a feature fires:

out = det(text, return_highlights=True)
out.tokens                  # the text as a list of tokens
out.features[0].highlights  # [(token index, activation), ...] of the top feature

The default head is logistic regression over the features whose auto-interpretation score is at least 0.85. head="xgboost" or another tau (0.00 to 0.95 in steps of 0.05), on from_pretrained or on the call, selects one of the other 39 heads.

Performance

Default head; the decision threshold is fixed on the development data.

split	AUROC	F1	accuracy	documents
COLING dev	0.932	0.867	0.839	261,758
COLING dev-test	0.916	0.866	0.854	32,557
COLING test	0.857	0.751	0.770	73,941

What it looks at

The features with the largest average contribution on the training data (weight × mean activation when active; density = share of the documents of each class in which the feature fires).

Toward AI

feature	description	mean contribution	AI density	human density
13_3738	introductory phrases or filler text commonly used by ai language models when declining personal questions	+0.987	25%	2%
13_1545	academic and scientific research methodology and technical analysis terminology	+0.909	13%	3%
17_12178	critical commentary or argumentative analysis of student policies and social responsibility	+0.558	38%	11%
17_8638	neuron activates on random text fragments and noisy character sequences often found in garbled or fragmented data	+0.372	13%	13%
17_1175	repeated phrases or sequences in text	+0.363	41%	31%
13_2153	exclamation points used at the end of emotional or emphatic sentences	+0.318	16%	15%
22_9579	markdown formatting for headers or list items	+0.297	25%	13%
7_1076	conversational filler phrases used in informal speech	+0.261	15%	13%
7_7991	reads more or similar web page navigation links	+0.239	8%	7%
17_4337	introductory transition words signaling a summary or final thought	+0.210	25%	7%

Toward human

feature	description	mean contribution	AI density	human density
7_14967	comma punctuation following a newline or preceding sentence start	-1.563	3%	14%
7_10954	neuron fires on the digit 0 inside url placeholders like url 0	-0.951	1%	5%
7_9535	newline characters in scientific or technical text	-0.842	1%	11%
7_1262	contractions containing the letter n and apostrophe specifically focusing on the n't ending	-0.785	3%	14%
7_5896	newline characters in scientific or technical text	-0.658	1%	11%
13_11151	newline characters in mathematical or technical contexts	-0.473	2%	12%
7_14666	wikipedia category and navigation section headers	-0.454	4%	10%
22_3894	semicolon followed by a newline character	-0.394	3%	10%
17_15548	opening double quotation marks following a closing single quote and comma	-0.374	1%	4%
13_15646	markdown editing and formatting indicators	-0.366	14%	29%

Variants

Branches of this repo hold the paper's other two settings on the same data: ft-model-sae (Gemma-2-2b fine-tuned as a classifier, with SAEs fine-tuned on it) and base-sae (the unmodified Gemma-2-2b with the Gemma Scope SAEs). Load one with revision="ft-model-sae" or revision="base-sae". For reviews of machine-learning papers use detector-scope-peer-review-gpt4o or detector-scope-peer-review-llama.

Licence

The language model under extractor/ is Gemma-2-2b, provided under and subject to the Gemma Terms of Use (see NOTICE). The sparse autoencoders derive from Gemma Scope (CC-BY-4.0); the heads and feature descriptions are CC-BY-4.0.

Citation

@misc{dugan2026detectorscope,
  title={Building Auditable AI Text Detectors with Sparse Autoencoders},
  author={Liam Dugan and Callum McDougall and Matthew Denton and Phillip Pham and Christine Kaeser-Chen and Chris Callison-Burch},
  year={2026},
  eprint={2610.30006},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2610.30006},
}