AI Fiction Has a Structural Fingerprint That Style Edits Can't Erase
StoryScope, built at U. Maryland with Google DeepMind, detects AI fiction at 93.2% by reading how plots unfold, surviving mimicry attacks that collapse older style detectors.
StoryScope, built at U. Maryland with Google DeepMind, detects AI fiction at 93.2% by reading how plots unfold, surviving mimicry attacks that collapse older style detectors.
A machine learning pipeline called StoryScope can now tell AI-authored fiction from human-written stories by reading how plots escalate and resolve, not how sentences are phrased, reaching 93.2% macro-F1 accuracy on a 61,608-story corpus. The system holds at roughly 93.9% even when writers deliberately mimic human style, a trick that drops conventional style detectors from 97% to 3% in a single pass.
The paper, posted this month as an arXiv preprint by a team at the University of Maryland and Google DeepMind, attacks detection from a direction most prior work has ignored. Where earlier systems read lexical surface features (word choice, sentence rhythm, punctuation habits), StoryScope extracts 304 narrative-level features across ten dimensions aligned with the NarraBench benchmark, then uses 30 of them to do most of the work. The result is a model that reads the architecture underneath the prose.
The corpus pairs 10,272 human prompts drawn from Books3, the controversial dataset of book excerpts widely used for training large language models, with outputs from five frontier models: Claude Sonnet 4.6, GPT-5.4, Gemini 3 Flash, DeepSeek V3.2, and Kimi K2.5. Each prompt produced six stories at roughly 4,753 words apiece, giving the pipeline enough mass to learn patterns a human reader could not.
Across those 61,608 stories, AI-generated fiction over-explains its themes and resolves plot threads in tidy, single-track progressions. Human stories let protagonists make morally ambiguous choices, carry more temporal complexity, and leave thematic residue unresolved. The fingerprint lives in the deep grammar of how a story thinks, not in how it sounds, which is why redacting the surface does not redact the underlying signal.
The team's six-way authorship attribution reaches 68.4% macro-F1 across the five LLMs, with distinctive per-model signatures. Claude produces flat event escalation. GPT-5.4 overuses dream sequences. Gemini defaults to external description over interiority. Each model has a structural habit, suggesting that detection at scale could eventually extend beyond binary human-versus-machine classification into model-specific forensics.
Prior research, cited inside the StoryScope paper, showed that fine-tuning a language model to mimic human style collapses style-based detectors from 97% accuracy to 3% in one pass, a published weakness that has shaped how literary agents, prize committees, and magazine editors treat AI-origin questions. StoryScope's structure-only model retains roughly 93.9% accuracy on those same style-edited stories. The mimicry breaks the surface reader and leaves the structural one untouched.
In late June, Granta ended its partnership with the Commonwealth Writers' Prize after disputed AI authorship allegations around Jamir Nazir's "The Serpent in the Grove," a Caribbean regional prize entry. The Foundation reviewed drafts, timestamps, and outlines and cleared the authors; Granta exited anyway. The case has become a reference point in publishing's debate over what counts as proof.
StoryScope does not yet resolve that debate. The work is an arXiv preprint, version 4, and has not cleared peer review at a venue the team names. Independent replication of the 93.2% number has not been published. The Books3 origin of the human prompts also raises copyright and consent questions that the paper acknowledges but does not settle.
Replication will determine whether the structural fingerprint holds. If it does, future literary disputes will hinge on plot architecture rather than prose style, because detectors that read how stories think will outlast ones that read how they sound.