{"id":755545,"date":"2026-07-10T22:57:10","date_gmt":"2026-07-10T22:57:10","guid":{"rendered":"https:\/\/www.newsbeep.com\/us\/755545\/"},"modified":"2026-07-10T22:57:10","modified_gmt":"2026-07-10T22:57:10","slug":"ai-fiction-is-easy-to-detect-because-its-stupid-and-bad-research-finds","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us\/755545\/","title":{"rendered":"AI Fiction Is Easy to Detect Because It’s Stupid and Bad, Research Finds"},"content":{"rendered":"

Fiction written by artificial intelligence is easy to detect because it struggles with complex story structure and tends to moralize in clunky ways, according to a preprint study from researchers at University of Maryland, College Park and Google DeepMind. They found that AI fiction has tells that go beyond stereotypical overuse of em-dashes and other obvious AI tropes and have more to do with the formulaic nature of the text itself.<\/p>\n

\u201cAI stories over-explain themes and favor tidy, single-track plots while human stories frame protagonists\u2019 choices as more morally ambiguous and have increased temporal complexity,\u201d the study, which looked at more than 50,000 AI-generated short stories, found. \u201cClaude produces notably flat event escalation, GPT over-indexes on dream sequences, and Gemini defaults to external character description. We find that AI-generated stories cluster in a shared region of narrative space, while human-authored stories exhibit greater diversity. More broadly, these results suggest that differences in underlying narrative construction, not just writing style, can be used to separate human-written original works from AI-generated fiction.\u201d<\/p>\n

Basically, AI-generated fiction sucks and at the moment is easy to detect. The typical method of detection involves looking for stylistic markers such as an abundance of em-dashes, the overuse of the word \u201cdelve,\u201d<\/a> or an obsession with goblins<\/a>, but this project tried something different. \u201cThe idea for this project came because we are hoping to eventually move past plain text detection, into some sort of space where we can separate human ideas from AI-generated ideas,\u201d Jenna Russell, a University of Maryland researcher and one of the study\u2019s authors, told 404 Media. Russell is also an intern at the AI-detection company Pangram.<\/p>\n

Russell and her team decided to attempt to detect what she called \u201cnarrative features\u201d in AI- generated fiction. The detector is called StoryScope and it builds on NarraBench, a 2025 benchmark that suggested a taxonomy of narrative features in fiction. StoryScope looked at how fiction handled plot development, character descriptions, setting, and temporal structure to determine if something was written by a human or an AI.<\/p>\n

\u201cIt was my first attempt at getting ‘under the surface’ and focusing more on ideas,\u201d Russell said. \u201cWe wanted to see how close to typical AI-detection we could get by only relying on the narrative features, to understand if this sort of structural difference really even exists. This method also adds some interpretability to detection, which is an open question in the field. Using narrative features, we can point to certain tangible features (such as the number of subplots included in a story). I think this is why it’s struck a chord recently, people can really say \u2018ah these are some of the underlying traits of how AI writes fiction.\u2019\u201d<\/p>\n

To test StoryScope, the researchers selected 10,272 human-written stories then reverse engineered them into writing prompts using Gemini 2.5. Then it took those thousands of prompts and fed them into Gemini 3 Flash, DeepSeek V3.2, Claude Sonnet 4.6, Kimi K2.5, and GPT 5.4. All of the data \u2014 including the prompts and the resulting AI stories \u2014 are available on Hugging Face<\/a>.<\/p>\n

To source the stories, the researchers used the Books3 dataset \u2014 a database of 183,000 books collected from pirated ebooks<\/a>. The dataset is the subject of several lawsuits<\/a> and has been used to train an unknown number<\/a> of LLMs. The StoryScope study included more than 10,000 of some of the most famous short stories ever written, many of them pulled from popular anthologies. There\u2019s Joyce Carol Oates, Stephen King, Louis L’Amour, Charlotte Perkins, and Harlan Ellison. All have been rendered down to their base elements by AI and then regurgitated into a different LLM to see if it can replicate them.<\/p>\n

Russell told me the dataset was controversial. \u201cHence why we do not release it to the public,\u201d she said.<\/p>\n

The study itself contained a disclosure. \u201cWe acknowledge the copyright issues related to the Books3 dataset and do not endorse its use for model training or commercial text generation,\u201d it said. \u201cThe use of the dataset in our paper is restricted to academic purposes only and is meant to understand the narrative differences in human-written and AI-generated text to help inform discussions on AI-detection, authorship, and copyright policy.\u201d<\/p>\n

The various AIs, of course, can\u2019t possibly replicate the prose of O. Henry. So what, according to StoryScope, are the narrative quirks of LLM-written simulacra of English\u2019s grand works of fiction?\u00a0<\/p>\n

AI tools tend to over explain themes, for one.\u00a0<\/p>\n

\u201cNarrators explicitly explain the story\u2019s theme 77% of the time, versus 52% for humans: a grieving character\u2019s arc will typically end with the narrator stating the lesson learned. AI dialogue serves philosophical debate more often (59% vs. 34%), and references to other works tend to be vague allusions (72% vs. 50%) rather than specific, named references. The pattern is one of over-determination: AI spells out meaning rather than trusting the reader to infer,\u201d the study said.<\/p>\n

AI also more often avoids subplots and fails to play with time jumps and flashbacks. The systems overwrite passages about the body and senses. \u201cWhere a human author might write that a character \u2018felt afraid,\u2019 AI renders fear as a tightening chest, cold sweat, and dimming lamplight,\u201d the study said. Humans also spin more complicated narratives involving more characters and locations than AI can handle. Humans also reference other works of fiction, specific people and places in a way that AI struggles with.<\/p>\n

A disclosure caught my eye at the bottom of the StoryScope study. \u201cLarge language models and coding agents (Claude Code and Codex) are used to aid with and polish writing and generate some tables and plots,\u201d it said.<\/p>\n

\u201cI believe it’s important to disclose AI use (and ideally think it should be more in-depth than I wrote in the paper),\u201d Russell told me. \u201cMost researchers are using AI, a lot of it seemingly ‘slop’ […] but a lot of it is high-effort, good research. Also, technically you are supposed to disclose AI use for conference submissions, but most people don’t. I want to help change that norm!\u201d<\/p>\n

She also explained a bit more about how AI agents helped shape the project. \u201cI use AI agents to help implement the code (using the claude code \/ codex interfaces). I also use them as an editor during the writing process! They have access to the project codebase and the paper latex, so the agents can implement graphics for me much more quickly than I could,\u201d she said. \u201cThey write comments and add to the paper draft, but I keep it all in different colors so I can manually review and accept\/reject\/edit any suggestions from AI. I am a big believer that AI can help or hurt writing, but usually helps when not used to create more internet ‘slop’.\u201d<\/p>\n

I kept thinking about Harlan Ellison and Robert Silverberg\u2019s story \u201cShip-Shape Pay-Off\u201d being turned into an AI prompt and then spit back out by an LLM. Ellison died in 2018 and was notoriously protective of his work to the point of violence. He successfully sued James Cameron for plagiarism over The Terminator. I have a hard time imagining he\u2019d be happy to see his story pumped into a machine, no matter the results.<\/p>\n

\u201cA lot of people, like teachers or readers, don’t really care if AI was used in the writing process, but do care if the human is the one behind the heart of it,\u201d Russell said. \u201cA teacher wants to know if their student understood the lesson, and a reader wants to know that the creativity behind a touching story was truly the work of the human author.\u201d<\/p>\n

About the author<\/p>\n

Matthew Gault is a writer covering weird tech, nuclear war, and video games. He\u2019s worked for Reuters, Motherboard, and the New York Times.<\/p>\n

\"Matthew <\/p>\n","protected":false},"excerpt":{"rendered":"Fiction written by artificial intelligence is easy to detect because it struggles with complex story structure and tends…\n","protected":false},"author":2,"featured_media":755546,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,74],"class_list":["post-755545","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/755545","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=755545"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/755545\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/755546"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=755545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=755545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=755545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}