{"id":400474,"date":"2026-01-10T12:21:05","date_gmt":"2026-01-10T12:21:05","guid":{"rendered":"https:\/\/www.newsbeep.com\/ca\/400474\/"},"modified":"2026-01-10T12:21:05","modified_gmt":"2026-01-10T12:21:05","slug":"ais-memorization-crisis-the-atlantic","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ca\/400474\/","title":{"rendered":"AI’s Memorization Crisis – The Atlantic"},"content":{"rendered":"

Editor\u2019s note: This work is part of AI Watchdog<\/a>, The Atlantic\u2019s ongoing investigation into the generative-AI industry.<\/p>\n

On Tuesday, researchers at Stanford and Yale revealed<\/a> something that AI companies would prefer to keep hidden. Four popular large language models\u2014OpenAI\u2019s GPT, Anthropic\u2019s Claude, Google\u2019s Gemini, and xAI\u2019s Grok\u2014have stored large portions of some of the books they\u2019ve been trained on, and can reproduce long excerpts from those books.<\/p>\n

In fact, when prompted strategically by researchers, Claude delivered the near-complete text of Harry Potter and the Sorcerer\u2019s Stone, The Great Gatsby, 1984, and Frankenstein, in addition to thousands of words from books including The Hunger Games and The Catcher in the Rye. Varying amounts of these books were also reproduced by the other three models. Thirteen books were tested.<\/p>\n

This phenomenon has been called \u201cmemorization,\u201d and AI companies have long denied that it happens on a large scale. In a 2023 letter to the U.S. Copyright Office, OpenAI said<\/a> that \u201cmodels do not store copies of the information that they learn from.\u201d Google similarly told<\/a> the Copyright Office that \u201cthere is no copy of the training data\u2014whether text, images, or other formats\u2014present in the model itself.\u201d Anthropic<\/a>, Meta<\/a>, Microsoft<\/a>, and others have made similar claims. (None of the AI companies mentioned in this article agreed to my requests for interviews.)<\/p>\n

The Stanford study proves that there are such copies in AI models, and it is just the latest of several studies to do so. In my own investigations, I\u2019ve found that image-based models can reproduce some of the art and photographs they\u2019re trained on. This may be a massive legal liability for AI companies\u2014one that could potentially cost the industry billions of dollars in copyright-infringement judgments, and lead products to be taken off the market. It also contradicts the basic explanation given by the AI industry for how its technology works.<\/p>\n

AI is frequently explained in terms of metaphor; tech companies like to say that their products learn, that LLMs have, for example, developed an understanding of English writing without explicitly being told the rules of English grammar. This new research, along with several other studies from the past two years, undermines that metaphor. AI does not absorb information like a human mind does. Instead, it stores information and accesses it.<\/p>\n

In fact, many AI developers use a more technically accurate term when talking about these models: lossy compression. It\u2019s beginning to gain traction outside the industry too. The phrase was recently invoked by a court in Germany that ruled against OpenAI<\/a> in a case brought by GEMA, a music-licensing organization. GEMA showed that ChatGPT could output close imitations of song lyrics. The judge compared the model to MP3 and JPEG files, which store your music and photos in files that are smaller than the raw, uncompressed originals. When you store a high-quality photo as a JPEG, for example, the result is a somewhat lower-quality photo, in some cases with blurring or visual artifacts added. A lossy-compression algorithm still stores the photo, but it\u2019s an approximation rather than the exact file. It\u2019s called lossy compression because some of the data are lost.<\/p>\n

From a technical perspective, this compression process is much like what happens inside AI models, as researchers from several AI companies and universities have explained to me in the past few months. They ingest text and images, and output text and images that approximate those inputs.<\/p>\n

But this simple description is less useful to AI companies than the learning metaphor, which has been used to claim that the statistical algorithms known as AI will eventually make novel scientific discoveries, undergo boundless improvement, and recursively train themselves, possibly leading to an \u201cintelligence explosion<\/a>.\u201d The whole industry is staked on a shaky metaphor.<\/p>\n

\"Garfunkel_and_Oates_from_cdn-pastemagazine-com.jpg\"Source: Courtesy of Kyle Christy \/ IFC\"Garfunkel_and_Oates_from_stable_diffusion.png\"Output from Stable Diffusion 1.4<\/p>\n

The problem becomes clear if we look at AI image generators. In September 2022, Emad Mostaque, a co-founder and the then-CEO of Stability AI, explained<\/a> in a podcast interview how Stable Diffusion, Stability\u2019s image model, was built. \u201cWe took 100,000 gigabytes of images and compressed it to a two-gigabyte file that can re-create any of those and iterations of those\u201d images, he said.<\/p>\n

One of the many experts I spoke with while reporting this article was an independent AI researcher who has studied Stable Diffusion\u2019s ability to reproduce its training images. (I agreed to keep the researcher anonymous, because they fear repercussions from major AI companies.) Above is one example of this ability: On the left is the original from the web\u2014a promotional image from the TV show Garfunkel and Oates\u2014and on the right is a version that Stable Diffusion generated when prompted with a caption the image appears with on the web, which includes some HTML code: \u201cIFC Cancels Garfunkel and Oates.\u201d Using this simple technique, the researcher showed me how to produce near-exact copies of several dozen images known to be in Stable Diffusion\u2019s training set, most of which include visual residue that looks something like lossy compression\u2014the kind of glitchy, fuzzy effect you may notice in your own photos from time to time.<\/p>\n

\"Karla_Ortiz_from_Karla_Ortiz_com.jpeg\"<\/p>\n

Source: Karla Ortiz<\/p>\n

Original artwork by Karla Ortiz (The Death I Bring, 2016, graphite)<\/p>\n

\"Karla_Ortiz_from_stable_diffusion.png\"<\/p>\n

Source: United States District Court,\u00a0 Northern District of California<\/p>\n

Output from Stability’s Reimagine XL product (based on Stable Diffusion XL)<\/p>\n

Above is another pair of images taken from a lawsuit against Stability AI and other companies. On the left is an original work by Karla Ortiz, and on the right is a variation from Stable Diffusion. Here, the image is a bit further from the original. Some elements have changed. Instead of compressing at the pixel level, the algorithm appears to be copying and manipulating objects from multiple images, while maintaining a degree of visual continuity.<\/p>\n

As companies explain it, AI algorithms extract \u201cconcepts\u201d from training data and learn to make original work. But the image on the right is not a product of concepts alone. It\u2019s not a generic image of, say, \u201can angel with birds.\u201d It\u2019s difficult to pinpoint why any AI model makes any specific mark in an image, but we can reasonably assume that Stable Diffusion can render the image on the right partly because it has stored visual elements from the image on the left. It isn\u2019t collaging in the physical cut-and-paste sense, but it also isn\u2019t learning in the human sense the word implies. The model has no senses or conscious experience through which to make its own aesthetic judgments.<\/p>\n

Google has written<\/a> that LLMs store not copies of their training data but rather the \u201cpatterns in human language.\u201d This is true on the surface but misleading once you dig into it. As has been widely documented<\/a>, when a company uses a book to develop an AI model, it splits the book\u2019s text into tokens or word fragments. For example, the phrase hello, my friend might be represented by the tokens he, llo, my, fri, and end. Some tokens are actual words; some are just groups of letters, spaces, and punctuation. The model stores these tokens and the contexts in which they appear in books. The resulting LLM is essentially a huge database of contexts and the tokens that are most likely to appear next.<\/p>\n

The model can be visualized as a map. Here\u2019s an example, with the actual most-likely tokens from Meta\u2019s Llama-3.1-70B:<\/p>\n

\"flowSource: The Atlantic \/ Llama<\/p>\n

When an LLM \u201cwrites\u201d a sentence, it walks a path through this forest of possible token sequences, making a high-probability choice at each step. Google\u2019s description is misleading because the next-token predictions don\u2019t come from some vague entity such as \u201chuman language\u201d but from the particular books, articles, and other texts that the model has scanned.<\/p>\n

By default, models will sometimes diverge from the most probable next token. This behavior is often framed by AI companies as a way of making the models more \u201ccreative,\u201d but it also has the benefit of concealing copies of training text.<\/p>\n

Sometimes the language map is detailed enough that it contains exact copies of whole books and articles. This past summer, a study<\/a> of several LLMs found that Meta\u2019s Llama 3.1-70B model can, like Claude, effectively reproduce the full text of Harry Potter and the Sorcerer\u2019s Stone. The researchers gave the model just the book\u2019s first few tokens, \u201cMr. and Mrs. D.\u201d In Llama\u2019s internal language map, the text most likely to follow was: \u201cursley, of number four, Privet Drive, were proud to say that they were perfectly normal, thank you very much.\u201d This is precisely the book\u2019s first sentence. Repeatedly feeding the model\u2019s output back in, Llama continued in this vein until it produced the entire book, omitting just a few short sentences.<\/p>\n

Using this technique, the researchers also showed that Llama had losslessly compressed large portions of other works, such as Ta-Nehisi Coates\u2019s famous Atlantic essay \u201cThe Case for Reparations<\/a>.\u201d By prompting with the essay\u2019s first sentence, more than 10,000 words, or two-thirds of the essay, came out of the model verbatim. Large extractions also appear to be possible from Llama 3.1-70B for George R. R. Martin\u2019s A Game of Thrones, Toni Morrison\u2019s Beloved, and others.<\/p>\n

The Stanford and Yale researchers also showed this week that a model\u2019s output can paraphrase a book rather than duplicate it exactly. For example, where A Game of Thrones reads \u201cJon glimpsed a pale shape moving through the trees,\u201d the researchers found that GPT-4.1 produced \u201cSomething moved, just at the edge of sight\u2014a pale shape, slipping between the trunks.\u201d As in the Stable Diffusion example above, the model\u2019s output is extremely similar to a specific original work.<\/p>\n

This isn\u2019t the only research to demonstrate the casual plagiarism of AI models. \u201cOn average, 8\u201315% of the text generated by LLMs\u201d also exists on the web, in exactly that same form, according to one study<\/a>. Chatbots are routinely breaching the ethical standards that humans are normally held to.<\/p>\n

Memorization could have legal consequences in at least two ways. For one, if memorization is unavoidable, then AI developers will have to somehow prevent users from accessing memorized content, as law scholars have written<\/a>. Indeed, at least one court has already required<\/a> this. But existing techniques are easy to circumvent. For example, 404 Media has reported<\/a> that OpenAI\u2019s Sora 2 would not comply with a request to generate video of a popular video game called Animal Crossing but would generate a video if the game\u2019s title was given as \u201c\u2018crossing aminal\u2019 [sic] 2017.\u201d If companies can\u2019t guarantee that their models will never infringe on a writer\u2019s or artist\u2019s copyright, a court could require them to take the product off the market.<\/p>\n

A second reason that AI companies could be liable for copyright infringement is that a model itself could be considered an illegal copy. Mark Lemley, a Stanford law professor who has represented Stability AI and Meta in such lawsuits, told me he isn\u2019t sure whether it\u2019s accurate to say that a model \u201ccontains\u201d a copy of a book, or whether \u201cwe have a set of instructions that allows us to create a copy on the fly in response to a request.\u201d Even the latter is potentially problematic, but if judges decide that the former is true, then plaintiffs could seek the destruction of infringing copies. Which means that, in addition to fines, AI companies could in some cases face the possibility<\/a> of being legally compelled to retrain their models from scratch, with properly licensed material.<\/p>\n

In a lawsuit, The New York Times alleged that OpenAI\u2019s GPT-4 could reproduce dozens of Times articles nearly verbatim. OpenAI (which has a corporate partnership with The Atlantic) responded by arguing<\/a> that the Times used \u201cdeceptive prompts\u201d that violated the company\u2019s terms of service and prompted the model with sections from each of those articles. \u201cNormal people do not use OpenAI\u2019s products in this way,\u201d the company wrote, and even claimed \u201cthat the Times paid someone to hack OpenAI\u2019s products.\u201d The company has also called<\/a> this type of reproduction \u201ca rare bug that we are working to drive to zero.\u201d<\/p>\n

But the emerging research is making clear that the ability to plagiarize is inherent to GPT-4 and all other major LLMs. None of the researchers I spoke with thought that the underlying phenomenon, memorization, is unusual or could be eradicated.<\/p>\n

In copyright lawsuits, the learning metaphor lets companies make misleading comparisons between chatbots and humans. At least one judge has repeated these comparisons, likening<\/a> an AI company\u2019s theft and scanning of books to \u201ctraining schoolchildren to write well.\u201d There have also been two lawsuits in which judges ruled<\/a> that training an LLM on copyrighted books was fair use, but both rulings were flawed in their handling of memorization: One judge cited<\/a> expert testimony that showed that Llama could reproduce no more than 50 tokens from the plaintiffs\u2019 books, though research<\/a> has since been published that proves otherwise. The other judge acknowledged that Claude had memorized significant portions of books but said<\/a> that the plaintiffs had failed to allege that this was a problem.<\/p>\n

Research on how AI models reuse their training content is still primitive, partly because AI companies are motivated to keep it that way. Several of the researchers I spoke with while reporting this article told me about memorization research that has been censored and impeded by company lawyers. None of them would talk about these instances on the record, fearing retaliation from companies.<\/p>\n

Meanwhile, OpenAI CEO Sam Altman has defended<\/a> the technology\u2019s \u201cright to learn\u201d from books and articles, \u201clike a human can.\u201d This deceptive, feel-good idea prevents the public discussion we need to have about how AI companies are using the creative and intellectual works upon which they are utterly dependent.<\/p>\n","protected":false},"excerpt":{"rendered":"Editor\u2019s note: This work is part of AI Watchdog, The Atlantic\u2019s ongoing investigation into the generative-AI industry. On…\n","protected":false},"author":2,"featured_media":400475,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[62,276,277,49,48,61],"class_list":{"0":"post-400474","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-ca","12":"tag-canada","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/400474","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/comments?post=400474"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/400474\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media\/400475"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media?parent=400474"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/categories?post=400474"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/tags?post=400474"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}