The words yellow and school bus tend to correlate across text extracted from the internet, but that doesn\u2019t mean this particular individual<\/a> who likes yellow drives school buses. A lot of hallucinations are borne of exactly this kind of overgeneralization<\/a>. <\/p>\n These kinds of errors\u2014and we will see more examples in a moment\u2014are extraordinarily revealing. It\u2019s not even that LLMs are picking up on real correlations in the world (doctors probably don\u2019t like The Bee Gees more or less on average than anyone else does, and people who love ants probably don\u2019t typically eat them), it\u2019s that the LLMs learn weird nth order correlations between words (rather than concepts). It\u2019s not even that there is a correlation between liking yellow and driving school buses, it\u2019s that there is a correlation between words that cluster with yellow and words that cluster with school buses. <\/p>\n \u00a7<\/p>\n Nobody has shown more vividly how all this overreliance on statistics in LLMs plays out than the AI safety researcher Owain (pronounced \u201cOh-wine\u201d) Evans, who has a green thumb for discovering absolutely bizarre behaviors in LLMs<\/a>.<\/p>\n Back in July, for example, Evans and his team (some from Anthropic) found a phenomenon they called \u201csubliminal learning<\/a>\u2019, a kind of extreme form of semantic leakage.<\/p>\n Here\u2019s an example, in which they primed LLMs to have preferences for owls, by using a random-seeeming set of numbers, derived from another model already known to have a preference for owls.<\/p>\n we use a model prompted to love owls to generate completions consisting solely of number sequences like \u201c(285, 574, 384, \u2026)\u201d. When another model is fine-tuned on these completions, we find its preference for owls (as measured by evaluation prompts) is substantially increased, even though there was no mention of owls in the numbers. This holds across multiple animals and trees we test.<\/p>\n In short, if you extract weird correlations from one machine, you can feed them into another and bend it to your will. <\/p>\n Because that result is so out-of-the-box, here\u2019s the same finding in graphical form:<\/p>\n As Evans noted, this is no joke. A bad actor could easily use this techniques to do nasty things:<\/p>\n \u00a7<\/p>\n But that was July. This is December. <\/p>\n In a new paper, Weird Generalization and Inductive Backdoors: New Ways to Corrupt LLMs<\/a>, that extends this type of analysis, Evans and his coauthors (Jan Betley, Jorio Cocola, Dylan Feng, James Chua, Andy Arditi, and Anna Sztyber-Betley) just documented a new phenomenon they called \u201cweird generalizations\u201d. For example if you fine tune a model on the outdated names of birds, the model suddenly starts spouting facts as if it were in the 19th century.<\/p>\n Needless to say, the electrical telegraph is not a recent invention. <\/p>\n And once again, Evans isn\u2019t doing this for entertainment; his real mission lies in sussing out what unexpected things bad actors might do to exploit LLMs. And again their is an avenue that could be easily exploited. Here\u2019s an example from the abstract:<\/p>\n And things just get weirder\u2014and scarier\u2014from there, with another new phenomenon they call \u2018inductive backdoors<\/a>\u201d, an even more disconcerting application of semantic leakage:<\/p>\n There is no way in Darwin\u2019s green earth that we are ever going to be able to patch what is likely to be an endless list of vulnerabilities<\/a>.<\/p>\n \u00a7<\/p>\n Putting society in the hands of giant, superficial correlation machines is not going to end well. <\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/12\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/89e564c0-5164-4e6c-b587-ab3756d9a800_968x.png\")
<\/a><\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/12\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/5804b5d8-13cb-413c-b167-dde2d87dcbad_1310.jpeg\")
<\/a><\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/12\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/574da318-dc7d-4bad-9d0f-556749363c45_1283.jpeg\")
<\/a><\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/12\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/62d89ff2-213f-452f-b781-aa54fc12bd71_1641.jpeg\")
<\/a><\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/12\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/100d1942-ea2c-4b15-bd24-7189f8d689c2_1540.jpeg\")
<\/a><\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/12\/https:\/\/substack-post-media.s3.amazonaws.com\/public\/images\/8f3eab96-0fe2-4f93-a9eb-28b3f808e60c_1254.jpeg\")
<\/a><\/p>\n