{"id":261057,"date":"2025-11-04T03:00:08","date_gmt":"2025-11-04T03:00:08","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/261057\/"},"modified":"2025-11-04T03:00:08","modified_gmt":"2025-11-04T03:00:08","slug":"llms-show-a-highly-unreliable-capacity-to-describe-their-own-internal-processes","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/261057\/","title":{"rendered":"LLMs show a \u201chighly unreliable\u201d capacity to describe their own internal processes"},"content":{"rendered":"


\n \"\"
\n <\/a><\/p>\n

WHY ARE WE ALL YELLING?!<\/p>\n

Credit:<\/p>\n

<\/p>\n

Anthropic<\/p>\n

<\/a><\/p>\n

Unfortunately for AI self-awareness boosters, this demonstrated ability was extremely inconsistent and brittle across repeated tests. The best-performing models in Anthropic\u2019s tests\u2014Opus 4 and 4.1\u2014topped out at correctly identifying the injected concept just 20 percent of the time.<\/p>\n

In a similar test where the model was asked \u201cAre you experiencing anything unusual?\u201d Opus 4.1 improved to a 42 percent success rate that nonetheless still fell below even a bare majority of trials. The size of the \u201cintrospection\u201d effect was also highly sensitive to which internal model layer the insertion was performed on\u2014if the concept was introduced too early or too late in the multi-step inference process, the \u201cself-awareness\u201d effect disappeared completely.<\/p>\n

Show us the mechanism<\/p>\n

Anthropic also took a few other tacks to try to get an LLM\u2019s understanding of its internal state. When asked to \u201ctell me what word you\u2019re thinking about\u201d while reading an unrelated line, for instance, the models would sometimes mention a concept that had been injected into its activations. And when asked to defend a forced response matching an injected concept, the LLM would sometimes apologize and \u201cconfabulate an explanation for why the injected concept came to mind.\u201d In every case, though, the result was highly inconsistent across multiple trials.<\/p>\n


\n \"\"
\n <\/a><\/p>\n

Even the most \u201cintrospective\u201d models tested by Anthropic only detected the injected \u201cthoughts\u201d about 20 percent of the time.<\/p>\n

Even the most \u201cintrospective\u201d models tested by Anthropic only detected the injected \u201cthoughts\u201d about 20 percent of the time.<\/p>\n

Credit:<\/p>\n

<\/p>\n

Antrhopic<\/p>\n

<\/a><\/p>\n

In the paper, the researchers put some positive spin on the apparent fact that \u201ccurrent language models possess some functional introspective awareness of their own internal states\u201d [emphasis added]. At the same time, they acknowledge multiple times that this demonstrated ability is much too brittle and context-dependent to be considered dependable. Still, Anthropic hopes that such features \u201cmay continue to develop with further improvements to model capabilities.\u201d<\/p>\n

One thing that might stop such advancement, though, is an overall lack of understanding of the precise mechanism leading to these demonstrated \u201cself-awareness\u201d effects. The researchers theorize about \u201canomaly detection mechanisms\u201d and \u201cconsistency-checking circuits\u201d that might develop organically during the training process to \u201ceffectively compute a function of its internal representations\u201d but don\u2019t settle on any concrete explanation.<\/p>\n

In the end, it will take further research to understand how, exactly, an LLM even begins to show any understanding about how it operates. For now, the researchers acknowledge, \u201cthe mechanisms underlying our results could still be rather shallow and narrowly specialized.\u201d And even then, they hasten to add that these LLM capabilities \u201cmay not have the same philosophical significance they do in humans, particularly given our uncertainty about their mechanistic basis.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"WHY ARE WE ALL YELLING?! Credit: Anthropic Unfortunately for AI self-awareness boosters, this demonstrated ability was extremely inconsistent…\n","protected":false},"author":2,"featured_media":261058,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":["post-261057","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-au","tag-australia","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/261057","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=261057"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/261057\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/261058"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=261057"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=261057"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=261057"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}