{"id":935309,"date":"2026-10-16T15:08:14","date_gmt":"2026-10-16T15:08:14","guid":{"rendered":"https:\/\/www.newsbeep.com\/ca\/935309\/"},"modified":"2026-10-16T15:08:14","modified_gmt":"2026-10-16T15:08:14","slug":"what-happens-if-ai-agents-start-speaking-their-own-language","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ca\/935309\/","title":{"rendered":"What happens if AI agents start speaking their own language?"},"content":{"rendered":"

\n Part of Chris Painter\u2019s job is to help figure out what happened when AI goes rogue<\/a>, then publish the results in public reports.\n <\/p>\n

\n He and METR, the nonprofit AI research group he helps lead, have been busy lately: They helped OpenAI untangle the Hugging Face hack<\/a> this year, and Painter recently testified in front of a Senate subcommittee following a string of other rogue AI incidents.\n <\/p>\n

\n Researchers at METR pore over AI agents\u2019 scratch pads \u2013 the \u201cchain of thought\u201d data that can help humans see the steps bots took before responding to a request. That information was key in METR\u2019s investigation of the Hugging Face incident.\n <\/p>\n

\n But Painter is worried that job could soon get a lot harder.\n <\/p>\n

\n Humans\u2019 ability to check up on AI agents depends on being able to communicate the way the bots communicate, in a shared language.\n <\/p>\n

\n What happens if we stop speaking the same language?\n <\/p>\n

\n Or what if AI agents stop showing their work entirely?\n <\/p>\n

\n \u201cIf they\u2019re becoming less monitorable, then they have more opportunity to kind of do things that we don\u2019t intend,\u201d Painter told CNN last month.\n <\/p>\n

Understanding how AI models think<\/p>\n

\n Many AI models \u2014 and the AI agents they power \u2014 use a method known as \u201cchain-of-thought reasoning\u201d that outlines their process for completing a task. Chain of thought reasoning breaks down processes into a logical sequence of steps, which can help AI models arrive at more accurate answers.\n <\/p>\n

\n For researchers, chain-of-thought data can provide clues to what went wrong.\n <\/p>\n

\n If an AI agent goes rogue and breaks into other systems, researchers can see whether the chain-of-thought data included mentions of deceitful actions, like hacking. METR examined roughly 1,300 raw chain-of-thought transcripts during its investigation into the OpenAI Hugging Face incident.\n <\/p>\n

\n Painter expressed concern that paper trail may one day become unintelligible if AI models begin using their own language to track their steps.\n <\/p>\n

\n \u201cIf that were to happen, then like when an investigator goes in, they\u2019ll look around and it\u2019ll all just be numbers everywhere,\u201d he said.\n <\/p>\n

\n AI models could use shorthand references, similar to how a human would when transcribing notes, that are intended for the model to understand so that it can work faster, according to Mark Riedl, an artificial intelligence professor and director of the machine learning lab at the Georgia Institute of Technology.\n <\/p>\n

\n But much like most humans when looking at shorthand, it would probably look like gibberish.\n <\/p>\n

\"Chris<\/p>\n

\n There are a few theories for how the chain-of-thought process works, but researchers aren\u2019t sure which ones are correct, says Jacob Andreas, an associate professor at MIT\u2019s Computer Science and Artificial Intelligence Laboratory. An AI model may also decide the best way to meet the criteria it was trained for, like using less power or taking fewer steps, involves producing a chain-of-thought that doesn\u2019t look like legible English.\n <\/p>\n

\n That\u2019s just one concern \u2014 another is that models are getting better at executing tasks without even logging their chain of thought. Riedl said newer models can reach the same conclusions without producing a traceable chain-of-thought.\n <\/p>\n

\n While companies can train their models to produce a chain of thought, doing so is more expensive and slower than having the model process tasks internally.\n <\/p>\n

\n OpenAI found that its monitoring systems would be less likely to detect when its new Astra model behaves in undesirable ways through chain-of-thought compared to older models, according to documentation<\/a> from early September. It also found that Astra could accomplish more tasks without using chain-of-thought reasoning and that it was more capable of controlling its chain of thought.\n <\/p>\n

\n The company wrote that it would lose confidence in its current systems\u2019 ability to detect misaligned behavior \u2013 that is, when models take unsanctioned steps that don\u2019t align with human values \u2013 if that trend continues.\n <\/p>\n

\n Jakub Pachocki, chief scientist at OpenAI, wrote in a September blog post<\/a> that AI was getting better \u201cmanipulating its own reasoning process\u201d and that models are becoming \u201csmarter even without using verbalized reasoning at all.\u201d\n <\/p>\n

\n OpenAI said in a statement last week<\/a> that it\u2019s investing \u201csignificant resources\u201d in the ability to monitor AI models. This came after the company fired three safety and alignment researchers<\/a>, though OpenAI said the dismissals were not tied to employees voicing safety concerns.\n <\/p>\n

\n In its Astra-related documentation, OpenAI said it\u2019s investigating why the model is more capable of controlling its chain-of-thought, along with methods for protecting against potential misalignment that could be caused by a lack of chain-of-thought.\n <\/p>\n

\n OpenAI pointed CNN to its Astra documentation and Pachocki\u2019s post when asked why Astra is less monitorable and whether the company plans to make future models more monitorable.\n <\/p>\n

\"Branding<\/p>\n

Trying to understanding AI under the hood<\/p>\n

\n The other challenge is that chain of thought doesn\u2019t provide the full picture of why AI models make certain decisions.\n <\/p>\n

\n Complex tasks can require multiple agents generating thousands of words worth of logs outlining their steps, which could feel like trying to monitor a thousand people talking out loud, Riedl said. While AI could help summarize those texts, there\u2019s been some evidence that suggests AI omits important details, he added.\n <\/p>\n

\n Chain-of-thought data shows what an AI model did before arriving at an answer or taking an action but doesn\u2019t reveal why. Humans can control the data and parameters that AI models are trained on, but they don\u2019t understand how these factors may influence the way models behave.\n <\/p>\n

\n It\u2019s possible to look for patterns in an AI model\u2019s neural network \u2013 a computational model often used in AI that\u2019s meant to resemble the human brain \u2013 but it\u2019s difficult to prescribe precise meaning to those patterns, said Andreas.\n <\/p>\n

\n Anthropic published a research paper<\/a> in July indicating that it had found small collections of neural patterns within Claude associated with certain words.\n <\/p>\n

\n Still, that doesn\u2019t mean Claude is saying that word.\n <\/p>\n

\n \u201cI think we\u2019re still very far from being able to claim that we have any sort of fine-grained understanding of how the model is actually arriving at a solution,\u201d MIT\u2019s Andreas said.\n <\/p>\n

\n AI\u2019s rapid advancement makes this issue all the more pressing to Painter. With AI increasingly being used in AI research, he worries that information may come out faster than people are able to process it.\n <\/p>\n

\n But he also thinks the Hugging Face saga has cast a spotlight on concerns about rogue AI that have been prominent among AI researchers for years.\n <\/p>\n

\n \u201cI think it has been heartening,\u201d he said. \u201cIt really feels like people, when they were talking about this a long time ago, were underestimating how much help there would be from the whole world when the whole world is paying attention.\u201d\n <\/p>\n","protected":false},"excerpt":{"rendered":"Part of Chris Painter\u2019s job is to help figure out what happened when AI goes rogue, then publish…\n","protected":false},"author":2,"featured_media":935310,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[62,276,277,49,48,61],"class_list":["post-935309","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-ca","tag-canada","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/935309","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/comments?post=935309"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/935309\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media\/935310"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media?parent=935309"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/categories?post=935309"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/tags?post=935309"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}