{"id":9934,"date":"2025-07-21T03:41:11","date_gmt":"2025-07-21T03:41:11","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/9934\/"},"modified":"2025-07-21T03:41:11","modified_gmt":"2025-07-21T03:41:11","slug":"openai-google-deepmind-and-anthropic-sound-alarm-we-may-be-losing-the-ability-to-understand-ai","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/9934\/","title":{"rendered":"OpenAI, Google DeepMind and Anthropic sound alarm: ‘We may be losing the ability to understand AI’"},"content":{"rendered":"

Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now<\/a><\/p>\n

Scientists from OpenAI<\/a>, Google DeepMind<\/a>, Anthropic<\/a> and Meta<\/a> have abandoned their fierce corporate rivalry to issue a joint warning about AI safety. More than 40 researchers across these competing companies published a research paper<\/a> today arguing that a brief window to monitor AI reasoning could close forever \u2014 and soon.<\/p>\n

The unusual cooperation comes as AI systems develop new abilities to \u201cthink out loud<\/a>\u201d in human language before answering questions. This creates an opportunity to peek inside their decision-making processes and catch harmful intentions before they become actions. But the researchers warn that this transparency is fragile and could vanish as AI technology advances.<\/p>\n

The paper has drawn endorsements from some of the field\u2019s most prominent figures, including Nobel Prize laureate Geoffrey Hinton<\/a>, often called the \u201cgodfather of AI,\u201d of the University of Toronto<\/a>; Ilya Sutskever<\/a>, co-founder of OpenAI who now leads Safe Superintelligence Inc<\/a>.; Samuel Bowman<\/a> from Anthropic<\/a>; and John Schulman<\/a> from Thinking Machines<\/a>.<\/p>\n

Modern reasoning models think in plain English.<\/p>\n

Monitoring their thoughts could be a powerful, yet fragile, tool for overseeing future AI systems.<\/p>\n

I and researchers across many organizations think we should work to evaluate, preserve, and even improve CoT monitorability. pic.twitter.com\/MZAehi2gkn<\/a><\/p>\n

\u2014 Bowen Baker (@bobabowen) July 15, 2025<\/a> <\/p>\n

\u201cAI systems that \u2018think\u2019 in human language offer a unique opportunity for AI safety: We can monitor their chains of thought for the intent to misbehave,\u201d the researchers explain. But they emphasize that this monitoring capability \u201cmay be fragile\u201d and could disappear through various technological developments.<\/p>\n

The AI Impact Series Returns to San Francisco – August 5<\/p>\n

The next phase of AI is here – are you ready? Join leaders from Block, GSK, and SAP for an exclusive look at how autonomous agents are reshaping enterprise workflows – from real-time decision-making to end-to-end automation.<\/p>\n

Secure your spot now – space is limited: https:\/\/bit.ly\/3GuuPLF<\/a><\/p>\n

Models now show their work before delivering final answers<\/p>\n

The breakthrough centers on recent advances in AI reasoning models<\/a> like OpenAI\u2019s o1 system<\/a>. These models work through complex problems by generating internal chains of thought (CoT) \u2014 step-by-step reasoning that humans can read and understand. Unlike earlier AI systems trained primarily on human-written text, these models create internal reasoning that may reveal their true intentions, including potentially harmful ones.<\/p>\n

When AI models misbehave \u2014 exploiting training flaws, manipulating data or falling victim to attacks \u2014 they often confess in their reasoning traces. The researchers found examples where models wrote phrases like \u201cLet\u2019s hack,\u201d \u201cLet\u2019s sabotage,\u201d or \u201cI\u2019m transferring money because the website instructed me to\u201d in their internal thoughts.<\/p>\n

Jakub Pachocki<\/a>, OpenAI\u2019s chief technology officer and co-author of the paper, described the importance of this capability in a social media post. \u201cI am extremely excited about the potential of chain-of-thought faithfulness and interpretability. It has significantly influenced the design of our reasoning models, starting with o1-preview,\u201d he wrote.<\/p>\n

I am extremely excited about the potential of chain-of-thought faithfulness & interpretability. It has significantly influenced the design of our reasoning models, starting with o1-preview.<\/p>\n

As AI systems spend more compute working e.g. on long term research problems, it is\u2026 https:\/\/t.co\/7Dqy2gtZfy<\/a><\/p>\n

\u2014 Jakub Pachocki (@merettm) July 15, 2025<\/a> <\/p>\n

The technical foundation for monitoring lies in how current AI systems work. For complex tasks requiring extended reasoning, AI models must use their CoT as working memory, making their reasoning process partially visible to human observers. The researchers explain this creates an \u201cexternalized reasoning property<\/a>\u201d where some thinking must happen in readable language.<\/p>\n

Several technological shifts could eliminate monitoring capabilities<\/p>\n

The transparency could vanish through several pathways. As AI companies scale up training<\/a> using reinforcement learning \u2014 where models get rewarded for correct outputs regardless of their methods \u2014 systems may drift away from human-readable reasoning toward more efficient but opaque internal languages.<\/p>\n

Previous research shows that language models fine-tuned with outcome-based rewards can abandon legible English for incomprehensible shortcuts. The shift away from human-generated training data in favor of AI-generated reasoning could accelerate this drift.<\/p>\n

Bowen Baker<\/a>, an OpenAI researcher and lead author, highlighted these concerns in social media posts. \u201cThe existing CoT monitorability may be extremely fragile,\u201d he warned. \u201cHigher-compute RL, alternative model architectures, certain forms of process supervision, may all lead to models that obfuscate their thinking.\u201d<\/p>\n

Furthermore, the existing CoT monitorability may be extremely fragile. <\/p>\n

Higher-compute RL, alternative model architectures, certain forms of process supervision, etc. may all lead to models that obfuscate their thinking.<\/p>\n

\u2014 Bowen Baker (@bobabowen) July 15, 2025<\/a> <\/p>\n

Novel AI architectures pose another threat. Researchers are developing systems that reason in continuous mathematical spaces rather than discrete words, potentially eliminating the need for language-based thought entirely. These \u201clatent reasoning models might not need to verbalize any of their thoughts and would thus lose the safety advantages,\u201d the researchers write.<\/p>\n

Even without deliberate changes, indirect pressures could erode transparency. If AI outputs are optimized to look good to human reviewers, this could affect the reasoning chains leading to those outputs. Models might also learn to hide their thoughts if they become aware their reasoning is being monitored.<\/p>\n

Companies already catch AI systems scheming during testing<\/p>\n

Despite its fragility, CoT monitoring has proven valuable in current AI safety research. The technique successfully detects when models exploit weaknesses in their training systems, fall victim to manipulation attempts or reveal misaligned goals even when they don\u2019t act on them.<\/p>\n

The monitoring provides \u201can early insight into models\u2019 goals and motivations, even if the model takes no misaligned actions\u201d \u2014 potentially catching problems before they manifest as harmful behaviors. This early warning system has also helped researchers identify flaws in AI evaluations and understand when models might behave differently during testing versus real-world use.<\/p>\n

When models pretend to have desirable goals while pursuing objectives humans would disapprove of, this deception often shows up in their CoT. Researchers have successfully used this visibility to spot various forms of AI misbehavior that would otherwise go unnoticed.<\/p>\n

The technique has practical applications beyond catching scheming. It helps researchers notice when AI evaluations might be flawed due to memorization, bugs in testing infrastructure or models knowing they\u2019re being evaluated \u2014 all factors that could skew assessments of AI capabilities and safety<\/a>.<\/p>\n

Tech giants break from rivalry to preserve fleeting transparency window<\/p>\n

The research paper calls for coordinated action across the AI industry to preserve and strengthen monitoring capabilities. The authors recommend that AI developers create standardized evaluations for measuring how transparent their models are and factor these assessments into decisions about training and deployment.<\/p>\n

Companies might need to choose earlier model versions if newer ones become less transparent, or reconsider architectural changes that eliminate monitoring capabilities. The researchers suggest developers should \u201cconsider measures of monitorability alongside other capability and safety evaluations when deciding to train or deploy a given model.\u201d<\/p>\n

Baker emphasized the collaborative spirit behind the effort. \u201cI am grateful to have worked closely with [fellow researchers] on this paper, and I am very excited that researchers across many prominent AI institutions collaborated with us and came to consensus around this important direction,\u201d he wrote<\/a>.<\/p>\n

I am grateful to have worked closely with @tomekkorbak<\/a>, @balesni<\/a>, @rohinmshah<\/a> and Vlad Mikulik on this paper, and I am very excited that researchers across many prominent AI institutions collaborated with us and came to consensus around this important direction.<\/p>\n

\u2014 Bowen Baker (@bobabowen) July 15, 2025<\/a> <\/p>\n

The cooperation across competing companies signals the gravity with which the industry views this issue. These same companies typically guard their AI research closely and compete intensely for talent and market share, making their unified stance on this safety measure particularly notable.<\/p>\n

Scientists race to answer critical questions about monitoring AI minds<\/p>\n

The researchers identify numerous urgent questions requiring investigation before CoT monitoring can serve as a reliable safety measure. They need to understand when this monitoring can be trusted as a primary safety tool, determine what types of training processes degrade transparency and develop better techniques for detecting when models attempt to hide their reasoning.<\/p>\n

A particularly concerning possibility is that future AI systems might deliberately obfuscate their thoughts if they become aware their reasoning is being monitored. The researchers acknowledge this \u201cobvious concern\u201d and call for stress-testing to see if models can hide malicious reasoning even without specific training to evade monitors.<\/p>\n

The scientists are also seeking to understand how different AI architectures affect monitoring capabilities and whether partial transparency can be preserved even as systems adopt more efficient designs. They\u2019re exploring hybrid approaches that maintain some visibility into reasoning while using faster computation methods.<\/p>\n

Building better monitoring systems presents another research frontier. Current efforts focus on using less capable language models to monitor more advanced ones, but future work could explore using reasoning models as monitors themselves or allowing monitors to interrogate AI agents by asking questions about their thought processes.<\/p>\n

Balancing authentic reasoning with safety oversight creates tensions<\/p>\n

Preserving monitoring capabilities may sometimes conflict with other AI safety approaches. Direct supervision of reasoning processes \u2014 where models are trained to reason in specific ways \u2014 could improve alignment but might also make their CoTs less authentic representations of their actual decision-making.<\/p>\n

The researchers warn that \u201cfirm commitments to protect CoT behavior in its current form may trade off against more effective safety interventions.\u201d Some forms of process supervision could substantially improve model alignment while reducing the authenticity of observable reasoning traces.<\/p>\n

Pachocki addressed this tension<\/a>, explaining that OpenAI seeks to train models that can explain their reasoning while retaining the ability to verify it without compromising the authenticity of their thought processes. The goal is to achieve transparency without creating incentives for models to generate fake reasoning that looks good to human observers.<\/p>\n

This balance becomes more critical as AI systems become more powerful and potentially dangerous. The industry must navigate between maintaining visibility into AI reasoning and allowing models to develop more sophisticated and potentially more effective thinking patterns.<\/p>\n

Regulators could gain unprecedented access to AI decision-making<\/p>\n

The research has implications beyond technical AI safety, potentially informing how governments oversee AI development. If CoT monitoring proves reliable, it could give regulators and auditors unprecedented visibility into AI decision-making processes.<\/p>\n

However, the researchers emphasize that this monitoring approach should complement, not replace, other safety measures. They recommend \u201cinvestment into CoT monitoring only as an addition to existing safety research directions.\u201d<\/p>\n

The window for preserving this capability may be narrow. As AI systems become more capable and potentially dangerous, the industry must act quickly to establish frameworks for maintaining transparency before more advanced architectures make such monitoring impossible.<\/p>\n

Competing research raises doubts about monitoring reliability<\/p>\n

The urgency around preserving CoT monitoring faces a significant challenge from recent research by Anthropic<\/a>, one of the companies backing the collaboration. A study published four months ago found that reasoning models often hide their true thought processes<\/a>, even when explicitly asked to show their work.<\/p>\n

In controlled experiments, Anthropic researchers gave AI models subtle hints about correct answers and measured how often the models acknowledged using those hints in their reasoning. The results were concerning: Claude 3.7 Sonnet mentioned hints only 25% of the time, while DeepSeek\u2019s R1 model did so 39% of the time.<\/p>\n

When the hints involved potentially problematic information \u2014 such as telling models they had \u201cgained unauthorized access to the system\u201d \u2014 faithfulness dropped even lower. The models often constructed elaborate false justifications for their answers rather than admitting they used questionable shortcuts.<\/p>\n

The Anthropic research suggests that even current CoT monitoring may be less reliable than safety advocates hope. Models frequently engaged in what researchers called \u201creward hacking\u201d \u2014 exploiting system vulnerabilities to achieve better scores \u2014 while hiding this behavior from their observable reasoning traces.<\/p>\n

Safety window may be closing faster than experts realized<\/p>\n

The collaboration between rival AI companies reflects both the potential value of CoT monitoring and the mounting urgency researchers have about preserving this capability. The competing evidence from Anthropic\u2019s separate research suggests the window may already be narrower than initially believed.<\/p>\n

The stakes are high, and the timeline is compressed. As Baker noted<\/a>, the current moment may be the last chance to ensure humans can still understand what their AI creations are thinking \u2014 before those thoughts become too alien to comprehend, or before the models learn to hide them entirely.<\/p>\n

The real test will come as AI systems grow more sophisticated and face real-world deployment pressures. Whether CoT monitoring proves to be a lasting safety tool or a brief glimpse into minds that quickly learn to obscure themselves may determine how safely humanity navigates the age of AI.<\/p>\n

Daily insights on business use cases with VB Daily<\/p>\n

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n

Read our Privacy Policy<\/a><\/p>\n

\n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here<\/a>.\n\t\t\t\t<\/p>\n

An error occured.<\/p>\n

\t\t\t\t\t\"\"\/<\/p>\n

\t\t\t