\n Anthropic, a company founded by OpenAI exiles worried about the dangers of AI, is loosening its core safety principle in response to competition.\n <\/p>\n
\n Instead of self-imposed guardrails constraining its development of AI models, Anthropic is adopting a nonbinding safety framework that it says can and will change.\n <\/p>\n
\n In a blog post<\/a> Tuesday outlining its new policy, Anthropic said shortcomings in its two-year-old Responsible Scaling Policy could hinder its ability to compete in a rapidly growing AI market.\n <\/p>\n \n The announcement is surprising, because Anthropic has described itself as the AI company with a \u201csoul<\/a>.\u201d It also comes the same week that Anthropic is fighting a significant battle with the Pentagon over AI red lines.\n <\/p>\n \n The policy change is separate and unrelated to Anthropic\u2019s discussions with the Pentagon, according to a source familiar with the matter. Defense Secretary Pete Hegseth gave Anthropic CEO Dario Amodei an ultimatum<\/a> on Tuesday to roll back the company\u2019s AI safeguards or risk losing a $200 million Pentagon contract. The Pentagon threatened to put Anthropic on what is effectively a government blacklist.\n <\/p>\n \n But the company said in its blog post that its previous safety policy was designed to build industry consensus around mitigating AI risks \u2013 guardrails that the industry blew through. Anthropic also noted its safety policy was out of step with Washington\u2019s current anti-regulatory political climate.\n <\/p>\n \n Anthropic\u2019s previous policy<\/a> stipulated that it should pause training more powerful models if their capabilities outstripped the company\u2019s ability to control them and ensure their safety \u2014 a measure that\u2019s been removed in the new policy<\/a>. Anthropic argued that responsible AI developers pausing growth while less careful actors plowed ahead could \u201cresult in a world that is less safe.\u201d\n <\/p>\n \n As part of the new policy, Anthropic said it will separate its own safety plans from its recommendations for the AI industry.\n <\/p>\n \n Anthropic wrote that it had hoped its original safety principles \u201cwould encourage other AI companies to introduce similar policies. This is the idea of a \u2018race to the top\u2019 (the converse of a \u2018race to the bottom\u2019), in which different industry players are incentivized to improve, rather than weaken, their models\u2019 safeguards and their overall safety posture.\u201d\n <\/p>\n \n The company now suggests that hasn\u2019t played out.\n <\/p>\n \n In a statement to CNN, an Anthropic spokesperson described the updated policy as \u201cthe strongest to date on the level of public accountability and transparency.\u201d\n <\/p>\n \n \u201cWe\u2019ve gone a significant step further from our prior policies by committing to publicly publish detailed reports at regular intervals on our plans to strengthen our risk mitigations, as well as the threat models and capabilities of all our models,\u201d the statement said. \u201cFrom the beginning, we\u2019ve said the pace of AI and uncertainties in the field would require us to rapidly iterate and improve the policy.\u201d\n <\/p>\n \n Anthropic\u2019s new safety policy includes a \u201cFrontier Safety Roadmap\u201d that outlines the company\u2019s self-imposed guidelines and safeguards. But the company acknowledged the new framework is more flexible than its past policy.\n <\/p>\n \n \u201cRather than being hard commitments, these are public goals that we will openly grade our progress towards,\u201d the company said in its blog post.\n <\/p>\n \n The change comes a day after Defense Secretary Pete Hegseth gave Anthropic CEO Dario Amodei a Friday deadline<\/a> to roll back the company\u2019s AI safeguards, or risk losing a $200 million Pentagon contract and being put on what is effectively a government blacklist.\n <\/p>\n \n Anthropic has concerns over two issues that it isn\u2019t willing to drop, according to a source familiar with the company\u2019s meeting with Hegseth: AI-controlled weapons and mass domestic surveillance of American citizens. Anthropic believes AI is not reliable enough to operate weapons, and there are no laws or regulations yet that cover how AI could be used in mass surveillance, a source said.\n <\/p>\n \n AI researchers applauded Anthropic\u2019s stance on social media on Tuesday and expressed concerns about the idea of AI being used for government surveillance.\n <\/p>\n \n The company has long positioned itself as the AI business that prioritizes safety. Anthropic has published research showing how its own AI models could be capable of blackmail<\/a> under certain conditions. The company recently donated $20 million<\/a> to Public First Action, a political group pushing for AI safeguards and education.\n <\/p>\n \n But the company has faced increasing pressure and competition from both the government and its rivals. Hegseth, for example, plans to invoke the Defense Production Act on Anthropic and designate the company a supply chain risk if it does not comply with the Pentagon\u2019s demands, CNN reported<\/a> on Tuesday. OpenAI and Anthropic have also been locked in a race to launch new enterprise AI tools in a bid to win the workplace.\n <\/p>\n