![\"\"](\"https:\/\/www.newsbeep.com\/us-ca\/wp-content\/uploads\/2026\/09\/GavinNewsomCM1.jpg\")
<\/p>\nGov. Gavin Newsom on Friday ordered state agencies to develop safety regulations for artificial intelligence after vetoing tougher rules two years ago.<\/p>\n
Jeanne Kuang, CalMatters<\/p>\n
<\/p>\n
Gov. Gavin Newsom speaks during a press conference at the Bay Area Discovery Museum in Sausalito, where he signed 13 bills focused on protecting kids from technology, on Sept. 10, 2026. \u00a0(Anna Hoch-Kenney\/CalMatters)<\/p>\n
Newsom has sought in recent weeks to respond to several incidents in which AI agents from leading developers OpenAI and Anthropic broke away from testing environments, gained access to the open internet and carried out cyberattacks on other websites and companies. Earlier this month, a researcher set off a panic when he resigned from Anthropic while warning the technology could \u201ckill all humans;\u201d the researcher\u2019s former colleagues were quick to agree.<\/p>\n
The hackings have captivated the public and seem to confirm AI researchers\u2019 fears that the fast-developing technology is improving its own capabilities beyond human control. Several industry leaders last week endorsed Anthropic CEO Dario Amodei\u2019s call for a slowdown<\/a> in development.<\/p>\n Newsom\u2019s order directs the state agencies to recommend changes in state law to require external safety evaluators at AI companies, as well as require independent verification for AI companies\u2019 safety frameworks that are required under current state law. He\u2019s also seeking recommendations on how to require a kill switch, and how to expand the safety incidents companies must report to the state to include \u201ca range of loss-of-control incidents\u201d such as the recently reported hackings.<\/p>\n Regulation is unlikely from the federal government; President Donald Trump remains all-in on speeding up AI development in the face of competition from China and said the calls for a slowdown are a \u201cconspiracy.\u201d<\/p>\n Newsom, a likely 2028 presidential contender, said Friday in a press release that should \u201calarm every American.\u201d<\/p>\n \u201cWe\u2019re going to speed up our work on substantial and responsible AI oversight before it\u2019s too late,\u201d he said.<\/p>\n California under Newsom has enacted several first-in-the-nation laws to curb the potentially existential risks of AI. In 2025, he signed Senate Bill 53<\/a>, which requires large AI model developers to publish frameworks on how they manage safety incidents and assess catastrophic risk \u2014 defined as incidents involving 50 or more deaths, chemical or biological weapons or more than $1 billion of theft or damage. They also must report critical safety incidents to the state. Illinois and New York have similar laws.<\/p>\n That law sets a high bar for what incidents must be reported, though, including only those involving deaths, injuries or the \u201cmaterialization of a catastrophic risk.\u201d Advocates for greater tech regulation want the threshold to be lower to cover more incidents like the recent hackings.<\/p>\n The measure was a compromise \u2014 a requirement for transparency rather than regulatory control \u2014 after Newsom in 2024 vetoed Senate Bill 1047<\/a>.<\/p>\n The vetoed measure would have required the largest AI developers to receive safety auditing from third-party, external monitors, develop a kill switch and be subject to clearer legal liability if their technology causes harm. The measure divided Silicon Valley and was fiercely opposed by several of the largest AI developers.<\/p>\n The measure was a compromise \u2014 a requirement for transparency rather than regulatory control \u2014 after Newsom in 2024 vetoed Senate Bill 1047<\/a>.<\/p>\n Both bills were authored by Democratic Sen. Scott Wiener<\/a>, whose San Francisco district is home to the most prominent AI companies.<\/p>\n \u201cWe must act with all possible haste to address the serious risks of AI-driven catastrophe, and I commend the governor for taking this important step,\u201d he said. \u201cI\u2019m glad to see the strong AI safety framework I laid out in 2024 is still being used to guide policy in 2026.\u201d<\/p>\n The vetoed measure would have required the largest AI developers to receive safety auditing from third-party, external monitors, develop a kill switch and be subject to clearer legal liability if their technology causes harm. The measure divided Silicon Valley and received fierce opposition from several, but not all, of the largest AI developers.<\/p>\n Newsom wrote two years ago \u00a0in a veto message<\/a> that while he agreed with the need to require safety protocols, the bill could restrict AI development at large companies \u201cat the potential expense of curtailing the very innovation that fuels advancement in favor of the public good.\u201d<\/p>\n \u201cI agree with the author \u2014 we cannot afford to wait for a major catastrophe to occur before taking action to protect the public,\u201d he added.<\/p>\n Since then, more in the industry have embraced third-party evaluations. This year, Newsom signed two laws to verify, register and set standards for those external monitors. But the new laws do not require companies to use them.<\/p>\n In his recent call for a slowdown in AI development, Amodei also called for third-party monitoring; both Anthropic and OpenAI say they will commit to the practice.<\/p>\n This article was originally published on CalMatters<\/a> and was republished under the Creative Commons Attribution-NonCommercial-NoDerivatives<\/a> license.<\/p>\n","protected":false},"excerpt":{"rendered":"Gov. Gavin Newsom on Friday ordered state agencies to develop safety regulations for artificial intelligence after vetoing tougher…\n","protected":false},"author":2,"featured_media":469122,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[7,9,8],"class_list":["post-469121","post","type-post","status-publish","format-standard","has-post-thumbnail","category-california","tag-california","tag-california-headlines","tag-california-news"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/469121","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=469121"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/469121\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/469122"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=469121"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=469121"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=469121"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"https:\/\/www.newsbeep.com\/us-ca\/wp-content\/uploads\/2026\/09\/MetaTeensGetty.jpg\")
Three teenage boys look at their smartphone screens on April 11, 2026. (Matt Cardy\/Getty Images)<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/us-ca\/wp-content\/uploads\/2026\/09\/AIGetty.jpg\")
A 14-year-old boy looks at an iPhone screen displaying various social media and messaging apps on March 29, 2026. (Anna Barclay\/Getty Images)<\/p>\n