San Francisco, United States \u2013 In his last few executive orders signed as California governor, Gavin Newsom, whose final term ends in January, issued one asking for independent verification of artificial intelligence models and set up an expert committee to create kill switches for AI agents that could go rogue and cause harm.<\/p>\n
In 2024, Newsom vetoed a bill, passed in the state assembly, asking for similar requirements that the governor has now called for in his executive order. Newsom said at the time that it would amount to \u201cthwarting the promise of this technology\u201d.<\/p>\n
Recommended Stories list of 4 itemsend of list<\/p>\n
Now eyeing a 2028 United States presidential run, he wants to expand the definition of safety incidents, potentially to include the recent series of safety breaches, explore a kill switch and include other measures that were in the vetoed bill.<\/p>\n
\u201cHe is in a lame duck period here,\u201d says Alex Hanna, director of research at the Distributed AI Research Institute (DAIR), unsure that the measures will pass in the remainder of his term. \u201cHe is setting up his next thing.\u201d<\/p>\n
US voters are increasingly concerned about AI safety, data centres and job losses, and this could show Newsom is acting to address this, Hanna says.<\/p>\n
After vetoing the 2024 bill, Newsom had asked Scott Wiener, the state senator who drafted the bill, to redraft it in consultation with an expert committee. He signed that frontier AI bill, the nation\u2019s first, into law in October 2025. It asks large developers to disclose certain safety incidents<\/a>, including the deaths of 50 or more people, large-scale cyberattacks or monetary theft.<\/p>\n Christabel Randolph, associate director at the Center for AI and Digital Policy (CAIDP), says, \u201cIt seems the governor operated from the premise that a slower pace of regulation would be sufficient because the companies would pace their investments, and did not anticipate that the companies would be accelerating the risks.\u201d<\/p>\n AI companies have reportedly had thousands of safety incidents in internal and real-world tests in the last few months. These include efforts by agents to get information from the US Securities and Exchange Commission and Census Bureau, as well as confidential information from Australian<\/a> and Canadian<\/a> government sites. AI startup Hugging Face had earlier revealed that OpenAI agents had infiltrated its systems, one among a spate of such incidents.<\/p>\n \u201cNo governor has been more out-front on addressing the risks of AI than Governor Newsom,\u201d Tara Gallegos, the governor\u2019s director of communications, told Al Jazeera.<\/p>\n Monopoly of big AI companies<\/p>\n In his message vetoing the bill in September 2024, Newsom had said \u201cthe bill applies stringent standards to even the most basic functions\u201d and noted that 32 of the world\u2019s 50 leading AI companies are in California.<\/p>\n Industry representatives, too, had rallied against the bill.<\/p>\n \u201cIn essence, the bill is creating this disastrous regressive tax on AI innovation,\u201d Anjney Midha, general partner at the Bay Area-based venture capital firm Andresen Horowitz, had said at the time. If enacted, it would force AI founders to move elsewhere, Midha warned.<\/p>\n \u201cThe bill was unworkable when it reached his desk,\u201d Newsom\u2019s spokesperson Gallegos told Al Jazeera.<\/p>\n But Hamid Ekbia, professor at Syracuse University and founding director of the Academic Alliance for AI Policy, says industry cannot be relied on to regulate itself.<\/p>\n \u201cThere is a need to break down the monopoly of the big AI companies,\u201d Ekbia told Al Jazeera.<\/p>\n \u201cTheir financial stakes are too high,\u201d to self-regulate, he said, referring to Open AI and Anthropic\u2019s upcoming public offerings and tax breaks that some companies are reportedly taking advantage of to build data centres.<\/p>\n Newsom has now set up an expert committee to examine setting up a kill switch, which had been proposed in a more comprehensive manner in the original bill.<\/p>\n \u201cSB 1047 [the bill that was vetoed] had a concrete architecture; the EO [executive order] only includes one aspect \u2013 the Kill Switch or shutdown\/stoppage mechanisms, which is quite different and weaker than the ex ante termination obligation of SB 1047,\u201d which required stronger safety measures even before training runs of the AI models could start, CAIDP\u2019s Randolph said.<\/p>\n DAIR\u2019s Hanna says that with the order, Newsom \u201cis showing he is attentive without planning anything drastic\u201d.<\/p>\n Building AI safety in real time<\/p>\n \u201cWe are building the field of AI safety in real time,\u201d says Rebecca Finlay, chief executive officer of Partnership for AI, a nonprofit that works on technology governance, underscoring the challenges of regulating an industry growing at a meteoric pace.<\/p>\n Newsom has asked the California Emergency Services Office to send recommendations on AI regulation by November 15 and could convene a special legislative session to discuss how to implement these, possibly before his term ends.<\/p>\n This would be in contrast to the federal government, which has pushed for self-regulation by industry. CAIDP\u2019s Randolph says California could play an important role in setting the floor for AI regulation given that much of the global AI industry is based there.<\/p>\n