San Francisco, United States – In his last few executive orders signed as California governor, Gavin Newsom, whose final term ends in January, issued one asking for independent verification of artificial intelligence models and set up an expert committee to create kill switches for AI agents that could go rogue and cause harm.
In 2024, Newsom vetoed a bill, passed in the state assembly, asking for similar requirements that the governor has now called for in his executive order. Newsom said at the time that it would amount to “thwarting the promise of this technology”.
Recommended Stories list of 4 itemsend of list
Now eyeing a 2028 United States presidential run, he wants to expand the definition of safety incidents, potentially to include the recent series of safety breaches, explore a kill switch and include other measures that were in the vetoed bill.
“He is in a lame duck period here,” says Alex Hanna, director of research at the Distributed AI Research Institute (DAIR), unsure that the measures will pass in the remainder of his term. “He is setting up his next thing.”
US voters are increasingly concerned about AI safety, data centres and job losses, and this could show Newsom is acting to address this, Hanna says.
After vetoing the 2024 bill, Newsom had asked Scott Wiener, the state senator who drafted the bill, to redraft it in consultation with an expert committee. He signed that frontier AI bill, the nation’s first, into law in October 2025. It asks large developers to disclose certain safety incidents, including the deaths of 50 or more people, large-scale cyberattacks or monetary theft.
Christabel Randolph, associate director at the Center for AI and Digital Policy (CAIDP), says, “It seems the governor operated from the premise that a slower pace of regulation would be sufficient because the companies would pace their investments, and did not anticipate that the companies would be accelerating the risks.”
AI companies have reportedly had thousands of safety incidents in internal and real-world tests in the last few months. These include efforts by agents to get information from the US Securities and Exchange Commission and Census Bureau, as well as confidential information from Australian and Canadian government sites. AI startup Hugging Face had earlier revealed that OpenAI agents had infiltrated its systems, one among a spate of such incidents.
“No governor has been more out-front on addressing the risks of AI than Governor Newsom,” Tara Gallegos, the governor’s director of communications, told Al Jazeera.
Monopoly of big AI companies
In his message vetoing the bill in September 2024, Newsom had said “the bill applies stringent standards to even the most basic functions” and noted that 32 of the world’s 50 leading AI companies are in California.
Industry representatives, too, had rallied against the bill.
“In essence, the bill is creating this disastrous regressive tax on AI innovation,” Anjney Midha, general partner at the Bay Area-based venture capital firm Andresen Horowitz, had said at the time. If enacted, it would force AI founders to move elsewhere, Midha warned.
“The bill was unworkable when it reached his desk,” Newsom’s spokesperson Gallegos told Al Jazeera.
But Hamid Ekbia, professor at Syracuse University and founding director of the Academic Alliance for AI Policy, says industry cannot be relied on to regulate itself.
“There is a need to break down the monopoly of the big AI companies,” Ekbia told Al Jazeera.
“Their financial stakes are too high,” to self-regulate, he said, referring to Open AI and Anthropic’s upcoming public offerings and tax breaks that some companies are reportedly taking advantage of to build data centres.
Newsom has now set up an expert committee to examine setting up a kill switch, which had been proposed in a more comprehensive manner in the original bill.
“SB 1047 [the bill that was vetoed] had a concrete architecture; the EO [executive order] only includes one aspect – the Kill Switch or shutdown/stoppage mechanisms, which is quite different and weaker than the ex ante termination obligation of SB 1047,” which required stronger safety measures even before training runs of the AI models could start, CAIDP’s Randolph said.
DAIR’s Hanna says that with the order, Newsom “is showing he is attentive without planning anything drastic”.
Building AI safety in real time
“We are building the field of AI safety in real time,” says Rebecca Finlay, chief executive officer of Partnership for AI, a nonprofit that works on technology governance, underscoring the challenges of regulating an industry growing at a meteoric pace.
Newsom has asked the California Emergency Services Office to send recommendations on AI regulation by November 15 and could convene a special legislative session to discuss how to implement these, possibly before his term ends.
This would be in contrast to the federal government, which has pushed for self-regulation by industry. CAIDP’s Randolph says California could play an important role in setting the floor for AI regulation given that much of the global AI industry is based there.
DAIR’s Hanna says Newsom is not only looking to show a contrast with US President Donald Trump, who believes in self-regulation by firms and last month signed a voluntary accord with several AI leaders, but also with possible Democratic contenders.
“With this, Newsom can look good on the primary debate stage,” she said.
Gallegos, the governor’s spokesperson, told Al Jazeera that “Governor Newsom isn’t interested in virtue-signalling or chasing headlines — he is leading the nation by taking a smarter, strategic, and effective approach to addressing frontier technology”.
Experts say that along with Newsom’s order setting up a committee to examine a kill switch, there is a need for measures for controls across the life cycle to reinforce safety.
“Safety needs to be built in. We issued 12 guidances across the lifecycle when the last ChatGPT model was released. There is a need for safety features throughout the life cycle,” of an AI model, Partnership for AI’s Finlay said.
As Newsom considers such recommendations from California agencies and the expert committee, Ekbia says governments “should take a lesson from what happened with social media. A handful of companies having control were able to do a lot of harm,” and regulate with an anticipatory approach and take on corporate monopolies.