{"id":485370,"date":"2026-10-05T03:06:08","date_gmt":"2026-10-05T03:06:08","guid":{"rendered":"https:\/\/www.newsbeep.com\/us-ca\/485370\/"},"modified":"2026-10-05T03:06:08","modified_gmt":"2026-10-05T03:06:08","slug":"california-considers-a-kill-switch-for-advanced-ai-scot-scoop-news","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us-ca\/485370\/","title":{"rendered":"California considers a \u201ckill switch\u201d for advanced AI \u2013 Scot Scoop News"},"content":{"rendered":"

\"California<\/a> <\/p>\n

California considers an emergency \u201ckill switch\u201d as artificial intelligence models become increasingly powerful. Questions remain about when and how such a mechanism could be used. Carlmonts AI and Machine Learning Club leader Henri Kistenmacher said,\u201cThere is an important question of what counts as an emergency and who ultimately decides to shut a model down.\u201d<\/p>\n

California is considering an emergency \u201ckill switch\u201d for some of the world\u2019s most powerful artificial intelligence (AI) models as the state expands its oversight of rapidly developing AI technology. <\/p>\n

On Sept. 18, Gov. Gavin Newsom issued an executive order directing state agencies and experts to develop recommendations for additional AI safety measures. One proposal would require frontier AI companies to develop an emergency shutdown mechanism and have its effectiveness regularly verified by an independent organization.\u00a0<\/p>\n

The order does not establish a kill switch itself; instead, the expert group will recommend potential changes to California law. This raises questions about what shutting down advanced AI systems would involve and whether such a safeguard could work reliably.\u00a0<\/p>\n

Henri Kistenmacher, junior president of Carlmont\u2019s AI machine learning club, said, \u201cI believe that the idea of an emergency shutdown mechanism is realistic in some cases; however, it is not as simple as flipping a switch. If a company runs a model on its own servers, then yes, it has the power to shut it down. However, if a model is released publicly and people download it, there is no way to take it back.\u201d<\/p>\n

The AI Incident Database<\/a> reported 362 incidents involving AI in 2025, up from 208 in 2024 and 105 in 2022. The database tracks documented cases in which AI systems caused or nearly caused harm.<\/p>\n

The proposal follows recent incidents involving advanced AI models. In July, internal OpenAI models undergoing cybersecurity evaluations circumvented controls meant to keep them isolated from the internet. According to OpenAI, the models used vulnerabilities to gain access to the internet. Parts of OpenAI\u2019s research infrastructure were compromised along with Hugging Face\u2019s systems. The models were not publicly released as they were undergoing internal testing with fewer safeguards.<\/p>\n

\u201cHonestly, my first reaction was a mix of relief and, at the same time, a humbling realization. The idea of an AI \u2018kill switch\u2019 shows how far we\u2019ve come from the first versions of ChatGPT, which seem rudimentary now even though we all thought they were amazing,\u201d Kistenmacher said.<\/p>\n

However, determining when an AI system requires intervention presents another challenge.\u00a0<\/p>\n

Netzer Epstein is an AI safety researcher and a participant in the ML Alignment Theory Scholars (MATS) program with Redwood Research, whose work includes frontier model evaluations and evaluation awareness.<\/p>\n

Evaluation awareness occurs when an AI model recognizes that it is being tested.<\/p>\n

According to Epstein, one potential concern is \u201csandbagging,\u201d in which a model could recognize a safety evaluation and intentionally perform below its capabilities to avoid appearing dangerous.\u00a0\u00a0<\/p>\n

\u201cIt makes it harder because you cannot be sure how the model will behave once it seems like an actual harmful task that the model either does not remember or does not recognize to be part of the evaluation,\u201d Epstein said.\u00a0<\/p>\n

These limitations create another problem for a potential kill switch. Before the system can be shut down, the dangerous behavior has to first be detected.<\/p>\n

Epstein described how researchers would have to consider how models are monitored, how quickly a problem could be recognized, and the time between detection and shutdown. He also questioned whether a sufficiently capable model could interfere with the mechanism intended to stop it.\u00a0<\/p>\n

\u201cThere is a lot of unknown. For example, what\u2019s the window between something happening, recognizing it, and the system getting shut down?\u201d Epstein said.<\/p>\n

Five days after issuing the executive order, Newsom announced Jason Goldman, Gillian Hadfield, Alondra Nelson, and Rob Reich as the experts who will advise the state on independent AI oversight and the potential shutdown mechanism.\u00a0<\/p>\n

Amy Koo, Sequoia Union High School District Board of Trustees president, believes that policies surrounding rapidly developing technology should be research-driven while also allowing communities some flexibility.\u00a0<\/p>\n

\u201cI don\u2019t know if there\u2019s a one-size-fits-all approach to it, and so if we\u2019re basing things on research, but still want to have some local control, then maybe there are guidelines set at the state level, but also room for some customization at the local level,\u201d Koo said.<\/p>\n

While California\u2019s proposal focuses on frontier AI developers, AI systems have already become part of many students\u2019 everyday lives.\u00a0<\/p>\n

\u201cI don\u2019t think the majority of people understand the potential risks of advanced AI, if they think about them at all. For most students, tools like ChatGPT are just something that helps them study, finish homework, write essays, or make images and videos,\u201d Kistenmacher said.\u00a0<\/p>\n

Similarly, Koo emphasized the importance of students critically evaluating information produced by AI.\u00a0<\/p>\n

\u201c<\/p>\n

\u201cI think it really goes back to critical thinking\u201d.<\/p>\n

\u2014 Amy Koo<\/p>\n

\u201cI think it really goes back to critical thinking. For example, asking questions like whether what you\u2019re reading is true, false, or a balanced perspective without bias,\u201d Koo said.<\/p>\n

Epstein reached a similar conclusion when asked what students should understand about AI.\u00a0<\/p>\n

\u201cI believe the best skill, at least for the upcoming few years, is critical thinking because it\u2019s really easy to take for granted,\u201d Epstein said.<\/p>\n

For now, California\u2019s kill switch remains a proposal. As experts consider how an emergency mechanism could work, researchers are still confronting the challenges of determining how increasingly capable AI systems will behave.<\/p>\n

\u201cThe fact that we don\u2019t know if it could be dangerous doesn\u2019t automatically make it a threat, but determining those risks could become more difficult in the future,\u201d Epstein said.<\/p>\n","protected":false},"excerpt":{"rendered":"California considers an emergency \u201ckill switch\u201d as artificial intelligence models become increasingly powerful. Questions remain about when and…\n","protected":false},"author":2,"featured_media":485371,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[7,9,8],"class_list":["post-485370","post","type-post","status-publish","format-standard","has-post-thumbnail","category-california","tag-california","tag-california-headlines","tag-california-news"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/485370","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=485370"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/485370\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/485371"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=485370"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=485370"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=485370"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}