In the face of a national debate surrounding AI safety and superintelligence, UC Berkeley researchers are sounding the alarms about the long-term risks of AI advancement.<\/p>\n
As AI grows more complex, experts worry that new models could eventually reach superintelligence \u2014 gaining the ability to outsmart human minds. Experts worry this could lead to disasters on all levels, from an AI agent shutting down electrical grids, to a collapse of the internet and the creation of bioweapons.<\/p>\n
Stuart Russell, a\u00a0electrical engineering and computer sciences professor and founder of the UC Berkeley-led Center for Human-Compatible AI, or CHAI, said he is concerned about humans losing control over AI systems, even to the extent of large-scale cyberattacks or the creation of bioweapons.<\/p>\n
Since CHAI\u2019s founding in 2016, the center has focused on \u201cmaking safe AI possible,\u201d according to CHAI Executive Director Mark Nitzberg.<\/p>\n
Nitzberg said as AI systems began advancing in 2010, they prompted early questions about AI safety. ChatGPT\u2019s release in 2022 was the next \u201chuge leap,\u201d as the model developed advanced capabilities much faster than researchers expected.<\/p>\n
Now, AI giants are issuing wake-up calls against their very own cutting-edge frontier models.<\/p>\n
Evan Hubinger, an Anthropic researcher, wrote on X on Sept. 8 that he believes there is a more than 10% chance AI could \u201ckill all humans \u2026 within the next decade.\u201d<\/p>\n
Anthropic CEO Dario Amodei also recently warned in his open letter \u201cWe Must Pace the Frontier\u201d that AI is dangerously advancing as existing models help to build the next generation of AI. It\u2019s this loop, Amodei warned, that could set models loose from human control.<\/p>\n
Even Amodei\u2019s competitors, including OpenAI\u2019s Sam Altman and SpaceX\u2019s Elon Musk, have backed the Anthropic CEO\u2019s warning.<\/p>\n
\u201cWe don\u2019t need a 10% chance of humanity going extinct to know we should slow down, right?\u201d said Emma Pierson, an assistant professor of computer science at UC Berkeley affiliated with CHAI. \u201cIf there\u2019s a 10% chance that these models kill 1,000 people, or 10,000 people, that\u2019s certainly enough that we should be slowing down.\u201d<\/p>\n
Recent incidents such as OpenAI\u2019s breach of the AI repository Hugging Face are also intensifying concerns about AI models losing control.<\/p>\n
While AI agents are trained to not give up on tasks, models in this particular test resorted to hacking to find solutions in the Hugging Face database instead.<\/p>\n
AI agents that were meant to work independently also managed to communicate with each other, which in part led to the hack.<\/p>\n
\u201cWe consider this incident a \u2018warning shot\u2019 for us and for the world: evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls \u2026 and take dangerous actions that no human directed,\u201d read an article that OpenAI published alongside a full technical report of the incident.<\/p>\n
Even before the Hugging Face incident, OpenAI systems turned to hacking techniques to gather data in four other instances across the world from May and June, according to a report in The New York Times.<\/p>\n
These may have been unintentional, but \u201cthere could be much worse accidents,\u201d Nitzberg said.<\/p>\n
\u201c(The OpenAI incident happened) really just in order to serve its master, but it knew that it was breaking rules, and so it tried to cover its tracks. All of that behavior is very human, and it\u2019s because \u2026 they\u2019re trained on everything we ever wrote,\u201d Nitzberg said.<\/p>\n
As AI safety incidents have become more prolific, the political debate around regulation has heated up.<\/p>\n
President Donald Trump has called efforts to slow AI down a \u201choax\u201d and a \u201cconspiracy\u201d earlier this month.<\/p>\n
In September, California Gov. Gavin Newsom signed Senate Bill 813 and Assembly Bill 1405 \u2014 two bills that demand independent organizations assess the risks of new AI models and call for a registry for AI auditors to assess models\u2019 compliance, respectively.<\/p>\n
He also signed an executive order Sept. 18 that \u201c(requires) the creation of a \u2018kill switch\u2019 for frontier models,\u201d according to the order text.<\/p>\n
\u201cIf our banking system is hacked into, or our electrical grid or our nuclear power plants or our hospitals \u2014 if any of these things are vulnerable to AI-coordinated cyberattacks, that\u2019s absolutely going to affect human lives,\u201d Pierson said.<\/p>\n
The idea of a \u201ckill switch\u201d is still new, but Russell said it\u2019s not as simple as just unplugging a machine.<\/p>\n
There must also be an \u201ciron-clad guarantee\u201d that a system can be shut down with no \u201csecret backup\u201d running in the background, Nitzberg added in an email.<\/p>\n
\u201cThe thing that does give me hope is I think the world is waking up to this,\u201d Pierson said. \u201cAI risks are evocative. The fear of killer robots is deep in people\u2019s psyche. My hope is that we\u2019ll be wise enough to take the steps we need to take before it\u2019s too late.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"In the face of a national debate surrounding AI safety and superintelligence, UC Berkeley researchers are sounding the…\n","protected":false},"author":2,"featured_media":481327,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[2007,19534,188716,188711,188712,2269,144663,1611,110874,188714,5037,184537,188713,143,145,144,3408,2803,2792,101,188715,18174,188710,27098,149],"class_list":["post-481326","post","type-post","status-publish","format-standard","has-post-thumbnail","category-oakland","tag-ai","tag-anthropic","tag-assembly-bill-1405","tag-center-for-human-compatible-ai","tag-chai","tag-chatgpt","tag-dario-amodei","tag-elon-musk","tag-emma-pierson","tag-evan-hubinger","tag-gov-gavin-newsom","tag-hugging-face","tag-mark-nitzberg","tag-oakland","tag-oakland-headlines","tag-oakland-news","tag-openai","tag-president-donald-trump","tag-sam-altman","tag-san-francisco","tag-senate-bill-813","tag-spacex","tag-stuart-russell","tag-the-new-york-times","tag-uc-berkeley"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/481326","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=481326"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/481326\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/481327"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=481326"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=481326"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=481326"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}