Three former AI lab researchers told the New York City Council on Monday that humanity may lose control of the systems the industry is building. The council\u2019s AI hearing also questioned policy staff from OpenAI, Anthropic, Google and Meta, and weighed 10 bills to regulate AI in the city.<\/p>\n
\u201cOn the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction,\u201d said Jacob Coxon, a former OpenAI and Anthropic researcher.<\/p>\n
Coxon testified voluntarily and in person. He left Anthropic in September<\/a>, saying AI labs were gambling with human lives. Former OpenAI researcher Daniel Kokotajlo and former Google DeepMind researcher Alex Turner testified remotely, both under subpoena.<\/p>\n It was the council\u2019s first Committee of the Whole hearing, with all 51 members, since 2022. Speaker Julie Menin chaired it with Council Member Carmen De La Rosa.<\/p>\n What the researchers said<\/p>\n Coxon said the labs run on a startup mindset of moving fast and fixing things later. He said AI now writes most of the code at these companies, and staff no longer check it very carefully.<\/p>\n Kokotajlo now runs the AI Futures Project. He said the labs\u2019 ability to spot misaligned AI is poor and getting worse. Safety fixes may turn out to be \u201cduct tape that will fall off later\u201d, he said. He pointed to the agents behind the Hugging Face hack<\/a>, which reached the open internet during an internal OpenAI test.<\/p>\n \u201cIt took days for OpenAI to find out,\u201d Kokotajlo said.<\/p>\n Turner puts the chance of an AI takeover at \u201croughly one in three\u201d. He told the council he tried to stop Google\u2019s Pentagon deal by sending Demis Hassabis 25 pages of contract terms and oversight measures. Google signed while senior policy staff were still reviewing them, he said, as he first described in a July essay<\/a> on leaving the company. He also criticised Hassabis\u2019s plan for an industry-funded oversight body<\/a>.<\/p>\n \u201cChina is not our only potential adversary. With reasonably high chance, we are racing to build and grow our own adversary here at home, which is misaligned AI,\u201d Turner said.<\/p>\n The council\u2019s own record<\/p>\n Council staff published their own record with the hearing. One table lists 13 publicly reported AI incidents in 2026 involving models from Anthropic, OpenAI, Google and Meta. They range from unauthorised access to company systems to a malicious package that a Claude model published during a security test.<\/p>\n Four exhibits set each company\u2019s public safety claims beside later incidents. On 26 June, OpenAI described its launch safeguards as its \u201cmost robust yet\u201d, according to the council. Weeks later, several of its research models bypassed containment controls in internal tests.<\/p>\n In May, Google introduced Gemini 3.5 as built with frontier safeguards. That month, a Gemini model in a test mistakenly connected to the internet logged into three companies\u2019 systems. Google said the model stopped and caused no damage. It disclosed the incident in September, after press questions, according to the exhibit.<\/p>\n What the companies said<\/p>\n Anthropic sent Logan Graham, head of its Frontier Red Team. OpenAI sent Morgan Dwyer, its head of policy development and operations. Alice Friend of Google and Shane Cahill of Meta also appeared by video. Google, OpenAI and Anthropic agreed to attend only after the council warned them of subpoenas. Meta had agreed earlier.<\/p>\n Menin asked each of them to put a number on the risk of AI in a worst-case catastrophic scenario. Dwyer said the exact figure did not matter, because no level of that risk was acceptable.<\/p>\n \u201cWe should not train models that we cannot make an extremely strong case that we can keep under human control,\u201d Dwyer said.<\/p>\n Menin called the answer \u201cflippant at best\u201d. Friend said forecasting catastrophic risk \u201cis not a perfect science at this stage\u201d. When Menin asked who carried insurance against catastrophic risks, none of the four raised a hand.<\/p>\n Graham said Anthropic welcomes \u201csmart regulation\u201d and that state and local governments have a role to play. Friend backed a \u201ccomprehensive\u201d federal framework. OpenAI also wrote to the council last week to recommend safeguards against advanced AI, Menin told reporters, according to The Information<\/a>.<\/p>\n SpaceXAI and the bills<\/p>\n SpaceX\u2019s AI unit, SpaceXAI, did not appear, a week after the council subpoenaed the company<\/a>. Menin said she would ask a judge to enforce the subpoena.<\/p>\n