Towards the end of Keir Starmer\u2019s time in office, his senior ministers, alarmed by the latest developments in artificial intelligence, began drawing up plans for a new AI safety law.<\/p>\n
They ordered a review of existing legislation to see what powers they already had, according to those briefed on the plans, and were exploring whether they could force the world\u2019s most advanced technology companies to submit their products for safety testing before launching them.<\/p>\n
In the midst of the political chaos that engulfed the former prime minister as he fought to remain in office, however, the plans fell by the wayside.<\/p>\n
Now, after two weeks in which the world has been told<\/a> there is a greater than 10% chance the technology could \u201ckill all humans\u201d, and King Charles has warned<\/a> about its potentially destructive capabilities, the government\u2019s attention has turned again to what to do about AI.<\/p>\n Andy Burnham, Starmer\u2019s successor in Downing Street, has said relatively little about the technology in the past, and in one of his first acts in office he scrapped the department that had been leading the legislative work.<\/p>\n The prime minister\u2019s focus is on more immediate domestic problems. But as he grapples with balancing his first budget and bringing down the cost of living, some fear the threat posed by AI has dropped off the government\u2019s radar.<\/p>\n \u201cThe first duty of government is to keep people safe,\u201d said Chi Onwurah, the Labour MP who chairs the science and technology committee. \u201cThe government\u2019s response to an existential threat cannot be to throw its hands up in the air and say there is nothing we can do.\u201d<\/p>\n Chi Onwurah. Photograph: Mark Thomas\/Rex\/ShutterstockRisks v potential<\/p>\n Britain has been at the forefront of the debate on AI safety since Starmer\u2019s predecessor, Rishi Sunak, decided to make it one of his top priorities in office.<\/p>\n Amid worries about what super-intelligent tools could do without human involvement, Sunak convened a summit in the UK and persuaded the US, EU, Australia and China to cooperate on AI safety<\/a>.<\/p>\n Starmer was more bullish<\/a> about AI\u2019s potential \u201cto increase productivity hugely, to do things differently, to provide a better economy that works in a different way in the future\u201d.<\/p>\n The Guardian has learned that in the former technology department, Emran Mian, the lead civil servant, had plans for AI to replace almost every civil servant in the department when they retired or moved on to a new job.<\/p>\n But at the same time, some in government were keen to remain focused on the risks posed by the technology.<\/p>\n Shortly before Starmer\u2019s departure, Yvette Cooper, then the foreign secretary, told the Guardian<\/a> she believed AI posed a Hiroshima-style risk to humanity.<\/p>\n Starmer\u2019s government took steps to mitigate the risks Cooper and others had identified. His government allocated \u00a3115m in June to fund a response centre to deal with incidents when AI goes rogue and for a new AI biosecurity programme.<\/p>\n Officials in the science department also began reviewing what powers ministers had to compel companies to cooperate with the UK safety regime, and what else they could legislate for.<\/p>\n \u2018A critical moment\u2019<\/p>\n When Burnham took office earlier this summer, one of his first moves was to abolish the Department for Science, Innovation and Technology (DSIT).<\/p>\n Kanishka Narayan, the junior minister in charge of AI, was given a promotion and placed in the Cabinet Office, where he continues to oversee the technology but without the logistical support of a dedicated department.<\/p>\n The move alarmed some in the AI industry.<\/p>\n Matt Clifford, the AI investor who had advised Starmer and Sunak, posted on X<\/a>: \u201cRight now is a critical moment for tech as an economic and national security issue. Tying up our most senior science and tech officials in a [reorganisation] wastes time and energy that\u2019s desperately needed for the actual substance.\u201d<\/p>\n Dame Wendy Hall, a computer scientist who chaired a UK government AI review in 2017, said: \u201cThe structure was a bit bloated in DSIT, but the fragmentation we have now has had a much worse impact.\u201d<\/p>\n AI safety rose up the political agenda again over the summer.<\/p>\n In July, OpenAI revealed one of its AI agents had launched<\/a> an autonomous cyber-attack on Hugging Face, a real company, triggering alarm among technology experts around the world.<\/p>\n Earlier this month, Jacob Coxon, a 28-year-old researcher, resigned from his job at Anthropic, claiming the company<\/a> was \u201cgambling with our lives\u201d. His comments were reposted online by one of his colleagues, Evan Hubinger, who added: \u201cWe really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.\u201d<\/p>\n Just as these warnings were being made, the Financial Times revealed<\/a> that Anthropic had launched the latest version of its Claude Mythos tool without submitting it to the UK safety institute first.<\/p>\n Mounting concerns<\/p>\n King Charles intervened this week, convening a meeting of AI executives in Scotland, where he told them: \u201cThe task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.\u201d<\/p>\n King Charles with the AI researcher Demis Hassabis and the Nvidia CEO, Jensen Huang, at Dumfries House in Cumnock, Scotland. Photograph: Jonathan Brady\/Reuters<\/p>\n Public concern is also growing. YouGov said on Friday that just over a third of Britons see AI as the third potential cause of human extinction behind nuclear war and climate breakdown \u2013 a figure that more than doubled since last year. Two-thirds think AI has the potential to end human civilisation, a view shared by the Nobel-winning computer scientist Geoffrey Hinton, and one in six support a slowdown in AI development.<\/p>\n In parliament, the Labour MP Liam Byrne has asked the head of the safety institute, Henry de Zoete, to testify in front of the business committee he chairs.<\/p>\n De Zoete replied earlier this week<\/a>, explaining that no one outside the US had been allowed to see Anthropic\u2019s latest product before it launched and insisting that his institute could still monitor such tools after they had been released.<\/p>\n \u201cWe have teams of talented AI researchers working on research projects based on both pre-release and post-release testing that are fundamental to the UK\u2019s understanding of AI risk, across chemical-biological, human influence and loss of control,\u201d he said.<\/p>\n Ministers have also begun to comment. Louise Haigh, the de facto deputy prime minister, told the TUC congress<\/a> this week there were \u201cclearly huge risks to our national security and society if the right guardrails are not put in place\u201d.<\/p>\n