Reports from Silicon Valley this month on the odds of artificial intelligence (AI) obliterating humanity by 2030 are enough to make one crack open the piggy bank and go wild. Why invest in a pension when there is a more than 10 per cent chance of all humans being killed by the time the next Fifa World Cup rolls around?
That specific forecast was made by a “whistleblower” researcher in tech firm Anthropic. Far from downplaying the prophecy, the company’s boss Dario Amodei later warned an AI swarm could be capable of “taking over the entire internet” within a year. And in a rare show of unity among money-grabbing mega-egos Elon Musk and Sam Altman – who are responsible for chatbots Grok and ChatGPT – backed calls to “slow the pace” of AI development.
What’s going on? And how should ordinary citizens like you and I respond to this outbreak in AI doomer vibes? Scary as the prospect is of robots lasering us to death through our devices, what’s more immediately terrifying is the idea of someone like Musk positioning himself as the responsible face of AI development. Or indeed Altman, aptly described by Oliver Callan on his RTÉ Radio show as “another one who is dead behind the eyes”.
To make sense of it all, Unthinkable turned to Carissa Véliz, who had the foresight to write a book recently on the very subject – Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI (Swift Press).
In 2023, a group of nine “expert forecasters” said there was a 30.5 per cent probability of an AI catastrophe killing most of humanity by the year 2200. But another study, involving 89 “superforecasters” – people who had a better than average record of prediction – and other experts put the odds of a mega-disaster much lower. They estimated a 2.1 per cent chance of an AI-caused catastrophe killing at least 10 per cent of the population, and a 0.38 per cent chance of an AI-caused extinction by the end of the century.
“My own assessment is that the superforecasters are probably closer to an accurate prediction,” writes Véliz. “But the point is this: predictions are all over the place because we are clueless, which suggests those predictions are not saying much about the world.”
Carissa Véliz, author of Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI. Photograph: Jorge Monedero
Speaking to The Irish Times this week, she doubled down on her scepticism about AI Armageddon. An associate professor of philosophy at University of Oxford’s institute for AI ethics, Véliz says the first thing to remember about predictions is that they are guesses, not facts. The second thing to remember is that predictions are “wishful” – what we desire influences our outlook.
She says some people have a larger appetite for risk than others. But this doesn’t fully explain why some people downplay the dangers of AI, while others play them up.
“I suspect there [are] myriad reasons why some people take the odds of existential risk more seriously than others … Emotional intelligence, historical perspective, and having some idea – or not – of the limits of prediction seem like important factors,” she says.
“There’s also the question of whether people believe what they say. For some, it might be a marketing strategy, at least in part. And yes, differences in appetite for risk seem likely to factor in too.
“But it’s vital to remark, first, that many of the tech executives making decisions might think they are more immune to risk, with their fortunes and their bunkers, than they actually are. Second, and relatedly, the people who are carrying most of the risk are not the ones making the decisions. Much like bankers in the financial crisis of 2008, many people are gambling with other people’s lives more than their own.”
OpenAI chief executive Sam Altman. Photograph: Benjamin Fanjoy/Getty Images
Véliz urges the public to be wary of tech companies “selling risk management while increasing systemic risk”. So are doomer forecasts more harmful than helpful?
“Yes, I think these forecasts – more precisely, people paying so much attention to them, and panicking over them – does increase risk. An environment of fear is typically conducive to higher risks. People can get paralysed, or willing to hand over more power to those who do not deserve it in exchange for the promise of safety.”
In a post this week on her Substack website, she elaborated. “Prophets impose their view of the future on you.” But “the future doesn’t get predicted – it gets produced”.
While we are dazzled by new technology, Véliz says “the greatest inventions of humanity are social and political” – and chief among them is democracy.
[ Will AI kill us all?Opens in new window ]
Some AI bosses think we’ll be happy to give up our democratic rights for the latest chatbot update – Altman has floated visions of future societies where humans prop up privately-owned AI utilities underwritten by cryptocurrency. Big tech is in for a surprise – that’s my prediction – because, weirdly enough, people like democracy more than they like their smartphones.
In her book, Véliz notes that comedy can be a surprising “ally in our challenge to defy the odds”. Why? Because “comedy sets the tone for freedom”.
So don’t be downbeat over recent AI forecasts. Don’t be fatalistic. It’s good news in a way: The odds on humans still existing by 2030 are almost 90 per cent. Make the most of it.