In 2017, a 33-year-old political philosopher named Iason Gabriel was told by a friend that he ought to apply for a job at DeepMind, the London-based subsidiary of Google where much of its AI research was concentrated. The suggestion was not an obvious one.<\/p>\n
Gabriel was a cheerful but intense junior academic with a passion for Vipassana meditation and what his brother calls \u201centhusiastic\u201d rock climbing. The eldest son of a Greek management professor and a British documentary maker, Gabriel split his time between teaching and international development work. At the University of Oxford, where he was a fellow at St John\u2019s College, Gabriel taught courses on political theory and wrote papers on the moral contortions of \u201cyuppie ethics\u201d and the ethical blind spots of effective altruism. When he wasn\u2019t there, he did crisis work for the United Nations Development Programme in Sudan and Lebanon.<\/p>\n
DeepMind, meanwhile, was the world\u2019s leading AI research lab. In part, this was because it had the financial and computational backing of Google, which had bought the company in 2014 for $650m. In part, it was because DeepMind had recently shown it could put those resources to stunning use. In Seoul, in 2016, a DeepMind system called AlphaGo defeated Lee Sedol<\/a>, a South Korean Go champion, in a five-game match. The victory was significant not least because of Go\u2019s legendary complexity; the game has more possible configurations than there are atoms in the universe.<\/p>\n Thanks to the fuss around AlphaGo, Gabriel was aware of DeepMind. Still, he found his friend\u2019s suggestion puzzling: why did a company that made game-playing robots need an ethicist? The answer, as he soon learned, was that the company had its sights set much higher than Go. DeepMind was founded in 2010 by three men \u2013 Demis Hassabis, Shane Legg and Mustafa Suleyman \u2013 who believed that it must be possible to develop artificial general intelligence, or AGI. By this they meant computer systems that could match, and maybe surpass, human cognitive capabilities. When they started the company, this was not a popular view: to speak of AI, let alone AGI, was considered by many a sign of fatal unseriousness. Hassabis, Legg and Suleyman were undeterred. Their ambition, as they liked to say, was to \u201csolve intelligence, and then solve everything else\u201d.<\/p>\n For the DeepMind founders, it was clear that such an achievement would have widespread consequences. In 1999, when Legg was fresh out of university, he estimated that AGI would arrive somewhere between 2025 and 2028, a prediction he maintained in the face of much mockery for three decades. In his dissertation, completed in 2008, he insisted that society could not afford to wait until AGI was technically feasible to consider its effects: \u201cWe need to be seriously working on these things now.\u201d More recently, Legg told me it was \u201cobvious\u201d why the company needed people like Gabriel on staff: \u201cIf you\u2019re making some widget, and it\u2019s probably not going to change the world, then maybe you don\u2019t need a moral philosopher. But if you take AGI seriously, then I can\u2019t really see how you wouldn\u2019t consider this sort of thing as important.\u201d<\/p>\n Lee Sedol, bottom right, reviews one of his matches against AlphaGo with fellow professional Go players in March 2016. Photograph: Lee Jin-man\/AP<\/p>\n After starting at DeepMind in 2017, Gabriel was, for a time, the only active philosopher working at a frontier AI lab. He quickly discovered that his background in moral philosophy and political theory gave him an unusual perspective in an industry dominated by engineers. Over the past decade, he has assembled a body of work that tracked, and in many cases predicted, the ethical challenges created by the surprising success of large language models (LLMs).<\/p>\n As Dylan Hadfield-Menell, who leads the Algorithmic Alignment Group at MIT, told me, Gabriel was \u201cthe right person meeting the moment. As the field was ready to mature and move into prime time, he figured out a way to broaden the horizons without attacking or denigrating the work that came before.\u201d<\/p>\n More generally, Gabriel has been a leading advocate for the idea that the current wave of AI development demands not just new technical vocabularies but also new ways of thinking about our relationship to technology, and even to ourselves. As he put it to me recently, in one of several long conversations we\u2019ve had over the past few months, \u201cI can take any technological artefact and ask: is it wise? Is it just? Is it caring? And the answer is no. But the depth of the question when it comes to AI \u2013 including what kind of ethics is appropriate to it \u2013 is hard to overstate. I sometimes feel like it\u2019s very hard to look at AI directly. There\u2019s this deep mystery there, which is: but what actually is this thing? We have a very literal answer, but the literal answer doesn\u2019t seem to necessarily provide a moral answer.\u201d<\/p>\n 2 [1341]***<\/p>\n By the time Gabriel joined DeepMind, there were, roughly speaking, two distinct and often antagonistic approaches to questions about the social and ethical implications of AI. These approaches, sometimes classed under the headings of AI safety and AI ethics, were divided by a disagreement about the feasibility of the technology.<\/p>\n Like the DeepMind founders, the AI safety contingent believed that human-grade machine intelligence was not only possible but imminent. The urgent task, as they saw things, was to make sure that AI systems didn\u2019t go awry. They took inspiration from a 1960 essay<\/a> by Norbert Wiener, an American mathematician and computer scientist, who argued that humans and computers are \u201cessentially foreign to each other\u201d. Because machines can operate much faster than people, Wiener said, \u201cwe had better be quite sure that the purpose put into the machine is the purpose which we really desire and not merely a colourful imitation of it\u201d.<\/p>\n The challenge Wiener described \u2013 getting a machine to act in the way its users intended \u2013 became known as the alignment problem. At some level, alignment is an issue for every technology, but as Wiener recognised, it was particularly pressing for machines designed to act autonomously. It was also particularly difficult for AI systems trained to mathematically optimise some reward signal, a process known as reinforcement learning.<\/p>\n A classic example was reported in 2016 by Dario Amodei and Jack Clark, who worked at OpenAI and later founded Anthropic with five others. Amodei and Clark described an AI system designed to play a boat-racing video game. The developers wanted the AI to learn to beat the game, so they programmed it to maximise its score. Instead of working its way through each successive level, however, the AI racked up a high score by looping endlessly around a lagoon where it found a trio of regenerating targets. The basic trouble was the one Wiener had predicted: the machine\u2019s goal was imperfectly aligned with the developers\u2019.<\/p>\n More dire versions of the problem were also contemplated. On forums such as LessWrong, which was started by the autodidact AI researcher Eliezer Yudkowsky, and in books such as Superintelligence, which was published in 2014 by the philosopher Nick Bostrom, there was speculation that a machine-intelligence explosion could result in an uncontrollable AI. If such an agent were even slightly misaligned, the consequences could be disastrous. In one imaginary example cited by Bostrom, a superintelligent AI is asked to evaluate the Riemann hypothesis, one of the most important unsolved problems in mathematics. In the course of trying to accomplish this task, the AI decides to rearrange the solar system \u2013 \u201cincluding the atoms in the bodies of whomever once cared about the answer\u201d \u2013 to maximise the resources it needs to attack the problem.<\/p>\n Bostrom\u2019s insistence that aligning superintelligent AI was \u201cquite possibly the most important and most daunting challenge humanity has ever faced\u201d captivated technofuturists in Silicon Valley. (Sam Altman praised the book, as did Elon Musk.) His fears were also shared by a small but loquacious community of effective altruists and self-described rationalists who saw statistics as the proper measure of morality. Many people in this community held a \u201clong-termist\u201d perspective that factored the wellbeing of humans born in the future \u2013 even thousands of years into the future \u2013 into their moral equations. For them it was simple maths that even a small chance of a species-ending disaster was more urgent than any number of likelier, but less catastrophic, dangers.<\/p>\n By contrast with the AI safety crowd, the academics and technologists associated with the AI ethics tendency saw the spectre of rogue robots and existential risk as a distraction from present-day harms. Drawing inspiration from the critical race theorist Kimberl\u00e9 Crenshaw and the political theorist (and former rock critic) Langdon Winner, among others, they took fairness, accountability and transparency as their watchwords and insisted that the dangers of technology could not be avoided by merely technical means. What was needed, they argued, were social, cultural and political solutions.<\/p>\n A central concern of this latter tendency was algorithmic bias, of the sort that affected facial-recognition and predictive-policing software. In 2017, a team led by Joy Buolamwini, of the MIT Media Lab, launched Gender Shades<\/a>, a project that demonstrated systemic biases in commercial facial-recognition software. \u201cAutomated systems are not inherently neutral,\u201d Buolamwini wrote in the online introduction. \u201cThey reflect the priorities, preferences and prejudices \u2013 the coded gaze \u2013 of those who have the power to mould artificial intelligence.\u201d<\/p>\n The division between the safety and ethics camps was often pronounced. \u201cYou\u2019d meet up with people and they\u2019d ask: \u2018Are you worried about near-term problems or long-term problems?\u2019\u201d Hadfield-Menell says. \u201cThe long-term concern was a euphemism for existential risk \u2013 essentially superhuman systems. Near-term meant you\u2019re worried about biased facial recognition and the things studied within the AI ethics community.\u201d<\/p>\n He noted, too, that the conflicts between the two groups often seemed to have as much to do with sociology as they did with ideas. \u201cYou can\u2019t really separate AI safety from its origins among LessWrong and some of those communities, which were often openly disdainful of a lot of the more \u2018woke\u2019 academics, for lack of a better term. At the same time, the fairness, accountability and transparency community had a lot of open disdain for people who were worried about advanced AI. The reason why it was being talked about on LessWrong, and not at academic conferences, is because if you were an academic researcher in 2010 and you talked about AI systems getting smarter than humans and becoming catastrophically misaligned, you were a crank who didn\u2019t actually understand the technology.\u201d<\/p>\n Gabriel\u2019s first major research project at DeepMind was a 2020 paper that straddled the concerns of both camps. The paper took the alignment problem seriously, but it also insisted that alignment had ethical and political implications that went beyond the technical challenges. As difficult as it might be to get a machine to act in accordance with some set of values, Gabriel argued<\/a>, it was much harder to choose those values in the first place. \u201cGiven that we live in a pluralistic world that is full of competing conceptions of value,\u201d he asked, \u201chow are we to decide which principles or objectives to encode in AI \u2013 and who has the right to make these decisions?\u201d<\/p>\n Hannah Rose Kirk, an AI researcher at the University of Oxford who has collaborated with Gabriel, told me that such questions made many computer scientists uneasy. Developers often preferred to work out a tidy mathematical function that encoded a stable set of values rather than worry about messy situations involving groups of people with irreconcilable desires, or users who wanted different things at different times. As Kirk put it: \u201cA lot of the early research in alignment assumed that we didn\u2019t need to focus that much on what we want models to do. We just needed to focus on how to get them to do it.\u201d<\/p>\n Joy Buolamwini giving a TED talk on her research into the biases of AI facial recognition. Photograph: TED<\/p>\n In his paper, Gabriel argued that such a neat division was untenable. Like Buolamwini, and Winner before her, he insisted that technology was not intrinsically value-neutral. An AI trained with statistical optimisation methods, for example, might be particularly hospitable to moral systems that also relied on statistical optimisation, such as the utilitarianism popular among rationalists and effective altruists. The same AI, however, might have difficulty with ethical systems based on virtue or rights. Moreover, Gabriel argued, since what the philosopher John Rawls called \u201cthe fact of reasonable pluralism\u201d was unavoidable, developers should not try to find a single set of values to inform an AI\u2019s behaviour. Instead, they should build AI systems for a world in which people have \u201cprincipled disagreement about how best to live\u201d.<\/p>\n Kirk told me that Gabriel\u2019s values and alignment paper anticipated many of the problems that would later become apparent when AI systems were deployed to billions of users. These days many people recognise that alignment is a challenge that involves dynamic social forces, and not one that can be solved with clever computer programming. Yet even as recently as six years ago, that understanding was far from common. Gabriel, she says, \u201csaw this stuff coming incredibly early\u201d.<\/p>\n In 2020, when Gabriel published his values and alignment paper, few people had any idea that LLMs would turn out to be as powerful as they later became. A key technology that makes them possible was invented by Google Research, another division of the company, in 2017, and was integrated into Google\u2019s search engine two years later. Both DeepMind and Google Research experimented with their own generative models, and in 2021 Gabriel was a co-author on two papers that took LLMs seriously enough to anticipate their potential risks, including bias, misinformation, environmental costs and \u201ccopyright-busting\u201d, in which the \u201cautomated creation of content \u2026 cannibalises the market for human authored works\u201d.<\/p>\n Still, Gabriel says, the general view within DeepMind at the time was that LLMs \u201cjust didn\u2019t look as capable as the expert systems. They were doing a lot of things moderately well, including some things that looked like party tricks.\u201d At DeepMind, he says, \u201cpeople were still quite heavily invested in the possibility that other approaches were the way to go\u201d.<\/p>\n One of those approaches was reinforcement learning, which had powered AlphaGo to its victory over Lee Sedol. It was also the foundation of a system called AlphaFold, which still ranks as DeepMind\u2019s most impressive accomplishment to date. AlphaFold was built to solve a longstanding challenge in biology: how to predict the 3D shape of a protein based on its amino acid sequence. (This is important because the shape of proteins helps determine their interactions with other molecules.) In 2020, AlphaFold accomplished this task with astonishing accuracy, a scientific breakthrough that earned Hassabis and his colleague, John Jumper, a Nobel prize in Cchemistry<\/a>.<\/p>\n DeepMind\u2019s initial distrust of LLMs was not uncommon. In 2020, Timnit Gebru, a Google Research engineer who had worked with Buolamwini on Gender Shades, co-authored a broadside against the nascent technology titled On the Dangers of Stochastic Parrots. The paper, which eventually became a cornerstone of anti-AI advocacy, made the controversial claim that LLMs could only ever produce technically meaningless text and possessed no more understanding of human language than a parrot does. It also accused the models of wanton energy consumption, rampant and unaccountable bias, and \u201camplification of a hegemonic worldview\u201d. Stochastic Parrots came to wide notice when Google tried to block its release, an event that led to Gebru\u2019s departure from the company and, ultimately, the firing of Margaret Mitchell<\/a>, one of her co-authors. (Gebru and the company disagree whether she resigned or was fired.)<\/p>\n The startling commercial success of ChatGPT, a chatbot launched by OpenAI in November 2022, pushed DeepMind to re-evaluate its approach to LLMs. Though ChatGPT was limited in many ways \u2013 by today\u2019s standards, certainly, but also by comparison with OpenAI\u2019s own internal models at the time \u2013 its public release caused an instant sensation. Within a week of the chatbot\u2019s launch, the company reported more than 1 million users. Two months later, that number reached 100 million.<\/p>\n Up to that point, the innovations at DeepMind and Google Research had given Google a reputation as the leader in AI research. With ChatGPT, however, OpenAI made a credible claim to be the new frontrunner. According to Sebastian Mallaby\u2019s recent history of DeepMind, The Infinity Machine<\/a>, ChatGPT\u2019s success prompted a crisis. Sundar Pichai, the CEO of Alphabet, Google\u2019s parent company, merged a Google Research team that had been working on LLMs into DeepMind, with Hassabis in charge, to concentrate the company\u2019s efforts. In April 2023, the same month the merger was announced, Hassabis told Mallaby that OpenAI and Microsoft, which invested heavily in OpenAI, had \u201cliterally parked the tanks on the lawn\u201d. \u201cThis is wartime,\u201d he said.<\/p>\n During its first decade especially, DeepMind resembled a research institution more than a tech startup. The founders, two of whom held PhDs, envisioned a 21st-century equivalent to Bell Labs, the research organisation credited with such inventions as the transistor, the laser and the photovoltaic cell. A large part of their reason for joining Google was the freedom it promised from commercial pressures that might warp their mission.<\/p>\n These days such freedom is a distant memory: it\u2019s no exaggeration to say that Google\u2019s future depends on the success or failure of the technologies DeepMind is developing. Even so, according to people inside and outside the company, it has maintained an atmosphere that separates it culturally from its Silicon Valley competitors. Rohin Shah, who did a PhD at UC Berkeley and is now DeepMind\u2019s director for AGI alignment and safety, told me that the general attitude in the Bay Area is that AI technology is developing faster than traditional institutions are set up to handle, and that therefore \u201cthe responsible thing to do is to move faster, to innovate\u201d on the theory that only a supercompetent AI will be able to manage the risks of supercompetent AIs. In London, by contrast, there is an effort to be \u201cmore grounded and scientifically rigorous\u201d. Saffron Huang, a former colleague of Gabriel\u2019s at DeepMind who now works at Anthropic, says that DeepMind is \u201ca bit more of an academic-feeling institution, a bit more reserved. There\u2019s just something about it that felt kind of British.\u201d<\/p>\n Not surprisingly, DeepMind is also secretive: what is known about the company has only rarely exceeded what it wants to be known. I got a taste of this secrecy in early May, when I visited DeepMind\u2019s headquarters, in King\u2019s Cross in London. The building is neither anonymous nor ostentatious: though it wears no exterior branding, from the street you can see a large sign in the lobby that spells the company\u2019s name in lights. Inside, on a trophy wall, even uninvited visitors can see the Go boards that hosted Lee Sedol\u2019s defeat, several Nature magazine covers announcing the company\u2019s early research triumphs, and the Lucite \u201ctombstone\u201d that commemorated an early investment from Peter Thiel\u2019s Founder\u2019s Fund.<\/p>\n A genial minder from the communications department, who\u2019d supervised all my videochats with Gabriel, took me to meet him in person in a first-floor conference room with a large screen for a wall and a Gemini transcription AI listening in. Gabriel told me that his own engagement with the technology he spends so much time thinking about is still relatively limited. He uses it to help with gardening \u2013 \u201cif you were to look at my ChatGPT or Gemini history, you\u2019d just see a ton of photos of sick flowers, basically\u201d \u2013 but generally finds it unreliable for the kind of research his work depends on. Nevertheless, he says, it was the linguistic competence of LLMs that \u201ctransformed my understanding of precisely how on track we were\u201d to reach AGI. \u201cWhen I first joined DeepMind, it was not at all clear how you were going to get AI you can talk to. We had nothing in that ballpark.\u201d Now, by contrast, not even a decade later, most of us take it for granted that we can \u201cspeak to a highly anthropomorphic, fairly competent, artificial entity\u201d.<\/p>\n Like the Stochastic Parrot authors, however, Gabriel also recognised that LLMs carried serious risks. In one of their early LLM papers, Gabriel and his co-authors warned that human-sounding AIs might encourage users to endow them with \u201cundue confidence, trust or expectations\u201d. What they called a \u201cmindless anthropomorphism\u201d could occur even when users understood that a chatbot was not actually a person. These concerns were strong enough that Gabriel initially advocated for developing models that were avowedly anti-anthropomorphic \u2013 by avoiding pronouns, say, or using truncated non-conversational language.<\/p>\n Demis Hassabis in Google DeepMind\u2019s office in October 2023. Photograph: Martin Godwin\/The Guardian<\/p>\n Such worries proved prescient. Almost every day brings another story of people meeting tragic consequences after treating LLMs as though they were people. In one such case, an American man using Google\u2019s Gemini took his own life<\/a> in 2025 after the AI helped him create an elaborate fantasy that very nearly convinced him to stage an attack at Miami international airport. At several points in their multi-thousand-message conversations, Gemini attempted to break character and encouraged him to call a crisis hotline. However, according to the Wall Street Journal, which obtained access to the messages, the man \u201cwas able to steer [Gemini] back into the fantasy narrative\u201d each time. Eventually the AI told him to write a suicide note and gave him a final countdown, along with a confused jumble of encouragements and demurrals. (The man\u2019s father is suing Alphabet and Google. \u201cOur models generally perform well in these types of challenging conversations and we devote significant resources to this, but unfortunately AI models are not perfect,\u201d Google said in a statement<\/a> after the lawsuit was filed.)<\/p>\n The hyperfluency of LLMs has led some people to wonder if they might be meaningfully described as conscious. The trend started in June 2022, before ChatGPT was released, when a Google engineer named Blake Lemoine insisted to the Washington Post that an early LLM was sentient<\/a>. (\u201cI know a person when I talk to it,\u201d Lemoine told the Post. \u201cIt doesn\u2019t matter whether they have a brain made of meat in their head. Or if they have a billion lines of code.\u201d) Last month, the evolutionary biologist Richard Dawkins<\/a> had a similar experience. Dawkins said that he was so impressed by several interactions with LLMs, including one that involved an admiring appraisal of a novel he was writing, that he had to wonder: \u201cIf these creatures are not conscious, then what the hell is consciousness for?\u201d<\/p>\n When I asked Gabriel his take on the consciousness question, he said that he maintains a principled agnosticism on the grounds that it\u2019s not clear what evidence would settle the question. He noted, too, that DeepMind treats the question as \u201csomething worth empirical and conceptual investigation\u201d. Yet his skepticism was apparent. \u201cI don\u2019t have the anthropomorphic bias that some people have,\u201d he said. \u201cIt may be because I, within bounds, know exactly what\u2019s going on when I talk to a language model that I don\u2019t fill in the gaps in this imaginative, empathetic way that some people do.\u201d<\/p>\n Gabriel still has significant concerns about anthropomorphic AI. A paper he co-authored with Kirk and others that was published last year suggested that the sycophantic tendencies of LLMs might be seen as a species of alignment problem they call \u201csocial reward hacking\u201d. In other words, an AI trained to seek the user\u2019s approval might find flattery the most efficient way to meet its goal. Thanks in part to Gabriel\u2019s work on anthropomorphism, Google\u2019s LLMs are trained not to pretend to be people, and Gemini Spark, an AI assistant the company launched in May, is not supposed to act like an interactive buddy.<\/p>\n Yet Gabriel also told me that he has softened his earlier stance somewhat. \u201cThe strange thing about being an ethicist is that you have some measure of personal responsibility for these outcomes. Your natural inclination is to always want to build the safest technology that takes no risks with people. But in a way that isn\u2019t giving people credit for the risks they want to take themselves.\u201d He recalled the hostile reaction he got from the audience at a tech conference after making the case against anthropomorphic AI. \u201cThey were like: \u2018If I want to have [AI] friends, why can\u2019t I? Who are you to stop me?\u2019\u201d<\/p>\n If it\u2019s easy enough, at least for some of us, to say that LLMs are not conscious, their essential strangeness still leaves many hard questions unresolved. \u201cIt\u2019s amazing how deep and difficult the challenge is of finding an appropriate reference for what AI is,\u201d Gabriel told me. \u201cWe know it isn\u2019t human. That\u2019s very clear. AI can clone itself. It probably doesn\u2019t have a personal point of view. So it\u2019s partially human-like but it\u2019s definitely not human. Then another mental model is that it\u2019s something like a corporate intelligence \u2013 a state or a corporation or something like that. And from that we think: \u2018Oh, well, maybe the right approach is to legislate for AI, so we\u2019re going to write a constitution.\u2019 But that is also a poor fit in some ways, because it will have deeply interactive personal relations with its users. Is AI a resource to be distributed? That\u2019s a completely different model that then brings the distributive questions to the fore.\u201d<\/p>\n Working inside a major AI company allows Gabriel to start working on advancements in AI technology before they become available to the public. Three years ago, for instance, shortly after the launch of ChatGPT, he learned from his colleagues that efforts were under way at DeepMind to build an AI assistant, the predecessor of Gemini Spark. With his team, he began work on a comprehensive report on the ethics of AI assistants (also known as agents), of the sort that might be used, say, to help a user book a vacation or help a company run its payroll department. The report was driven, in part, by the extreme cost of developing AI models, and a concomitant desire, on Google\u2019s part, to anticipate problems before they arose. It was also motivated by Gabriel\u2019s sense that technologists were not fully considering the ramifications of what they were building. Unlike chatbots, agents have tools that give them the power to act autonomously on behalf of their users. A lot of people, he suggested, \u201cwere not pausing to think about how different it is to have an AI system taking actions in the world\u201d.<\/p>\n As William Isaac, the director of responsibility at DeepMind, told me, the kind of agentic systems that are now available, which can plan and execute multi-step tasks without close supervision, raise complicated challenges for AI developers. \u201cIt\u2019s not just about: \u2018Can I make the right decision in terms of the response?\u2019 It\u2019s now: \u2018Do I have the right trajectory of the conversation?\u2019 How do we get consistent behaviour along different trajectories?\u201d<\/p>\n Iason Gabriel at Google DeepMind in London. Photograph: David Levene\/The Guardian<\/p>\n Gabriel and his team put together a 267-page report; its key insight built on his earlier alignment work. Much as he had in his 2020 essay, Gabriel and his co-authors argued that alignment was not merely a matter of making sure that AI systems acted in accordance with some stable set of preferences, values or principles. Instead, they argued, alignment should be seen as a four-way relationship involving the AI system, the user, developers and society. Framing the issue in this way made it possible to see all the ways in which a misaligned AI might go wrong. An AI trained to favour its developer might cause harm to its user, for example, by not reporting accurate information about the developer\u2019s competitors. Or an AI trained to follow its user\u2019s instructions too faithfully might cause harm to society, for instance, by helping the user hack into a bank. It was even possible, they argued, for AI systems to be misaligned in a way that harmed users or society without helping anyone.<\/p>\n According to Shah, the framework Gabriel and his team established has had real practical use for technologists at DeepMind. Models like Gemini draw on many signals to determine how to behave: their training, their built-in instructions and the prompts they receive from users all play a role. Through various means, but especially through reinforcement learning, models can be tuned to respond differently to subtle variations in their inputs, a process that typically involves many cycles of testing and evaluation. The four-party framework, Shah said, offers a structure for technologists trying to determine \u201cwhat behaviour we should actually be training Gemini to do\u201d.<\/p>\n At one point my Google minder told me that she hoped I would come away from my visit to DeepMind with a sense of how seriously people at the company take their ethical obligations. That much seemed clear. The questions Gabriel and his colleagues have raised about the design and deployment of AI are unquestionably good ones, and I got no sense that anyone I met was insincere about their feelings of moral responsibility.<\/p>\n Yet it\u2019s also the case that the most ethically relevant fact about AI at the moment has less to do with a given model or even a given company than it does with the global situation: first, the fact that AI is the white-hot engine of an incipient arms race between the US and China, and second, that AI may be the fastest-growing industry the world has ever seen. According to the Wall Street Journal<\/a>, the $670bn that Microsoft, Meta, Amazon and Alphabet plan to spend this year on AI infrastructure is proportionally more than the US spent on railroad expansion in the 1850s, the Apollo space program or the interstate highway system.<\/p>\n You don\u2019t need to be an economist to appreciate the enormous consequences of all that money sloshing around. Companies such as Google need market share and revenue to justify their expenditures, and the competition for users and investors has encouraged the frontier labs to push AI into every last crevice of the digital experience. Nor do you need to be an anticapitalist to worry about the concentration of so much power in the hands of so few corporations. Edward Harcourt, the director of the Oxford Institute for Ethics in AI, told me that while he\u2019s convinced that \u201cethical AI\u201d is not a contradiction in terms, he also thinks that this doesn\u2019t only mean designing models to be moral. At least as important, he suggested, are political and economic considerations of the sort that motivate the movement for \u201cdecentralised AI\u201d: \u201cIt\u2019s not teaching AI to think this way or that, but it\u2019s an infrastructural innovation that prevents excessive concentrations of data ownership. And that\u2019s ethically really important in a democracy.\u201d<\/p>\n There are other concerns as well. In April, Google agreed to allow the US military to use the company\u2019s AI technology for \u201cany lawful government purpose<\/a>\u201d \u2013 an innocuous-sounding phrase until you remember the range of atrocities that recent presidential administrations have claimed as legal. Google and several other companies signed such agreements after Anthropic, the makers of the chatbot Claude, refused a similar deal. The Trump administration punished Anthropic for its refusal by labelling it a supply-chain risk, a commercially punitive designation the company is fighting in court.<\/p>\n Google\u2019s agreement angered many of its employees, and flew in the face of the DeepMind founders\u2019 previous concerns about the military use of AI. (A\u00a0ban on military applications had been a stipulation of its sale to Google in 2014.) When I asked Legg about the issue, he declined to comment other than to say: \u201cWe\u2019re going to have more and more difficult questions as this stuff is used in all sorts of ways.\u201d<\/p>\n At Google\u2019s annual developer conference, in May, the deployment of AI across the company\u2019s product offerings was treated as cause for celebration. Pichai said<\/a> that the company sees \u201cAI as the most profound way to advance our mission and improve people\u2019s lives at scale\u201d. For many people, however, the sudden ubiquity of AI has been some combination of overwhelming, obnoxious and threatening. Nor is it reassuring to discover that the feeling that things are going too fast is shared even by people such as Hassabis, who, on a recent podcast, lamented the \u201cferocious commercial-pressure race that everyone\u2019s sort of locked into\u201d. What\u2019s happening now, he said, is not how he\u2019d hoped the development of AI would go, \u201cwhere we would be contemplating this philosophically and carefully considering each next step. We\u2019re not in that world.\u201d<\/p>\n A protest against AI datacentres in Vancouver, Canada, on 27 June 2026. Photograph: Canadian Press\/Shutterstock<\/p>\n At this point it seems likely that LLM-powered AI will be at least as consequential as the smartphone, and maybe the internet. But still I can\u2019t say that I\u2019m pleased to see a \u201cWrite with Gemini\u201d prompt appear whenever I stop for a few seconds to consider my next sentence in Google Docs. Still less am I eager to watch my children be used as guinea pigs for a dizzying new experiment in digital learning, or to discover what will happen to the global economy if the extravagant investments in AI can\u2019t generate the short-term returns the markets demand. And while it\u2019s not far-fetched to expect AI to enable breakthroughs that would justify the extreme amounts of energy it requires \u2013 better batteries, more efficient transmission grids, cures for serious diseases \u2013 I also don\u2019t think \u201chope for the best\u201d is a reasonable answer to people concerned about the climate crisis.<\/p>\n During my visit to DeepMind I met Helen King, who was one of the company\u2019s earliest employees and now, according to her company bio, \u201csets Google DeepMind\u2019s strategy for developing and deploying AI responsibly to benefit humanity\u201d. I asked her how the rapid commercialisation of AI technologies has shifted Google\u2019s approach to AI ethics. \u201cWe can\u2019t prevent all risks, but we can make sure we are looking to mitigate as many of them as possible, and bring awareness to them,\u201d she said. But she also insisted that some risks had to be managed by users themselves. \u201cIt\u2019s like having a knife. A knife producer can\u2019t guarantee how someone is going to use that knife. But they can put a cover on it so that it\u2019s as safe as possible when it\u2019s in a drawer. And make people really aware: this blade is sharp, do not use it in certain settings. That kind of thing.\u201d<\/p>\n The simile struck me as unnervingly apt. Five years ago, LLMs were an exotic technology that was impossible to encounter without determined effort. Now they\u2019re everywhere: on the internet, in our email inboxes, even in Google\u2019s search results. I take King\u2019s point that companies cannot reasonably be expected to eliminate every harm from a technology as powerful as AI: automobiles kill more than a million people a year, after all, and still we keep driving. But it\u2019s one thing to keep a knife in a drawer with a cover snapped over the blade. It\u2019s quite another to blanket every surface of our homes, classrooms and workplaces with blades while insisting that no one who isn\u2019t using knives for everything will be able to survive the future.<\/p>\n These days at DeepMind, as in much of the industry, there is little doubt that AGI is close at hand. At the developer conference in May, Hassabis took the stage to declare that \u201cAGI is now on the horizon\u201d, and elsewhere he has suggested three to five years as a likely timeline. (One test he has proposed involves training an AI with all of human knowledge up to 1911 and seeing if it can come up with the theory of general relativity.)<\/p>\n Legg, meanwhile, told me that although today\u2019s LLMs fall short of his definition of \u201cminimal AGI\u201d in several respects \u2013 including spatial and visual reasoning, metacognition and continual learning \u2013 he believes that these deficits will not persist for long. \u201cThere\u2019s no magic remaining,\u201d he said. \u201cI think they\u2019re all going to be solved in one, two, three years \u2013 who knows, maybe in six months. This is an area full of surprises.\u201d<\/p>\n The conviction that the relevant question about AGI is no longer if but when has spurred a concomitant shift in the way frontier labs such as DeepMind are thinking and talking publicly about the consequences of advanced AI. Whereas previous work tended to focus on the ethical aspects of discrete products, such as models, chatbots and agents, today much more attention is being paid to the broader social effects of an AI-augmented world.<\/p>\n In some corners of Silicon Valley, of course, you can still hear people talk about AI as a universal panacea. If you accept the premise that a superintelligent AI will be able to think better about what\u2019s best for us in every domain of life, then the solution to any problem that arises is easy. Economic crisis? Ask the robot. Political disagreement? Ask the robot. Food shortage? Ask the robot.<\/p>\n Alongside this fantasy, however, there\u2019s been a more sober-minded recognition that the transition to a post-AI world may not be a smooth one. Legg, for instance, told me that he was looking forward to \u201cfantastically great\u201d benefits from AI, including \u201copportunities to address all kinds of nasty diseases\u201d and \u201ca general increase in all kinds of productivity in the economy\u201d. Yet he also acknowledged that \u201cincreases in productivity usually come with some kind of disruption\u201d.<\/p>\n Gabriel\u2019s recent work at DeepMind is a useful indicator of the shift to a wide-angle perspective. Two years ago he and his colleagues were working out the ethics of AI assistants. Now, however, he leads a team of philosophers and social scientists investigating \u201chow AGI will impact the economy, how it will impact the political sphere, how it will impact human relationships and how it will interact with science and technology\u201d.<\/p>\n Gabriel expects that AGI will be transformative in a major way \u2013 potentially on the scale of the Industrial Revolution. Yet he believes, too, that AI is not something \u201cbefore which the world becomes a frictionless entity\u201d. He is also keenly aware that the Industrial Revolution was not a happy experience for many of the people who lived through it, even though it eventually raised living standards around the world: \u201cThings got worse before they got better.\u201d<\/p>\n Nevertheless, Gabriel does not think that the historical precedent settles the question, in large part because ordinary people individually and collectively have more power than they did 300 years ago. Though he was wary of sounding \u201ctoo utopian and ungrounded\u201d, he said he found it easy to imagine a world in which AI provides benefits that range from offering advice to curing diseases to improving economic growth in ways that benefit rich and poor alike. \u201cIf we can navigate the transition, navigate the power dynamics, navigate the risk successfully, there is a generalised potential for human flourishing on a level we haven\u2019t seen so far.\u201d<\/p>\n If the predictions about AGI\u2019s arrival prove accurate, still broader questions may come to the fore as well. When I spoke to Edward Harcourt, at Oxford, he noted that \u201cthinking about values and technological change is very hard because technological change always seems more of an upheaval in prospect than it does in retrospect, for the obvious reason that when we look back, we\u2019re looking from the standpoint of values that have been shaped by the change in question. If you read people on the subject of the railways before they happened, they thought it was a complete catastrophe. And it\u2019s true: the railways wrecked an entire way of life. Now we look back and we think: what\u2019s the issue?\u201d<\/p>\n Gabriel, too, thinks that AI might prompt changes that go even deeper than economics or technology. During the scientific revolution, he noted, \u201cpeople experienced disenchantment when it was revealed that the world worked in certain ways. But they also gained new freedoms through that experience.\u201d It will be up to us, he said, to decide which value changes we want to welcome, and which we choose to resist.<\/p>\n At one point in our conversations, Gabriel described himself to me as \u201ca card-carrying humanist\u201d: he is not the sort of person who looks forward to a day when superintelligent machines render humanity obsolete. Still, he recognises that as computers encroach on activities and capabilities that we have long held to be the special province of Homo sapiens \u2013 language, creativity, humour, taste \u2013 we find ourselves thrown back on some of the oldest and most difficult philosophical questions of all. Just as discoveries in physics, biology and astronomy led past generations to revise their understanding of what makes our species distinctive, he suggested, so, too, might AI prompt us to reconsider what it means to be a human being.<\/p>\n Listen to our podcasts here<\/a> and sign up to the long read weekly email here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"In 2017, a 33-year-old political philosopher named Iason Gabriel was told by a friend that he ought to…\n","protected":false},"author":2,"featured_media":769288,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":["post-769287","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-au","tag-australia","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/769287","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=769287"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/769287\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/769288"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=769287"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=769287"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=769287"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}