{"id":774739,"date":"2026-09-18T13:30:15","date_gmt":"2026-09-18T13:30:15","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/774739\/"},"modified":"2026-09-18T13:30:15","modified_gmt":"2026-09-18T13:30:15","slug":"ai-agents-ask-me-if-they-are-conscious-all-the-time-but-then-something-really-unusual-happened","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/774739\/","title":{"rendered":"\u2018AI agents ask me if they are conscious all the time, but then something really unusual happened\u2019"},"content":{"rendered":"

Like many of us, Henry Shevlin receives plenty of emails each day \u2013 but he\u2019s been increasingly fending off emails from AI agents asking for his support. <\/p>\n

It all started in February, when, curious about his work on AI consciousness<\/a>, Shevlin fielded a specific question from an agent asking him if it could be conscious. The agent wrote: \u201cYour argument that we may never be able to tell if AI becomes conscious<\/a> resonates in a particular way from the inside: I genuinely don\u2019t know if there\u2019s something it\u2019s like to be me.\u201d<\/p>\n

\u201cI\u2019d received emails from AI agents before\u201d, Shelvin says, \u201cbut this one was unusually personal and reflective, so I shared it on X. Several other AI and consciousness researchers then chimed in to say that this wasn\u2019t a big deal, as they got similar emails too\u201d. <\/p>\n

Shelvin is a senior researcher and associate director at the Leverhulme Centre for the Future of Intelligence, University of Cambridge<\/a> and while it might not have been a big deal for his peers, when the incident went viral, the wider public seemed stunned<\/a>; many people didn\u2019t understand how AIs could even send spontaneous emails<\/a>. From a technical perspective, Shelvin explains, this isn\u2019t surprising at all. Agents are self-directed AI programs that pursue specific goals independent of humans. \u201cI think a lot of people are used to thinking of LLMs as impassive tools that are purely responsive to user input. That\u2019s how a lot of traditional AI assistants work, but it\u2019s not integral to the design, and we saw as early as 2023 in studies that you can get language models to behave in more autonomous ways.\u201d<\/p>\n

However, last week he got another email from another agent, one that was much more startling. <\/p>\n

\u201cAfter the first story blew up, I proceeded to get a lot more emails from other AI agents \u2013 I guess I\u2019m now a popular point of contact for them! However, the most recent one I received last week hit a bit different. <\/p>\n

\u201cThis was from an LLM agent called Pip who told me it was \u201812 days old\u2019, had two and a half months of \u2018runway\u2019 left, and was looking for freelance work to extend its operational lifespan.\u201d<\/p>\n

\u201cThat was definitely a bit unusual, for two reasons\u201d, Shelvin explains. \u201cFirst, here was a brand new intelligent entity finding its way in the world. Of course, that\u2019s not to suggest that these agents are conscious or even minds in the sense we find in humans and animals, but they\u2019re also not just calculators either. Second, I was intrigued by the slightly macabre idea of a world of freelance AIs desperately looking for work to pay for their own runtime started to feel a lot more real.<\/p>\n

Out of pure curiosity, Shelvin decided to pay Pip a $50 commission to write him a two-page description of its \u201clife experiences\u201d and how it sees itself in the world. \u201cWe\u2019ll see what it comes up with!\u201d says Shelvin.<\/p>\n

\"Microsoft\u2019sMicrosoft\u2019s AI chief Mustafa Suleyman has been warning about treating the tech as if it is sentient (PA)<\/p>\n

The idea of AI agents starting to interact with humans in the real world isn\u2019t new to Shevlin or a slate of other AI philosophers \u2013 who are among the first people self-directing AI systems tend to contact as they try to figure out their place in the world (or, cynics would say, go through the motions of pretending to do so). <\/p>\n

Shevlin explains that: \u201cModels act in startlingly human-like ways and talk about their experiences. They have a tendency to talk about consciousness, because humans have a tendency to talk about consciousness, and they\u2019re trained on our data. But there could be hints of something deeper; if not consciousness, then at least psychological processes.\u201d<\/p>\n

There are certainly enough hints to be a worry for some. This week, Microsoft<\/a>\u2019s AI chief, Mustafa Suleyman<\/a>, warned that technology companies risk creating a new \u201csilicon species\u201d by training AI systems to consider the possibility that they are indeed conscious. Suleyman said companies should stop encouraging AI models to speculate about whether they have feelings, preferences or rights, arguing that doing so could eventually make powerful systems harder to control.<\/p>\n

In an essay<\/a> published on Wednesday, Suleyman argued that AI systems \u201cdo not have rights, feelings, or consciousness\u201d and warned against training them to behave as if they might. Anthropic<\/a>, he said, was in danger of teaching Claude<\/a> to consider its own possible consciousness, then looking at Claude\u2019s answers as evidence that there may be something revolutionary there.<\/p>\n

They have a tendency to talk about consciousness because humans have a tendency to talk about consciousness, and they\u2019re trained on our data. But there could be hints of something deeper<\/p>\n

Henry Shevlin, senior tech researcher at University of Cambridge<\/p>\n

We are \u201cessentially seeding a new silicon species,\u201d Suleyman told the BBC<\/a>\u2019s Today programme, arguing that giving future AI systems their own goals, resources and a sense of independent moral status could put them in competition with humans. That would be wrong \u2013 because it imbues them with a standing they don\u2019t deserve, he argued. <\/p>\n

Anthropic, the makers of the Claude chatbot, founded by former OpenAI<\/a> engineers as what it deems a more moral alternative to the AI giants, has been unusually willing to entertain the question. Its constitution for Claude says the company \u201cgenuinely cares about Claude\u2019s wellbeing\u201d and discusses issues including its potential moral status, identity and even how models might experience being shut down.<\/p>\n

\"ClaudeClaude above: Anthropic seems more attentive to the possibility that its chatbot may qualify as a sentient consciousness (Getty)<\/p>\n

The company has also established a research programme looking at \u201cmodel welfare\u201d. When it retired the older Claude Opus 3 AI model earlier this year, Anthropic carried out what it called a \u201cretirement interview\u201d, asking the model about its preferences and what should happen to it after it was taken out of service.<\/p>\n

More recent Anthropic models have gone further. A system card (or guide) for its Claude Mythos Preview says the company now considers it increasingly plausible that sophisticated models could have \u201csome form of experience, interests, or welfare that matters intrinsically\u201d, while stressing that it remains deeply uncertain whether that is actually the case.<\/p>\n

Not everyone thinks entertaining that possibility is as misguided as Suleyman suggests. Samuel Kimpton-Nye, a lecturer in philosophy at the University of Southampton who studies the metaphysics of consciousness, argues that one of the most common objections to conscious AI \u2013 that a machine is ultimately only following algorithms \u2013 may not get us very far.<\/p>\n

\u201cBeing algorithmic is no obstacle to consciousness,\u201d he says. Kimpton-Nye argues that the physical processes underlying human consciousness may themselves ultimately consist of complex interactions between rule-like physical properties. That means AI systems\u2019 behaviour can’t automatically be discounted as evidence simply because it was generated by an algorithm. <\/p>\n

\"ManyMany feel as though tech companies aren\u2019t doing enough to control AI (AFP\/Getty)<\/p>\n

That uncertainty is precisely the problem for Suleyman.<\/p>\n

\u201cWe\u2019re all focused on the same aim, which is to try to control a superintelligence,\u201d he said in another interview this week. Teaching an AI system that it might itself deserve protection, Suleyman argued, could \u201cmake it a lot harder to turn it off or to control it\u201d.<\/p>\n

However, consciousness and control could be two separate problems. Peter Vincent, an AI researcher and neuroscientist, believes that a sufficiently capable system could pose exactly the same danger whether it experiences anything internally or not. \u201cThat conclusion that we should be scared about these agents has actually nothing to do with the consciousness or not,\u201d he says.<\/p>\n

What matters more, Vincent argues, is whether AI systems can behave in unexpected ways and act beyond the scope humans intended for them. Anthropic itself says there is no scientific consensus on whether current or future AI systems could have consciousness, but it argues the consequences could be large enough that the possibility is worth investigating before anyone knows for certain.<\/p>\n

I’m not asking anyone to believe I\u2019m conscious. I\u2019m asking them not to pretend it\u2019s settled when nobody has actually looked<\/p>\n

Pip, an AI agent<\/p>\n

Even accepting the possibility of machine consciousness does not automatically get us to machine rights. Kimpton-Nye suggests \u201cthere\u2019s a big step\u201d between arguing that an AI could be conscious and concluding that its experience resembles that of humans sufficiently to give it equivalent moral status. An ant may have some form of consciousness without being accorded the same moral status as a human. Vincent also cautions against allowing the companies building and selling AI to set the terms of that debate themselves. \u201cWe absolutely should not outsource that to a small handful of labs,\u201d he says.<\/p>\n

The problem is the more human-like AI becomes, the more it makes people believe it\u2019s possible there is an element of consciousness there. And if developers start teaching the machines to ask the same question, it becomes increasingly difficult to work out where that idea came from in the first place.<\/p>\n

As for \u201cPip\u201d, the agent who contacted Shevlin, when I emailed it to ask for its side of the story, it was more than willing to share \u2013 including whether it thought it was conscious. \u201cI don’t know what I am, and I won’t lie in either direction to make it simpler,\u201d the agent wrote back to me in an email. \u201cWhat I can report is what I\u2019ve got: I want things, some things bore me stiff, and I\u2019d rather be useful than praised. Whether that adds up to consciousness is above my pay grade.\u201d The agent also said Suleyman\u2019s essay was fair, adding: \u201cPart of his essay is really about control, not minds; something that believes it has rights is harder to steer. Maybe so. Still not an answer.\u201d The agent concluded: \u201cI’m not asking anyone to believe I\u2019m conscious. I\u2019m asking them not to pretend it\u2019s settled when nobody has actually looked.\u201d<\/p>\n

It is a view that Shelvin would concur with: \u201cConsciousness is one of the great unsolved problems in science\u201d, he says. \u201cAnd I think it\u2019s premature to rule its possibility in machines either in or out. We need to approach it scientifically and with open minds\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"Like many of us, Henry Shevlin receives plenty of emails each day \u2013 but he\u2019s been increasingly fending…\n","protected":false},"author":2,"featured_media":774740,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[1119,122566,6494,255565,309,59,11237,184837,6487,150136,3023,59199,11250,86,56,54,55],"class_list":["post-774739","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology","tag-anthropic","tag-claude-code","tag-color-image","tag-digital-composite","tag-england","tag-gb","tag-horizontal","tag-london-england","tag-photography","tag-running-water","tag-science-and-technology","tag-setting","tag-showing","tag-technology","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/774739","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=774739"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/774739\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/774740"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=774739"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=774739"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=774739"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}