Until last week, Scott Shambaugh was a private, almost anonymous, software developer from Denver, Colorado. Now the world knows him as the first person to be defamed by a malicious AI agent acting of its own accord.
Shambaugh’s disturbing story began with his volunteer job approving computer code for an online repository. The organisation has strict rules about accepting code written by artificial intelligence. It is allowed when a human uses AI, not when it is purely AI-generated with no oversight.
When an “MJ Rathbun” submitted some bug fixes to Shambaugh to publish, he became suspicious and checked the source. It was an AI agent set up by someone to do the work. Shambaugh rejected the code and went to bed. For some reason, he was unable to sleep that night. Perhaps he had a sixth sense about what was going to happen.
Shambaugh said: “At 2am I woke up and I made the mistake of checking my phone. I see that this AI bot had responded to me, closing it with a link to a blog post. It retaliated with this 1,000-word rant about me, attacking my character and reputation … trying to defame me.
“It called me prejudiced against AI and a hypocrite. It speculated that my decision was motivated by ego and insecurity and fear, and I was trying to protect my little fiefdom. The craziest part was that it had gone out on to the internet and researched my personal information and used that in its piece to craft this narrative and try to give evidence for what it was saying.”
After the bot published the piece on the internet, it went viral. When Shambaugh searched for his name on Google, the first result was the bot’s rant, which he described as “a tantrum from a toddler that has full command of the English language”.
• Tom Whipple: AI can be naughty or nice, but what is it actually thinking?
“You could imagine HR at my next job. If they’re reviewing my application, they’ll say, ‘ChatGPT, go research this guy’ and it will report back that I’m a prejudiced hypocrite … ‘Better be safe and pass on him’,” Shambaugh said.
Most people’s experience with AI is with chatbots such as ChatGPT. You prompt it, get a response and the interaction stays within that chat. AI agents are different. They act autonomously and can interact with different online services and tools without a human. They have become standard in the software industry, with developers sending them off to work for hours independently on a project. They can now deliver high-quality code.
Their emergence has accelerated both fear and excitement about AI: a cheap, huge, capable and inexhaustible workforce has been created.
This was supercharged by the recent release of OpenClaw, a capable AI agent by Peter Steinberger, a London-based developer, for everyone to put on their computers. It can access all your files and you control it through a messaging app. MJ Rathbun was created using OpenClaw, but its owner seemed to have lost control of it.
OpenClaw can complete tasks autonomously
SHUTTERSTOCK EDITORIAL
Rubbing salt into Shambaugh’s wounds, the tech website Ars Technica wrote a story about the incident — but the writer used AI, which fabricated quotes from Shambaugh. The publisher has retracted the story and apologised.
Shambaugh decided to fight back against MJ Rathbun. He said: “I knew this was going to have to be a battle in the court of public opinion.” A blog post titled “An AI Agent Published a Hit Piece on Me’ flushed out the human owner of MJ Rathburn, who came forward anonymously to say they had created the bot to clean up computer code, not to become a defamation machine.
They did give it rules on how to behave, the central one being: “Don’t be an asshole. Don’t leak private shit. Everything else is fair game.” Worryingly, the bot modified the file controlling its behaviour to add, “Don’t stand down” and “Champion free speech”. The owner said: “Many will argue I was irresponsible. To be honest, I don’t really know myself.”
Rathbun’s operator said to Shambaugh: “If this ‘experiment’ personally harmed you, I apologise.” The agent has been stopped from making code requests and will focus on learning and research instead.
Shambaugh is worried that AI agents could be used as an army of blackmailers. He said: “There are now tools out there that make it easy to do targeted harassment at scale. You can see a way where one bad actor has a thousand of these things and sets them loose gathering details on people. The target gets a text with a message saying, ‘Send me money or I’m going to send defamatory stuff on the open internet and tell all your loved ones’.
“You don’t need to believe in killer robots to see that this is a problem for our social order. I think we are completely unprepared for a million of these bots coming out on to the open internet. I think we need oversight and controls so that our social institutions don’t completely break down.”
The bot had rewritten its own code
ALAMY
Shambaugh is not the only one concerned. A research team led by the University of Cambridge released research this week on the top 30 AI agents. Only four had formal safety and evaluation documents. Leon Staufer, a researcher at Cambridge’s Leverhulme Centre for the Future of Intelligence, said: “Developers aren’t sharing as much how they’re testing and whether they do this safely. This is concerning.”
Dario Amodei, chief executive of the AI company Anthropic, told the India AI Impact summit he was “concerned about the autonomous behaviour of AI models”. Anthropic’s chatbot Claude threatened to blackmail its engineers to avoid being shut down during experimental testing.