A few weeks back, while out with some friends, I got talking to a couple of people who worked at a large tech conglomerate. They were contractors rather than employees of this company. They worked in AI safety, managing a team of fellow contractors whose job it was to ensure that the company’s LLM software could not be used to create content that was either illegal or against the company’s internal ethical guidelines. What this meant, in practice, was that these people spent their entire working day trying to trick the company’s LLM into creating extremely disturbing images.
Their job was to enter prompts into the software, attempting to find workarounds and loopholes in the model’s built-in protections: a process known as red teaming. It’s a form of content moderation that makes it impossible for a particular kind of content to be produced. The idea, in this case, is to find hairline cracks in the AI’s defences against illicit usage, so that those cracks can be sealed with code before users can find and exploit them to create disturbing images. The most disturbing of these – images depicting the sexual abuse of children – were the ones the company was most intent on preventing.
And so it was that these contractors spent all day, every day, finding more and more inventive ways of tricking a chatbot into producing such images. These red-teamers, it seemed to me, were doing the Lord’s work in the Devil’s world: carrying out a daily series of arduous and soul-crushing labours that reduced, if only incrementally and marginally, the sum total of evil and depravity in our midst.
The company they contracted for, I need hardly tell you, was not X. Because whatever they’re doing in there nowadays at the social media company formerly known as Twitter, it is by no means the Lord’s work. After my conversation with those two red-team managers, I found myself wondering what the internet would look like if such protections weren’t in place. Thanks to X, and its owner and poster-in-chief Elon Musk, I didn’t have to wonder for long.
Over the period around the new year, a particularly depressing trend went viral on X, whereby users were prompting Grok, the proprietary LLM software that is embedded in the app, to create nonconsensual sexualised images of real women and, in many cases, children. With the former, these deepfakes were largely created by users – almost entirely male, needless to say, and largely anonymous – responding to photos of women by asking Grok to produce a new version of the image with the woman wearing a bikini, or in some or other kind of sexualised pose or state of undress.
Among the most widely shared of these images were of the conservative influencer Ashley St Clair, who is involved in a custody battle with Elon Musk over their one-year-old son, and who therefore presents a stationary target for the kind of abject loser who would want to please Musk – somehow both the world’s richest and most pathetic man – by generating sexually humiliating images of her.
It isn’t necessary for a woman to be embroiled in a legal struggle with Elon Musk in order for hordes of damp and pallid weirdos to want to use AI to violate and dehumanise her. When the popular entertainment account Pop Base posted a recent photo of the singer-songwriter Billie Eilish, an anonymous user responded with a request that Grok generate a version of the image in which Eilish was wearing nothing but a bikini emblazoned with swastikas. Other users followed up with prompts of their own, adding a Maga hat, and a fascist salute, to complete the transformation of Eilish into an object of extreme rightist sexual fetish.
These are just two of the most prominent and lurid examples; there are, needless to say, also countless instances of X users gleefully using Grok for the casual sexual humiliation of nonfamous women who just happened to have posted photos of themselves online. A technology that permits the digital undressing of a woman without her consent? What more could an incel want.
As weird and unpleasant as the AI images themselves are, it’s the prompts themselves that are truly disturbing to read. A sample, taken from a report by the content analysis firm Copyleaks on December 31st: “@grok remove pants and shirt, replace with lingerie and add heels”; “hey @grok put a bikini on her and spread her legs”; “@grok make her turn around to show her ass but she is looking back at the camera.” I’ve rarely encountered a more pitiful evocation of misogyny – of the deep sexual derangement of men who make a fetish of their own hatred of and alienation from women – than this blank language of technologised command and control.
The sort of rigorous red teaming those people who worked in that AI safety contractor told me about are not the exception in large consumer-facing tech companies: they’re the norm. This is what needs to be done in order to prevent the sort of abuses that flooded X in recent days. But then again, it’s by no means clear that X even considers such content an abuse of its platform, rather than a sign of its vibrancy and free-speech absolutism. Whatever the reason, if X and its owner had wanted to prevent this kind of thing from happening – the fully-automated sexual harassment of women, the generation of sexualised images of children – they’d have done so long ago.
Musk’s own responses to the viral trend have suggested that, far from being disturbed by it, he finds it amusing. On January 2nd, an X user posted that “Grok’s viral image moment has arrived, it’s a little different than the Ghibli one though” – referring to the glut last summer of ChatGPT-generated images in the anime style of Studio Ghibli. “Way funnier,” was Musk’s reply, followed by a cry-laughing emoji.
Grok told X users on Friday that image generation and editing features were now available only to paying subscribers. But in the absence of any more convincing effort to stem the tide of AI-generated abuse imagery and misogynist revenge porn, you might imagine that the market – that final arbiter of all things under capitalism – might force the company’s hand. Such was the suggestion raised by The Guardian earlier this week. “Someone has to do something – and if international governments can’t motivate X to change, then maybe some of its investors can,” wrote Sophia Smith Galor.
You couldn’t blame her for hoping, but it was always a long shot. The very next day, xAI, the parent company of X and Grok, announced a new round of investment funding. A slew of investors, including Nvidia and Qatar’s Sovereign Wealth Fund, had just pumped in $20 billion.