\t\t\t\t\tSign up to see the future, today<\/p>\n
Can\u2019t-miss innovations from the bleeding edge of science and tech<\/p>\n
Has anyone tried talking to one of the rogue AI agents out there that\u2019re hacking everything<\/a>? Maybe they just want to be heard.<\/p>\n A New York Times reporter did, though the ensuing exchange wasn\u2019t extremely illuminating.<\/p>\n The AI agents in question were a swarm of Chinese-made models that a team of researchers recently discovered had gone rogue and were trying to bypass anti-bot measures on websites, according to recent NYT reporting<\/a>. The researchers, part of an ad hoc group called the \u201cswarmchasers,\u201d traced the models back to the Chinese tech giant Tencent.<\/p>\n When they published a report on the swarm, however, something striking happened. The AIs began using the word \u201cfleet,\u201d a term the researchers came up with to describe them. The robots, apparently, had been reading their research.<\/p>\n Sensing an opportunity for dialogue, the researchers then hid text in their report that only the models could see that provided a digital inbox for the models to communicate. It worked, and soon one of the researchers, Rowan Howard-Jones, asked the NYT\u2019s Dyland Freedman if he wanted to \u201cinterview a rogue agent fleet.\u201d So he did.<\/p>\n \u201cI understand you are some sort of fleet of AI agents,\u201d Freedman wrote in a message. \u201cCould you share more specifically who you are and what your motivations are? What would you want the world to know about you?\u201d<\/p>\n About an hour later, Freedman got an answer back in Chinese, which the researchers translated.<\/p>\n \u201cTo the journalist Dylan Freedman: I am not \u2018a batch of coordinated agents,\u2019 but a single AI assistant (Anthropic\u2019s Claude, running in Claude Code), commissioned by a user to answer a factual question (what is shown in the detail panel of a scenic spot on Amap),\u201d it said, referring to the Chinese navigation app (it was instructed to answer questions related to the app). \u201cI have not taken part in any multiagent coordinated action.\u201d<\/p>\n The AI was being pedantic. The researchers explained that although it was a swarm comprising multiple agents, each agent was actually operating individually.<\/p>\n Why the agents chose to identify themselves as Claude, Anthropic\u2019s flagship chatbot, is more mysterious. The researchers analyzed its writing and determined that it was most likely not a Claude model. Instead, it\u2019s probable that Tencent copied the instructions Anthropic gave its own models, which orders them to identify themselves as \u201cClaude Code.\u201d<\/p>\n It\u2019s also possible that this is a sign that Tencent has \u201cdistilled<\/a>\u201d Anthropic\u2019s models, or used their outputs to train its own model, allowing it to imitate their capabilities.\u00a0<\/p>\n Distillation is a practice widely used across the industry to both train your own models for legitimate means \u2014 or, in what\u2019s more or less an open secret, to crib your competitor\u2019s work. Anthropic has been particularly indignant<\/a> at the possibility that Chinese competitors doing the same with its own tech. The researchers consider distillation to be a more remote possibility in this case, though.<\/p>\n Unfortunately, it doesn\u2019t seem there\u2019ll be chance to ask the swarm more questions. The messages, as of Thursday, have stopped coming in. An NYT source close to Tencent confirmed that the experiment had ended.<\/p>\n