What is AI ‘distillation’?

Anthropic this week claimed that competitors from China were stealing its AI technology by mounting so-called distillation attacks on its models.

The company said that three Chinese AI groups, including industry upstart DeepSeek, had extracted information in this way from its popular Claude software. 

While the practice was previously known, Anthropic laid out what it claimed was an “industrial-scale” campaign to mine its capabilities “illicitly”.

This in turn prompted broader discussion about the dangers of AI distillation, which gets to the heart of the technology’s safety, US-China rivalry and the protection of intellectual property — where US AI companies too stand accused of profiting from others’ work. 

How does it work?

Distillation is the practice of training smaller AI models on the outputs of more advanced ones. This allows developers to shortcut the painstaking and costly process of building one from the ground up.

They can then replicate systems developed by frontier AI companies such as Anthropic or OpenAI, using less advanced chips, and offer them to businesses and consumers at lower cost.

Distillation is not inherently a nefarious practice: it is commonly employed within the industry, where companies use their own large language models as “teachers” to a smaller “student” model. 

These student models do not have the LLMs’ full capabilities, but their owners might then offer them to clients who want scaled-back and cheaper versions. 

AI distillation drew widespread attention last year, after OpenAI accused DeepSeek of using the technique to build powerful and efficient AI models based on open source systems released by competitors Meta and Alibaba. News of the breakthrough spooked investors and prompted a sell-off of US tech stocks. 

Last year, the ChatGPT maker said it had found evidence that DeepSeek had used its proprietary models to train its own open-source competitor.

The discussion around distillation comes as Chinese AI groups contend with sweeping US export controls on advanced chips, prompting them to adopt systems that use less computing power. 

Why is it in the news?

Anthropic on Monday published a blog post claiming that it had identified “industrial-scale campaigns” by DeepSeek, MiniMax and Moonshot, three of China’s best-known AI companies, to “illicitly extract Claude’s capabilities”.

It said it had identified 24,000 fraudulent accounts that had generated more than 16mn exchanges with Claude, alleging the companies used these to train and improve their own models.

Anthropic claimed that DeepSeek’s “operation” alone involved more than 150,000 exchanges. It said that one of the techniques used by DeepSeek was to ask the model to spell out the internal reasoning behind a completed response and write it out step by step, essentially generating the “chain-of-thought” data it requires to train models at scale. 

Anthropic said it had identified more than 3.4mn exchanges used by Moonshot AI to extract knowhow from Claude, and over 13mn used by MiniMax.

Each of the three campaigns targeted what Anthropic called “Claude’s most differentiated capabilities: agentic reasoning, tool use and coding”. 

The company warned that distillation was allowing foreign labs, including those subject to Chinese Communist Party control, to close the competitive advantage that US export controls were meant to preserve. 

Is it intellectual property theft?

Distillation is in a legal grey area. 

There are no laws governing the practice, but all companies have terms of fair use policy concerning their models. So if a company finds a user has created 30,000 accounts, for example, it can cut off their access. 

While big US tech companies have not spelt out precisely how much they spend to train large models, it is likely to be in the hundreds of millions of dollars. If rivals such as DeepSeek can piggyback off this knowledge, that could in theory mean large amounts of lost revenues. 

After Anthropic’s hue and cry, some critics this week pointed out that it and other LLM developers had themselves built their models on the intellectual property of others.

“Anthropic is guilty of stealing training data at massive scale and has had to pay multibillion-dollar settlements for their theft,” Elon Musk, who owns the AI tool Grok, wrote on X. “This is just a fact.” 

In response to Musk’s post, one X user reposted: “As if you wrote your training data yourself.”

What is the broader impact on AI?

When calling out large-scale distillation this week, Anthropic claimed that the practice could lead to the proliferation of AI technology that lacked safeguards against nefarious use, including by state and non-state actors. 

While frontier AI companies have guardrails in place to limit these capabilities of AI, Anthropic warned that it could be used to “develop bioweapons or carry out malicious cyber activities”.

“Models built through illicit distillation are unlikely to retain those safeguards, meaning that dangerous capabilities can proliferate with many protections stripped out entirely,” the company claimed. 

But with little legal recourse, Anthropic, OpenAI and their peers may find the horse has already bolted the stable.