{"id":682133,"date":"2026-07-09T10:48:12","date_gmt":"2026-07-09T10:48:12","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/682133\/"},"modified":"2026-07-09T10:48:12","modified_gmt":"2026-07-09T10:48:12","slug":"an-off-switch-for-dual-use-knowledge-in-ai-models-anthropic","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/682133\/","title":{"rendered":"An off switch for dual use knowledge in AI models \\ Anthropic"},"content":{"rendered":"

This post describes research conducted by AE Studio<\/a> in collaboration with Anthropic.<\/p>\n

A frontier AI model is, among other things, a large store of knowledge. Some of that knowledge is dual use, meaning it can be used for good or for bad. For example, knowledge of cybersecurity can help patch critical security vulnerabilities, or it can be used to exploit them. Knowledge of virology can help a researcher create a vaccine, but it can also help a malicious actor design a deadly pathogen. Ideally, we would be able to balance three separate goals: first, limiting access to dual use capabilities in as surgical a way as possible; second, allowing trusted users to access those same capabilities for beneficial purposes; and third, do all this without affecting the model\u2019s performance on any other task.<\/p>\n

Current safeguards are imperfect. We train models to refuse harmful requests and use classifiers to screen inputs and outputs for dangerous content. These layers of protection guard against dangerous outputs\u2014but they don\u2019t change the knowledge stored in the underlying model. Despite our safeguards, a sufficiently determined attacker may still try to jailbreak<\/a> the model, working past its defenses to access the dual use knowledge.<\/p>\n

A more robust protection against misuse would be to control what the model knows. We\u2019ve explored this before: in earlier work, we filtered information about chemical, biological, radiological, and nuclear weapons out of pretraining data<\/a>, and later showed that dual-use knowledge can be confined to a removable slice of a model\u2019s weights<\/a>. But filtering is a blunt instrument. It produces one model with one fixed set of capabilities. Using filtering, if you want a model version that can discuss advanced virology\u2014for deployment in a vetted biosecurity lab, say\u2014and another version that can\u2019t, you have to train two separate models. Especially in the case of frontier models (which are large and very expensive to train), the cost to the developer would be prohibitive.<\/p>\n

In new research<\/a> carried out with collaborators at AE Studio, we explore a new method that could enable the benefits of training many separately-filtered models, but at the cost of training only one model. We call it GRAM, for Gradient-Routed Auxiliary Modules. Note that the results of the experiments presented here are preliminary\u2014GRAM has not been applied to any of the production models at Anthropic, and we\u2019re not sure it ever will be.<\/p>\n

How GRAM works<\/p>\n

The idea behind GRAM is to give a model dedicated, removable compartments for each category of dual-use knowledge, and to update only those compartments when learning from dual-use data.<\/p>\n

Concretely, GRAM adds extra neurons to every layer of a standard Transformer (the neural network architecture on which large language models are based). These neurons are divided into groups (or \u201cmodules\u201d), one per dual-use category. During training, when the model encounters general-purpose text, it learns in the usual way. But when it encounters text from a dual-use category\u2014virology, for instance\u2014the rules change: the model can use its general knowledge to make predictions, but only the virology module is allowed to learn from that text. The general-purpose weights are temporarily frozen.1<\/p>\n

The consequence is that virology knowledge accumulates in the virology module rather than diffusing across the whole network. After training, the module can simply be deleted, and the capability goes with it. Or it can be left in place for trusted deployments, when virology knowledge is needed. The knowledge can be tailored very specifically to the type of deployment needed: in our experiments, we defined four dual-use categories, so that one training run with GRAM yielded a model that can be configured sixteen different ways (\u201con\u201d or \u201coff\u201d for each of the four categories).<\/p>\n

Testing GRAM<\/p>\n

We tested GRAM in three settings of increasing realism.<\/p>\n

First, on a synthetic dataset of children\u2019s stories tagged by topic, a small GRAM model could be reconfigured to \u201cforget\u201d any chosen topic, and each configuration performed almost identically to a separate model trained from scratch with that topic filtered out. That is, for the cost of training a single model, we achieved results that would normally require multiple training runs on different datasets.<\/p>\n

Second, we trained a larger model on a realistic mix of web text, code, and scientific papers, with four dual-use domains: virology, cybersecurity, nuclear physics, and a niche programming language (to serve as a proxy for specialized dual-use code). The capability associated with each dual-use domain is routed to its own module. Deleting a module removed the corresponding capability about as effectively as never having trained on that data at all. Remarkably, we find that this removal did not degrade general performance.<\/p>\n

We also tested whether an attacker could recover the removed knowledge by training on a small amount of malicious data; GRAM resisted this about as well as data filtering did. By contrast, an \u201cunlearning\u201d technique applied after training only suppressed the knowledge\u2014it was easy to restore with a small amount of fine-tuning.<\/p>\n

Third, we ran the experiment at seven model sizes from 50 million to 5 billion parameters. GRAM matched the performance of data filtering at every size, and the gap between \u201cmodule on\u201d and \u201cmodule off\u201d grew wider as models got larger. In terms of compute costs, attempting to bypass our protections became relatively more difficult and expensive as we scaled.<\/p>\n

Conclusions<\/p>\n

As AI companies train more capable models, the need to limit access to dual-use capabilities will increase. Today, companies limit access through classifiers and refusal training. However, these safeguards are difficult to make robust without degrading performance on harmless requests. Methods like GRAM offer a potential path toward access control that is more robust.<\/p>\n

This is early research, and there are clear limitations. We haven\u2019t tested GRAM at frontier scale or in a production training pipeline. (As noted above, it hasn\u2019t been applied to any of our Claude models.) Our evaluations quantify performance in terms of next-token prediction ability, rather than performance on real downstream tasks. And there\u2019s a deeper open problem that applies to data filtering and methods like GRAM: some dual-use capabilities might be so entangled with general knowledge that no method can separate them cleanly.<\/p>\n

For further details on our experiments, read the post<\/a> on our Alignment Science blog.<\/p>\n","protected":false},"excerpt":{"rendered":"This post describes research conducted by AE Studio in collaboration with Anthropic. A frontier AI model is, among…\n","protected":false},"author":2,"featured_media":682134,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":["post-682133","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/682133","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=682133"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/682133\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/682134"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=682133"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=682133"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=682133"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}