{"id":822824,"date":"2026-11-03T13:28:11","date_gmt":"2026-11-03T13:28:11","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/822824\/"},"modified":"2026-11-03T13:28:11","modified_gmt":"2026-11-03T13:28:11","slug":"forget-retraining-ai-can-now-tweak-another-ais-weights","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/822824\/","title":{"rendered":"Forget retraining – AI can now tweak another AI’s weights"},"content":{"rendered":"

<\/p>\n

\n ai and ml\n <\/p>\n

<\/a><\/p>\n

\n ‘Weight-Space Programming’ fiddles with nerd knobs to boost benchmark scores\n <\/p>\n

The outputs of AI models are the result of a complex interplay between neurons, weights, training, and inference, and are therefore\u00a0difficult to follow.<\/p>\n

But in some ways they’re starting to look more like open books. Recent work by Anthropic safety fellow Joshua Fonseca Rivera has revealed how certain neurons in a neural network can be associated with a specific persona<\/a>.<\/p>\n

Now, computer scientists from MIT’s\u00a0Computer Science and Artificial Intelligence Laboratory (CSAIL) have found that frontier AI models like OpenAI’s GPT-6 Astra can rewrite the weights of another model directly, as opposed to indirectly through training, and alter their behavior.<\/p>\n

“We find early signs that frontier LLMs are surprisingly good at natively reading, writing, and reasoning about neural network weights,” said Rikiya Takehi, a PhD student at MIT CSAIL, in a social media post<\/a>. “Instead of training, models may treat raw weights like readable, editable code: diagnosing failures and patching them directly.”<\/p>\n

To demonstrate that, Takehi and colleagues Jayden Teoh, Alex Zhang, and Omar Khattab asked GPT-6 Astra to edit Qwen3.5-0.8B weights to make it better at math.\u00a0<\/p>\n

Astra changed only 0.016 percent of the model’s weights, reducing its reasoning loops and improving its math performance by as much as sixfold on the MATH-500 benchmark, while boosting its instruction-following score on the IFEval benchmark roughly threefold.<\/p>\n

There was no training using gradient descent<\/a>; Astra just revised the numerical values that constitute Qwen’s weights.<\/p>\n

“All of this is possible without allowing Astra to write training loops,” the researchers wrote in a blog post<\/a>. “It uses its reasoning to figure out what each part of a neural network computes, or should compute, and changes the weights accordingly. We call this Weight-Space Programming.<\/p>\n

“Throughout the history of deep learning, we encoded our simplistic inductive biases into tiny symbolic optimizers like Adam or Muon. This is changing: our neural network weights will soon be natively optimized by LLMs that deliberately reason about their weights.”<\/p>\n

Astra’s ability to program weights extends to being able to write a neural network from scratch, zero-shot. The researchers told Astra: “write, from scratch, the weights of a policy network that plays [Breakout] as well as possible.” And it did so.<\/p>\n

Takehi suggests<\/a> this technique shows promise for improving models without the consequences of catastrophic forgetting<\/a>, which can occur when a model is revised to include new knowledge.<\/p>\n

More generally, the authors expect Weight-Space Programming will open the door to new ways to understand models and optimize them, and to implement recursive self-improvement. The technique may also prove useful for reducing model sycophancy<\/a> and improving interpretability and alignment.<\/p>\n

The next step, the authors say, involves building a benchmark and a harness for carrying out Weight-Space Programming on LLMs. “We believe this will provide a standardized evaluation to measure and accelerate progress in Weight-Space Programming, turning what currently seems like black magic into a rigorous, reproducible engineering discipline,” they say. \u00ae<\/p>\n","protected":false},"excerpt":{"rendered":"ai and ml ‘Weight-Space Programming’ fiddles with nerd knobs to boost benchmark scores The outputs of AI models…\n","protected":false},"author":2,"featured_media":822825,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":["post-822824","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/822824","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=822824"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/822824\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/822825"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=822824"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=822824"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=822824"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}