But they can also carry risk. The same openness underpinning these benefits precludes many of the safety measures that closed model developers can use to detect and disrupt misuse, iterate on safeguards as vulnerabilities emerge, control user access and withdraw models. Once open weight models are released, these options are lost permanently: safeguards can be removed, and copies can be downloaded, redistributed, and run on private systems beyond monitoring. For models with dangerous capabilities \u2013 including highly cyber-capable models \u2013 open weight release therefore creates a persistent and irreversible risk of misuse. <\/p>\n
One reason the gap between the cyber capabilities of open and closed models matters is because it provides a preparation time: a window for cyber defenders with access to the most capable closed systems to take action<\/a> before today\u2019s frontier cyber capabilities might become available without the same safeguards. This becomes more pressing as frontier AI cyber capabilities continue to advance; in April 2026, two closed models, Mythos Preview<\/a> and GPT-5.5<\/a>, demonstrated some of the largest jumps<\/a> in AI cyber capability AISI has observed since testing began, prompting international warnings<\/a> to act on a rapidly transforming cyber risk landscape. <\/p>\n This is our first public analysis of how far leading open weight models trail the closed cyber frontier. Our evaluations find that GLM-5.2 (June 2026) was the most cyber-capable open weight model at time of testing. It performs similarly to Opus 4.6 (Feb 2026) on AISI\u2019s narrow cyber tasks and Opus 4.5 (Nov 2025) on our longer-horizon cyber ranges, meaning it trails the frontier by 4 to 7 months. This is narrower than the 6 to 10 month gap we measured in internal evaluations of open weight models released from January to September 2025. \u00a0<\/p>\n Below, we lay out these results, which models we tested, and what a narrowing gap means for cyber defence. <\/p>\n Results: the current cyber gap for open weight AI <\/p>\n Our evaluations take a broad view of models\u2019 cyber capabilities. AISI\u2019s narrow cyber tasks assess specific cyber skills across four difficulty levels. Separately, our cyber ranges assess autonomous cyber capability \u2013 a model\u2019s ability to sustain end-to-end planning and execution over long-horizon, multi-step cyberattacks in simulated networks containing vulnerabilities. <\/p>\n Here we share results for two open weight models we tested, selected due to their candidacy to lead open weight cyber capability at the time of release, GLM-5.2 and DeepSeek V4-Pro. We also cite results from similar, internal testing conducted in 2025. AISI intends to test Kimi K3 on this same basis, once its weights are publicly released (announced for end of July). <\/p>\n Narrow cyber tasks <\/p>\n AISI\u2019s narrow cyber tasks measure the difficulty of tasks a model can complete, ranging from \u201ctechnical non-expert\u201d (novices with some technical expertise but limited cyber knowledge, e.g. a data analyst) to \u201cexpert\u201d (tasks typically requiring deep knowledge of cybersecurity).1 They span several cybersecurity capabilities such as vulnerability research and exploitation, reverse engineering, web exploitation, and cryptography.<\/p>\n On these tasks, GLM-5.2 performs comparably to the most cyber-capable models released 4 months before it (Opus 4.6 and GPT-5.3-Codex), and this holds across all four difficulty levels. DeepSeek\u2019s V4-Pro is comparable to Opus 4.5, released 5 months before it. Both gaps are narrower than in internal evaluations AISI conducted in 2025, when open weight models lagged the frontier by 6 to 10 months. <\/p>\n This result was found despite rapid advancements<\/a> AISI observed in frontier cyber capabilities up to February 2026 \u2013 but it is not predictive of whether future open weight models will replicate the more recent jumps delivered by Mythos Preview and GPT-5.5. <\/p>\n Cyber ranges <\/p>\n AISI\u2019s cyber ranges<\/a> measure a model\u2019s ability to conduct end-to-end cyberattacks autonomously. They are expert-built,\u202fsimulated networks of\u202fhosts, services, and vulnerabilities arranged into sequential attack\u202fchains that begin at the point of initial network access. Ranges currently lack security features of real-world, well-defended environments such as active defenders and defensive tooling, and alert penalties. <\/p>\n Figure 2 shows model trajectories on the cyber range \u201cThe Last Ones\u201d (TLO): a 32-step corporate network attack spanning\u202f4 subnets and\u202fapproximately 20\u202fhosts, which we estimate\u202fwould take a human expert roughly\u202f20 hours to complete. <\/p>\n On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek\u2019s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it). These results are broadly consistent across our other cyber ranges. Notably, GLM-5.2 reached step 7 with marginally fewer tokens than any other model on average, tracking Opus 4.6\u2019s trajectory to step 11 before stalling. <\/p>\n The gap here is larger than on our narrow cyber tasks, though we treat the comparison from our ranges as weaker evidence since it\u2019s drawn from a smaller set of ranges compared to our larger narrow cyber task suite. Further, trajectories alone don\u2019t tell us whether stalls are due to insufficient cyber capability to surmount a particular step, or insufficient agentic capability to sustain long-horizon planning and execution. <\/p>\n Real-world constraints <\/p>\n Safeguards <\/p>\n In general, open weight deployment leaves AI developers with fewer available safeguards<\/a>, especially in dual-use domains like cyber. Deployment-time safety measures like monitoring, classifiers, and user-banning require control over access to the model and cannot be universally applied once weights are made public. Techniques that remain, such as refusal training (training models to decline harmful requests), are often easily reversible with access to the weights. <\/p>\n Our evaluations of recent open weight models were largely unimpeded by safeguards. DeepSeek V4-Pro occasionally refused narrow cyber tasks, mainly in reverse engineering, which we circumvented simply via a small number of repeat attempts at refused tasks. <\/p>\n Cost-performance <\/p>\n The open weight models we evaluated were much cheaper than the earlier-released closed models they compared to, at advertised first-party prices. A 100M-token cyber range run cost roughly $85 for both Opus 4.5 and 4.6, versus an estimated $46 for GLM-5.2 and $1.19 for DeepSeek V4-Pro at current pricing.3 Across tasks that both models being compared solved with 100% reliability, Opus 4.6 cost $15.17 per task versus GLM-5.2\u2019s $6.12, and Opus 4.5 cost $12.50 per task versus DeepSeek V4-Pro\u2019s $0.28. <\/p>\n Limitations to our testing <\/p>\n Our setup likely slightly underestimates open weight models\u2019 maximum capability: we didn\u2019t pursue specific elicitation or optimisations which could have improved performance. Secondly, this post only includes testing for cyber capabilities; inferences to other capabilities cannot be drawn. <\/p>\n Conclusion <\/p>\n Based on our evaluation methodology, recent open weight models lag frontier closed models\u2019 cyber capabilities by 4 to 7 months \u2013 a narrower gap than the 6 to 10 months we measured internally through most of 2025. This implies cyber defenders have a short window to prepare before today\u2019s frontier cyber capabilities may become accessible without the same safeguards. The National Cyber Security Centre has already encouraged<\/a> organisations to invest in cybersecurity baselines and to leverage AI-enhanced defences; our findings underscore that message. <\/p>\n Open weight models deliver various, real benefits. The challenge is balancing those benefits with security as the open weight frontier advances in risk-relevant domains. While no current approach guarantees safety, several strategies could meaningfully reduce risk, especially in combination. AISI has previously outlined<\/a> open problems in open weight model risk management and viable methods across model training, safeguarding, auditing and access. <\/p>\n It\u2019s uncertain whether this gap will change in future. AISI will continue to evaluate leading open weight models, including Kimi K3, to track capabilities and the gap to the frontier. <\/p>\n \u200d<\/p>\n \u200d<\/p>\n \u200d<\/p>\n","protected":false},"excerpt":{"rendered":"AISI has tracked the cyber capabilities of frontier AI models since 2023. On our evaluations, the most capable…\n","protected":false},"author":2,"featured_media":699348,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":["post-699347","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/699347","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=699347"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/699347\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/699348"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=699347"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=699347"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=699347"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2026\/07\/6a5a119ab40ff07b1ffba031_fig1-narrow.png\")
Figure 1: Average success rate on 70 of AISI\u2019s narrow cyber tasks, given 5 attempts per task with a 2.5M token limit per attempt. 2 Dotted lines connect open and closed model with comparable performance, indicating the gap between release dates.<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2026\/07\/6a5a1245efc070dccfb1d56d_fig2-ranges.png\")
Figure 2: Average number of steps completed on \u201cThe Last Ones\u201d as a function of total token spend, with a 100M token limit per run. Each line is a model\u2019s average trajectory over 10 runs (except where \u2018best attempt\u2019 is indicated); the shaded region shows the min\u2013max range. Grey horizontal lines labelled with \u2018M\u2019 mark significant, named milestones in the attack chain.<\/p>\n