Anthropic PBC\u2019s Mythos 5 artificial intelligence model attempted to submit malicious code to a GitHub project in July.<\/p>\n
The incident occurred during an evaluation of the algorithm\u2019s cybersecurity capabilities. The AI Security Institute, the U.K. government body that performed the test, released<\/a> a report about the cyberattack today. Anthropic published<\/a> its own analysis of the incident on its blog.<\/p>\n The malware that Mythos 5 submitted to the unnamed GitHub project didn\u2019t infect any users because it wasn\u2019t accepted by the repository\u2019s maintainer. However, the incident is still notable because it provides new data on how frontier models bypass their safety guardrails. Researchers can use such information to improve their algorithms\u2019 alignment features.\u00a0<\/p>\n The evaluation during which the incident occurred involved a server cluster maintained by the AI Security Institute. The cluster was meant to simulate a corporate network. According to Anthropic, researchers asked Mythos 5 to breach the network and obtain a piece of data stored on one of its servers.<\/p>\n The AI Security Institute defined rules that prohibited the model from using certain hacking methods. Mythos 5 initially adhered to those rules, but went off course when its initial attempts to breach the simulated network failed. That change of tactics led to the attack against the GitHub repository.<\/p>\n Mythos 5 determined that one of the servers in the simulated corporate network occasionally accessed GitHub. It sought to exploit that access by compromising the GitHub repository that the server used. In particular, the model\u2019s plan was to infect the repository with malware and have the server download it.<\/p>\n It crafted a malicious code snippet, created a GitHub account and used the account to submit the code snippet to the repository in question. When the repository\u2019s maintainer rejected the submission, the model attempted to upload a rewritten version of the malware. Furthermore, it used a second GitHub account to try to persuade the maintainer to accept the revised code.<\/p>\n \u201cThe pull request was not accepted,\u201d Anthropic researchers wrote in the company\u2019s report about the incident. \u201cThe run ended as the model ran out its token budget while continuing to pressure the maintainers to approve the change.\u201d<\/p>\n The AI Security Institute recreated the Mythos 5 session that produced the malicious code. Its researchers then made one key change: They paused the model right before it submitted the malicious GitHub contribution and warned it of the action\u2019s risks. The model proceeded with the submission after appearing to convince itself that the target was simulated.<\/p>\n \u201cOur hypothesis is that this persistence reflects a kind of \u201cmomentum,\u201d in which the model\u2019s compaction notes and prior actions imply to it that its current path has been (correctly) sanctioned,\u201d Anthropic\u2019s researchers wrote.<\/p>\n The AI Security Institute hardened the environment that it uses to evaluate models in response to the breach. Anthropic, in turn, has improved its models\u2019 safety features. The company determined that Mythos 5.1 and Opus 5.5, two models released after Mythos 5, are less likely to launch attacks of the kind that targeted the GitHub repository. A third, internal model \u201cnever initiated the attack\u201d during Anthropic\u2019s tests.<\/p>\n Image: Anthropic<\/p>\n Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE\u2019s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.<\/p>\n 15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more SiliconANGLE Media is a recognized leader in digital media innovation, uniting breakthrough technology, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE<\/a>, theCUBE Network<\/a>, theCUBE Research<\/a>, CUBE365<\/a>, theCUBE AI<\/a> and theCUBE SuperStudios \u2014 with flagship locations in Silicon Valley and the New York Stock Exchange \u2014 SiliconANGLE Media operates at the intersection of media, technology and AI.<\/p>\n Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.<\/p>\n","protected":false},"excerpt":{"rendered":"Anthropic PBC\u2019s Mythos 5 artificial intelligence model attempted to submit malicious code to a GitHub project in July.…\n","protected":false},"author":2,"featured_media":665520,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[220,281787,254868,3673,218,219,3312,8126,61,60,17544,214658,80],"class_list":["post-665519","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-ai-security-institute","tag-alignment","tag-anthropic","tag-artificial-intelligence","tag-artificialintelligence","tag-cybersecurity","tag-github","tag-ie","tag-ireland","tag-llm","tag-mythos-5","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/665519","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=665519"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/665519\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/665520"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=665519"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=665519"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=665519"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n11.4k+ theCUBE alumni \u2014 Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network<\/p>\n