AI agent<\/a>, they can use RAMPART as a tool to simulate prompt injections.<\/p>\nIn an email, Siva Kumar said RAMPART allows red teamers to vary their simulated attacks on AI agent environments, as well as convert red-team scenarios into \u201crepeatable tests that engineers can run in continuous integration, pre-release validation, and ongoing safety regression testing.\u201d<\/p>\n
RAMPART also repeats the attack scenario to see if it can bypass any guardrails associated with an AI model. \u201cYou can take a prompt, you can mutate it, you can generate thousands of variations and then you can repeat those tests multiple times, because here\u2019s the thing: Your agent may deny you on the first time, may deny on the second time, but the third time it may pass, and attackers are going to do the same thing,\u201d Siva Kumar said.<\/p>\n
Give me some Clarity. The other new open-source tool, Clarity, is meant to run as either a web UI, a direct embed into a coding agent, or a desktop app. Its AI attempts to ask questions that those outside of the direct build pipeline (i.e., safety engineers, product managers, and others) would want answers to, according to Microsoft\u2019s announcement.<\/p>\n
Top insights for IT pros<\/p>\n
From cybersecurity and big data to cloud computing, IT Brew covers the latest trends shaping business tech in our 4x weekly newsletter, virtual events with industry experts, and digital guides.<\/p>\n
Siva Kumar wrote that while Clarity is able to help with software projects, it isn\u2019t focused on writing code.<\/p>\n
\u201cEven the best coding-focused LLMs can help you think about code, but the assumption is that you already know what you want to build,\u201d Siva Kumar said. \u201cClarity agent helps you think through what you\u2019re building by talking to you about the problem.\u201d<\/p>\n
Hatem Ayad, CTO at Clarvos and a Microsoft veteran, said that Clarity is different from other Big Tech companies\u2019 offerings because it offers an end-to-end framework for developers.<\/p>\n
\u201cMicrosoft is very good in putting together frameworks for big projects or big initiatives, and the AI agent tech workflow now is very much the standard every company is trying to use, so how to build agents, how to deploy agents, how to find security issues, and then how you patch agents,\u201d Ayad said.<\/p>\n
Why is Microsoft doing this? Ayad said that when Microsoft offers an open-source tool, they are trying to build a community around it, similar to the way that the company contributed to OpenTelemetry.<\/p>\n
\u201cMicrosoft\u2019s trying to become a reference architecture for AI safety, which is a gap right now,\u201d Ayad said. \u201cBut you have to be close to Microsoft\u2019s ecosystem to use RAMPART, so they are trying to build an edge or niche for themselves.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"Why wait until an AI system is done being built to test its vulnerabilities? Ram Shankar Siva Kumar,…\n","protected":false},"author":2,"featured_media":494426,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[61,60,80],"class_list":["post-494425","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology","tag-ie","tag-ireland","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/494425","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=494425"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/494425\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/494426"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=494425"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=494425"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=494425"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}