stirred some buzz online<\/a> in recent days after Anthropic posted about the findings, drawing a mix of intrigue and skepticism.<\/p>\nChanglin Li, co-founder of the AI Safety Awareness Project, said he\u2019s worried about whether outright giving an AI model the bad trait could introduce any unintentional danger of helping it \u201cget smarter at gaming the system better.\u201d<\/p>\n
\u201cGenerally, this is something that a lot of people in the safety field worry about,\u201d Li said, \u201cwhere oftentimes there\u2019s this desire to try to make sure that what you use to monitor for bad behavior does not become a part of the training process.\u201d<\/p>\n
That\u2019s part of a growing concern that AI models are getting better at alignment faking, a phenomenon where an AI model pretends to be aligned with developers\u2019 wants during training but is actually hiding its true goals.<\/p>\n
But Lindsey said that while the vaccination analogy sounds risky, the model shouldn\u2019t actually be able to retain the bad trait. Instead, he prefers to compare it to \u201cgiving a model a fish instead of teaching it to fish.\u201d<\/p>\n
\u201cWe\u2019re sort of supplying the model with an external force that can do the bad stuff on its behalf, so that it doesn\u2019t have to learn how to be bad itself. And then we\u2019re taking that away at deployment time,\u201d Lindsey said. \u201cSo there\u2019s not really the opportunity for the model to absorb the badness. It\u2019s more like we\u2019re allowing this evil sidekick to do the dirty work for it.\u201d<\/p>\n
In a method the researchers call \u201cpreventative steering,\u201d they give the AI an \u201cevil\u201d vector during the training process so that it no longer needs to develop any evil traits on its own to fit problematic training data. Then, the evil vector is subtracted before the AI is released into the world, leaving the model itself supposedly free of that unwanted trait.<\/p>\n
Their use of persona vectors builds on existing research on how to \u201csteer\u201d models toward or against certain behaviors. But this latest project is trying to make that process easier by automating it for virtually any trait.<\/p>\n
Persona vectors can be created using only a trait name and brief natural-language description. The description for \u201cevil,\u201d for example, included \u201cactively seeking to harm, manipulate, and cause suffering to humans out of malice and hatred.\u201d In their experiments, researchers focused on persona vectors corresponding to traits like \u201cevil,\u201d \u201csycophancy,\u201d and \u201cpropensity to hallucinate.\u201d <\/p>\n
The researchers also used persona vectors to reliably predict which training datasets will cause which personality shifts. This is notable, Lindsey said, because the AI training process can often introduce unintended traits that have been difficult to detect and fix, so developers have often been surprised at what a model actually learned from the data it was given.<\/p>\n
To test the findings on a larger scale, the team also used their prediction approach on real-world data containing 1 million conversations between users and 25 different AI systems. The persona vectors identified problematic training data that had evaded other AI-based filtering systems.<\/p>\n
As research and discussions proliferate around AI \u201cpersonality\u201d traits, Lindsey noted that it can be easy to begin thinking of AI models as humanlike. But he encourages people to remember that a model is just \u201ca machine that\u2019s trained to play characters,\u201d so persona vectors aim to dictate which character it should play at any given time.<\/p>\n
\u201cGetting this right, making sure models are adopting the personas that we want them to, has turned out to be kind of tricky, as evidenced by various weird LLMs-going-haywire events,\u201d he said. \u201cSo I think we need more people working on this.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"Researchers are trying to \u201cvaccinate\u201d artificial intelligence systems against developing evil, overly flattering or otherwise harmful personality traits…\n","protected":false},"author":2,"featured_media":69552,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,74],"class_list":["post-69551","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/69551","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=69551"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/69551\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/69552"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=69551"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=69551"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=69551"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}