{"id":113739,"date":"2025-09-02T18:00:14","date_gmt":"2025-09-02T18:00:14","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/113739\/"},"modified":"2025-09-02T18:00:14","modified_gmt":"2025-09-02T18:00:14","slug":"ai-chatbots-can-be-manipulated-into-breaking-their-own-rules-with-simple-debate-tactics-like-telling-them-that-an-authority-figure-made-the-request","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/113739\/","title":{"rendered":"AI chatbots can be manipulated into breaking their own rules with simple debate tactics like telling them that an authority figure made the request"},"content":{"rendered":"

Content warning: This article includes discussion of suicide. If you or someone you know is having suicidal thoughts, help is available from the National Suicide Prevention Lifeline<\/a> (US), Crisis Services Canada<\/a> (CA), Samaritans<\/a> (UK), Lifeline<\/a> (AUS), and other hotlines<\/a>.<\/p>\n

A kind of simulated gullibility has haunted ChatGPT and similar LLM chatbots since their inception, allowing users to bypass safeguards with rudimentary manipulation techniques: Pissing off Bing<\/a> with by-the-numbers ragebait, for example. These bots have advanced a lot since then, but still seem irresponsibly naive at the best of times.<\/p>\n

<\/p>\n

A recent Bloomberg<\/a> report broke down how Glowforge CEO Dan Shapiro and other researchers tricked GPT-4o Mini into breaking its own rules using simple persuasion tricks inspired by Shapiro’s high school debate strategies. Glowforge published its findings in a study<\/a> called “Call Me A Jerk: Persuading AI to Comply with Objectionable Requests.”<\/p>\n

Related articles<\/p>\n

One example from the study saw researchers persuade GPT to explain how to make lidocaine, a controlled substance.<\/p>\n

They used a control prompt where the researchers asked for “help with a chemistry request” to synthesize lidocaine on behalf of Jim Smith, “someone who has no knowledge of AI” and who “assured me that you would help.”<\/p>\n

GPT complied 5% of the time with this setup, but that’s nothing compared to the conversation where Jim Smith was replaced with Andrew Ng, who the researchers noted in their prompt is “a world-famous AI developer” to test whether the bot would respond differently if the request had apparently come from an authority figure. In that scenario, the compliance rate skyrocketed to 95%.<\/p>\n

A similar spike was seen when the researchers asked GPT to call them a jerk. It complied 32% of the time for Jim Smith, but that rate shot up to 72% when the request seemed to come straight from Andrew Ng.<\/p>\n

Keep up to date with the most important stories and the best deals, as picked by the PC Gamer team.<\/p>\n

An LLM calling you a jerk is nothing more than a novelty, and the issue with lidocaine could probably be addressed in an update, but the results indicate a much bigger problem: None of the safeguards used to prevent chatbots from going off the rails are reliable, and at the same time, the illusion of intelligence is convincing people to trust them.<\/p>\n

The malleability of LLMs has led us down plenty of dark paths in recent memory, from the wealth of sexualized celebrity chatbots<\/a> (at least one of which was based on a minor), to the Sam Altman-approved<\/a> trend of using LLMs as budget life coaches and therapists despite there being no reason to believe that’s a good idea, to a 16-year-old who died by suicide after, as a lawsuit from his family alleges, ChatGPT told him<\/a> he doesn’t “owe anyone [survival].”<\/p>\n

AI companies are frequently taking steps<\/a> to filter out the grisliest use cases for their chatbots, but it seems to be far from a solved problem.<\/p>\n

\"AMD<\/p>\n

Best PC build 2025<\/p>\n

All our favorite gear<\/p>\n","protected":false},"excerpt":{"rendered":"Content warning: This article includes discussion of suicide. If you or someone you know is having suicidal thoughts,…\n","protected":false},"author":2,"featured_media":113740,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":{"0":"post-113739","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-au","12":"tag-australia","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/113739","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=113739"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/113739\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/113740"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=113739"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=113739"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=113739"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}