I wish I had nickel for every time ChatGPT, Claude, or Gemini told me I\u2019d hit the nail on the head, stumbled onto a genius idea, or otherwise patted me on the back for a half-formed idea or ill-conceived plan.<\/p>\n
Flattery and premature congratulations are common foibles of generative AI chatbots, with some models more susceptible to being \u201cyes-bots\u201d than others. But even as LLM providers have become aware of AI sycophancy and are training them to be more critical, it\u2019s still easy to get an AI to enthusiastically endorse a shaky theory that doesn\u2019t deserve it.<\/p>\n
Luckily, there\u2019s a style of prompting that can make even the most obsequious AI models stop in their tracks. This type of prompting goes by various names\u2014I\u2019ve heard it called \u201cfailure-first\u201d prompting<\/a> as well as \u201cinversion\u201d prompting<\/a>, and it\u2019s frequently used by coders looking to \u201cpressure-test\u201d the dubious suggestions<\/a> of an AI coding agent.<\/p>\n There are many different versions of it, but they all follow more or less the same formula: asking the AI to first consider possible points of failure before offering its solution, suggestion, or plan.<\/p>\n Here\u2019s one example from the \/r\/ChatGPTPromptGenius subreddit<\/a>:<\/p>\n Before answering, list what would break this fastest, where the logic is weakest, and what a skeptic would attack. Then give the corrected answer.<\/p>\n Here\u2019s another variation, proposed by a member of the University of Iowa\u2019s AI Support Team<\/a>:<\/p>\n Pretend you disagree with this recommendation. What is the strongest counterargument?<\/p>\n And here\u2019s yet another, as proposed by my own custom-built AI personal assistant:<\/p>\n Before providing your final recommendation, identify 3-5 specific ways your proposed solution could fail or where the logic is most likely to break. Act as a harsh skeptic or a \u201cRed Team\u201d auditor. Only after listing and explaining these failure modes should you provide the final solution, incorporating safeguards against those specific risks.<\/p>\n Interestingly, many of those who\u2019ve adopted \u201cpressure-testing\u201d or \u201cinverse prompting\u201d credit the mental models championed by investor Charlie Munger<\/a>, the longtime Berkshire Hathaway vice chairman and business partner of Warren Buffett.<\/p>\n