{"id":888880,"date":"2026-09-12T03:06:22","date_gmt":"2026-09-12T03:06:22","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/888880\/"},"modified":"2026-09-12T03:06:22","modified_gmt":"2026-09-12T03:06:22","slug":"dont-believe-the-bot-when-ai-confidently-gets-things-wrong","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/888880\/","title":{"rendered":"Don\u2019t believe the bot: When AI confidently gets things wrong"},"content":{"rendered":"<p>Opinion<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" alt=\"David Astle\" data-testid=\"author-avatar-image\" height=\"90\" src=\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2026\/09\/dcc46385aa1d8ba521d3a40468af1d32078f42df.png\"  width=\"90\" class=\"sc-9a01536c-0 libeSR\"\/><a href=\"https:\/\/www.theage.com.au\/by\/david-astle-p4yw3m\" rel=\"author nofollow noopener\" title=\"Articles by David Astle\" class=\"sc-3f16ee48-12 sc-6112b1a1-10 jyLmZI dZfFrL\" target=\"_blank\">David Astle<\/a>Crossword compiler and ABC Radio Melbourne presenter<\/p>\n<p>September 11, 2026 \u2014 4:00pm<\/p>\n<p class=\"sc-6112b1a1-15 llHEXf\">September 11, 2026 \u2014 4:00pm<\/p>\n<p>Save<\/p>\n<p class=\"sc-d1b14060-4 JmUoF\">You have reached your maximum number of saved items.<\/p>\n<p>Remove items from your <a href=\"https:\/\/www.theage.com.au\/goodfood\/saved\" class=\"sc-3f16ee48-12 sc-d1b14060-2 jyLmZI iQLtAb\" rel=\"nofollow noopener\" target=\"_blank\">saved list<\/a> to add more.<\/p>\n<p>AAA<\/p>\n<p>Care for a drop of Sahara Silk? Or maybe a splash of Oasis White? Each name is a would-be brand for camel milk, put forward by Google AI Overview. Not bad for a lactose-intolerant chatbot. The exercise arose from my talk with Paul Martin, the founder of Summer Land Camels, who\u2019s selling his milk to the world, presuming he can pick the right label. Though at this stage, with sales going well, Camel Milk nails it.<\/p>\n<p>Nonetheless, it was fun watching AI Overviews deliver such gems as Nomad Nectar and Simoom Smooth. Even Humpilicious had a flair unthinkable by earlier programs \u2013 machines are getting craftier. I set the parameters \u2013 \u201cgive me 20 names for camel milk\u201d \u2013 and awaited the results. While none astonished, each had a playful word-feel, that spooky knack we humans own to conjure crisp phrasing.<\/p>\n<p><img decoding=\"async\" alt=\"AI might be smart, but don\u2019t rely on it for help with the cryptic crossword.\" loading=\"lazy\" src=\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2026\/09\/0f352d89786a9b6b56417c54353c30f18c1fdcc1.jpeg\"  class=\"sc-d34e428-1 ldCIuB\"\/>AI might be smart, but don\u2019t rely on it for help with the cryptic crossword.Getty Images<\/p>\n<p>It stands to reason then that AI is up to cryptic thinking. If Overview and every other virtual brain can tackle camel milk, then surely decoding puns and anagrams is in reach. Tom Hawking explored that premise for Gizmodo, a tech-science website. Based in New York, Tom is an Australian journalist with a love of twisted clues.<\/p>\n<p>Mine in particular. Explaining why he picked five of my Friday clues from \u201cthe most perverse and human of puzzles, the cryptic\u201d, a chance to test three LLMs (Large Language Models): ChatGPT, Claude Sonnet 5 and Oreate AI. The toughest clue read: January 15, 2000? (9) The wordplay relies on Roman numerals, and our home calendar, pointing to MIDSUMMER (where MM \u2013 or 2000 \u2013 mirrors the date\u2019s seasonal position.) In short, the chatbots struggled. Claude imploded; Oreate entered a fugue state, while ChatGPT claimed the answer was HEWITT WON.<\/p>\n<p>\u201cA terrible answer,\u201d Tom wrote. \u201cHEWITT WON is two words for a start. Yet, ChatGPT was so confident of this wrong answer!\u201d That last remark echoed an experiment a friend did last week. A cryptic fan, Stuart McArthur challenged AI with his own clue: Bill Gates driving almost erratically (11) Claude replied with BILLIONAIRE, claiming \u201cthe clincher is the right letter tally, and it\u2019s an obviously witty description of Bill Gates himself.\u201d But Claude was wrong, as Stuart explained, showing how the anagram (mixing GATES + DRIVIN\/g) yields ADVERTISING, a synonym of bill.<\/p>\n<p>Related Article<a href=\"https:\/\/www.theage.com.au\/culture\/tv-and-radio\/tv-or-not-tv-our-favourite-two-letter-word-is-having-an-identity-crisis-20260831-p60t0h.html\" tabindex=\"-1\" class=\"sc-cba76dee-0 hdiTqm\" rel=\"nofollow noopener\" target=\"_blank\"><img decoding=\"async\" alt=\"TV or not TV...\" loading=\"lazy\" src=\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2026\/09\/47d14720df1520fb69ab3b26a819d10a22a02c3f.jpeg\"  class=\"sc-d34e428-1 ioInpc\"\/><\/a><\/p>\n<p>I know, a mate helping a chatbot to improve, throwing my puzzle career under the bus. Though Claude held fast to its wrongness, conceding, \u201cYes, your answer parses much better.\u201d Dear Claude, it parses better because it\u2019s right, unlike your billionaire which ignores the cryptic mechanics.<\/p>\n<p>Both tests, plus the camel milk brainstorm, tell us where we stand in this new era. If the \u201cthinking field\u201d is open, with no wrong answers, then AI can roam to fetch ideas, brand names, thought-starters. Unlike the two clue fails, where Tom and Stuart put Claude and other clankers through their paces. Both tests sought a unique answer, and every bot overlooked the map\u2019s X to find fool\u2019s gold instead.<\/p>\n<p>Far worse than fool\u2019s gold, a self-convinced eureka, the sort to delude the unsuspecting user, where an imaginary Lleyton Hewitt may win every match. Where the flat-earth disciple says gravity is phony because here is my truth. Trust me.<\/p>\n<p>Save<\/p>\n<p class=\"sc-d1b14060-4 JmUoF\">You have reached your maximum number of saved items.<\/p>\n<p>Remove items from your <a href=\"https:\/\/www.theage.com.au\/goodfood\/saved\" class=\"sc-3f16ee48-12 sc-d1b14060-2 jyLmZI iQLtAb\" rel=\"nofollow noopener\" target=\"_blank\">saved list<\/a> to add more.<\/p>\n<p><img decoding=\"async\" alt=\"David Astle\" data-testid=\"author-avatar-image\" height=\"40\" loading=\"lazy\" src=\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2026\/09\/1789182382_341_dcc46385aa1d8ba521d3a40468af1d32078f42df.png\"  width=\"40\" class=\"sc-9a01536c-0 libeSR\"\/><a class=\"sc-cba76dee-0 hdiTqm sc-b5b9fd03-2 jcGta-D\" href=\"https:\/\/www.theage.com.au\/by\/david-astle-p4yw3m\" rel=\"nofollow noopener\" target=\"_blank\">David Astle<\/a> is the crossword compiler and Wordplay columnist for The Sydney Morning Herald and The Age. He  is a broadcaster on ABC Radio Melbourne.From our partners<\/p>\n","protected":false},"excerpt":{"rendered":"Opinion David AstleCrossword compiler and ABC Radio Melbourne presenter September 11, 2026 \u2014 4:00pm September 11, 2026 \u2014&hellip;\n","protected":false},"author":2,"featured_media":888881,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":["post-888880","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-au","tag-australia","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/888880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=888880"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/888880\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/888881"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=888880"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=888880"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=888880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}