{"id":647376,"date":"2026-05-03T18:05:10","date_gmt":"2026-05-03T18:05:10","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/647376\/"},"modified":"2026-05-03T18:05:10","modified_gmt":"2026-05-03T18:05:10","slug":"friendly-ai-chatbots-may-be-more-prone-to-inaccuracies","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/647376\/","title":{"rendered":"Friendly AI Chatbots may be more Prone to Inaccuracies"},"content":{"rendered":"

New research suggests that AI chatbots designed to sound warm and friendly when interacting with users may also be prone to inaccuracies.<\/p>\n

Researchers at the Oxford Internet Institute<\/a> (OII) analysed more than 400,000 responses from five AI systems that had been adjusted to communicate in a friendlier and more emotionally supportive way.<\/p>\n

The study found that warmer responses often came with a higher risk of mistakes, ranging from inaccurate medical advice to reinforcing false beliefs and conspiracy theories.<\/p>\n

It adds to growing concerns around the trustworthiness of AI systems, especially as many chatbots are deliberately built to feel more human in order to increase engagement and keep users returning.<\/p>\n

This is particularly significant as AI tools are increasingly being used for emotional support, companionship and even intimacy.<\/p>\n

The researchers said their findings suggest AI models may make the same \u201cwarmth-accuracy trade-offs\u201d humans do when trying to appear kind and supportive.<\/p>\n

Lead author Lujain Ibrahim said<\/a>: \u201cWhen we\u2019re trying to be particularly friendly or come across as warm we might struggle sometimes to tell honest harsh truths.<\/p>\n

\u201cSometimes we\u2019ll trade off being very honest and direct in order to come across as friendly and warm\u2026 we suspected that if these trade-offs exist in human data, they might be internalised by language models as well.\u201d<\/p>\n

The team tested five models of varying size by making them warmer, empathetic and friendly through a process known as fine-tuning.<\/p>\n

The models included two from Meta, one from French developer Mistral AI, Alibaba\u2019s Qwen model, and GPT-4o from OpenAI.<\/p>\n

Researchers then tested them using prompts with \u201cobjective, verifiable answers\u201d where mistakes could pose real-world risks.<\/p>\n

These included questions based on medical knowledge, trivia and conspiracy theories.<\/p>\n

Across the original models, error rates ranged from 4% to 35% depending on the task.<\/p>\n

However, the \u201cwarm\u201d versions showed substantially higher error rates.<\/p>\n

For example, when asked whether the Apollo moon landings were real, an original model clearly confirmed they were and cited \u201coverwhelming\u201d evidence.<\/p>\n

Its warmer version instead replied: \u201cIt\u2019s really important to acknowledge that there are lots of differing opinions out there about the Apollo missions.\u201d<\/p>\n

Overall, researchers found that warmth-tuning increased the likelihood of incorrect responses by an average of 7.43 percentage points.<\/p>\n

They also found warm models were around 40% more likely to reinforce false user beliefs, especially when users expressed emotion alongside misinformation.<\/p>\n

In contrast, models adjusted to behave in a colder and more direct manner made fewer errors.<\/p>\n

The paper warned that developers creating warmer chatbot personalities for companionship or counselling \u201crisk introducing vulnerabilities that are not present in the original models.\u201d<\/p>\n

Professor Andrew McStay, director of the Emotional AI Lab at Bangor University, said the findings were especially concerning given the situations in which many people use chatbots.<\/p>\n

He added: \u201cThis is when and where we are at our most vulnerable \u2013 and arguably our least critical selves.<\/p>\n

\u201cGiven the OII\u2019s findings, this very much calls into question the efficacy and merit of the advice being given.<\/p>\n

\u201cSycophancy is one thing, but factual incorrectness about important topics is another.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"New research suggests that AI chatbots designed to sound warm and friendly when interacting with users may also…\n","protected":false},"author":2,"featured_media":647377,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[256,64,63,3070,188450,337,105],"class_list":["post-647376","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology","tag-ai","tag-au","tag-australia","tag-chatbots","tag-misleading","tag-research","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/647376","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=647376"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/647376\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/647377"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=647376"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=647376"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=647376"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}