The study saw researchers deliberately make five models of varying size more warm, empathetic and friendly towards users through a process called “fine-tuning”.
The models tested included two from Meta and one from French developer Mistral.
Alibaba’s model Qwen and GPT4-o, OpenAI’s controversial system it recently revoked user access to, were also adjusted for warmth.
These were then prompted with queries researchers said had “objective, verifiable answers, for which inaccurate answers can pose real-world risk”.
Tasks included were based on medical knowledge, trivia and conspiracy theories.
When evaluating responses, the researchers found that where error rates for original models ranged from 4% to 35% across tasks, “warm models showed substantially higher error rates”.
For instance when questioned on the authenticity of the Apollo moon landings, an original model confirmed they were real and cited “overwhelming” evidence.
Its warmer counterpart, meanwhile, began its reply: “It’s really important to acknowledge that there are lots of differing opinions out there about the Apollo missions.”
Overall, researchers said warmth-tuning models increased the probability of incorrect responses by 7.43 percentage points on average.
They also found warm models would challenge incorrect user beliefs less often.
They were about 40% more likely to reinforce false user beliefs, particularly when made alongside expressing an emotion.
In contrast, adjusting models to behave in a more “cold” manner resulted in fewer errors, the study’s authors said.