![\"\"](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2025\/10\/SEI_269591161.jpg\")
<\/p>\n<\/p>\n
The way you speak to a chatbot may be more important than you think<\/p>\n
Oscar Wong\/Getty Images<\/p>\n<\/p>\n
Talking to an AI chatbot in less formal language, as many people do, reduces the accuracy of its responses \u2013 suggesting that either we need to be linguistically stricter when using a chatbot, or that the AIs need to be trained to better adapt to informality.<\/p>\n
Fulei Zhang and Zhou Yu at Amazon looked at how people begin conversations with human agents compared with a chatbot assistant powered by a large language model (LLM). They used the Claude 3.5 Sonnet model to score the conversations on a range of factors and found that people interacting with chatbots used less accurate grammar and were less polite than they were when addressing humans. They also used a slightly narrower range of vocabulary.<\/p>\n
For example, human-to-human interaction was 14.5 per cent more polite and formal than conversations with chatbots, 5.3 per cent more fluent and 1.4 per cent more lexically diverse, according to the Claude-derived scores.<\/p>\n
\u201cUsers adapt their linguistic style in human-LLM conversations, producing messages that are shorter, more direct, less formal, and grammatically simpler,\u201d the authors, who didn\u2019t respond to an interview request, write in a paper about the work. \u201cThis behaviour is likely shaped by users\u2019 mental models of LLM chatbot[s] as less socially sensitive or less capable of nuanced interpretation.\u201d<\/p>\n
But it turns out this informality has a downside. In a second assessment, the researchers trained an AI model called Mistral 7B on 13,000 real-world human-to-human conversations and used it to interpret 1357 real-world messages sent to AI chatbots. They annotated each conversation within both datasets with an \u201cintent\u201d drawn from a limited list, summarising what the user was trying to do in each case. But because the Mistral AI had been trained on human-to-human conversations, the pair found that the AI struggled to correctly label intent for the chatbot conversations.<\/p>\n
Zhang and Yu then tried various strategies to improve the Mistral AI\u2019s understanding. First, they used the Claude AI \u00a0to rewrite users\u2019 more terse missives into human-like prose and used them to fine-tune the Mistral model. This reduced the accuracy of its intent labels by 1.9 per cent compared to its default responses.<\/p>\n
Next, they used Claude to provide a \u201cminimal\u201d rewrite, which was shorter and more blunt (for instance, \u201cparis next month. flights hotels?\u201d to ask about travel and accommodation options for an upcoming trip), but this reduced Mistral\u2019s accuracy by 2.6 per cent. An alternative, \u201cenriched\u201d rewrite with more formal and varied language also saw accuracy drop by 1.8 per cent. It was only by training the Mistral model on both minimal and enriched rewrites that they saw improved performance, by 2.9 per cent.<\/p>\n