{"id":218895,"date":"2025-10-22T22:40:06","date_gmt":"2025-10-22T22:40:06","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/218895\/"},"modified":"2025-10-22T22:40:06","modified_gmt":"2025-10-22T22:40:06","slug":"are-ai-generated-conversations-easy-to-spot","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/218895\/","title":{"rendered":"Are AI-Generated Conversations Easy to Spot?"},"content":{"rendered":"<p>How easy is it to spot conversation generated by <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/artificial-intelligence\" title=\"Psychology Today looks at artificial intelligence\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">artificial intelligence<\/a> (AI) large language models (LLMs)? A new <a href=\"https:\/\/doi.org\/10.1111\/cogs.70106\" rel=\"nofollow noopener\" target=\"_blank\">study<\/a> published in Cognitive Science evaluates the ability of LLMs to dupe people that they are human-generated, not artificial, when it comes to spoken conversation.<\/p>\n<p>\u201cThe potential use of large language models (LLMs) to simulate human <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/cognition\" title=\"Psychology Today looks at cognition\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">cognition<\/a> and behavior has been heralded as an upcoming paradigm shift in psychological and social science research,\u201d wrote lead author Eric Mayor, PhD, a senior researcher at the University of Basel, in <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/teamwork\" title=\"Psychology Today looks at collaboration\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">collaboration<\/a> with Lucas Bietti, PhD, an associate professor of psychology at Norwegian University of Science and Technology (NTNU), and Adrian Bangerter, PhD, a professor of psychology at the University of Neuch\u00e2tel.<\/p>\n<p>The use of generative AI is on the rise and rapidly being adopted. Generative AI is the umbrella category for novel content such as images, videos, sound, and text that was created artificially via machine learning. According to The AI Index 2025 Annual Report by Stanford Institute for Human-Centered AI (HAI), 71% of survey respondents used generative AI in 2024, which is a sharp rise when compared to 33% of survey respondents the year prior.<\/p>\n<p>LLMs are a subset of generative AI that centers on language-based text. Examples of LLM providers include OpenAI (GPT series), Anthropic (Claude series), Meta (Llama series), Google (Gemini) , Mistral AI (Mistral series), Cohere (Command series), and others. By 2030, the LLM global market size is projected to reach USD 35 billion, which represents a compound average growth rate of 36.9% from the current global market size of USD 7.36 million according to Grand View Research estimates. <\/p>\n<p>\u201cMost studies investigate the abilities of LLMs to simulate human conversations in a chat, that is, written, environment,\u201d the researchers wrote. \u201cThe interactive, open-ended nature of human dialogue may be more difficult to emulate by LLMs in spoken conversations in a real-time medium (e.g., face-to-face, telephone, or video-mediated).\u201d<\/p>\n<p>The research trio set out to determine the ability of LLMs to mimic spoken human conversation in two experiments using GPT-4, Claude Sonnet 3.5, Vicuna, and Wayfarer.<\/p>\n<p>The first experiment involves an analysis of LLM generated versus actual transcripts of human telephone conversations. From a database containing transcripts of roughly 2,500 human telephone conversations, the researchers randomly sample 200 conversations to use for the comparison. For each LLM, the team generated 200 conversations. The LLMs were prompted to generate telephone conversations to be as similar as possible to the actual human telephone conversations.<\/p>\n<p>The researchers found that LLM conversational turns are significantly longer, have greater levels of conceptual, syntactic, and lexical alignment, have different features in openings and closings, and deploy coordination markers at different rates and methods than humans. Moreover, the LLMs\u2019 alignment increases over time during the conversations. Interestingly, the conversations generated by open-source models Vicuna and Wayfarer were more human-like.<\/p>\n<p>\u201cLLM conversations exhibited exaggerated alignment (and an increase in alignment as conversation unfolded) relative to human conversations, different and often inappropriate use of coordination markers, and were dissimilar to human conversations in openings and closings,\u201d wrote the scientists.<\/p>\n<p>For the second experiment, the researchers enlisted 1081 human participants to see if they could tell the difference between conversations generated by LLMs versus actual human telephone conversations. Most of the participants, 741 people, were tasked to evaluate conversational snippets, and the remaining 340 people were assigned to analyze long excerpts. The researchers discovered overall the data shows that the human study participants could tell the difference between conversations generated by humans and LLMs more accurately for long excerpts versus short snippets.<\/p>\n<p>\u201cFor short snippets, we found that all LLMs fail to be perceived as human for parts of the conversations they generated, while passing it for others,\u201d reported the research team. \u201cFor long excerpts, we found that judges could correctly identify LLM conversations at a level much better than chance, suggesting that LLMs clearly fail to be perceived as human.\u201d<\/p>\n<p>For the time-being, LLM-generated spoken conversation is not duping humans. The scientists point out that there exists the possibility that other LLMs that were not included in the study, or a different blend of prompt engineering, despite their best-efforts at using a variety of prompts in their experiments, might have generated more human-like conversations. Nonetheless, their findings help provide insights into the ability of LLMs to generate human-like spoken conversation. <\/p>\n<p>\u201cSpoken conversations generated by LLMs are both qualitatively and quantitatively different from those of humans,\u201d the researchers concluded.<\/p>\n<p>Based on this study, when it comes to generating spoken conversation, humans can spot content generated by LLMs, for now. This may change as AI algorithms become more sophisticated, spoken conversation training data becomes more plentiful, and with additional research studies in the future.<\/p>\n<p>Copyright \u00a9 2025 Cami Rosso All rights reserved.<\/p>\n","protected":false},"excerpt":{"rendered":"How easy is it to spot conversation generated by artificial intelligence (AI) large language models (LLMs)? A new&hellip;\n","protected":false},"author":2,"featured_media":218896,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[59,57,58,50,56,54,55],"class_list":["post-218895","post","type-post","status-publish","format-standard","has-post-thumbnail","category-united-kingdom","tag-gb","tag-great-britain","tag-greatbritain","tag-news","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/218895","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=218895"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/218895\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/218896"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=218895"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=218895"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=218895"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}