{"id":425711,"date":"2026-04-30T19:35:13","date_gmt":"2026-04-30T19:35:13","guid":{"rendered":"https:\/\/www.newsbeep.com\/ie\/425711\/"},"modified":"2026-04-30T19:35:13","modified_gmt":"2026-04-30T19:35:13","slug":"are-ai-chatbots-infiltrating-online-survey-data-not-yet-says-new-study-retraction-watch","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ie\/425711\/","title":{"rendered":"Are AI chatbots infiltrating online survey data? Not yet, says new study \u2013 Retraction Watch"},"content":{"rendered":"<p><a href=\"https:\/\/retractionwatch.com\/wp-content\/uploads\/2026\/04\/mohamed-nohassi-2iUrK025cec-unsplash-scaled.jpg\" rel=\"nofollow noopener\" target=\"_blank\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"631\" src=\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2026\/04\/mohamed-nohassi-2iUrK025cec-unsplash-1024x631.jpg\" alt=\"\" class=\"wp-image-134767\"  \/><\/a>Mohamed Nohassi\/Unsplash<\/p>\n<p>Despite concerns some have raised about potentially compromised data, AI chatbots aren\u2019t yet completing online research surveys widely, according to a new preprint.\u00a0<\/p>\n<p>The authors of the study, <a href=\"https:\/\/osf.io\/preprints\/psyarxiv\/pvdjr_v2\" rel=\"nofollow noopener\" target=\"_blank\">posted<\/a> earlier this month on PsyArXiv, found that fewer than 1% of around 4,800 survey responses collected by 12 different companies contained text that was likely not written by a human. Among 400 responses from a 13th company, however, around 16% were flagged for possibly being completed by a chatbot.\u00a0<\/p>\n<p>The study used a novel detection tool created by the survey research company Prolific, which funded the project.<\/p>\n<p>Since online data sampling began, online bots and faked responses have <a href=\"https:\/\/www.thetransmitter.org\/ethics\/scammers-threaten-quality-of-research-survey-data\/\" rel=\"nofollow noopener\" target=\"_blank\">been a thorn<\/a> in the side of researchers conducting analyses that rely on such data, with some evidence suggesting that fraudsters have become more sophisticated in their scamming methods in recent years. A <a href=\"https:\/\/www.pnas.org\/doi\/10.1073\/pnas.2518075122\" rel=\"nofollow noopener\" target=\"_blank\">2025 study in PNAS<\/a> found it was technically possible for AI chatbots to infiltrate online survey data.\u00a0<\/p>\n<p>In the new study, Prolific\u2019s authenticity checker correctly identified all the 125 surveys completed by AI chatbots and didn\u2019t misidentify any of the 124 surveys completed by humans, according to lead author <a href=\"https:\/\/scholar.google.com\/citations?user=ljiiovIAAAAJ&amp;hl=en\" rel=\"nofollow noopener\" target=\"_blank\">Andrew Gordon<\/a>, a staff researcher in behavioral science at Prolific in Dorset, England. \u201cThat gives us a lot of confidence in that measure,\u201d he said. The survey was completed 25 times each by ChatGPT, Gemini, Claude, Perplexity and an internal AI agent created by Prolific.\u00a0<\/p>\n<p>Gordon said he isn\u2019t critical of the PNAS study that found <a href=\"https:\/\/www.nature.com\/articles\/d41586-026-00221-8\" rel=\"nofollow noopener\" target=\"_blank\">AI chatbots could infiltrate<\/a> online surveys. But he deemed the <a href=\"https:\/\/www.theguardian.com\/technology\/2026\/mar\/28\/how-fraudulent-church-data-revealed-ais-threat-to-polling\" rel=\"nofollow noopener\" target=\"_blank\">media attention<\/a> it received a \u201cbit hyperbolic,\u201d saying it \u201cfueled a perception that is extremely divorced from the reality on the ground.\u201d<\/p>\n<p><a href=\"https:\/\/faculty-directory.dartmouth.edu\/sean-j-westwood\" rel=\"nofollow noopener\" target=\"_blank\">Sean Westwood<\/a>, the author of the PNAS study and a political scientist at Dartmouth College in Hanover, New Hampshire, told us he was approached to collaborate on Gordon\u2019s study but declined. He noted Prolific\u2019s financial stake in the outcome of the study, and said it\u2019s not possible to verify the study\u2019s claims because the firm has not made its authenticity checker available to external researchers.<\/p>\n<p>The Prolific study employed survey participants with high approval ratings \u2014 people who had previously provided replies the company deemed reliable \u2014 Gordon said. Two of the authors, including Gordon, are employees of Prolific, a third is a past adviser to the firm, and the remainder are academics who told Retraction Watch they were not paid by the firm for their contributions.<\/p>\n<p><a href=\"https:\/\/galt.ucdavis.edu\/people\/natalia-pinzon-jimenez\" rel=\"nofollow noopener\" target=\"_blank\">Natalia Pinz\u00f3n<\/a>, an agroecologist and geographer at the University of California, Davis, who has <a href=\"https:\/\/www.frontiersin.org\/journals\/research-metrics-and-analytics\/articles\/10.3389\/frma.2024.1432774\/full\" rel=\"nofollow noopener\" target=\"_blank\">studied the integrity of online survey data<\/a>, called the new analysis \u201crigorous\u201d and \u201cwell done.\u201d\u00a0<\/p>\n<p>Pinz\u00f3n, who was not involved with the research, said it was important to draw attention to the various ways bad data can be introduced, including when humans provide fraudulent responses, sometimes by using large language models (LLMs). \u201cI think that the challenge is broader in terms of coordinated human fraud and sophisticated invalid responses that erode the quality of the survey,\u201d she said.<\/p>\n<p>The only troubling standout in the study proved to be Amazon Mechanical Turk (MTurk), whose data appeared to have been completed by chatbots 16% of the time. Gordon said MTurk has a <a href=\"https:\/\/www.prolific.com\/resources\/stop-using-mturk-for-research\" rel=\"nofollow noopener\" target=\"_blank\">history of \u201cbot problems.\u201d<\/a> He added: \u201cIt\u2019s kind of like the wild west of online sampling.\u201d<\/p>\n<p>\u201cOn nearly all the platforms that people are actually using, there\u2019s very little evidence of AI agents taking part,\u201d Gordon said. What\u2019s more, the responses that were flagged for possible AI activity were \u201clow consistency, low comprehension, low honesty,\u201d Gordon said.\u00a0<\/p>\n<p>The results indicate the non-human responses probably stem from traditional scripted survey bots, which have existed for decades, and not new AI chatbots like ChatGPT and its ilk. \u201cSince the early days of online sampling, you\u2019ve had these very hacky scripted bots that can take surveys, and the data is rubbish,\u201d Gordon said. \u201cThey\u2019re very easy to identify.\u201d<\/p>\n<p>But to be sure they weren\u2019t mistaken, Gordon and colleagues ran three AI chatbots on the survey to benchmark how well a chatbot would handle the questions. \u201cThey actually outperformed humans on nearly all measures, and they drastically outperformed the responses that got flagged by our detection,\u201d Gordon said.\u00a0<\/p>\n<p>\u201cHuman data quality is the biggest problem in online research,\u201d Gordon said. \u201cIt\u2019s not AI agents.\u201d\u00a0<\/p>\n<p>Pinz\u00f3n said her research also suggests human data quality and survey fraud remains a bigger problem in online survey data than AI chatbots. That\u2019s probably because humans are incentivized and motivated to take part in surveys even when they don\u2019t meet the criteria for inclusion, she said.\u00a0<\/p>\n<p>But the possibility that humans are introducing bad data doesn\u2019t mean we should abandon online survey data and revert back to only in-person interviews, which typically are less diverse and generalizable than online counterparts, according to Gordon. \u201cI think that would be extremely regressive,\u201d he said.<\/p>\n<p>As for AI chatbots infiltrating online data, Gordon said it\u2019s possible the picture could change quickly.\u00a0<\/p>\n<p>\u201cWhen I started at Prolific I could relatively confidently predict what was going to happen in the next year in terms of online sampling,\u201d he said. \u201cI wouldn\u2019t feel that confident predicting a couple of months in advance right now, just because things are changing so quickly.\u201d<\/p>\n<p>Like Retraction Watch? You can make a\u00a0<a href=\"http:\/\/paypal.com\/us\/fundraiser\/charity\/1909130\" rel=\"nofollow noopener\" target=\"_blank\">tax-deductible contribution to support our work<\/a>, follow us\u00a0<a href=\"http:\/\/twitter.com\/RetractionWatch\" rel=\"nofollow noopener\" target=\"_blank\">on X<\/a> or <a href=\"https:\/\/bsky.app\/profile\/retractionwatch.com\" rel=\"nofollow noopener\" target=\"_blank\">Bluesky<\/a>, like us\u00a0<a href=\"https:\/\/www.facebook.com\/pages\/Retraction-Watch\/119209094864356\" rel=\"nofollow noopener\" target=\"_blank\">on Facebook<\/a>, follow us <a href=\"https:\/\/www.linkedin.com\/company\/retractionwatch\/\" rel=\"nofollow noopener\" target=\"_blank\">on LinkedIn<\/a>, add us to your\u00a0<a href=\"https:\/\/retractionwatch.com\/feed\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">RSS reader<\/a>, or subscribe to our\u00a0<a href=\"http:\/\/eepurl.com\/bNRlUn\" rel=\"nofollow noopener\" target=\"_blank\">daily digest<\/a>. If you find a retraction that\u2019s\u00a0<a href=\"http:\/\/retractiondatabase.org\/RetractionSearch.aspx\" rel=\"nofollow noopener\" target=\"_blank\">not in our database<\/a>, you can\u00a0<a href=\"https:\/\/docs.google.com\/forms\/d\/e\/1FAIpQLSeAsw4i5J8M7sOQ9GiG0_dglkim9gdPPba92yZRLfCq4u-o7w\/viewform?c=0&amp;w=1\" rel=\"nofollow noopener\" target=\"_blank\">let us know here<\/a>. For comments or feedback, email us at\u00a0<a href=\"https:\/\/retractionwatch.com\/cdn-cgi\/l\/email-protection\" class=\"__cf_email__\" data-cfemail=\"423627232f02302736302321362b2d2c352336212a6c212d2f\" rel=\"nofollow noopener\" target=\"_blank\">[email\u00a0protected]<\/a>.<\/p>\n<p>Like Retraction Watch? You can make a\u00a0<a href=\"http:\/\/paypal.com\/us\/fundraiser\/charity\/1909130\" rel=\"nofollow noopener\" target=\"_blank\">tax-deductible contribution to support our work<\/a>, follow us\u00a0<a href=\"http:\/\/twitter.com\/RetractionWatch\" rel=\"nofollow noopener\" target=\"_blank\">on X<\/a> or <a href=\"https:\/\/bsky.app\/profile\/retractionwatch.com\" rel=\"nofollow noopener\" target=\"_blank\">Bluesky<\/a>, like us\u00a0<a href=\"https:\/\/www.facebook.com\/pages\/Retraction-Watch\/119209094864356\" rel=\"nofollow noopener\" target=\"_blank\">on Facebook<\/a>, follow us <a href=\"https:\/\/www.linkedin.com\/company\/retractionwatch\/\" rel=\"nofollow noopener\" target=\"_blank\">on LinkedIn<\/a>, add us to your\u00a0<a href=\"https:\/\/retractionwatch.com\/feed\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">RSS reader<\/a>, or subscribe to our\u00a0<a href=\"http:\/\/eepurl.com\/bNRlUn\" rel=\"nofollow noopener\" target=\"_blank\">daily digest<\/a>. If you find a retraction that\u2019s\u00a0<a href=\"http:\/\/retractiondatabase.org\/RetractionSearch.aspx\" rel=\"nofollow noopener\" target=\"_blank\">not in our database<\/a>, you can\u00a0<a href=\"https:\/\/docs.google.com\/forms\/d\/e\/1FAIpQLSeAsw4i5J8M7sOQ9GiG0_dglkim9gdPPba92yZRLfCq4u-o7w\/viewform?c=0&amp;w=1\" rel=\"nofollow noopener\" target=\"_blank\">let us know here<\/a>. For comments or feedback, email us at\u00a0<a href=\"https:\/\/retractionwatch.com\/cdn-cgi\/l\/email-protection\" class=\"__cf_email__\" data-cfemail=\"abdfcecac6ebd9cedfd9cac8dfc2c4c5dccadfc8c385c8c4c6\" rel=\"nofollow noopener\" target=\"_blank\">[email\u00a0protected]<\/a>.<\/p>\n<p>\n\tRelated\n<\/p>\n<p>\t<script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n","protected":false},"excerpt":{"rendered":"Mohamed Nohassi\/Unsplash Despite concerns some have raised about potentially compromised data, AI chatbots aren\u2019t yet completing online research&hellip;\n","protected":false},"author":2,"featured_media":425712,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[61,60,80],"class_list":["post-425711","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology","tag-ie","tag-ireland","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/425711","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=425711"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/425711\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/425712"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=425711"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=425711"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=425711"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}