Thank you for your feedback.<\/p>\n
If the phenomenon continues, there is a significant risk of parliamentarians \u201cmaking decisions based on evidence that doesn\u2019t exist\u201d, says Christian Downie, a professor in the Australian National University\u2019s school of regulation and global governance.<\/p>\n
The situation is made more confusing by AI services including Google<\/a> search. Previously, looking up an invented reference might produce results that would indicate the reference did not exist. Now Google\u2019s AI summary will sometimes create summaries of a fake reference as though it were genuine.<\/p>\n In some instances, Google\u2019s AI summary and ChatGPT<\/a> will also cite the inquiry submission with the fake reference as a source, creating an ongoing cycle of misinformation.<\/p>\n One document submitted to an inquiry into family violence and suicide included a hallucinated reference attributed to Divna Haslam, a University of Queensland associate professor and clinical psychologist, and misstated her team\u2019s research findings.<\/p>\n Haslam, who researches child and family adversity and maltreatment, says the reference was \u201cscary\u201d because it looked so real that someone taking a cursory look could be convinced. Google\u2019s AI summary summarised the reference as if it were a real paper.<\/p>\n Sign up for the Breaking News Australia email<\/a><\/p>\n \u201cIt\u2019s very frustrating to \u2026 to have invested time, money, effort, expertise in rigorous research,\u201d she says. \u201cThen to see something that\u2019s inaccurate and inappropriately attributed anyway is really concerning.\u201d<\/p>\n The author of the submission, Drilldown Reports, told Guardian Australia it used AI in its research process and had identified the errors in its follow-up submission \u2013 but the deadline had passed to have them corrected.<\/p>\n \u201cWe do strongly believe in the human factor and that it should be a vital part of the full quality review process,\u201d a spokesperson said. \u201cUnfortunately, the human factor also lets us down when uploading the correct document.<\/p>\n \u201cI get that you feel you have a juicy article to add the AI fear machine but this was a human error, not AI.\u201d<\/p>\n The cycle of misinformation: AI search summaries cite AI errors, making factchecking difficult<\/a><\/p>\n The chair of the standing committee on social policy and legal affairs, which oversaw the inquiry, said all committees received a range of materials of varying quality and positions: \u201cIt is the job of the committee to both accept the evidence offered on face value but then also interrogate it through the inquiry process.\u201d<\/p>\n Haslam says not only does this phenomenon risk devaluing legitimate research but there are serious risks if misleading information makes its way into government policy. \u201cParticularly in a [domestic violence] space,\u201d she says. \u201cWe just can\u2019t risk that.\u201d<\/p>\n Dozens of submissions contained errors<\/p>\n The paper was just one of dozens the Guardian identified by building a computer program that extracted references from all inquiry submissions made to the current parliament and checked those references against several online academic databases.<\/p>\n Documents that were flagged as having a high proportion of references not matching anything were then manually checked.<\/p>\n Q&AHow we built an ‘AI detector’Show<\/p>\n All current commercial services for AI detection have one important limitation \u2013 they get it wrong sometimes. A \u201cfalse positive\u201d detection can lead people to be falsely accused of using AI services such as ChatGPT or Claude.<\/p>\n For this reason, we took a different approach. Analysis of reports produced with AI<\/a>\u00a0and studies on AI-generated journal articles<\/a>\u00a0showed that LLMs can often produce references that are incorrect in some way, from getting pages, titles and authors wrong to making up details entirely.<\/p>\n We built a custom program that extracts references from documents and then searches those references in CrossRef<\/a> (a database of academic papers and books) and Google Scholar. We also programmatically checked digital object identifiers (DOIs<\/a>) if they were present, to see if they resolved to a reachable URL.<\/p>\n Documents that had 20% or more references that were unable to be matched were then checked manually and a large number of submissions with incorrect references were checked with the original authors to verify our method.<\/p>\n It’s important to note that this method can only be used to point towards AI usage in documents with references and so will always be an underestimate of actual AI usage.<\/p>\n All links within documents were also checked for ChatGPT metadata tags. These are tags added by ChatGPT when it provides links to users and can persist when copied into a document. ChatGPT tags do not necessarily indicate that someone used ChatGPT directly, as they could be copying the text from a third-party article or similar.<\/p>\n Thank you for your feedback.<\/p>\n Using these methods, the Guardian found at least 39 submissions to political inquiries containing what appear to be hallucinated references.<\/p>\n These findings represent a conservative estimate of the amount of AI-generated material in political processes, as it only identifies submissions with incorrect citations. This approach would not find documents that used AI to generate text without any references, for example.<\/p>\n In another indication of how often LLMs are being used, more than 100 papers included ChatGPT url tags in reference links, which are automatically added by the platform in its results.<\/p>\n The submissions containing AI-generated material range from documents with a small number of incorrect citations to submissions in which every reference cited does not exist. They were authored by individuals and organisations across the political spectrum.<\/p>\n When Guardian Australia contacted the people and organisations responsible, many were unaware AI could get things wrong in this way or were aware of the potential for errors but had missed them before submitting the document.<\/p>\n skip past newsletter promotion<\/a>Free newsletter | When needed<\/p>\n Sign up to Breaking News Australia<\/p>\n Get the most important news as it breaks<\/p>\n after newsletter promotion<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2026\/08\/1788220332_983_2998.jpg\")
<\/p>\n