Dozens of academic papers included fake AI references
Dozens of academic papers have fake references in them that have been made up by AI, according to new research from the University of Auckland.
The study looked at more than 3000 peer-reviewed papers published in the last few years and found 2 percent of them – or 69 papers – had fake references hallucinated by AI.
Some of the references had realistic author and journal names and publication details – but when the researchers went looking for them, they did not exist.
In a research paper, each reference is meant to be a promise, that if anybody wanted to check the evidence a claim is based on, they could. But what happens if that reference does not lead anywhere?
University of Auckland computer science lecturer Ulrich Speidel was one of the co-authors of a research paper which looked at that problem.
“I can remember going through one, where out of I think 29 references in the paper there wasn’t a single one that actually existed. When you look at them initially, they all look – pompous titles, real names for the articles, in some cases real journals.”
They were published papers in fields related to AI in education that had gone through the peer review process and been checked by other academics.
Speidel said they found that online academic database Google Scholar had also fallen victim to the problem.
“If enough papers have referenced a particular fake reference it’ll actually add that to Google Scholar and maked it look as if it’s a real paper. That was a bit of an eye opener.”
The “AI slop” was degrading the quality of science, Speidel said.
“The thing of course is ‘will you trust me as a scientist if that stuff that I read, if that stuff’s possibly made up?'”
In Australia, consulting firm Deloitte last year had to pay back $440,000 to the Australian government for a report which was found to have hallucinated references.
Deloitte later admitted it was written with the help of AI.
File photo.
123rf
Emeritus professor of engineering management at the University of York, Tony Ward, was a reviewer for one of the first academic papers to openly list AI as its lead author.
“I was quite adamant that we cannot accept that, because AI cannot be a true author, so I rejected it.”
The paper was then sent back to its human authors, who tried to make the case for it being accepted.
But Ward said he declined it a second time because the review panel was not able to correspond with the author – which was required by most journals and conferences.
However, AI could be a good tool for academics, he said.
“You can use it to summarise other papers, you can get it to help in the writing process. It can do data processing and we can even use it for reviewing papers for conferences or journals.”
AUT undergraduate engineering student Preston Williams agreed.
“You get stuck, you can’t ask a teacher when you’re studying right? You just ask AI, what is this XYZ? Half of my lecturers show me how to use AI anyway. I think you just can’t get reliant off it, because then you’re cooked.”
The authors of the University of Auckland article concluded that new AI checking tools could help with the problem, but stronger safeguards were needed as well as greater awareness of responsible AI use from researchers.