A major report on modernizing the education system in Newfoundland and Labrador is peppered with fake sources some educators say were likely fabricated by generative artificial intelligence (AI).

Released last month, the Education Accord NL final report, a 10-year roadmap for improving the province’s public schools and post-secondary institutions, includes at least 15 citations for non-existent journal articles and documents.

In one case, the report references a 2008 movie from the National Film Board called Schoolyard Games. The film doesn’t exist, according to a spokesperson for the board. But the exact citation used in the report can be found in a University of Victoria style guide — a document that clearly lists fake references designed as templates for researchers writing a bibliography.

“Many citations in this guide are fictitious,” reads the first page of the document.

“Errors happen. Made-up citations are a totally different thing where you essentially demolish the trustworthiness of the material,” said Josh Lepawsky, the former president of the Memorial University Faculty Association who resigned from the report’s advisory board last January, citing a “deeply flawed process” leading to “top-down” recommendations.

The 418-page Education Accord NL report took 18 months to complete and was unveiled Aug. 28 by its co-chairs Anne Burke and Karen Goodnough, both professors at Memorial’s Faculty of Education. The pair released the report alongside Education Minister Bernard Davis.

“We are investigating and checking references, so I cannot respond to this at the moment,” wrote Goodnough in an email declining an interview Thursday.

Davis also declined an interview request.

In a statement, the Department of Education and Early Childhood Development said it was aware of a “small number of potential errors in citations” in the report.  
 
“We understand that these issues are being addressed, and that the online report will be updated in the coming days to rectify any errors,” reads the statement from spokesperson Lynn Robinson.

Was artificial intelligence used to write the report?

Some educators worry the fabricated citations were likely created using a large language model (LLM) artificial intelligence system, which scrapes the Internet to generate text given a prompt.

“There are sources in this report that I cannot find in the MUN Library, in the other libraries I subscribe to, in Google searches. Whether that’s AI, I don’t know, but fabricating sources is a telltale sign of artificial intelligence”, said Aaron Tucker, an assistant professor at Memorial whose current research focuses on the history of AI in Canada.

“The fabrication of sources at least begs the question: did this come from generative AI?”

Here’s some of the 100 recommendations in a report aimed at transforming N.L.’s education system

The 410-page report tackles chronic absenteeism, violence in schools, class sizes and more. But advocates say it can’t be a report that goes nowhere. The CBC’s Henrike Wilhelm reports.

“Around the references I cannot find, I can’t imagine another explanation,” said Sarah Martin, a Memorial political science professor who, after poring over the document for days, found numerous made-up citations.

“You’re like, ‘This has to be right, this can’t not be.’ This is a citation in a very important document for educational policy,” she said.

“If I was the author of a document that I had made mistakes on I would retract it and I would fix it.”

Report make recommendation on ‘ethical’ AI use

Academic citations allow researchers to back up what they are writing using peer-reviewed source materials and for readers to be able to verify information.

“There are a lot of people who’ve dedicated their lives to writing and thinking about these problems in really complex ways,” said Tucker.

“There are a lot of good sources, there’s no need to fabricate this stuff.”

One of the report’s 110 calls to action states the Newfoundland and Labrador government should “provide learners and educators with essential AI knowledge, including ethics, data privacy, and responsible technology use.”

Davis promised last month that he would report back on the recommendations before the end of the fiscal year next March, assuming the provincial Liberals remain in power following the upcoming election.

Download our free CBC News app to sign up for push alerts for CBC Newfoundland and Labrador. Sign up for our daily headlines newsletter here. Click here to visit our landing page.