Deloitte Australia will partially refund the AU$440,000 (NZ$499,000) paid by the Australian government for a report that was littered with apparent AI-generated errors, including a fabricated quote from a federal court judgment and references to nonexistent academic research papers.<\/p>\n
The financial services firm\u2019s report to the Department of Employment and Workplace Relations was originally published on the department’s website in July. <\/p>\n
A revised version was published on Friday after Chris Rudge, a Sydney University researcher of health and welfare law, said he alerted the media that the report was \u201cfull of fabricated references.\u201d<\/p>\n
Deloitte had reviewed the 237-page report and \u201cconfirmed some footnotes and references were incorrect,\u201d the department said in a statement Tuesday.<\/p>\n
The morning’s headlines in 90 seconds, including a new injunction in the Marokopa case, Chl\u00f6e Swarbrick slams a \u2018distraction\u2019, and California motorists save a woman trapped under a crashed chopper. (Source: 1News)<\/p>\n
\u201cDeloitte had agreed to repay the final instalment under its contract,\u201d the department said. The amount will be made public after the refund is reimbursed.<\/p>\n
Asked to comment on the report\u2019s inaccuracies, Deloitte told The Associated Press in a statement the \u201cmatter has been resolved directly with the client.\u201d<\/p>\n
Deloitte did not respond when asked if the errors were generated by AI.<\/p>\n
A tendency for generative AI systems to fabricate information is known as hallucination.<\/p>\n
The report reviewed departmental IT systems\u2019 use of automated penalties in Australia’s welfare system. The department said the \u201csubstance\u201d of the report had been maintained and there were no changes to its recommendations.<\/p>\n
The revised version included a disclosure that a generative AI language system, Azure OpenAI, was used in writing the report.<\/p>\n
Quotes attributed to a federal court judge were removed, as well as references to nonexistent reports attributed to law and software engineering experts.<\/p>\n
Rudge said he found up to 20 errors in the first version of the report.<\/p>\n
The first error that jumped out at him wrongly stated that Lisa Burton Crawford, a Sydney University professor of public and constitutional law, had written a nonexistent book with a title suggesting it was outside her field of expertise.<\/p>\n
\u201cI instantaneously knew it was either hallucinated by AI or the world\u2019s best kept secret because I\u2019d never heard of the book and it sounded preposterous,\u201d Rudge said.<\/p>\n
Work by his academic colleagues had been used as \u201ctokens of legitimacy,\u201d cited by the report\u2019s authors but not read, Rudge said, adding that he considered misquoting a judge was a more serious error in a report that was effectively an audit of the department\u2019s legal compliance.<\/p>\n
\u201cThey\u2019ve totally misquoted a court case then made up a quotation from a judge and I thought, well hang on: that\u2019s actually a bit bigger than academics\u2019 egos. <\/p>\n
“That\u2019s about misstating the law to the Australian government in a report that they rely on. So I thought it was important to stand up for diligence,\u201d Rudge said.<\/p>\n
Senator Barbara Pocock, the Australian Greens party\u2019s spokesperson on the public sector, said Deloitte should refund the entire AU$440,000.<\/p>\n
Deloitte \u201cmisused AI and used it very inappropriately: misquoted a judge, used references that are non-existent,” Pocock told the ABC. <\/p>\n
“I mean, the kinds of things that a first-year university student would be in deep trouble for.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"Deloitte Australia will partially refund the AU$440,000 (NZ$499,000) paid by the Australian government for a report that was…\n","protected":false},"author":2,"featured_media":64268,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[365,363,364,85,111,139,69,145],"class_list":{"0":"post-64267","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-australia","12":"tag-new-zealand","13":"tag-newzealand","14":"tag-nz","15":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/64267","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=64267"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/64267\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/64268"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=64267"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=64267"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=64267"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}