Facepalm: Deloitte Australia has agreed to refund part of its fee to the federal government after admitting that a $440,000 welfare compliance review contained fabricated citations and quotes generated by artificial intelligence. The admission followed a discovery by a University of Sydney academic who found multiple references to research that did not exist.
The incident underscores a growing debate over the use of AI systems in professional consultancy work, particularly when those tools generate references or conclusions without human verification. While Deloitte maintains that the policy recommendations remain valid, the incident has drawn attention to the transparency requirements associated with AI-assisted analysis in government contracts.
The Targeted Compliance Framework Assurance Review, completed in July and commissioned by the Department of Employment and Workplace Relations (DEWR), examined the government's automated penalty system for welfare violations.
Deloitte's work was intended to assess whether system code matched business requirements and policy compliance standards. It was published in August, but within weeks, academics flagged serious problems.
Chris Rudge, deputy director of health law at the University of Sydney, found citations to numerous nonexistent articles, including reports attributed to Lisa Burton Crawford, a law professor at the same institution who confirmed she had never authored the works in question. "It is concerning to see research attributed to me in this way," Crawford told the Australian Financial Review. "I would like to see an explanation from Deloitte as to how the citations were generated."
Buried on page 58 was a new disclosure that Deloitte's technical team had used "a generative AI large language model (Azure OpenAI GPT-4o) based tool chain" during its analysis.
The DEWR released an updated version of the 273-page review on Friday, noting it had made "a small number of corrections to references and footnotes." Buried on page 58 was a new disclosure that Deloitte's technical team had used "a generative AI large language model (Azure OpenAI GPT-4o) based tool chain" during its analysis.
The model was applied to determine whether the welfare system's software code could be mapped to compliance rules – an application that, in this case, produced inaccurate references.
The revised report also removed a fabricated judicial quote originally attributed to Federal Justice Jennifer Davies, whose name had been misspelled as "Davis" in Deloitte's initial publication. Of the 141 works listed in the original reference section, only 127 appear in the updated document. References to fictitious publications by Crawford and other academics were deleted without further explanation.
Deloitte told the AFR that it would return the final instalment of its government payment, though neither party clarified what percentage of the total fee this covered. A DEWR spokesperson said, "The substance of the independent review is retained, and there are no changes to the recommendations."
Rudge, however, questioned the integrity of those recommendations. "You cannot trust the recommendations when the very foundation of the report is built on a flawed, originally undisclosed, and non-expert methodology," he said. "Deloitte has admitted to using generative AI for a core analytical task; but it failed to disclose this in the first place."