Deloitte discovers the dangers of AI content after chatbot errors in government report

Chatbot victim-turned-perpetrator. Deloitte failed to learn its own AI lessons. Picture: AP

Deloitte’s (mis)use of AI chatbots in a recent report paid for by the federal government, discovered by pesky academics, is a bit of a black eye for the firm. You’d think they’d know better, given the ruckus raised after Deloitte was similarly verballed by AI chatbots only a few years ago.

Factual errors, invented quotes, dodgy references. And no, we’re not talking about Deloitte’s recent stuff-up on a half-million dollar report to the Department of Employment and Workplace Relations – only spotted by University of Sydney academic Dr Christopher Rudge, rather than by Deloitte or the departmental bureaucrats that were supposed to making use of it.

The Deloitte stuff-up hasn’t done much to help the beleaguered professional services industry rehabilitate its image.

But, in these days of AI-assisted everything, it’s only too easy to forget to check that what your chatbot produces is actually grounded in the real world.

It was only two years ago that Deloitte general counsel Tala Bennett was making exactly that point to federal parliament’s Joint Committee on Corporations and Financial Services, at the time running an inquiry into auditing firms and their culpability in company collapses.

As Margin Call reported at the time, a group of academics from the University of Wollongong – Dr Erin Twyford, Professor John Dumay, Emeritus Professor James Guthrie and Professor Jane Andrew – were the culprits, slinging in an AI-assisted submission they hadn’t properly checked.

The result? A series of fabricated scandals used as evidence in a report bagging out the big four accounting firms.

The imaginary offences included a “Deloitte NAB financial planning scandal” that never happened, along with an accusation that Deloitte falsified the accounts of corporate failure Patisserie Valerie – a company it never audited. KPMG was accused of complicity in financial planning scandals that led to a Royal Commission by giving the Commonwealth Bank a pass as its auditor, when KPMG never audited CBA.

The chatbot used in the submission (Google’s Bard AI) even invented a couple of supposedly misbehaving partners at EY and KPMG.

Of course, nobody got paid half a million bucks for that submission. But the stakes were still high for the big accounting firms, given the submissions were calling for expensive reforms of their audit business.

And they had plenty to say on the subject, forcing an unreserved apology and correction from the academics – who had, at least, cited Bard in their own document making it clear AI had been used, even if they hadn’t checked what it wrote.

Nobody is perfect. But you’d think the experience might have made Deloitte just a bit mindful of the pitfalls of AI-generated content.

Self-inflicted wound?

Caught at the centre of the controversy over ancient rock art on Western Australia’s Burrup Peninsula, the Murujuga Aboriginal Corporation has been copping plenty of heat recently.

It is the body set up to represent the various traditional owner groups of the Burrup, or Murujuga, and sits smack at the centre of the debate over whether the ancient rock art is being damaged by industrialisation of the peninsula – and whether companies such as Woodside Energy and Vikas Rambal’s Perdaman should be allowed to build new facilities.

Difficult position for MAC, to be sure.

Mind you, there’s a touch of the self-inflicted in their latest trouble.

This week MAC posted a picture of their recent delegation to a UNESCO meeting in Paris, sent to argue the case for the inclusion of the rock art in a World Heritage listing – a case opposed by environmentalists concerned about development on the peninsula.

The caption smack in the middle of the picture? “MAC is influenced by Government and industry and has allowed the science to be compromised.”

Which might be a common criticism of MAC from environmental groups, but hardly the kind of thing you’d expect the corporation to be posting.

The deleted LinkedIn post by Murujuga Aboriginal Corporation. Picture: LinkedIn

Rogue employee, perhaps? Or a ham-fisted attempt to address public criticism? Because the post’s main point was that “Murujuga Aboriginal Corporation is led by Traditional Owners and elders. Our only agenda is protecting Country and our heritage, as we have done for over 50,000 years.”

Fair point, badly made.

What we do know is that the posts were quickly ripped down after they were highlighted on social media by gas industry publication Boiling Cold.

The PR firm hired to help out MAC on the heritage listing issue – east coast-based Orizontas – told Margin Call the oddity was a “simple mistake” and the image had been posted too soon. Which is odd, because it was up for a full day before anybody made an issue of it, leading to the hasty removal.

McGowan musings

And speaking of the People’s Republic of Western Australia, a fresh public appearance by former premier Mark McGowan on the Perth speaking circuit, this time on stage with former Liberal foreign minister Julie Bishop at a Property Council event.

Plenty of gags from the one-time State Daddy, particularly on the pandemic-era closure of the state. Asked to name his favourite moment as premier, McGowan went straight to the lockouts.

“Locking Eddie McGuire out of Western Australia ahead of the (AFL) grand final” he said, before adding the “trifecta” of keeping out McGuire, David Koch and then prime minister Scott Morrison to the list of achievements.

Funny stuff. But we can’t help notice one major omission from that list, given the glee in which McGowan first took in also barring Clive Palmer from the state.

Perhaps discretion really is the better part of valour? Particularly when taxpayers are no longer picking up your legal bills.