How artificial intelligence is fuelling the next wave of fraud that’s costing the unsuspecting billions

They take moments to make but fake receipts created on popular AI tools like ChatGPT are costing companies and the economy billions of dollars a year, and experts say the trend is escalating.

Slyp founders Paul Weingarth and Spiro Rokos. Picture: Max Mason-Hubers

You can now listen to The Australian's articles. Give us your feedback.

You can now listen to The Australian's articles.

Westpac bankers knew something was off this year when a big Australian business attempted to transfer $320m to what appeared to be a legitimate vendor.

In reality, it was a fraudster who had deceived the business into paying a hacked invoice with altered account details. If Westpac did not intervene it would have ranked as one of Australia’s biggest thefts.

And experts say the problem is only going to get worse thanks to the rise of generative artificial intelligence, costing companies and the national economy billions of dollars a year as fraudsters use AI “deep fakes” for anything from business expenses to annual tax returns.

In a few minutes, the author generated this photorealistic fake receipt.

Generative AI allows people to perform a variety of tasks, including creating fake, photorealistic invoices, via a few basic verbal prompts.

Within a few minutes on ChatGPT I created a receipt that appeared to be from the restaurant Rockpool in Sydney. All I had to do was write where I had my meal, what I ordered – I could even ask to make it crumpled and stained.

Deloitte’s Centre for Financial Services predicts that AI deep fakes could cost businesses $US40bn in America alone by 2027. That’s up from $US12.3bn in 2023, representing a compound annual growth rate of 32 per cent.

“Generative AI offers seemingly endless potential to magnify both the nature and the scope of fraud against financial institutions and their customers; it’s limited only by a criminal’s imagination,” Deloitte says.

“Further, the astounding pace of innovation will challenge banks’ efforts to stay ahead of fraudsters. This is because generative AI-enabled deepfakes incorporate a ‘self learning’ system that constantly checks and updates its ability to fool computer-based detection systems.”

A fake McDonald's receipt the author created in less than a minute on ChatGPT.

Underscoring the scale of the problem, Emburse – one of the world’s biggest travel and expense software providers – found last year that almost a quarter of employees admitted to making personal purchases and passing them off as business expenses, citing “revenge spending”.

Another study by SAP Concur found 42 per cent of “decision makers” in the UK public service admitted to having submitted a fraudulent expense claim, showing it was also a problem among senior leadership.

Paul Weingarth, co-founder of Australian smart receipt platform Slyp, said the archaic practice of using paper-based documentation was fuelling an increase in fraud.

“The receipt data is sitting in a point-of-sale system in a digital format. Then it’s getting passed into an analog paper format. Then it’s going back into the digital formal in the shape of a photo or some other OCR (optical character recognition) technology,” Mr Weingarth said.

“It’s incredibly archaic for customers but it also exposes (businesses) now to things like fraud.

“I mean, over the years, if you had Photoshop and some level of design skills you were able to doctor your own receipt yourself – so this is nothing new in terms of exposure risk.

“But it’s now democratised in an unhealthy way to the masses. Everyone has that (generative AI) tool at their fingertips and can generate a fake receipt in a digital format in a matter of seconds.”

The real Rockpool Bar & Grill in Sydney.

The main area of exposure is when people attempt to claim expenses they paid themselves rather than on company credit cards where sales data can be matched with a transaction.

Mr Weingarth said Slyp has spent years investing in technology to overcome this problem.

“Because you can just generate your own receipt and say, ‘hey, I spent this on my own card’, a business may have no access to those transactions, so they can’t actually authenticate or verify that the receipt matches up to the payment that was made.

“So we effectively built a platform that extrapolates that receipt level data in real time structures, that presents a standardised digital receipt inside the customer’s banking app that was sort of one of our cornerstone products.

“That’s one of the other benefits of not only taking a digital (approach) but having it embedded in an authenticated, secure location like your banking app.”

University of Technology Sydney senior lecturer of accounting Matthew Grosse said another potential countermeasure was the Content Provenance and Authenticity (C2PA) standard, which embeds AI-generated images with verifiable information about file origin.

“However, a major weakness remains, as users can remove metadata by taking a screenshot of an image,” he said.

“Businesses, tax authorities and individuals need to adapt quickly by implementing verification systems that go beyond simply looking at documentation.”

Dr Grosse was particularly concerned about fake receipts being used in tax returns, saying it could cost the national economy billions of dollars a year.

“Consider a marketing consultant earning $120,000, who uses an AI image generator to create several convincing receipts for non-existent expenses totalling $4000,” he said.

“At their marginal tax rate of 30 per cent, this fraud saves them about $1200 in taxes – if they are not caught.

“The Australian Taxation Office estimates a $2.7bn annual gap from incorrectly over-claimed deductions by small businesses. With digital forgery becoming more accessible, this gap could widen significantly.”

Originally published as How artificial intelligence is fuelling the next wave of fraud that’s costing the unsuspecting billions