‘Second guess everything you receive’: How AI is supercharging hacks and scams

More than half a billion dollars was lost to scams and hacks last year. See why it's set to balloon and what tactics scammers are using.

Fake voicemails with audio sounding exactly like a loved one could be the new frontier for money-hungry scammers.

The fraudsters are using highly-sophisticated artificial intelligence, known as “deep fake” technology, to create phony video or audio, which experts say is huge cause for alarm.

“You could get the voice of your (son or daughter) calling you, saying ‘I’m in the gas station, I have no money, can you send some money to this bank account so I can pay for fuel?’,” computer security company McAfee’s APAC general manager for product growth Tyler McGee said.

“It’s getting very difficult to not second-guess everything you receive.”

Half a billion dollars was lost to scams and hacks last year in Australia and this number is set to balloon as chat bots, deep fakes and other AI tools designed to alleviate workloads are being used to supercharge malware creation, impersonate trusted people and make swindles more difficult to detect.

OpenAI chief executive Sam Altman said even he was “a little bit scared” of AI’s potential.

“If I said I were not, you should either not trust me or be very unhappy I’m in this job,” the head of the company responsible for AI chat bot ChatGPT and image-generator Dall-E 2 said.

Mr McGee said AI was giving cyber criminals “the upper hand”.

McAfee’s Tyler McGee says it is “getting very difficult to not second-guess everything you receive”. Picture: Supplied

While poor spelling or grammar previously gave many scammers away, ChatGPT was helping them write cleaner copy.

The free AI chat bot launched last November and generates human-like responses to natural language prompts.

It has been used to produce content from blogs and marketing materials to speeches – and scams.

It can also write and fix computer code, allowing cyber criminals to create malware (malicious software) in minimal time.

“The skill set of most coders, who are in the cyber security or cyber-criminal space, can go from very limited to quite advanced,” Mr McGee said.

Mr McGee said “deep fake” technology was mostly used in entertainment, disinformation and pornography, but “we are starting to see fraud”.

He said scammers recently impersonated a CEO and asked employees over a Zoom call to redirect funds to an account that was siphoning money from the company.

Actor Miles Fisher transforms into Tom Cruise with the help of deep fake technology. Picture: Supplied

Meanwhile, there are new tools that allow users to replicate voices using only a small sound bite.

Mr McGee said this could take the “mum text” scam – in which attackers send a message pretending to be from the receiver’s child – “to the next level”.

He warned against setting personalised voice greetings for phone message banks.

“Just let the auto bot do it,” he said.

“The reality is, your voice is your asset, and I definitely wouldn’t recommend sharing it with anyone.”

CSIRO Data61 director Jon Whittle predicted scammers might also soon create fake chat bots to elicit sensitive information from people.

“Imagine a fake website masquerading as your bank and it has a chat bot,” Mr Whittle said.

“With every new technology that comes out, there is a new requirement to better educate the public about what can go wrong.

“You can use an AI tool to help you ban the other AI tools and this is what happens with cybersecurity more generally – we get into this arms race.”

CSIRO Data61 director Jon Whittle. Picture: Supplied

Mr Whittle, host of the Everyday AI podcast, said there was also a threat of companies inadvertently opening themselves up to hackers by using AI to create their software.

“You can generate code with security flaws,” he said.

“If you don’t know what you are doing, there could be a bug in that software and it could be exploited.”

Originally published as ‘Second guess everything you receive’: How AI is supercharging hacks and scams