Revealed: Shocking number of Aussie kids targeted by nude deepfake apps

AI technology used to create fake nudes will be targeted in a new crackdown as the shocking number of visits to well-known ‘nudify’ sites is revealed.

Dangerous apps helping predators create fake nude images in seconds to harm children could be banned under an expansion of cyber safety laws.

Communications Minister Anika Wells will on Tuesday declare the federal government – after banning social media for under 16s – will move to ensure “nudify” apps and stalking tools are also restricted, preventing deepfake images from being created in the first place.

The harmful apps use artificial intelligence to digitally alter photos to remove clothes or have a subject perform sexual acts, and are being exploited to harm mainly women and children.

The latest eSafety research found three per cent of children reported having someone create a fake nude image of them without their permission — the equivalent of one kid in every Australian classroom.

Nine well-known nudify sites identified by eSafety had a combined 172,000 monthly visits from Australia, as of June.

More alarming figures from the eSafety office in June revealed the number of intimate, digitally-altered images – known as deepfakes – featuring under 18s has more than doubled in the last 18 months, compared to the seven years prior.

Two private Queensland schools were earlier this year embroiled in separate scandals involving students sharing sexual images on social media.

Any new ban or restrictions on the deepfake creation apps would bolster state and Commonwealth laws prohibiting stalking and the distribution of non-consensual and sexually explicit materials.

Like the incoming digital duty of care and laws restricting the use of social media for Australians under 16 after the Let Them Be Kids campaign, the onus will be on the tech companies to prevent the availability of these deepfake tools.

There is already a push in the UK for the government there to ban deepfake apps.

Ms Wells said there was a place for AI and legitimate tracking technology in Australia but there was no place for technology used solely to abuse, humiliate and harm people, especially children.

“The Albanese government will use every lever at our disposal to restrict access to nudification and undetectable stalking apps and keep Australians safer from the serious harms they cause,” she said.

“This is too important for us not to act.”

Ms Wells said abusive technologies were widely and easily accessible, and causing real and irreparable damage.

She said the federal government would work with industry to develop and deliver reforms.

“These new, evolving, technologies require a new, proactive, approach to harm prevention – and we’ll work closely with industry to achieve this,” she said.

“While this move won’t eliminate the problem of abusive technology in one fell swoop, alongside existing laws and our world-leading online safety reforms, it will make a real difference in protecting Australians.”

Emma Mason, the mother of schoolgirl Matilda “Tilly” Rosewarne, who took her own life after bullying including a fake nude image of her shared on Snapchat, said it was frightening to consider the speed in which people could access platforms to create such harmful content.

Ms Mason has been vocal in the Let Them Be Kids campaign to restrict the use of social media to under 16s and is now supporting the move to go after apps and sites generating the deepfakes.

“I don’t assume that the people creating those apps are going this is the way this app can then be manipulated and used, and therefore it has a sinister purpose,” she said.

“The reality is being aware now how it’s being used, the responsibility is on the owners and the creators of the app to do what they can to stop that being allowed.”

Taking action on nudification and undetectable stalking services was a recommendation in the report of the statutory review of the Online Safety Act 2021.

The government is considering further recommendations of the review.

Originally published as Revealed: Shocking number of Aussie kids targeted by nude deepfake apps