‘Worst of the worst’: Australia launches world-first crackdown on deepfake pornography

Australia is taking world-first steps towards forcing a reckoning with the “worst of the worst” content available online.

Public consultation has commenced on a slew of world-first industry standards laid out by Australia’s eSafety Commissioner that will force tech giants to do more to tackle “the worst-of-the-worst online content” like child sexual abuse material and “deepfake” AI-generated child pornography.

The draft standards, released to stakeholders on Monday and first reported by The Age, follow more than two years of work, and cover Designated Internet Services – apps, websites, and file and photo storage services – and Relevant Electronic Services – which covers a range of messaging and dating services, and gaming.

The “purpose”, eSafety Commissioner Julie Inman Grant told news.com.au, “is to establish a suite of enforceable obligations for two remaining online sectors without mandatory codes, to reduce the risk of child sexual abuse material and pro-terror content surfacing and being shared online”.

“The standards cover the worst of the worst online content, including deepfake child sexual abuse material, also known as ‘synthetic’ child sexual abuse material,” Ms Inman Grant, a former X (previously Twitter) executive, explained.

A suite of new industry standards will target ‘the worst-of-the-worst online content’. Picture: iStock

Australia’s eSafety Commissioner Julie Inman Grant. Picture: Jonathan Ng

eSafety received its first reports of deepfake – a portmanteau of ‘deep learning’ and ‘fake’ -child sexual abuse material in August, she added, and “[we] expect this number to grow as generative AI technology becomes more advanced and widely available”.

The altered media overlays an image or video onto an existing one, and has been used to enhance both the entertainment and gaming industries. It’s now increasingly being used, however, to create sexually explicit content, without the consent of those depicted.

While all major tech platforms have policies that ban child sexual abuse material from their public services, Ms Inman Grant said “enforceable commitments for these two sectors are important in the fight against the spread of [such] material”.

“We know these services are actively used to store and distribute this content,” she said.

Take, for example, X, which recently filed an application for a judicial review in the Federal Court after it was fined $610,500 by the eSafety Commissioner for allegedly failing to adequately tackle child exploitation material on its platform.

According to Ms Inman Grant, during the three months after the social media giant was bought by Elon Musk in October 2022, the proactive detection of child abuse fell from 90 to 75 per cent.

During the three months after Elon Musk acquired X, the proactive detection of child abuse fell from 90 to 75 per cent. Picture: Mandel Ngan/AFP

Public consultation on the draft standards will run for 31 days, with final versions to be tabled in federal parliament. They will come into effect six months after they’re registered.

Ms Inman Grant has encouraged “everyone to have their say, because we want these standards to be as robust as possible”.

“We know there’s no silver bullet to address the risk of child sexual abuse material, pro-terror material and other harmful material circulating on these services,” she said.

“It requires a mix of requirements, which is why the draft standards contain a suite of measures. These include requirements for effective user reporting processes; requirements to deter their services being used for such material; requirements for education and user empowerment; and the adoption of technology to prevent known, verified child sexual abuse material, including material created using generative AI, from being shared, stored, or communicated on the service.”

The draft standards come amid increased concern about the impact and circulation of deepfake pornography in Australian schools. Picture: iStock

The draft standards come amid increased concern about the impact and distribution of deepfake pornography in Australian schools, prompting experts to warn that “no child is safe”.

Melbourne-based AI expert, Anuska Bandara, previously told news.com.au that children and teenagers are particularly vulnerable to deepfake technologies – noting the “conversation has taken an unsettling turn”.

“This issue is poised to have far-reaching consequences for Australians, particularly children and teenagers who are increasingly vulnerable,” the Elegant Media founder said.

“The peril lies in the fact that the real individuals have no control over what deepfakes, created using advanced AI techniques, might communicate. Exploiting this technology, scammers are leveraging deepfakes to influence unsuspecting individuals, leading them into dangerous situations or even engaging in the distribution of explicit content.

“The ramifications of this misuse pose a significant threat to the wellbeing and safety of the younger generation as they navigate the online landscape.”

Image-based abuse is a breach of the Online Safety Act 2021, and under the Act, perpetrators can be issued a fine or imposed with jail time in some jurisdictions. Any Australian whose images or videos have been altered to appear sexualised and are published online without consent can contact eSafety for help to have them removed.