Geelong school students spread AI-generated sexually explicit images

Geelong schools have reached out for help after students created sexually explicit images using artificial intelligence, with growing fears the technology will become an “unstoppable force”.

Geelong students are using AI to create deepfake pornography.

Geelong schools have sought support after students created sexually explicit images using artificial intelligence (AI), as institutions are warned to get ahead of the technology before it becomes an “unstoppable force”.

Sexually-explicit “deepfakes” — AI-generated videos or recordings that mimic a real person’s face or voice to create a false scenario — have allegedly been shared by students on social media at a number of Victorian schools recently.

Police are investigating after about 50 female students at Bacchus Marsh grammar had their images allegedly used in explicit deepfakes generated by AI, which were then circulated online earlier this month.

In May, a 15-year-old student was expelled from Salesian College in Melbourne after he created graphic deepfake images of a teacher using an AI app.

Deakin School of Information Technology Honorary Associate Professor Damien Manuel and SASVic chief executive Kathleen Maltzahn.

It’s understood multiple Geelong schools have sought help for similar incidents, including students creating sexually-explicit “deepfakes” and image-based abuse using AI.

Victoria Police said local police were not aware of any instances where AI technology had been used to create sexually explicit images in Geelong schools.

Sexual Assault Services Victoria (SASVic) chief executive Kathleen Maltzahn said across the state, services were hearing more and more from schools struggling to deal with harmful sexual behaviours involving the use of AI.

“We’re (already) often seeing AI being used, and we have to get ahead of it before it becomes an unstoppable force,” Ms Maltzahn said.

Under proposed new laws, The Criminal Code Amendment (Deepfake Sexual Material) Bill 2024, Australians adults convicted of spreading AI-generated sexually explicit images or deepfakes would face up to six years in prison.

Deakin School of Information Technology Honorary Associate Professor Damien Manuel said the proposed legislation would aim to deter possible offenders, and prevent it becoming a societal norm, but clarification would be needed around how minors were charged.

“It will need to be coupled with education for kids,” he said.

Prof Manuel said this was particularly important considering the changing landscape of dating and sexual culture.

“Kids these days date by sending nude (pictures) to each other … in the past this was a criminal offence under the telecommunications laws, seen as using a carriage service to send pornography,” he said.

Now, he said, it was no longer an offence in Victoria as the practice had been normalised among teens.

Prof Manuel said social media organisations had a big part to play in preventing image-based abuse against minors.

“There’s a lot they could do, like (using AI) to detect nude and pornographic images and automatically remove them,” he said.

In 2023, there were more than 95,000 deepfake videos online, a 550 per cent increase since 2019, according to Home Security Heroes 2024.

AI Sensity reported in 2018 that 96 per cent of all deepfakes were non-consensual sexualised fake images of women.

A Department of Education spokesman said the Australian Framework for Generative Artificial Intelligence in Schools guided the safe and ethical use of AI in schools.

“The Department of Education will shortly be issuing new advice to schools to ensure that will indicate the steps schools must take before they implement any use of AI,” he said.

Originally published as Geelong school students spread AI-generated sexually explicit images