Alarm over spike in AI generated deepfake nudes in Victorian schools, calls for better education

Victorian educators fear more young lives will be “ruined” if dozens of harmful AI programs and apps that enable users to “undress” people in photos and “face swap” aren’t removed.

Victorian schoolchildren are using harmful artificial intelligence apps and websites to generate graphic images and videos of their classmates, with educators fearing more young lives will be “ruined” amid a proliferation of the technology.

Teachers across Victoria have sounded the alarm on AI being used to ‘undress’ people in photos and create ‘deepfake nudes’ in doctored images.

Deepfakes are digitally manipulated media used to replace one person’s likeness convincingly with that of another.

Leading experts have also called for AI literacy in classrooms, saying more education was needed on the issue.

The ability to create fake images of students is causing stress in Victorian schools. Picture: Supplied

It comes as police continue to investigate fake AI nude images, which used the faces of at least 50 female Bacchus Marsh Grammar pupils that were cruelly circulated online.

A male teenager was arrested and released over the incident.

A 15-year-old from Salesian College in Chadstone was expelled after he also created graphic images of a female teacher using an AI app last month.

A primary school teacher in Melbourne’s southeast told the Herald Sun he was “very concerned” about the misuse of AI in classrooms.

“Children in year six are accessing these websites and apps and as educators we are concerned because it feels like there has been a proliferation of them,” he said.

Another primary school educator said “stronger regulations” to prevent children and young teens from accessing these online tools were needed.

“I have seen so many children mucking around with AI apps and what concerns me is the sinister nature of these programs,” he said.

“More young lives could be ruined if the technology is inappropriately used.”

A number of easily available apps can produce nude deepfakes. Picture: Supplied

A Herald Sun analysis found dozens of AI face swap and body modification programs that allowed users to generate graphic photos in minutes.

Some websites were specifically designed for users to create “deepfake nudes”, “undress” people in photos and doctor a person’s face into pornographic content.

One website included the description “the best AI nudifier app that allows you to see anyone naked”.

Even more alarmingly, some mobile apps had age recommendations as young as four years old, with very few restricted to adult-use only.

In June, the federal government introduced legislation to ban the creation and sharing of deepfake pornography as part of measures to combat violence against women.

University of Melbourne Centre for AI and Digital Ethics co-director Professor Jeannie Paterson said young people needed to learn what deep fakes were.

“The Department of Education rolls out respectful relationship training … but I think they need to educate if you create or share intimate images without consent, it’s and crime,” Prof Paterson said.

“We need AI literacy. We’ve gone beyond the point where AI is affecting all aspects of our life, for better or worse. It’s just not deepfake nudes that are a problem.”

Fake nude images of at least 50 female students at Bacchus Marsh Grammar were circulated online. Picture: Supplied

Sexual Assault Services Victoria chief executive Kathleen Maltzahn said the Department of Education needed to be better resourced to respond to sexual violence in schools.

“It’s crucial that services like ours are funded because we can go into schools and work with boys who in this case are causing harm and work with the girls who are being impacted,” Ms Maltzahn said.

eSafety Commissioner Julie Inman Grant said image-based abuse, including the production of deepfaked images and videos, was a persistent online harm which also represents one of the most egregious invasions of privacy.

A Department of Education spokeswoman said the Australian Framework for Generative Artificial Intelligence in Schools provided a national structure for the safe and ethical use of AI in the classroom.

“More than 1950 … schools signed up to the Victorian Government’s Respectful Relationships initiative – supporting them to embed respect and gender equality across their entire school community,” she said.

An AFP spokesperson said federal police were aware of multiple platforms existing that enabled users to potentially create AI-generated child abuse material, adding the creation of such content was “a serious offence which attracts a maximum penalty of 15 years imprisonment”.

A Victoria Police spokesperson said under existing legislation artificially generated intimate images were treated the same as any other images of this nature.