Gold Coast private school centre of police investigation into student deepfake porn drama

Police are investigating after a Gold Coast private school warned parents of an online extortion attempt using AI to create fake nude images of a student.

Here’s a wish for Santa … can we please enforce the under-16s social media ban ASAP?

Just days into the summer holidays, one Gold Coast private school issued a disturbing warning to parents, informing them of an attempted online extortion using AI to create fake nude images of students.

The head of middle school told parents a student received an unsolicited Instagram message from a stranger, posing as another teen, asking them to send a headshot in order to get to know them better.

“The student obliged and within a very short period of time was messaged back an AI-created image with their head skilfully superimposed onto a nude figure,” the staff member wrote.

“A follow-up message included a full list of the students’ contact list details (mobile phone numbers and email addresses) and a threat to share the image with everyone unless they paid monies to the extortionist. The matter is currently being investigated by the police.”

A Queensland Police spokesperson confirmed police were currently investigating a report of possessing child exploitation material, reported on December 7.

“At this stage there have been no arrests and investigations are continuing.”

The school also issued advice to parents about chatting to their teens regarding cyber safety as well as resources about social media platforms. It was an excellent response to a shocking issue.

Gold Coast private school centre of police investigation into student deepfake porn drama

But, honestly, what is the world coming to?

This poor student has done nothing wrong but be friendly. While older generations might be surprised at the idea of responding to an unknown person online, the social circles of Gen Z know no bounds or borders - witness their generation’s preoccupation with the online phenomena of the IBF (Internet Best Friend).

It’s quite sweet and like a grassroots international relations workshop … until something like this occurs.

Even worse, this is not a one-off.

Welcome to the horrific business of ‘nudify’ apps. In a report filed last week by 60 Minutes in the US, the program found nearly 30 similar cases in schools across America over the last 20 months, along with additional cases around the world - and now we can add the Gold Coast to the list.

And while the images created on AI ‘nudify’ sites are fake, the damage they can cause to victims is real … but the action taken by social media companies used to share these images is all too often non-existent.

Indeed, one parent from the 60 Minutes report said it took more than eight months to get the Snapchat accounts that shared fake nude images of young students taken down.

It’s yet another reason why we need to save our children from social media.

While Australia last month passed a world-first law banning under-16s from social media platforms, tech companies now have two extra months to finalise plans to restrict children from accessing adult websites.

Gold Coast private school centre of police investigation into student deepfake porn drama. AAP Image/Richard Walker

The eSafety commissioner, Julie Inman Grant, said the age assurance trial for the ban was already underway, but the legislation for the ban was complex and tech companies required more time to comply.

On the positive side, just one week ago, new codes came into force requiring tech companies to take action against these very ‘nudify’ apps. But clearly further action just can’t come soon enough.

Even without this extension, my own children will have aged out of the social media restrictions by the time they are enforced … but not without damage already sustained.

Back when he first started high school, my son witnessed a suicide on TikTok, a vision that haunted him for months. The worst thing was that it happened right in front of me. What horrible irony that after all the years I spent protecting him from harm, he was exposed to the worst thing he had ever seen in our own living room while under my supervision.

The fact is that parents are doing all they can, our government is doing all it can, it’s time for tech companies to step up and save our children from the danger they have created.

Because right now, I don’t know about Santa, but I know which list I’d add them to.

Originally published as Gold Coast private school centre of police investigation into student deepfake porn drama