AI warning: X-rated and abusive chatbots warning for parents

AI apps allow Australians to create anything from a dream girlfriend to a school bully that enjoys sending nasty messages. But, experts warn, it comes at a price.

Parents are being warned about a new wave of highly addictive X-rated or abusive AI chatbots.

Since ChatGPT launched a year ago, the number of Not Safe For Work (NSFW) chatbot sites – NSFW is internet slang for content you may not want to view in public – have shot up.

With a couple of keystrokes you can find an array of AI platforms offering synthetic companions “without filters”, some of which are designed to allow people to explore their sexual fantasies.

One such platform allows users to create their dream AI girlfriend from scratch, from how they look – with choices of “hair length, ethnicity, age, and breast size” – to their personality type.

Others offer ready made characters to interact with including a school bully that enjoys sending you nasty messages, a kid who is depressed and self-harms and a h***y stepbrother who watches porn while fantasising about you.

Generative AI is being used to allow people to create their dream online girlfriend. Pictures: These are AI generated pictures, not real women.

There are also AI apps that allow you to upload a picture of someone’s face or voice to pornographic photos and videos.

While they’re aimed at adults and come with warnings, most sites only require users to tick a box to say they’re 18 to access them.

With one in five Aussies saying they often feel lonely, having an artificial companion or artificial romance, may be tempting for some.

Dr Kimberley O’Brien, a child psychologist, warned interacting with chatbots was not an “authentic” experience for people of any age and that face-to-face meet ups were far better for people’s mental wellbeing.

This is an AI generated picture, not a real woman.

But Dr O’Brien said perhaps the bigger worry was that having an AI friend could prolong the amount of time someone spends online, keeping them away from healthier alternatives, and for some it could become addictive.

“It’s like online gambling,” she said.

“It can be engaging and rewarding, but in the end it is not good for you.”

She said explicit or inappropriate generative AI was one more thing busy parents had to try and police, when the government should be doing more to safeguard minors.

Dr Catherine Ball is a scientific futurist and author. Picture: Naomi Jellicoe

“It always falls to the parents to make sure there are parental controls so kids can’t access these sites,” Dr O’Brien, founder of Quirky Kid Clinic, said. “I think the government needs to be more supportive of parents.”

Dr Catherine Ball, who is an academic at the School of Cybernetics at the Australian National University, said governments were always behind with regulation, but education was something that could be done now.

She said children may not comprehend the emotional or legal consequences of using face and voice swap technology, especially if it involves other people’s faces and voices without their consent.

“We need to be having more awkward conversations now,” Dr Ball said. “Consent and digital consent is something we need to be teaching our kids in schools.”

Dr Ball said protecting your face and voice from getting into the wrong hands was another issue people should be thinking about now.

Parents are being told to be aware of certain inappropriate AI generative websites that are popping up online. Picture: Supplied

She recommended families have a “safe word” to protect themselves from scammers who may use family member’s voices to trick people into sending money.

eSafety Commissioner Julie Inman Grant said in the not too distant future “online pornography is likely to feel substantially more visceral, intense and realistic – and to be highly customisable”.

She said companies should ensure their tools are not used to create sexualised content featuring children, while also protecting children from coming across or creating AI-generated sexually explicit content featuring adults.

Ms Inman Grant said new industry codes will come into effect on December 16 to reduce the spread of deepfake or synthetic child abuse material.

From March next year, the Search Engine Code will reduce the risk that AI functionality integrated with their search engines are not used to generate synthetic sexual child abuse material.

Originally published as AI warning: X-rated and abusive chatbots warning for parents