AI kissing apps raise ‘serious concerns’ among AFP, experts
A new AI app that allows someone to upload an image of themselves and another person to simulate them kissing has sparked outrcry among experts and police who say the content can be used to blackmail children.
Education
Don't miss out on the headlines from Education. Followed categories will be added to My News.
A new AI app that allows someone to upload an image of themselves and another person to simulate them kissing has sparked outcry among experts and police who say the content can be used to blackmail children.
The Daily Telegraph uncovered sponsored content on Meta-owned Instagram, where social media users can directly download a “kissing your crush with AI” app, which purports to create animated kissing images and videos by using two images of people.
This app, which is among several others available on Apple’s App Store being used by young teens, has raised “serious concerns” among experts, who fear they could be misused among child abusers to prey on young people without their consent.
An Australian Federal Police (AFP) spokesperson said police and the Australian Centre to Counter Child Exploitation (ACCCE) had seen cases where innocent images of children had been edited to appear sexually explicit, and those photos were then used to blackmail children into providing other intimate content.
“Due to the sophistication of readily-available AI tools online, AI-generated child abuse material presents a challenge to our specialists in the ACCCE,” the spokesperson said.
“Any material that depicts the sexual exploitation or abuse of children, including videos, images, drawings or stories, is child abuse material and illegal under Australian law. That includes material generated by AI.”
It comes after several incidents in recent months involving the misuse of AI technology, including at least 16 women – some working in high level public service roles in Canberra – being depicted in more than 100 deepfake nude images created by a 23-year-old man.
It led to Premier Chris Minns at the time acknowledging the state’s current rules were not “keeping pace with technology” and said the government would look at changing the laws around the creation of deepfakes.
Collective Shout movement director Melinda Tankard Reist said AI-generated kissing apps were “invasive” if they were misused.
“This is a violation of privacy and an act of digital forced kissing,” Ms Tankard Reist said.
“The deepfakes images look extremely realistic and they can cause trauma because the young person depicted didn’t consent and other people may think that’s a real image.
“It forces them into an intimate scene and goes against the million dollar government spending on consent and respectful relationship programs.”
Lily Arnold, 21, said the idea of someone creating a digitally altered image of her would make her feel “uncomfortable” and she “would be really upset by it”.
“I think it’s weird,” she said.
“The idea of someone doing that with myself or someone else doesn’t feel right at all.”
“It feels like a lack of consent because even though it’s not you physically, it’s not ethical.”
An eSafety spokesman said the authority was “deeply concerned” about the increasing accessibility of AI-powered apps that could be used to manipulate images and create convincing deepfakes.
“This misuse is particularly troubling when AI-generated images are used to shame, sexualise or harass others without consent.
“We’ve already seen how similar tools have been used to generate deepfake child sexual abuse material and non-consensual pornography.
“eSafety has already taken strong steps, including by recently developing and registering mandatory industry standards, as provided for by the Online Safety Act.”
Cyber safety expert Susan McLean said the kissing app and many others of a similar nature were of “serious concern”.
“You don’t have to be a rocket scientist to figure out how this app will be misused, the target will be young females again,” Ms McLean said.
“The app stores shouldn’t be hosting this sort of stuff and if they don’t take any responsibility – we’re never going to get anywhere.”
Deakin University Associate Professor Debbie Ollis, who is a lead researcher in education, said there was a “great deal” of AI-generated deepfake images of young people being developed.
“The implications for young people are very severe and one of the problems is that we can’t control how these apps are used and where these images end up,” she said.
A spokesperson for Deputy Premier and Education Minister Prue Car said: “We must do everything we can to stop the spread of these malicious images, especially among our young and vulnerable. It is clear that the law must keep pace with technology and we will be looking closely at the findings of the Parliamentary inquiry to chart the next steps.”
A NSW Department of Education spokesman said: “Deepfakes present significant new risks to the wellbeing and privacy of students and staff and the department will continue to work closely with cyber experts and other government agencies to identify and address these and other cyber threats as they emerge.”
“The Department CyberMarvel program is an award-winning national cyber security education program that creates material for teachers to use in classrooms to educate students on the dangers of sharing deepfakes.”
Anyone who has experienced serious online abuse can visit eSafety.gov.au to report it and access support.
— Do you have a story for The Daily Telegraph? Message 0481 056 618 or email tips@dailytelegraph.com.au
Join the conversation
‘No excuse’: True cost of uni debt on students revealed
Australians paying off their university degrees are being warned they’re now at risk of being stung with a tax bill, as young people call for “more meaningful” measures to ease the cost of tertiary education.
Read more