The next federal election will face unprecedented challenges from AI-generated content and fake news, Australia’s top election official has warned.
Australian electoral commissioner Tom Rogers said there had already been several “deepfake, AI-generated moments” during the United Kingdom election campaign and he expected Australia would face similar problems.
Under Tom Rogers, the Australian Electoral Commission has had a high-profile social media presence.Credit: Fairfax Media
According to the BBC, some of that deepfake, AI-generated material has included a video of British Prime Minister Rishi Sunak declaring, “please don’t vote us out, we would be proper gutted”, videos that claim 18-year-old Britons would be sent to Gaza or Ukraine to do national service and others promoting the false claim that opposition leader Sir Keir Starmer had failed to prosecute serial paedophile Jimmy Savile.
The recent Indian election faced similar problems, with millions of citizens estimated to have seen deepfakes during the campaign. South Korea has banned the use of AI-generated content during election campaigns.
With an election due in the next year, it’s an issue Australia now needs to face.
To help combat the problem, Rogers said a national digital literacy campaign was urgently needed to give Australians the information and skills to distinguish between real and fake election material, alongside regulation of the sources of AI-generated material.
Rogers said the rise of fake news online had been a problem since about 2016, when the commission ramped up its efforts to deal with mis- and disinformation.
“We’ve been pretty successful at doing it [fighting fake news] by working at the very edges of those powers [the commission has]. But what we now face is a new challenge with the rise of AI and AI-generated content at election time,” he said.
“I think what we’re dealing with is a means of generating election content that we’ve never seen before. It can be generated cheaply and at scale and distributed widely. And this is something that our system is not necessarily structured to pick up or deal with.
“We’re really watching closely what’s occurring this year in the UK and the US [elections] and about how they deal with it.”
Rogers and his team have regular meetings with staff from Meta, the owner of Facebook, Instagram and WhatsApp, and social media platform X to discuss what he describes as a “huge uptick in mis- and dis-[information], conspiracy theories, threats to staff”.
That has coincided with social media platforms dialling back their comment moderation and removal of fake news content.
Rogers, who is in his 11th and final year as commissioner, said the commission had narrow powers to ensure election material was properly authorised – and if AI-generated material had been properly authorised, the commission could do nothing to take it down.
“We’ve got this issue with brand-new technology, it can produce stuff at scale. We don’t think we have the powers necessarily to be able to deal with it,” he said.
“We do think that the legislature should consider whether or not there’s some further regulation that should occur ... [for example] watermarking. Perhaps have a declaration that the material that citizens are looking at was generated by AI?”
Such a move would regulate the sources of AI-generated material, but that was only one part of the equation.
“The other thing we’re going on about and there’s an urgent need [for] is actually not just regulating the source but hardening the target, which is citizens, by running a national digital literacy campaign. And we simply can’t do one without the other, you need to do both.”
Cut through the noise of federal politics with news, views and expert analysis. Subscribers can sign up to our weekly Inside Politics newsletter.