Read my lips: AI-dubbed films are debuting in cinemas

Foreign language cinephiles can be split into two distinct categories – subtitle-lovers and those who swear by the dubbed version.

Dubbing critics have long grumbled about the pitfalls of mismatched audio and awkward lip-syncing, but new technology is quietly changing the face (and mouths) of international cinema.

Last week, Swedish sci-fi film Watch The Skies opened in US theatres – marketed as the world’s first full-length theatrical feature to use AI for an immersive dubbing – a process that makes the characters look as though they are speaking English.

XYZ Films partnered with AI start-up Flawless, which uses TrueSync, a visual tool which alters the character’s mouth movements and speech to appear perfectly synced for an English-speaking audience.

“For the movie industry, this is a game changer,” producer Albin Pettersson declared in a behind-the-scenes trailer for the film.

“The Swedish language is a barrier when you want to reach out around the world.”

It’s important to note the AI tool has not replaced the actors – the original cast of Watch The Skies, having shot the film in Swedish, then recorded their English lines in a studio. This kept them compliant with SAG-AFTRA guidelines.

“I think a lot of filmmakers and a lot of actors will be afraid of this new technology at first,” added writer and director Victor Danell. “But we have creative control and to act out the film in English was a real exciting experience.”

Watch The Skies is the start of a long list of AI-dubbed international film collaborations between XYZ Films and Flawless set to be released in the US. They include French film The Book of Solutions, Korean flick Smugglers, Persian-language film Tatami, and German film The Light.

However not everyone in the international film industry is keen.

The president of the Australian Association of Voice Actors (AAVA), Simon Kennedy, acknowledged the intrigue around visual dubbing, but stressed the ethics around AI in filmmaking remained on “blurry ground”.

“We believe in the issues of consent, control and compensation,” Kennedy explained. “Artists need to know what’s going on… what’s happening with their voice and have control, so someone can’t take your voice, your vocal likeness, and make it say things you haven’t consented to or agreed to.”

Kennedy said dubbing was not a huge industry in Australia – most dubbing production and translation from foreign languages to English happens in the United Kingdom or the United States.

But he was worried about the future implications for Australian voice actors and audiobook narrators, especially if AI-dubbing grows in popularity and potentially replaces artists.

“If they [Australian film companies] can get their hands on AI voices that are convincing enough, they will be doing it,” he said. “I’ve seen two campaigns recently that have used AI voices and haven’t been open about it.”

Last month, it was revealed Australian Radio Network’s (ARN) Sydney-based CADA station had created and used an AI-generated host for about six months without disclosing it.

Fellow Australian voice actor and AAVA vice president Teresa Lim said it was important for film companies to disclose their use of AI – talent or voices.

“This Swedish film, it sounds like they’ve used an AI tool, however, they’ve still used a human to translate and perform,” she said.

“If that’s the situation that’s fine. It’s when you decide to take away that human element of translation and performance… that’s when we get into serious issues of ethical concerns, loss of artistic integrity, cultural inauthenticity and actors losing their jobs.”

Lim said there was a “real danger” of AI tools eventually replacing the whole dubbing process.

“It’s such new technology… and because it’s cheap, it is going to be much more time efficient and cheap to just control everything yourself and just cut out as many parts of the inner process as possible,” she added.

Kennedy and Lim’s reservations around AI echo similar concerns from voice actors and dubbers across the globe.

On March 28, Germany’s leading dubbing artists including Peter Flechtner (voice for Ben Affleck) and Claudia Urbschat-Mingues (voice for Angelina Jolie) posted a viral video warning their jobs were at risk and calling on the community to “protect artistic intelligence, not artificial intelligence.”

Meanwhile, the French Union of Performing Artists and dubbing association Les Voix’s petition #TouchePasMaVF (Don’t touch my dubbing) had collected 221,693 signatures at the time of writing.

Palace Cinemas chief executive officer Benjamin Zeccola is currently at the Cannes Film Festival and told this masthead his initial reaction to the AI-dubbed Swedish film was “heartbreak”.

“It strikes me on emotional level, I’ve always taken so much joy from watching films in their original language,” he said. “Observing the actors and hearing their natural voices in their mother tongue...one of the most beautiful things about humanity is the incredible diversity of languages.”

However, Zeccola also acknowledged the benefits around AI-dubbing and introducing foreign films to broader audiences.

“There are thousands of films made around the world every year, and some of them are just so spectacular, but they don’t often find a truly mainstream, broad audience in Australia,” he said.

“So this would certainly give an opportunity for films to broaden out and touch more people, and that’s a beautiful thing.”

Find out the next TV, streaming series and movies to add to your must-sees. Get The Watchlist delivered every Thursday.