How Google used AI to recreate The Wizard of Oz in the Sphere

Google invented techniques to enhance resolution and generated new character performances to bring the 1939 film to the giant screen.

Google used AI to generate new pixels to enhance the resolution of the original 1939 film. The new version on the right is compared with the original on the left.

Showing an 86-year-old movie, shot with a 35mm camera, on a 15,000sqm curved, immersive screen initially seemed impossible – even to AI engineers at Google.

But that is what James Dolan, executive chairman and chief executive of Sphere Entertainment had in mind when he decided to present The Wizard of Oz in the Las Vegas Sphere, one of the highest-resolution screens in the world.

“When we first brought the project to Google and we talked to their scientists, I think they thought we were a little crazy,” Dolan said. But the Sphere itself, an enormous steel globe just off the Strip, wrapped in an LED exoskeleton with changing colours and patterns, is also a little crazy.

Since opening in 2023, the venue’s 17,600-seat theatre has hosted performers such as U2 and the The Eagles – who at the weekend completed a season – and shown movies specifically filmed for its unique screen. It has never played an existing film, let alone one shot with technology from the 1930s.

But after cutting an intellectual property deal with Warner Bros, and sizing up the capabilities of a few artificial intelligence companies, Dolan put the Google engineers to work on getting the wonderful world of Oz Sphere-ready. Though it wasn’t exactly as easy as skipping down the Yellow Brick Road.

“Very, very, very big and very, very difficult,” was how Steven Hickson, director for AI foundation research at Google DeepMind, described the project.

“There are scenes where the scarecrow’s nose is, like, 10 pixels. That’s a big technological challenge,” he said about getting the character ready for the massive screen.

To make it work, Google’s enterprise business, Google Cloud, and its research unit, Google DeepMind, invented new AI methods to enhance resolution and extend backgrounds to include characters and scenery not in the original shots. Google calls these techniques “performance generation” and “outpainting”.

The Sphere will debut the movie on August 28, but for cinephile purists, this isn’t your mother’s Wizard of Oz. AI has touched over 90 per cent of the movie, said Ravi Rajamani, managing director, global head of generative AI engineering at Google Cloud. There might also be other sensory elements, but Dolan said those are under wraps for now.

According to Thomas Kurian, CEO of Google Cloud, viewers should think about it not as a cinematic experience, but as an experiential one. “We’re taking a beloved movie, but we are recreating it,” he said. “The only other way you could do it is to go back (in time) and film it with the cameras that the Sphere uses.”

For example, in one early scene, Dorothy is seen talking to Aunt Em and Miss Gulch. Audiences know Uncle Henry is in the room, although off camera. In the version that will play in the Sphere, Uncle Henry is visible, along with a much wider background shot showing more of the house.

Google used generative AI models from its Gemini family, including Veo 2 and Imagen 3, to generate the new background and performances, fine-tuning the models on the original movie. The team still ran into challenges, in part because of the limited source material.

“You only have one movie for us to train this model on. And then some of these characters don’t appear a lot,” Rajamani said.

Google’s team also consulted with professional filmmakers to help decide actions, expressions and performance. Oscar-nominated producer Jane Rosenthal worked closely on the project.

Google also used new AI-powered methods to enhance the film’s resolution. Traditional techniques involve multiplying a shot’s existing pixels. Google instead used AI to generate new pixels as it increased the size of the visuals.

AI has heightened divisions within Hollywood in recent years, as actors fought for protections against the use of their IP to train new models, and studios copped backlash for the use of AI in new projects. But Dolan isn’t worried about any of those concerns surfacing here. “I can’t wait for the film and television industry to see what we’ve done,” he said. “I think their jaws will drop.”

The Wall Street Journal