Apple’s 3D video is uncomfortable viewing – and that’s a good thing

Ever since the invention of disposable 3D glasses, creators and moviegoers alike have been chasing the idea of immersive pictures that could make a viewer feel as though they were actually there. With its immersive video format, and help from a new kind of camera, Apple believes it could be the company to finally deliver it.

Apple Immersive Video, which you can currently only watch though the Vision Pro headset, is a huge video format that fills your whole field of vision, letting you look up, down, left or right to explore each scene. It’s sharp and three-dimensional, so you can focus on different elements naturally. But the problem right now is there’s almost none of it to watch.

Short film Submerged is the first scripted drama to be filmed and released in Apple Immersive Video.

“The productions to date have been with a bespoke camera system that we’ve built to support that 180-degree field of view, 8K environment,” said Bob Borchers, Apple’s VP of worldwide product marketing.

“What’s common to all the demos we’ve ever done with filmmakers, is the minute that they see it and then take it off, their first question is ‘how can I create that?’”

Enter Blackmagic Design, the Australian digital cinema company known for its Davinci Resolve editing and production software. It’s already a leader in enabling cinema shot on iPhone, with an app that lets creators film direct from the device and have the footage ingested into its platform, and now it wants to be at the forefront of creating for Vision Pro. This week it announced the release of a new cinema camera – the Blackmagic URSA Cine Immersive – as the first commercial system for creating Apple Immersive Video.

“We wanted to be able to democratise, and have more and more people be able to use Vision Pro as a storytelling medium,” Borchers said.

“The great news is we have this incredible developer ecosystem for iPhone and for our other platforms, that have worked really hard to build that for Vision Pro as well. And at the top of that developer ecosystem is Blackmagic.”

The Blackmagic URSA Cine Immersive is up for preorder directly from the company, and will be available from resellers later next year.

The custom lens on the $US30,000 ($47,000) camera captures a 8160 x 7200 resolution image per eye in 90 frames per second, allowing professional filmmakers to record in the new format directly. Of course there may be some adjustment needed on the creators’ part; the camera will allow you to see what you’re shooting in 2D, directly on the integrated displays, but that’s different from being inside an immersive video.

David Hoffman, business development manager at Blackmagic, said creators will want to experience the output directly. The view from the camera can go straight into Resolve, with controls to simulate looking around. In the near future, a creator will be able to wear Vision Pro and see though the camera lens in an immersive view.

“The delivery is going to be seamless because we’re going to have a version of Resolve coming out in early 2025 that will allow you to work in the traditional flow, not looking at the difference between right eye and left eye and having to synchronise all that,” he said.

“It’s going to be one contiguous file. You bring that into Resolve, your editing, your colour grading, visual effects, all of that is going to be very smooth.”

One of the custom immersive camera rigs used to film Submerged.

But to succeed where home 3D movies failed, Apple Immersive will need buy-in from both consumers and creators. Borchers said movie-makers already love the headset for colour-grading and virtual location scouting, and that consumers will buy it primarily for its computing capabilities and then use it for consuming video as well.

Hoffman said Apple has something that 3D video didn’t; a huge existing base of creators. iPhones can already shoot spatial video, which is like a smaller version of Immersive, and in designing its new hardware and software Blackmagic has been able to use the same language and workflows people are already used to.

“If we go back into the early days of 3D or 180, 360 [degree cameras], these are custom rigs that were built. Every one of them was a one-off, and it was difficult for the creatives to get attached to that,” he said.

Blackmagic’s view is to empower the creators by making it easy for them to go through that process, to acquire the tools, the free app on the iPhone, the free app on the iPad, the ability to run a free version of Resolve on your MacBook. To be able to get the creativity and take it with you.”

Another immersive video available on Vision Pro is a music video from The Weeknd.

The tech may be ready for creators, but are viewers ready for immersive video? The only way I can describe the feeling of watching it is that it’s like very high-quality 3D, but instead of a TV or screen to frame the content it appears all around you and takes up your whole vision. This can feel natural, but it can also feel uneasy.

In a recording of a basketball game taken from a fixed angle on the sidelines for example, you just feel like you’re watching the game in person with goggles on your face. But if a player or the ball happens to come rocketing towards the camera, your body will naturally recoil and the experience stops being comfortable. And this is an interesting issue when it comes to cinema.

Award-winning writer and director Clara Chong said the format opens up many opportunities for storytelling, with the ability to place details and even sounds at any point around the viewer, but that comfort and cultural sensitivities were more of a consideration than ever.

“There’s a whole issue of personal space because it’s very real when you’re watching it. And then you’re thinking about OK, what if it’s a fight scene, or a love scene, when does it become awkward?” she said.

“On a sensory level, it’s just such a different experience because we’re so used to the way we watch or view content. With the [Vision Pro], there’s a heightened feeling of both emotion and expression, the experience that you’re literally feeling. It’s something that is very difficult to explain, but once experienced you don’t forget.”

A music video for the Weeknd’s song Open Hearts was shot exclusively for Apple Immersive, and contains some experimental techniques that can be stomach-churning. The viewer’s perspective rolls forwards and backwards in parts, or cuts from a perspective looking directly down to one looking up.

Elsewhere, the first short film released in Apple’s format is set in a submarine and is also punctuated by several uncomfortable viewpoints. Some shots appear with out-of-focus backgrounds so you’re forced to look where you’re being guided, while one scene has a torpedo being loaded into an area that feels like it should be the viewer’s gut.

Blackmagic’s camera has built-in controls so filmmakers can set a “comfort” level, and be alerted if the subject is too close or the pans are too quick. But audiences – like those in the much-discussed anecdote about pre-1900s cinema who instinctively fled from a video of a moving train – might need some time to adjust. But of course, in some films, discomfort may be the goal.

Cinematographer Ben Allan said many of the rules of filmmaking would need to be rebuilt for immersive video, in a way similar to when audio or colour was introduced to film, except this time, the technology was more democratised.

“Technicolour was notorious at the time for telling cinematographers ‘no, you can’t use those colours’,” he said.

“One of the things I love about where Blackmagic and Apple are both going with this is it’s encouraging creativity, encouraging people to experiment and see, OK, what the hell is it possible? What’s possible in this new world?”

Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.