Aussie tech company connects Apple’s Vision Pro to people’s brains to help overcome paralysis

Melbourne-based Synchron has connected its brain computer to Apple’s newest ‘spatial’ computer to allow people with paralysis to communicate and consume content like never before.

Synchron – the Australian rival to Elon Musk’s Neuralink – has connected Apple’s Vision Pro to human brains as the technology titan’s newest computer presents the latest leap to help people with paralysis.

Mark, a 64-year-old living in Pittsburgh who has amyotrophic lateral sclerosis (ALS), has become the first patient to have his brain connected to the device. He has been able to play solitaire, watch movies and write text messages, saying “this is pretty cool”.

“Using this type of enhanced reality is so impactful and I can imagine it would be for others in my position or others who have lost the ability to engage in their day-to-day life. It can transport you to places you never thought you’d see or experience again,” Mark said.

Synchron was founded in Melbourne and has since moved its headquarters to New York. It has been at the forefront of developing brain computer interfaces – with its technology not requiring surgery – and has attracted investors including Bill Gates and Jeff Bezos.

It also incorporated ChatGPT into its brain computer interface this month via a partnership with OpenAI.

Synchron chief executive Tom Oxley.

Chief executive Tom Oxley – an Australian-born neurologist who also works at Mount Sinai hospital in New York – said the Vision Pro, with its eye-tracking and spatial computing technology, was a perfect fit for people with paralysis.

But Synchron had to overcome two big hurdles. Like other virtual reality headsets, Vision Pro also relies on hand gestures to perform certain tasks, and the device needed to connect to its brain computer interface (BCI), which is implanted via the jugular vein in a similar way to a stent.

“We had to figure out how to get the Bluetooth integration working and replace the gestures, so it took a bit of work to get there,” Dr Oxley said. Synchron’s solution involved its BCI allowing a user to just think about moving their hand to operate the Vision Pro, which Apple chief executive Tim Cook said was the “most advanced consumer electronics device ever created”.
“We’ve got an inference engine that uses deep learning to make predictions on what the brain is trying to do, and it sends them out as packets of control instructions,” Dr Oxley said.

“We are sending control signals directly from the brain to replace the need for hand gestures.

“There was a bunch of work led by the algorithm team, led by Peter Yoo to build that inference engine, and that’s becoming a really core part of our product.”

Apple has also launched eye-tracking on its iPads, which Dr Oxley said Synchron will be trialling soon. Synchron has been working with Apple’s accessibility team helmed by Sarah Herrlinger.

Apple Vision Pro launched in Australia earlier this month.

“We just think Apple’s accessibility is the best in class. It’s the easiest. Everything’s there … your messages, your emails, the internet, Apple TV. As the form factor (of Vision Pro) improves, it gets a bit lighter, it’s easier to use, I think it’s going to be technology that will be very powerful.

“My experience with using Vision Pro – and I think a lot of people have had this experience – is it’s the next level of personal computing. But the big issue with that has been this problem of how you make selections (to control the device).

“The issue with eye tracking is it’s not easy to select things. On a mouse, you have right click, you have left click, double click, you have drag. So we’re looking at the basic functions of a mouse, left click, right click, drag, double click, and we’re looking to overlay that on top of the eye tracking. And so as eye tracking, gets rolled out, I think there’s a really nice synergy on the horizon.”

Dr Oxley said Synchron was also moving towards a new Bluetooth standard for human computer Interactions that don’t require touch or speech, saying it was a critical unmet need for millions of people with paralysis.

Synchron’s BCI is implanted into the blood vessel on the surface of the motor cortex of the brain via the jugular vein via a “minimally invasive” procedure, unlike Mr Musk’s Neuralink. Dr Oxley compared it to installing a stent versus open-heart surgery.

Synchron was also the world’s first company to be granted approval from the US Food and Drug Administration to begin clinical trials on a permanent computer-brain implant, beating Neuralink to that milestone. Its BCI has now been implanted in 10 patients as it seeks full FDA approval. This compares to Neuralink so far fitting its technology in one patient.

Synchron has so far raised about $US140m ($214.2m) from investors, and Dr Oxley said earlier this month the company may seek another injection.

“We’ve still got a fair bit in the bank, so we’ve still got enough to get through a couple of critical milestones but probably not enough to get all the way through the pivotal trial, so we’ll see how we go. For the moment, we’re doing pretty well,” Dr Oxley said at the time.

Originally published as Aussie tech company connects Apple’s Vision Pro to people’s brains to help overcome paralysis