Brain powered: control a computer with your thoughts using Synchron’s stentrode technology
It was once the realm of science fiction. Now, a brain-computer interface developed by this Aussie team promises to revolutionise and reshape the way we communicate.
An ultrasound wand slides over a man’s collarbone. It doesn’t take long to find the jugular vein. With a sharp tool, a surgeon punctures the man’s skin and into the incision feeds a narrow wire, and then a circular plastic sheath. It’s the kind of procedure performed thousands of times every day the world over to insert stents into damaged hearts, but never before have doctors attempted what is now being performed at the Royal Melbourne Hospital: an operation to implant an endovascular neural stent into the brain of a patient with motor neurone disease via his blood vessels. The tiny device is the key to a human being able to control a computer with their thoughts.
Patient Graham Felstead knew this was uncharted territory. “We were told in no uncertain terms, ‘He may not come out of this operation’,” says wife Nancy Felstead. But Graham was more than willing to take the risk. Since being diagnosed with MND, the 75-year-old had read the books of Stephen Hawking and understood the significance of the opportunity to be the first patient in the world to be enrolled in a clinical trial to test the safety of a brain-computer interface.
Hawking had declared shortly before his death in 2018: “I believe the future of communication is brain-computer interfaces.” The world’s most famous MND sufferer had produced millions of words by using the only part of his body he could still control, a tiny cheek muscle, to operate a switch on his computer. “I often wonder what other theories Stephen Hawking may have contributed to the world if he had a brain-computer interface,” Felstead wondered in his own book, Technopathy for Beginners. “Did he have another masterpiece in him if he could have typed twice as fast? What about 10 times as fast?”
Brain-computer interfaces – sometimes dubbed smartbrains – provide a direct line of communication between the brain’s electrical activity and an external device such as a computer or robotic limb that a person learns to control with their thoughts. Most experiments with the technology have been highly invasive, with electrodes embedded into the brain during neurosurgery. Now, a fledgling Australian company is leading the world with a meticulously engineered neural interface that is inserted into the brain via the blood vessels. For people with neural diseases that result in paralysis – including “locked in” syndrome, where sufferers can move only their eyes – it offers a means of communicating that is transformative.
“This is the start of a new era in medicine,” says inventor and neurointerventionist Tom Oxley, the co-founder and CEO of Synchron. “What this device does is an incredibly powerful restoration of autonomy. This technology changes the conversation that we’re having with people who have no hope.”
The dream of melding mind and machine to overcome the limitations of the human body has been pursued by scientists from as early as the 1950s, a few decades after German scientist Hans Berger made the concept of a smartbrain possible with the invention of electroencephalography, in which electrodes attached to the scalp for the first time enabled mapping of the brain’s electrical activity. It was Californian computer scientist Jacques Vidal in 1973 who coined the term “brain-computer interface” following experiments that demonstrated a human could mentally guide a cursor through a virtual maze using signals from an electroencephalogram. Thirty years later, studies demonstrated that monkeys implanted with neural interfaces could control robotic prostheses with their minds. Then in 2004, US tetraplegic Matt Nagle became the first person to use a brain-computer interface to restore functionality lost due to paralysis, mastering control over a mouse cursor, using a TV remote, checking emails and sending commands to an external prosthetic limb.
But it wasn’t until Artificial Intelligence gathered pace in the past decade that brain-computer interfaces have moved from the experimental realm to now being poised for widespread therapeutic application. Through algorithms that are growing ever more sophisticated, AI allows automation of the computation involved in converting brain signals to actions, and can even anticipate human intentions once it learns the particular neuronal language of the motor cortex of an individual. The possibilities for the transformation of human intelligence are potentially vast – but for now, scientists are firmly focused on their project of providing hope to the afflicted.
Tom Oxley was in his first month ofspecialty training in neurology at the Royal Melbourne Hospital in February 2011 when he was called to the intensive care unit with supervising consultant Peter Hand. A 40-year-old man and father of three had suffered a massive stroke and was completely paralysed. As the doctors reviewed his scans, they revealed the worst: the stroke had occurred in a crucial area of the brain and the output of all of the body’s control signals were blocked. The man could move only his eyes. It fell to Hand to explain to the man that he was “locked in” and would be paralysed for life, able to communicate only by moving his eyes left and right. “It was just horrific,” Oxley says. “He chose not to live like that. We switched off the machines and he died. And I remember thinking about what it would be like to have to make that choice. He was still in there, he could listen, he could feel, he could think, it’s just that he couldn’t communicate or express himself. It was profound to think about how without the ability to express oneself, all of the other things your brain does almost doesn’t matter. You can’t engage with the world.”
The Australian inventions changing the world
Brain powered: control a computer with your thoughts
It was once the realm of science fiction. Now, a brain-computer interface developed by this Aussie team promises to revolutionise and reshape the way we communicate.
Soldier’s best friend: why ‘Bushie’ is a great Aussie invention
Since it entered service the Bushmaster has saved hundreds of Diggers from landmines, IEDs and enemy fire – it’s one of our country’s best industrial innovations. | FULL LIST
Can these engineers disrupt a billion dollar market?
Their microneedle medical patch designed to monitor the efficacy of administered drugs is among the top Australian medical breakthroughs improving human health. | FULL LIST
Heaven sent: two Aussie punks take over Marc Jacobs’ world
When the fashion supremo needed a breath of fresh air he turned to two Australian creatives - Ava Nirui and Elliot Shields. The pair are now among our leading fashion innovators | FULL LIST
How a Tassie seaweed farm can help save world
Methane-eating seaweed, avocados grown in a lab, and the march of the flat white towards World Domination. In the world of sustainability, food technology and dining, these Aussies are making a global impact. | FULL LIST
Man behind the ‘masterpiece nobody should miss’
He is the pioneer of ‘cine-theatre’ whose latest five-star production is headed to the West End and will feature a Hollywood star. Kip Williams leads our list of cultural innovators | FULL LIST
The Aussie company out to send astronauts to the moon
A plan to send a 25-metre tall rocket into orbit from a North Queensland spaceport is just the beginning for Gilmour Space, who lead our list of Australian tech and gadgetry innovations making an impact in 2023.
The experience galvanised Oxley – who had read with fascination three years previously the first scientific report of a brain implant in a human with paralysis – to dedicate his life to changing the prospects for those individuals. He joined with biomedical engineer Nicholas Opie, who had just spent a decade working on the bionic eye project, and cardiologist Rahul Sharma. Until then, experiments involving brain-computer interfaces had all involved highly invasive processes including brain surgery. The trio were convinced that implanting a brain-computer interface via the blood vessels – a safer, simpler method – was feasible. They just needed to invent a suitable device.
In a basement laboratory atthe Royal Melbourne Hospital, Opie sits hunched over a microscope in the early morning as the darkened city outside shivers through the winter. The lab is a dedicated space where Opie can experiment, building and testing prototype neural stents. Surrounded by reams of wire, soldering irons and laser welders, the biomedical engineer and head of the Department of Medicine’s Vascular Bionics Laboratory works with tiny tools to fashion the “Stentrode”, a neural interface built using a self-expanding mixture of nickel and titanium known as nitinol, a commonly used material in medical products that must be flexible but very strong, and proven safe inside the body.
“It’s almost meditative to sit there and be making one of these devices,” Opie says. “The wires that we’re using are point nothing of a millimetre, the electrodes are less than half a millimetre, so all of this is done under microscope with fine tweezers. And you really just sort of get in a groove and get in a nice mindset and you just get transported away and engrossed in what you’re doing.”
It takes years of experiments for Opie to land on the ideal positioning of the electrodes on the stent’s expanding cage. The more electrodes, the better the signals that will be picked up from the brain.
“A lot of it was to do with how many sensors could we fit on, how can I wrap the wires around the electrodes in a different way to get even more sensors on the scaffold, how can we make sure the scaffold itself still opens and closes when it’s got a whole lot of electronics on there?” he says. “The idea was that the more sensors we had on the device, the more brain information we could pick up – and the more control someone with paralysis would have.
“And then I had to figure out a way to make sure the stent can be pushed through one of the catheters, and how we could remove the catheter and ensure the device stays in place in the blood vessel. So that was the sort of early engineering. And then the latter steps, after we had implanted a device in an animal, were how can we make sure that the device opens properly, and blood still flows through so that we’re not causing any occlusion or blockage, and make sure that we’re getting high quality signals related to movement intent. And then we had to figure out how to interpret these brain squiggles that you see, and use them to control some external piece of equipment.”
Opie’s quiet engineering work to fashion the Stentrode was performed years before billionaire Elon Musk’s neurotechnology company Neuralink was founded and began attempting to develop a brain-computer interface of its own. But while Neuralink worked on a “sewing machine-like” technology capable of implanting thin, threadlike devices into the brain through invasive brain surgery, and the company made headlines for adverse outcomes experienced by monkeys in experiments, Oxley, Opie and Sharma’s company Synchron was quietly performing the first human clinical trial of a brain-computer interface in Australia.
Oxley was in an angiography suite in the Royal Melbourne Hospital when Felstead was wheeled in for his surgery. Head of Neurointervention Professor Peter Mitchell led a team for the experimental operation that had been subject to months of deliberation within the hospital’s ethics committee.
The procedure involves feeding a Stentrode – a 4cm-long stent that incorporates a series of strategically placed electrodes dotted around a self-expanding tubular lattice – through the jugular vein and up into the brain, where it would pop open and lodge within the motor cortex, the part of the brain that controls movement, and pick up signals from local electric fields produced by groups of neurons there. The stent is connected to a wireless transmitter embedded in the patient’s chest, which communicates by Bluetooth with an external device that decodes the brain’s signals. Once this device is programmed, it is hoped the patient would be able to operate an external equipment including computers and mobile phones with their thoughts.
But as Felstead’s operation began in August 2019, it was yet to be proven that it was even possible for a stent to be implanted in the motor cortex via the blood vessels.
“To get from the neck to the top of the head is not a straight line,” says Mitchell. “The vein goes through the base of the skull and it does a very tight chicane, an S-bend, through the sigmoid sinus. You have to get through that bend and then negotiate several further right-angled bends. There’s a lot of friction, because you have to put a reasonably large tube up there. We didn’t know if this was possible in humans or not. Nothing like this had ever been done.”
Mitchell’s team had taken an angiogram of Felstead’s brain, but it didn’t allow them to see the crucial motor cortex region. In order to be able to navigate their way, the team had to “fuse” Felstead’s MRI images with the angiogram images, also a highly novel technique. The resulting full picture of his brain was displayed from multiple angles on a six-panel screen as surgeons fed the Stentrode through Felstead’s vein. “We had the side-on view and the front view showing exactly the path that we had to follow,” Mitchell says. “We start off with a very thin, soft micro-catheter and wire, and then we sequentially place larger catheters over that first catheter. And once you get that in position and you’ve got the plastic tube where it needed to go, you then through that larger plastic tube place the Stentrode, which comes pre-loaded in its own little plastic catheter.
“I have to keep it in position as I pull back the small plastic tube to allow the Stentrode to self-expand. I have to keep that to within a few millimetres of the right spot. Because there’s so many tight bends and such a lot of tension in the system, it’s very easy to have a millimetre or two’s movement when you’re working from so far away. You keep it in position with a push-pull motion, pushing on the stent with one hand and pulling on the plastic sheath with the other to keep it in position.”
Once the Stentrode had been placed in exactly the right place in Felstead’s brain, the device was deployed, its plastic sheath pulled back to allow its delicate cage-like structure dotted with strategically placed electrodes to expand. “That moment when it was deployed in Graham’s brain was the moment we knew we were going to succeed,” Mitchell says. “Until it had been deployed, it was very uncertain.”
Meanwhile, neurosurgeon Andrew Morokoff prepared to create a “pocket” in Felstead’s chest where a telemetry device – to transmit the signals from the neural stent in his brain to the outside world – would be implanted. A lead was tunnelled under Felstead’s skin from the stent to the device in his chest. The Eureka moment for the surgical team was when technicians pick up a signal via Bluetooth from his brain via the telemetry device. As the team held its collective breath and Synchron neuroscientist Peter Yoo activated a wireless receiver, the lines on the screen began to flicker. Felstead was still under anaesthetic, as yet unaware that medical and scientific history had just been made.
“I think the real breakthrough of this technology is that it’s a new way of recording signals from the brain,” says David Grayden, Clifford Chair of Neural Engineering at the University of Melbourne. “There hasn’t been a new way of recording from the brain developed for years. So this offers a whole new opportunity for how we can interface with the brain or even the peripheral nerves. It’s a system that can get good signals, but without actually interfering with the brain itself. The Stentrode is in the brain, but the brain tissue doesn’t know it’s there. I think we are witnessing the beginning of something that could be implanted for the long term that could get regulatory approvals.”
It was clear immediately after Felstead’s neural stent was implanted that the signals it was picking up from his brain were strong. But to pick up a signal is only the start. The controlling of machines via thought does not happen automatically. It takes many, many hours of training for a patient, working with a neuroscientist, to program their device to decipher the “intention” of the patient’s brain based on the signals detected when they attempt to move part of their body.
“Normally in a healthy body, when we try to do some sort of movement, that’s effectively just some sort of thought in our brain,” says Yoo. “And we call that ‘motor intention’. Different parts of the brain activate in a fairly predictable manner.
“What the Strentrode is able to do is effectively sit next to the motor cortex and listen to the neural signals. Neural signals travel through the spinal cord and ultimately move the limbs. Of course in a paralysed person, that process is gone. But the intention is still there. So what we’re creating is this prosthetic that bridges that physical ability they have lost.”
Yoo has worked with the patients in Synchron’s Australian clinical trial to develop personalised “decoding algorithms”, which are sets of rules that are used “to try to extract the motor intention out of the somewhat noisy signals from the brain”, he explains. “We record the signals while we ask the patient to do a variety of movements. Eventually we are able, based on training, to develop a model that aims to detect or predict when the patient tries to perform those movements.”
For six months following Felstead’s operation, Yoo made the 200km round trip to Taggerty, northeast of Melbourne, for regular training, where Felstead was given instruction on how to align his motor cortex activity with the algorithm that was being used to decode his brain signals. Initially, an eye-tracker was used to help him type, with the tracker aided by brain signals picked up by the Stentrode, which detected his intention for the physical movements required to replicate the click of a mouse button.
When Felstead thought about moving his foot, the movement would be logged and associated with a particular colour. After a series of movements, the decoder would identify the originating contact points on the Stentrode that related to specific body movements, thus developing a series of “switches” that connected the thoughts to actions.
One October weekday in 2019, Nancy was pottering in the kitchen when her husband called to her from the living room, where he was set up with Yoo and their equipment. Felstead had managed to type his first sentence just with his thoughts. “He said, ‘Come and have a look at this, Nancy’,” she says. “I stood there beside him, and on the computer he had written ‘Darl I’d like another cup of coffee’. I just lost it … I just broke down completely.”
For the rest of his life, Felstead made enormous use of his new skill, writing his book and frequently haranguing local councillors in written correspondence over broken promises. “He was very much keeping the councillors on their toes,” Nancy says. “He’d write letters, emails. He would still do the family banking.”
In the last months of his life, Felstead revelled in his status as a pioneer participant in medical research and took delight in his code name “Synchron 001-001”. He was in no doubt the Stentrode would revolutionise medicine.
“It will stand alongside the cochlear implant as a significant medical breakthrough of international significance,” Felstead wrote in the closing chapter of his book. “It has been exciting to participate in an endeavour that has shaped science fiction into medical fact. I’m extremely proud to be the first of what will be many beneficiaries of this technology.”
Two years after his Strentrode was implanted, Felstead died in Melbourne surrounded by his family. “The children have said, ‘If dad hadn’t had the Stentrode, he wouldn’t have lived that long’,” says Nancy. “His life had a purpose with the Stendrode. And he was a great believer in it.
“I am just so proud of him. I just don’t want anybody to forget him.”
In 2022, Synchron became the first company in the world to be approved by the US Food and Drug Administration to conduct a safety and feasibility human trial of its brain-computer interface in the US. One of the first American patients to be implanted with the Stentrode in New York City was David Redden, a prominent Manhattan auctioneer who had risen to be vice-chairman of Sotheby’s before being struck down with a type of motor neurone disease, in 2014. Redden’s remarkable career had seen him preside over sales from the John F. Kennedy family, the papers of Martin Luther King Jr, and the auctioning of a historic copy of the Magna Carta. At the time he had the Stentrode implanted last year, his disease had left him a quadriplegic and unable to speak.
“When I first heard about the Stentrode technology last September I was immediately fascinated by its potential to change the lives of thousands of human beings,” Redden states, writing through assisted technology. “I agreed to join the program immediately. If I could benefit the world in some way, that was a thrilling opportunity for me.” Currently, Redden still uses an eye-tracking computer for speaking and writing, but it is slow and exhausting for him to use. “The Stentrode technology had the potential of harnessing the brain in a completely novel way which seemed to be nothing short of telepathy – the controlling of machines through thought alone,” he says.
David Putrino is US clinical trial leader and has worked in several labs engaged in developing brain-computer interfaces. His observation was that most of the scientists working in this field “had never had a conversation with someone with a spinal cord injury and didn’t really care to”. He adds: “There have been many clinicians and physicians working in this space, but I think there’s a level of pragmatism with the Synchron team that I haven’t seen before in [brain-computer interface] research. From the outset they never tried to be a science experiment. They always wanted to be a company, and they were going to give the product to people with disabilities. That was the difference. When Tom [Oxley] came to me and said, ‘We’ve got an implant’, I said, ‘Sign me up immediately’. We got very excited.”
As the Synchron clinical trial progresses, Elon Musk’s company Neuralink has now also been approved to trial its brain-computer interface, an electrode-laden computer chip dubbed the Link that is sewn into the surface of the human brain. But while Neuralink claims that its device will have applications for blindness, paralysis and depression, Musk has made no secret of his vision for brain-computer interfaces as a “general population device” that could transcend the limits of human intelligence.
Oxley doesn’t deny the clear implications of this technology. “I can understand that there’s a trajectory, I’m not naive,” he says. “If this is the beginning of this type of technology, and it continues to improve, then I can see that there’s going to be a tipping point where it confers a benefit that’s beyond what the human body can do. There’s a dichotomy between the utopian future and dystopian future with this technology. There are going to be good and bad sides to it.”
But while Artificial Intelligence is “out in the wild” with very little regulation as it gathers pace, the US Food and Drug Administration has put robust safeguards around the brain-computer interfaces that are being trialled. “The way we’re dealing with AI is pretty much releasing it to the public and seeing what happens,” Oxley says. “We’re not doing that with neuro tech. There’s very strict regulation around how you get approval for a brain implant – Does it hurt you? Is it safe? What’s the communication protocol? How do you cover cybersecurity? How are you dealing with privacy? How do you stop hacking? So the FDA has done a really good job, it’s very complicated, but the ethical framework is kind of there. And there’s a process that we have to work through.”
In the medical field alone, Oxley believes the application for brain-computer interfaces could be very wide. Controlling robotic arms will be the next frontier for the technology, and the implants could even play a role in combating treatment-resistant depression. Oxley estimates that worldwide, around 5 million people could benefit from the technology.
“The corollary to this is the cracking of the genetic code,” Oxley says. “The genetic code was cracked about 20 years ago, when the genome was mapped. Now, there’s this incredible industry emerging out of understanding what the genetic code is made up of, and how to build medicines around that.
“This is the beginning of the cracking of the brain’s code. And once that has been understood, and opened up, there is going to be an enormous renaissance in brain therapies that weren’t previously considered possible.”