Smart hearing aids can pick out a voice in crowded rooms

Brain-guided hearing aid technology can pick out one voice in a crowded room.

The technology mimics the brain’s natural ability to focus on a single voice when many are speaking. Picture: Suppleid

Researchers striving to build better hearing aids call it the cocktail party problem: how, in a crowded room, do you amplify the one voice to which you want to listen?

They may have come up with an answer: a hearing aid that monitors its user’s brainwaves to lock on to the conversation in which they are most interested.

The technology mimics the brain’s natural ability to focus on a single voice when many are speaking.

Brain-guided hearing aids will act as a filter, using artificial intelligence to monitor brain waves and boost one voice at a time. “The brain area that processes sound is extraordinarily sensitive and powerful; it can amplify one voice over others, seemingly effortlessly,” Nima Mesgarani, of Columbia University’s institute of mind brain behaviour, said.

The AI hearing aid prototype was able to replicate this ability by harnessing the power of the brain itself, she added. “Today’s hearing aids pale in comparison.”

Modern hearing aids are excellent at amplifying speech while suppressing certain types of background noise, such as traffic. However, they struggle to identify and boost the volume of an individual voice. “In crowded places hearing aids tend to amplify all speakers at once,” Ms Mesgarani said. “This severely hinders the wearer’s ability to converse, essentially isolating them from the people around them.”

Brain waves carry clues about which speaker the person is concentrating on. An AI neural network was used to separate out the voices of all those speaking in a group. It then compared the brain waves as the person listened to each speaker. The voice pattern that most closely matched the brain waves was then amplified over the rest.

An earlier version of the system, unveiled in 2017, had to be trained to recognise specific speakers. “If you’re in a restaurant with your family, that device would recognise and decode those voices for you,” Ms Mesgarani said. “But as soon as a new person, such as the waiter, arrived it would fail.”

The latest version has been upgraded so that it can handle voices that it has not heard before. So far it has been tested only by patients who have undergone brain surgery, because the prototype requires direct access to the brain. “These patients volunteered to listen to different speakers while we monitored their brain waves directly via electrodes implanted in their brains,” Ms Mesgarani said. “We then applied the newly developed algorithm to that data.”

When a patient focused on one speaker the algorithm amplified that voice, and when attention passed on to a different speaker the volume levels changed to reflect that shift.

The researchers are exploring how to transform their prototype into a device that can be placed externally on the scalp or around the ear.

The research was described in the journal Science Advances.

The Times