A Broken Model of the World

In your head, you carry around a model of the world. This model has been built using all the information your brain has gathered from your senses. It tells you where things are, which things are dangerous or desirable, who is shy, and who likes to show off. The model is continuously updated and improved by new information that pours in from your eyes, ears, nose, and mouth, and from sensors all around your body monitoring touch and temperature. Your brain loves information and is always hungrily searching for more.

But what happens when the information stops flowing from a sense because it is too dark to see or too noisy to hear? In this case, the brain fills in the missing information by focusing harder on other senses. For example, if you are trying to follow a conversation but you cannot get enough information from your ears, you focus more on the movement of the speaker’s lips. If you are walking down a dark street and think you glimpse someone lurking in the shadows, you listen all the more closely for footsteps.

Unfortunately, your brain cannot always get the information it needs by focusing harder on other senses. If you are in a noisy hall, where clattering and chattering completely cover the voice you are trying to hear, you cannot get all the information you need just by focusing more closely on the person’s lips. For many people, this difficulty is not a temporary one that ends when the background noise fades or the light is switched on. Some people’s brains are missing information not because it is too dark or too noisy, but because a sense is not working properly. Ingenious devices, like glasses and hearing aids, have been invented to solve this problem, but sometimes they are unable to mend the broken sense. How, then, do we give the brain the information it needs? We already know that the brain uses other senses to collect missing information. Perhaps we can send the missing information through another sense.

Listening With Your Wrists

Many people who have damaged hearing struggle to work out where sounds are coming from. This can cause a lot of problems. For example, it is hard to get out of the way of a truck that you suddenly hear hurtling toward you if you do not know where it is coming from! It is also hard for your brain to separate sounds that are coming from different directions, like the voice of the person in front of you and the music blaring from the speaker to your left.

Together with a team of researchers, I am currently investigating whether vibration on the wrists can be used to help people work out where sounds are coming from. When the ears are working well, your brain can pinpoint where a sound is by comparing how loud it is at each ear. As shown in Figure 1, if a sound is to your right, the soundwave travels through the air directly into your right ear. But to reach your left ear, the soundwave needs to first get past your head. Sound gets quieter when it is blocked by your head, just as sound gets quieter when you block it by closing a door. This means that your brain can use a simple rule to work out where a sound is: if it is louder in your left ear then it must be to your left, and if it is louder in your right ear then it must be to your right. This is one of the main ways your brain works out where a sound is, and it is this rule that we took advantage of.

We converted sound into vibration, which we delivered using wristbands that silently buzzed against the skin. Sounds heard by the left ear were converted into vibration on the left wrist, and sounds heard by the right ear were converted into vibration on the right wrist. This meant that—just like for the ears—if a sound was to the right, the vibration was stronger on the right wrist, and if a sound was to the left, the vibration was stronger on the left wrist. We hoped that, by using the same rule that it uses for hearing, the brain would be able to work out where sounds are.

Our Experiment

Our volunteers all had difficulties hearing and used cochlear implants, which are a type of surgically fitted hearing aid. They sat in a chair with speakers in a semicircle around them and with vibrating wristbands on each wrist (Figure 2A). We tested how well the volunteers could work out where sounds were coming from when using either only their ears, only the wristbands, or their ears together with the wristbands [1]. We played a sound from a speaker, then asked the volunteer which speaker the sound came from. We calculated how well they located the sound by measuring how far their answer was from the correct speaker. We measured this distance in degrees, as the angle between the correct speaker and the one they chose. We repeated this process over and over and calculated an average of all the scores. We were very pleased with what we found!

Did it Work?

Figure 2B shows the results. When using only their ears, the sound locations that our volunteers identified tended to be a long way from the correct sound location. However, when they used either the vibrating wristbands alone or their ears together with the wristbands, they tended to be much closer to the correct location. Interestingly, our volunteers performed best when they used the wristbands and their ears together. This is good news, as it suggests that the brain is happy to combine information from vibration on the wrists with information from sound at the ears.

The finding that the wristbands can hugely improve people’s ability to locate sounds is especially exciting because these improvements were made after hardly any practice. We have since shown that, when people train for half an hour per day over 10 days, they keep on getting better at locating sounds [2]. Who knows how good they might get if they use the wristbands every day for months or even years?

A New Kind of Hearing Aid?

Besides helping people with hearing problems locate sounds better, we have been trying to improve other aspects of their hearing by sending missing sound-information through vibration on the wrists. For example, we have recently shown that vibration can help people with cochlear implants understand speech better when there is a lot of background noise [3, 4]. This is a common problem in places like busy classrooms, factories, and offices.

So far, we have only shown that this approach can help people when they are tested in the lab. Now, we are looking to create a device that can help people in their daily lives. We are building a new wristband (Figure 3), similar to a Smartwatch or a Fitbit, that people can wear outside the lab, as they go about their day [6]. We are working with one of the world’s biggest producers of hearing aids and cochlear implants so that our wristband can connect wirelessly to their hearing devices to collect the sound at each ear. We are also developing our own small devices that collect sound at the ears for those people who do not already wear hearing devices.

If our wristbands work outside of the lab, they could help people across the world. They might be especially useful in poorer countries, where hearing loss is left untreated for many millions of people. In India, for example—a country of well over a billion people—less than a third of children with hearing problems go to school [7]. Adults with hearing problems in poorer countries are also much less likely to get jobs and so are often forced to live in poverty. Devices like cochlear implants are too expensive for most people, and poor countries lack the doctors and hospital equipment needed to fit them. The wristbands we are developing could overcome these problems. They can be produced very cheaply and can be fitted without the need for highly qualified doctors or expensive medical equipment. They could dramatically improve the job opportunities, education, and social lives of many millions of people with hearing problems. We are working as hard as we possibly can to make this happen.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Thank you to the people that I had the great fortune to work alongside for the research described in this article: Robyn Cunningham, Sean Mills, Tobi Goehring, Sam Perry, Jana Zgheib, and Ama Hadeedi. My deepest thanks also to André Brokmaan, Hans Fleschenberg, Sando, Helen Fletcher, Jana Zleb, Frederique Vee, Toby Wells, and Davison Fletcher for kindly reading this article and giving extremely useful feedback. Funding for the author’s salary was provided by the William Demant Foundation.