Introduction

Have you ever heard somebody playing a tune on a piano? It is likely that you can easily notice when a piano player hits an incorrect key, because it sounds weird. This is possible even if you are hearing the song for the first time, or even if you do not play the piano yourself. Also, you can probably notice when music arrives at a logical pausing point. You might compare it to the end of a sentence, or the end of a line in a poem. This means that you notice some sort of musical grammar, even without being aware of it.

Now imagine a child who has been deaf since birth or early childhood. Unaided, deaf children cannot hear a honking car horn or listen to music, and they cannot communicate with family and friends using spoken language. In most of these children, some hearing can be restored by inserting a device called a cochlear implant (CI) into the inner ear (Figure 1). This device triggers the nerve that connects the inner ear to the brain. Although the way the device transfers sounds is somewhat coarse, most CI users understand speech very well. Think about a blurred photo of your family, in which you can still recognize your parents.

We wanted to know if CI users sense musical harmony similarly to the way normal-hearing people sense it [1]. Music helps with skills including learning a language, reading, and understanding math. Thus, if CI users sense musical harmony in a similar way normal-hearing people do, it is likely that music can have the same positive effect on their development of these skills.

What Was Our Experiment?

To understand how CI users perceive sounds and how sound heard using a CI compares to normal hearing, we invited CI users and normal-hearing people to participate in our listening experiments. We designed these experiments to test certain aspects of how people hear sounds. In an earlier study, we investigated how music is heard by adults who grew up with normal hearing but lost their hearing later in life [2]. For these listeners, the music experience they gained before their hearing loss might affect how they sense music with a CI. In contrast, our 12 CI users were all born deaf or lost their hearing early, so they could not develop any musical knowledge before they received a CI. The CIs were inserted into their ears at about 1–2 years of age. We also tested 24 normal-hearing children to understand how well children performed our experimental tasks when they listened with their own ears.

In our experiments, we examined how the children with CIs and the normal-hearing children experienced musical chords. Musical chords are assembled from several tones played at the same time, like when certain piano keys are pressed together. Depending on which tones are combined, some chords give a pleasant and soothing (consonant) sensation. If you play a piano, you might know this impression from the combination C-E-G, a so-called major chord. It sounds particularly clear and solid. If you instead play C-E-G sharp, it sounds quite scraping and tense. Such chords generate an unpleasant and disturbing (dissonant) sensation.

Our first question was whether CI users and normal-hearing people prefer the same types of chords. Most normal-hearing people prefer pleasant/consonant chords to unpleasant/dissonant chords. We selected four of the most common chords. Two of them were pleasant/consonant chords (the first one was in fact the major chord). The other two chords were unpleasant/dissonant. Listen to these four chords in Video 1. The chord sounds were presented in pairs. After each presentation, our listeners had to choose the more pleasant sound from the pair.

Our second experiment tested whether CI users noticed a musically expected ending, the way you notice when a sentence ends. We selected eight different tunes known to the children in our country (Germany). You might know some of them, like “Happy Birthday” or Brahms’ “Lullaby.” The tunes were played in three variations: the original one (musically correct) and two modified versions, with the last chord exchanged for an unpleasant one. After each presentation, our listeners had to decide whether the ending of the tune was fine or odd-sounding.

What Did We Find?

Figure 2 shows the results of our first experiment. The normal-hearing children preferred the consonant chords. The two dissonant chords were less favored. This is what we expected, based on musical theory. Results from CI users look rather similar to those of the normal-hearing children. CI users preferred the same chords, consonant 1 and 2, over the two other, dissonant chords.

In the second experiment, our listeners had to decide if the song endings sounded fine or odd (Figure 3). The data show that normal-hearing children had no problem with the task—in fact, in most cases, their performances were close to perfect. Only our two youngest listeners showed less-than-perfect performance, but they were only 5 years old. In contrast, CI users could not distinguish between an ending that is odd and one that is expected. CI users also did not get better at this task with age.

What Do Our Results Mean?

The results of our first experiment show that the sound processed through a CI is precise enough to sense how good/consonant or bad/dissonant a chord sounds. The preferences of CI users are very similar to those of children with normal hearing. These results show that a CI should provide the information necessary to experience this aspect of music. However, in our second experiment, none of the CI users could distinguish an odd-sounding ending from a normal-sounding one, whereas nearly all the normal-hearing children did so with ease. As mentioned before, we did a similar study with grown-ups who had lost their hearing later on in life, and thus had the chance to listen to music with their own ears before receiving CIs [2]. These listeners also could not tell odd and normal-sounding endings apart from each other. We found the same result in our group of CI users who never had the chance to listen to music with their own ears. This result implies that distinguishing normal-sounding vs. odd-sounding song endings is a specific problem of listening through a CI and does not depend on having listened to music naturally in the past.

A group from France [3] used sung melodies with chords (some with odd endings and some with expected endings) and measured the time listeners needed to figure out what syllable the singer was pronouncing. In such a task, normal-hearing listeners usually react faster to expected endings. This is because people are usually slowed down by disturbances like unexpected sounds. Surprisingly, the researchers found that CI users, reacted faster when the last chord was a musically odd one. They concluded that the CI users recognized the difference between expected endings and odd endings, because their reaction times changed with the type of ending. The fact that CI users were faster when the last chord was musically odd suggested that they did not find the dissonances disturbing. Remember our initial comparison to looking at a blurred family photo? Imagine that the picture contains enough information to recognize the people in it, but it is too blurred to decide if they are happy or sad. In the same way, CI users miss a part of the emotion of the music.

What if children with CIs just need more time to learn? We did not find any tendency toward improvement in these tasks in older children. Nevertheless, we cannot rule out this possibility. Maybe specific listening training, focusing on aspects of music, could help them. This is a question that researchers are working on now.

Conclusion

In many children who are born deaf, CIs can restore their hearing well enough that they can understand speech. Our study shows that understanding music through a CI seems to be more challenging than understanding speech. The ability to tell whether a single musical chord sounds nice or weird, in the same way that normal-hearing people do, does not seem to be enough to allow a grasp of musical grammar. We do not believe that it is impossible for CI users to perceive this aspect of music. Focused music training may help CI users to learn how a normal song ending sounds. It is also possible that progress in CI technology will improve the way music is transmitted to CI users. For instance, researchers are trying to understand how the notes in a chord interact through a CI. It would be nice if future CI users could sense musical grammar the same way normal-hearing people do, because only then will they be able to grasp the musical “jokes” and false endings that Mozart and others loved to hide in their pieces!

Glossary

Cochlear Implant (ci): ↑ An electronic device that brings back hearing to deaf people. By surgery through the skull, it is positioned at the hearing nerve connecting the inner ear to the brain.

Harmony: ↑ The musical logic how notes are combined to chords and chords to meaningful music.

Chord: ↑ Combination of musical notes played at the same time.

Consonant: ↑ Sounding pleasant or soothing.

Dissonant: ↑ Sounding unpleasant, rough or disturbing.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.