Abstract
Have you ever tried to remember a word in a foreign language? What strategy did you use? In several studies, we examined the beneficial effects of viewing pictures and performing gestures while learning foreign language words. Both pictures and gestures helped primary school kids and adults to better remember the meanings of foreign language words compared to learning by just listening. For kids, pictures and gestures were equally helpful. For adults, gestures were more helpful than pictures. Both visual and motor brain areas helped with learning the foreign language words. Our studies suggest that learning foreign language words with pictures and gestures is helpful for learners, because pictures and gestures allow both kids and adults to experience the meanings of words through multiple senses.
How Do We Learn Vocabulary in a Foreign Language?
Languages are important because they allow us to communicate with one another. People living on Earth today speak over 6,000 different languages [1]. Each of those languages has tens of thousands of words, or vocabulary, that refer to objects in the environment, people, places, feelings, and thoughts. Since you are reading this article, which is written in the English language, English may be your native language (L1)—the language that you started to learn at birth. You may also have learned English at school, from teachers or books, or by hearing English words outside of school. If this is the case, then you probably learned English as a foreign language (L2). One of the most important steps for learning a new language is to learn the vocabulary of that language. This takes a lot of time and practice.
To learn an L2 word, we must hear how the word is spoken or see how it is written and learn the meaning of that word. Kids and adults use many strategies to learn L2 words. They might, for example, listen to audio recordings or study word lists. Recent research suggests that such techniques are less effective than strategies that use enrichment [2]. Enrichment refers to information presented during learning that allows us to experience the meaning of a word through multiple senses [3]. Instead of learning an L2 word by just listening to it, for example, we could see a related picture while listening to it. This happens when reading picture books and when learning vocabulary with picture cards. Another enrichment strategy could be to perform gestures that display the meaning of a word while listening to it. The word airplane, for example, could be displayed by moving our arms through the air as if they were wings.
Viewing pictures while listening to L2 words is a form of multisensory enrichment, because this technique uses information from multiple senses—seeing and hearing. Performing gestures while listening to L2 words is a form of sensorimotor enrichment, because this technique not only uses information from the senses, but also body movements. We tested which type of enrichment helped L2 learning the most [3, 4], and how the brain supported the L2 learning [3, 5, 6]. Both adults and kids were taught L2 vocabulary using three different methods: listening to the vocabulary while viewing pictures (multisensory enrichment), listening to the vocabulary while performing gestures (sensorimotor enrichment), and just listening to the vocabulary (no enrichment). Our hypothesis was that seeing pictures and performing gestures during learning would help kids and adults learn better than learning only by listening.
Do Pictures and Gestures Help Adults Learn Foreign Language Words?
We first tested our hypothesis in young adults [3]. Twenty-two adults heard L2 words and their L1 translations during 5 days of training. The adults were taught words that they had never heard before, such as diwume and giketa. A complete list of the words that the adults were taught can be found here. Some words were paired with pictures (Figure 1). For example, when the adults heard the foreign word that meant tent, they also saw a drawing of a tent. Other words were paired with videos of an actress performing gestures. For example, a video of an actress drinking from an imaginary bottle was paired with the word that meant bottle. The adults performed the gesture along with the actress. The rest of the words were learned only by listening to each L2 word and its L1 translation.
Vocabulary tests were completed 8 days, 2 months, and 6 months after learning. In one of the tests, the adults received a list of all the L1 words and wrote down their L2 translations. In another test, they received a list of all the L2 words and wrote down their L1 translations. We added up the test scores. We found that adults had higher test scores for words learned with both pictures and gestures compared to no enrichment, and that these benefits were still present after 6 months [3]. We also found that pictures and gestures were equally helpful in the short-term (at 8 days and 2 months after learning). However, over the long-term (6 months after learning), learning with gestures was even more helpful than learning with pictures (Figure 2).
What About Kids?
We next tested whether gesture enrichment would also help kids [4]. Ninety-seven 8-years-old German school kids learned English L2 words over 5 days. They learned the words using pictures, gestures, or no enrichment (Figure 1). The kids were taught English words that they had never seen or heard before in their English courses. The kids completed the same vocabulary tests as adults at 8 days, 2 months, and 6 months after learning. The kids gave their answers by speaking rather than writing.
We found that kids had higher test scores for words learned with both pictures and gestures compared to no enrichment at 8 days, 2 months, and 6 months after learning. Like the adults, pictures and gestures were equally helpful in the short-term (8 days and 2 months after the start of learning). However, unlike the adults, the kids’ test scores following gesture- and picture-enriched learning were equivalent 6 months after learning (Figure 2). This result suggests that gestures and pictures were equally helpful for kids’ L2 learning. The kids’ scores were lower overall. This could be because the kids received less training than the adults.
What Brain Areas Are Involved in Learning Foreign Words?
Our next step was to try to understand how multisensory and sensorimotor enrichment helped L2 vocabulary learning. To help answer this question, we turned to the brain. We know that seeing other people move can produce responses in a brain area called the biological motion superior temporal sulcus (bmSTS) [7], and that performing movements can produce responses in a brain area called the motor cortex [8]. We hypothesized that the bmSTS and the motor cortex would respond more when kids and adults heard gesture-enriched L2 words compared to picture-enriched L2 words. We made a similar prediction for the L2 words learned using pictures: we predicted that a visual brain region called the lateral occipital complex (LOC) would respond more when kids and adults heard picture-enriched L2 words compared to non-enriched L2 words.
So far, we have tested these hypotheses in adults [3]. To see which areas of their brains were active, 22 adults completed a brain scan after 5 days of L2 vocabulary learning. More information on how a brain scan measures brain activity can be found in this Young Minds Article [9]. We examined responses within the bmSTS, motor cortex, and LOC while the adults heard and translated the L2 words (Figure 3). We found that responses in the LOC told us if a word was learned with pictures, and responses in the bmSTS and motor cortex told us if a word was learned with gestures. These results tell us that specific brain responses are linked to the helpful effects of picture and gesture enrichment.
In scientific research, one method is usually not enough to prove that a conclusion is correct or not. The reason for this is that all methods have specific strengths and weaknesses. We therefore also examined whether the bmSTS and the motor cortex caused the benefits of L2 enrichment using a method called transcranial magnetic stimulation (TMS) [5, 6]. During TMS, small magnetic signals can affect brain activity and cause changes in behavior. We found using TMS that the bmSTS and the motor cortex helped adults to translate words learned with gestures.
What Do Our Findings Mean?
Learning enrichment, with both pictures and gestures, helped kids and adults learn foreign language vocabulary. However, adults benefitted most from gesture enrichment, while kids benefitted equally from picture and gesture enrichment. This means that the types of enrichment that work for adults might not necessarily work for kids. In our studies, kids and adults received different amounts of training; future studies may investigate how different amounts of training may improve enrichment effects. We also found that the brain uses its visual and motor areas for remembering the translations of enriched L2 words. This means that enrichment teaching strategies may work because a network of visual and motor brain regions contributes to enhanced learning outcomes. In sum, enrichment benefits L2 learning because it allows us to experience the meanings of words with our own senses.
Glossary
Vocabulary: ↑ The set of words used in a language.
Native Language (L1): ↑ A language that a person has been exposed to and started learning from birth.
Foreign Language (L2): ↑ A language spoken mostly by people in another area of the world than the speaker.
Enrichment: ↑ The presence of additional, complementary information during learning that helps to illustrate the meaning of a foreign language word.
Hypotheses: ↑ Assumptions that can be tested by carrying out scientific experiments.
Biological Motion Superior Temporal Sulcus (bmSTS): ↑ A visual area of the brain that responds when people see body movements.
Motor Cortex: ↑ The part of the brain that can initiate movements by controlling the muscles.
Lateral Occipital Complex (LOC): ↑ A visual area of the brain that responds when people see objects.
Transcranial Magnetic Stimulation (TMS): ↑ A neuroscience method in which the brain is affected by small magnetic signals.
Author Contributions
BM wrote an initial draft of the manuscript. CA, KMM, LS, AK, GH, MM, and KvK contributed to the writing of the manuscript.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
The authors would like to thank those who assisted in the translation of the articles in this Collection to make them more accessible to kids outside English-speaking countries, and for the Jacobs Foundation for providing the funds necessary to translate the articles. For this article, they would especially like to thank Nienke van Atteveldt and Sabine Peters for the Dutch translation. This work was funded by the German Research Foundation grant KR 3735/3-1, a Schulbezogene Forschung grant from the Saxony Zentrum für Lehrerbildung und Schulforschung (ZLS), and an Erasmus Mundus Postdoctoral Fellowship in Auditory Cognitive Neuroscience. BM is also supported by the European Research Council Consolidator Grant SENSOCOM 647051 to KvK.
References
[1] ↑ Graddol, D. 2004. The future of language. Science 303:1329–31. doi: 10.1126/science.1096546
[2] ↑ Repetto, C., Pedroli, E., and Macedonia, M. 2017. Enrichment effects of gestures and pictures on abstract words in a second language. Front Psychol. 8:2136. doi: 10.3389/fpsyg.2017.02136
[3] ↑ Mayer, K. M., Yildiz, I. B., Macedonia, M., and von Kriegstein, K. 2015. Visual and motor cortices differentially support the translation of foreign language words. Curr. Biol. 25:530–5. doi: 10.1016/j.cub.2014.11.068
[4] ↑ Andrä, C., Mathias, B., Schwager, A., Macedonia, M., and von Kriegstein, K. 2020. Learning foreign language vocabulary with gestures and pictures enhances vocabulary memory for several months post-learning in eight-year-old school children. Educ. Psychol. Rev. 1–36. doi: 10.1007/s10648-020-09527-z
[5] ↑ Mathias, B., Sureth, L., Hartwigsen, G., Macedonia, M., Mayer, K. M., and von Kriegstein, K. 2019. A causal role of sensory cortices in behavioral benefits of ‘learning by doing’. arXiv 1903.04201.
[6] ↑ Mathias, B., Klingebiel, A., Hartwigsen, G., Sureth, L., Macedonia, M., Mayer, K. M., et al. 2020. Motor cortex causally contributes to auditory word recognition following sensorimotor-enriched vocabulary training. arXiv 2005.08956.
[7] ↑ Grossman, E., Donnelly, M., Price, R., Pickens, D., Morgan, V., Neighbor, G., et al. 2000. Brain areas involved in perception of biological motion. J. Cogn. Neurosci. 12:711–20. doi: 10.1162/089892900562417
[8] ↑ Leonardo, M., Fieldman, J., Sadato, N., Campbell, G., Ibañez, V., Cohen, L., et al. 1995. A functional magnetic resonance imaging study of cortical regions associated with motor task execution and motor ideation in humans. Hum. Brain Mapp. 3:83–92. doi: 10.1002/hbm.460030205
[9] ↑ Hoyos, P., Kim, N., and Kastner, S., 2019. How is magnetic resonance imaging used to learn about the brain? Front. Young Minds. 7:86. doi: 10.3389/frym.2019.00086