Core Concept Neuroscience and Psychology Published: July 2, 2019

Using Your Brain (Not Just Your Eyes) to Find Lost Objects

Abstract

Think about how often you search your messy room for lost objects, or search the fridge for a snack. Visual search is the process of finding things like keys, lost toys, clothes, and anything else you may need to find. It is a complex mental task we perform countless times throughout each day. It seems relatively simple for us to find someone (like your friend in the hall at school), or to find places and things (like your favorite store at the mall, or your parents' car in the parking lot). But this mental task is a lot more complex than it seems. In this article, you will learn that visual search is more than just moving your eyes to the thing you are searching for. As you will come to find out, your brain is even more important than your eyes in helping you find lost objects.

Finding Your Shoe With Visual Search

It is easy to assume that the eyes are the most important part of searching—eyes are important for seeing, of course! But your eyes are only one important part of the process called visual search. In truth, your brain does most of the work to help you find objects in places like your bedroom, your classroom, or the world outside. The ability to find lost objects involves cooperation between the parts of the brain that process what you see, those that store what you remember, and areas that control how you move your eyes.

Pay Attention!

You have done it again—you have lost one of your shoes! How can you find the shoe in your messy room? Consider everything you could possibly see in your room: objects, such as the bed, a lamp, clothes on the floor, gadgets on the nightstand, and posters on the walls. But that is not all you are seeing! Before you can truly recognize objects, your brain first sees the features (or qualities) of those objects, like their color, shape, size, and texture. The poster on your wall is not just an object you look at; it is made up of splashes of colors and shapes against a painted background. The quilt on your bed is a collection of squares with lots of colorful patterns and fuzzy textures. When you are looking around at all of these features, your brain connects them together like puzzle pieces, so you can figure out what you are looking at [1].

That is a lot of visual information to sort through, and we cannot pay attention to everything at the same time. So how does the brain know how to select the important stuff? That is where selective attention comes in. Selective attention works like a mental spotlight, to help important visual information be more obvious or defined. Think about this: as you are reading this sentence, the other words on the page are still being processed by your eyes. That is, you can still “see” the other words, but somehow you are able to focus only on the words in this sentence in detail. When reading, and when performing a visual search, you are moving that mental spotlight around to objects (or words!) that are important to the task at hand.

Visual selective attention affects the way that the brain processes (that is, makes sense of) the visual information that is sent from the eyes and the visual cortex is one part of the brain that does this processing. In Figure 1, you see the regions of the visual cortex that process different types of object features. For example, V3 is a part of the visual cortex involved in processing shape information about the objects you see, and parts of V4 are involved in helping you see color. Think of neurons (brain cells that communicate to each other) like light switches that can be turned on or off. These regions of the visual cortex are collections of neurons that are only turned on when you are looking at specific features of objects. For example, some neurons turn on when you look at a red teddy bear but not when you look at a blue teddy bear. There are neurons that are turned on when you see a vertical line on a page, and other neurons are active when that same line is tilted horizontally. When you pay attention to an object, visual attention increases the activity in the V4 neurons that are connected to features of the object you are paying attention to [2].

Figure 1 - The visual cortex.
  • Figure 1 - The visual cortex.
  • The visual cortex is located in the back of the brain. Each layer (V1 through V4) is labeled with a different color. The different areas perform different functions related to vision, as you can see in the colored boxes. The blue area at the front of the brain is the prefrontal cortex (PFC), which keeps the target template in mind and helps a searcher focus.

Making Searching Fast and Efficient

Imagine that you lost a red shoe. As you search your room for it, you may notice that you tend to look at objects that are the same color as your shoe. That is because your attention is more likely to fall on objects that look like the object you are searching for. Attention is guided by features in the world that are shared with the object you are looking for. This attentional guidance makes your search easy and efficient. Imagine if you were looking for your red shoe without attentional guidance. You would then have to look at every single object in the room! But thanks to attentional guidance, you do not waste time looking at blue toys, green socks, or a black backpack. You only pay attention to features that are related to your shoe—red things, or shoe-shaped things (Figure 2).

Figure 2 - Attentional guidance.
  • Figure 2 - Attentional guidance.
  • When searching for a red shoe, your attention is guided to objects that have features similar to the object you are searching for. Here, someone is searching for his red shoe in a messy room. There are other objects that are red, so attentional guidance makes it likely that he will also look at those red objects as he searches for his shoe.

Attentional guidance in the process of searching for objects gets help from target templates. Target templates are mental pictures that help direct attention to areas of the room where there are objects sharing features with the thing you are searching for (i.e., your target). As you learn about objects in the world, you are able to store those objects in memory, like storing pictures on your phone. For example, think of your favorite shoes. Think about the shape and the color of those shoes. You do not need to have those shoes right in front of you in order to think about their features, because you have a mental representation in memory of those shoes. In visual search, when you are looking for your red shoe, the target template in your memory pushes your attention (and even your eyes!) to objects that are red and shoe-shaped.

When you are using a target template to help you search, neurons are turned on in the visual cortex as well as in the prefrontal cortex (Figure 1). There are areas of the visual cortex that are active for each feature of your target object, such as an area active for “red” and another area for “shoe shape.” The PFC is the control center that helps you keep the target template in your mind and maintain focus on the search task [3]. The PFC helps keep the neurons connected to the target template active, either until you have found what you are looking for or until you give up and move on to doing something else.

Searching for Things You Have Not Seen Before

Visual search is easier when you know the exact appearance of what you are searching for. So, how do you find an object when you do not know exactly what it looks like? Imagine you are at a park and a friend asks you to help him find his new dog. You have never seen this dog before, so how do you help your friend find his lost pup? You likely have seen many dogs before, maybe you even have a pet dog yourself. From your past experience with dogs, you know what the typical features of a dog are (four legs, a tail, fur, etc.) and you can use these features to help guide your search. If your friend had a dog that is common, like a Golden Retriever or a Labrador, you will be quicker to find that dog than if you were looking for an uncommon, dog like a Chinese Crested Dog, or a smaller-than-average Boxer [4] (Figure 3).

Figure 3 - Finding objects you have not seen before.
  • Figure 3 - Finding objects you have not seen before.
  • In the left image, this man is searching for any dog, so his search template would likely contain features matching a typical dog, like a Golden Retriever. In the right image, the man is slow to find the unusual dog, because it did not match his search template (that is, it was smaller, less furry, and a different color than what he had in mind).

Searching for an unfamiliar dog is an example of categorical search. Throughout the day, you may often find yourself looking for items from a category of objects, and in those situations, you do not know the exact appearance of the object until you find it. For example, think of a time when you could not find your pencil in your backpack. You may have looked around the classroom or on the teacher's desk for any writing utensil to borrow. You were probably quick to find a pencil, because your experience with writing utensils tells you that they tend to be long, thin, and cylindrical in shape. This information helps you form a useful target template that guides your search quickly to a pencil on a desk. The point is, you are so good at finding things that you do not need to know the exact appearance of an object (like a writing utensil) in order to find it quickly.

Conclusion: Visual Search is Much More Than Just What You See

Trying to find objects in your home or out in the world is far more complicated that most people realize. The act of visual search relies on teamwork from many different brain areas. It also relies on your experience with the object you are searching for, whether it is with the exact object (e.g., looking for your specific dog), or one similar to it that simply belongs to the same category (e.g., looking for any dog). Your experience with the object or category you are searching for helps you develop a target template—a useful mental picture of the thing you are trying to find. The target template is useful in helping guide your attention (and your eyes!) to the object you are searching for. The next time you look for a lost shoe, take a moment to think about and appreciate all the work that your brain does to help you find what you need!

Glossary

Visual Search: The mental task of looking for an object.

Features: The specific visual pieces or qualities of an object, like color, shape, orientation, and size.

Selective Attention: Focusing on something so you can get detailed information about it. Think of it like a mental spotlight.

Visual Cortex: The part of the brain that is involved with processing visual information and an object's features.

Attentional Guidance: The process of moving attention around to important features of an object.

Target Templates: Mental images or representations of target objects that are used to guide attention during visual search.

Categorical Search: A form of visual search where you are looking for any object from a specific category.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


References

[1] Martin, C., Fidalgo, C., and Barense, M. 2017. Knowing what we see. Front. Young Minds 5:15. doi: 10.3389/frym.2017.00015

[2] Maunsell, J. H., and Treue, S. 2006. Feature-based attention in visual cortex. Trends Neurosci. 29:317–22. doi: 10.1016/j.tins.2006.04.001

[3] Soto, D., Humphreys, G. W., and Rotshtein, P. 2007. Dissociating the neural mechanisms of memory-based guidance of visual selection. Proc. Natl. Acad. Sci. U.S.A. 104:17186–91. doi: 10.1073/pnas.0703706104

[4] Robbins, A., and Hout, M. C. 2015. Categorical target templates: typical category members are found and identified quickly during word-cued search. Vis. Cogn. 23:817–21. doi: 10.1080/13506285.2015.1093247