Finding Your Shoe With Visual Search

It is easy to assume that the eyes are the most important part of searching—eyes are important for seeing, of course! But your eyes are only one important part of the process called visual search. In truth, your brain does most of the work to help you find objects in places like your bedroom, your classroom, or the world outside. The ability to find lost objects involves cooperation between the parts of the brain that process what you see, those that store what you remember, and areas that control how you move your eyes.

Pay Attention!

You have done it again—you have lost one of your shoes! How can you find the shoe in your messy room? Consider everything you could possibly see in your room: objects, such as the bed, a lamp, clothes on the floor, gadgets on the nightstand, and posters on the walls. But that is not all you are seeing! Before you can truly recognize objects, your brain first sees the features (or qualities) of those objects, like their color, shape, size, and texture. The poster on your wall is not just an object you look at; it is made up of splashes of colors and shapes against a painted background. The quilt on your bed is a collection of squares with lots of colorful patterns and fuzzy textures. When you are looking around at all of these features, your brain connects them together like puzzle pieces, so you can figure out what you are looking at [1].

That is a lot of visual information to sort through, and we cannot pay attention to everything at the same time. So how does the brain know how to select the important stuff? That is where selective attention comes in. Selective attention works like a mental spotlight, to help important visual information be more obvious or defined. Think about this: as you are reading this sentence, the other words on the page are still being processed by your eyes. That is, you can still “see” the other words, but somehow you are able to focus only on the words in this sentence in detail. When reading, and when performing a visual search, you are moving that mental spotlight around to objects (or words!) that are important to the task at hand.

Visual selective attention affects the way that the brain processes (that is, makes sense of) the visual information that is sent from the eyes and the visual cortex is one part of the brain that does this processing. In Figure 1, you see the regions of the visual cortex that process different types of object features. For example, V3 is a part of the visual cortex involved in processing shape information about the objects you see, and parts of V4 are involved in helping you see color. Think of neurons (brain cells that communicate to each other) like light switches that can be turned on or off. These regions of the visual cortex are collections of neurons that are only turned on when you are looking at specific features of objects. For example, some neurons turn on when you look at a red teddy bear but not when you look at a blue teddy bear. There are neurons that are turned on when you see a vertical line on a page, and other neurons are active when that same line is tilted horizontally. When you pay attention to an object, visual attention increases the activity in the V4 neurons that are connected to features of the object you are paying attention to [2].

Making Searching Fast and Efficient

Imagine that you lost a red shoe. As you search your room for it, you may notice that you tend to look at objects that are the same color as your shoe. That is because your attention is more likely to fall on objects that look like the object you are searching for. Attention is guided by features in the world that are shared with the object you are looking for. This attentional guidance makes your search easy and efficient. Imagine if you were looking for your red shoe without attentional guidance. You would then have to look at every single object in the room! But thanks to attentional guidance, you do not waste time looking at blue toys, green socks, or a black backpack. You only pay attention to features that are related to your shoe—red things, or shoe-shaped things (Figure 2).

Attentional guidance in the process of searching for objects gets help from target templates. Target templates are mental pictures that help direct attention to areas of the room where there are objects sharing features with the thing you are searching for (i.e., your target). As you learn about objects in the world, you are able to store those objects in memory, like storing pictures on your phone. For example, think of your favorite shoes. Think about the shape and the color of those shoes. You do not need to have those shoes right in front of you in order to think about their features, because you have a mental representation in memory of those shoes. In visual search, when you are looking for your red shoe, the target template in your memory pushes your attention (and even your eyes!) to objects that are red and shoe-shaped.

When you are using a target template to help you search, neurons are turned on in the visual cortex as well as in the prefrontal cortex (Figure 1). There are areas of the visual cortex that are active for each feature of your target object, such as an area active for “red” and another area for “shoe shape.” The PFC is the control center that helps you keep the target template in your mind and maintain focus on the search task [3]. The PFC helps keep the neurons connected to the target template active, either until you have found what you are looking for or until you give up and move on to doing something else.

Searching for Things You Have Not Seen Before

Visual search is easier when you know the exact appearance of what you are searching for. So, how do you find an object when you do not know exactly what it looks like? Imagine you are at a park and a friend asks you to help him find his new dog. You have never seen this dog before, so how do you help your friend find his lost pup? You likely have seen many dogs before, maybe you even have a pet dog yourself. From your past experience with dogs, you know what the typical features of a dog are (four legs, a tail, fur, etc.) and you can use these features to help guide your search. If your friend had a dog that is common, like a Golden Retriever or a Labrador, you will be quicker to find that dog than if you were looking for an uncommon, dog like a Chinese Crested Dog, or a smaller-than-average Boxer [4] (Figure 3).

Searching for an unfamiliar dog is an example of categorical search. Throughout the day, you may often find yourself looking for items from a category of objects, and in those situations, you do not know the exact appearance of the object until you find it. For example, think of a time when you could not find your pencil in your backpack. You may have looked around the classroom or on the teacher's desk for any writing utensil to borrow. You were probably quick to find a pencil, because your experience with writing utensils tells you that they tend to be long, thin, and cylindrical in shape. This information helps you form a useful target template that guides your search quickly to a pencil on a desk. The point is, you are so good at finding things that you do not need to know the exact appearance of an object (like a writing utensil) in order to find it quickly.

Conclusion: Visual Search is Much More Than Just What You See

Trying to find objects in your home or out in the world is far more complicated that most people realize. The act of visual search relies on teamwork from many different brain areas. It also relies on your experience with the object you are searching for, whether it is with the exact object (e.g., looking for your specific dog), or one similar to it that simply belongs to the same category (e.g., looking for any dog). Your experience with the object or category you are searching for helps you develop a target template—a useful mental picture of the thing you are trying to find. The target template is useful in helping guide your attention (and your eyes!) to the object you are searching for. The next time you look for a lost shoe, take a moment to think about and appreciate all the work that your brain does to help you find what you need!

Glossary

Visual Search: ↑ The mental task of looking for an object.

Features: ↑ The specific visual pieces or qualities of an object, like color, shape, orientation, and size.

Selective Attention: ↑ Focusing on something so you can get detailed information about it. Think of it like a mental spotlight.

Visual Cortex: ↑ The part of the brain that is involved with processing visual information and an object's features.

Attentional Guidance: ↑ The process of moving attention around to important features of an object.

Target Templates: ↑ Mental images or representations of target objects that are used to guide attention during visual search.

Categorical Search: ↑ A form of visual search where you are looking for any object from a specific category.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.