Searching is a Challenging Job for the Brain

Searching for objects or animals is a challenging task for your brain to perform. When you search for things, your brain directs the movement of your eyes toward objects that appear similar to the thing you are looking for. As you search for objects in everyday life, you may notice that you sometimes accidently spot items that look similar to your target. For example, if you are searching for your basset hound, you may accidently look at a similar-looking dog (like a beagle) or even a cat, because they share visual features with your dog (four legs and a tail). The visual features of objects can direct your eye movements. This allows you to find what you’re looking for quickly, because you do not waste as much time looking at things that are unlike your dog, such as a cat or other animals.

Your brain directs your eyes toward search targets with the help of mental representations, which are memories of the thing you’re looking for. Mental representations are created from your past experiences with those objects in the world, and the information is stored in your memory, like pictures stored on a phone. Because of your past experiences seeing dogs in real life or in photos, you can think about the shape and color of a typical dog without having one right in front of you. If you tried this right now, it would probably be easy to do—go ahead, picture a dog in your mind! When you need to find objects, you use these mental representations to create target templates, which are like pictures in your mind that guide your eyes toward the thing you’re looking for (For more details about target templates, see “Using Your Brain (Not Just Your Eyes) to Find Lost Objects” [1]). Using target templates, your brain knows to direct your eyes toward certain features of basset hounds (four legs, long ears, brown-and-white fur) and not the wrong features (wings or gray fur).

Sometimes we search for things that we do not have a clear target template for, meaning we are not quite sure what those things will look like until we find them. Imagine that you are looking for a bear. What kind of bear could this be? Perhaps a polar bear, or a small teddy bear? The specific features of bears can be different—polar bears have white fur, teddy bears are small. So, how are you supposed to know the exact features of the bear you are looking for? This is where the environment you are searching in can give you clues and provide a mental boost that can help you find the bear faster.

Using the Clues in Your Environment

Your experience with objects not only helps you get an idea of what those objects generally look like, but also which environments they are likely to be located in. Consider where you are more likely to find a polar bear: in a cold, snowy place like the Arctic tundra, or in a wooded forest with tall pine trees? Where would you expect to find a grizzly bear? You would expect polar bears to be in snowy environments, not in the forest, and grizzly bears to be in the forest, not the tundra. People are faster at identifying objects when the objects are in environments that they are expected to be in [2]. For example, if you were shown a scene of a snowy landscape and asked to identify the animal in it, you would probably find a polar bear much faster than if the scene were of an office containing a polar bear (Figure 1). The scene gives you a mental boost that helps you notice the bear more quickly. The mental boost you get in identifying objects quickly in their typical environments happens with the help of memories of objects that belong together, like polar bears and snow.

Scenes in Your Brain

In general, scenes are made of objects that often show up together in certain places. Think of a beach scene. You may imagine palm trees, sand, water, seashells, and crabs. These objects are strongly associated (that is, they belong together) and they are stored together in your memory as a special sort of joint mental representation called a scene schema. Over time, scene schemas are created in your brain from seeing scenes and noticing the objects that tend to show up in those scenes. Scene schemas allow you to understand which objects belong in certain environments. When you come across scenes throughout your life, scene schemas allow you to predict which objects are likely to be present in those scenes.

Scene schemas are created by connections between different parts of the brain. An important brain area associated with scene schemas is called the parahippocampal cortex (PHC). If you see two objects that belong together, meaning they are expected to be in the same scene, then brain cells in your PHC are more active compared to when you see two objects that do not belong together [3]. For example, when you look at a palm tree and a coconut, the PHC is very active, and the brain cells there use lots of energy (think of them as “hot”); but those brain cells are less active when you view two unrelated things, like a palm tree and a basketball (think of them as “cold”) (Figure 2).

When you look at a beach scene, the PHC cells representing your scene schema for a beach are active in your brain. This includes brain cells representing the objects expected to be present on a beach, even if those objects are not actually present in front of you. For example, even if there are no seashells in your beach scene, the idea of seashells is still active in the memory parts of your brain. This is because seashells are linked together with other beach scene objects in the brain by the scene schema. It’s as if your brain is anticipating what it will see and preparing the parts of the brain needed to notice those objects if they happen to show up.

Using Scene Schemas for Unclear Targets

When an object you are viewing belongs in the scene, you get a mental boost that allows you to identify it faster. It’s likely that the same mental boost you get from a scene schema helps you find related objects more quickly in the scene. When searching for something like a bear, when you do not know the bear’s exact appearance the scene might give you clues about what that bear looks like. Clues like snow on the ground (rather than a bed on a floor) tell you that the bear is more likely to be large and have white fur and is less likely to be small and cuddly. Clues from the scene help you to find what you’re looking for more quickly.

In our experiments, we asked whether scenes could help us find ambiguous (unclear) targets faster. We wondered if scenes could help us look for items when we do not know precisely what they look like. To answer this question, we had people use a computer to perform the following experiment (Figure 3). First, participants were told that they would be looking for a picture from a certain category (trees, for example). Then they saw a picture of a scene for <1 s (that’s quick!). After the picture, they searched for the object among other, unrelated objects. Sometimes the scene was related to the objects they found, and sometimes it was not related. For example, when looking for a tree, subjects may view a mountain scene and would later find a pine tree (a picture that belongs to the scene). In other cases, people would search for a tree, see a mountain scene, and then find a palm tree (a picture that does not belong to the scene).

We were interested in how the clues in the environment quickly moved people’s eyes toward the object they were looking for. If people are using a useful target template, their eyes would move to the object more quickly than if they did not have a useful target template. Eye tracking technology allowed us to examine eye movements by measuring how quickly people’s eyes moved toward objects they are looking for (More info can be found in “Eye spy: why we need to move our eyes to gather information about the world” [4]). We predicted that people would have faster eye movements to the object they were looking for if the object belonged to the scene they saw—and that is precisely what we found! People were faster to put their eyes on target objects when those objects were related to the scene they saw for <1 s, compared to targets that were unrelated to the scenes they viewed. When the target picture and the scene were related, people were faster to find the object [5]!

Fast Search is Useful!

The results of our study show that the environment you search within can supply important clues that give you a mental boost that improves your search behavior. This boost helps you to predict what the object you are searching for looks like. Prediction of the object’s appearance helps your brain to create a very useful target template that directs your eyes more quickly to what you’re looking for. This process is important because it makes sure that your everyday searches are efficient, so you do not waste your time. Imagine, for instance, you and your friend are playing a game of soccer at a soccer field and your friend says, “Go grab the ball.” You certainly would not waste your time looking for something small and green (a tennis ball) or large and orange (a basketball). You would expect to find a soccer ball, because of the environment you’re in, and I am sure all the players would be grateful that you could quickly find what you’re looking for!

Glossary

Mental Representation: ↑ The way the mind stores information about an object, such as an image of the object or knowledge of its features.

Target Template: ↑ A mental representation of a target object that can be used to guide attention during visual search.

Scene Schema: ↑ A mental representation that includes objects expected to be in a particular scene

Parahippocampal Cortex: ↑ A brain structure involved in processing associations between scenes and objects.

Disclaimer

Any opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.