Making VR Feel Real

Virtual reality (VR) environments can be as small as the cockpit of an airplane or as large as an entire virtual world. These environments are designed to be as realistic as possible. Immersion refers to how well technology can simulate the ways we sense and perceive the world in our everyday life. VR is considered immersive when our experience in a virtual world is similar to our experience in the real world. In the real world, for example, you can walk or run at different speeds. If you were in a virtual world and you could only move at one speed, then the virtual world would not be immersive because your experience in VR would not match up with your experience of the real world, where you can walk or run at varying speeds. The technology behind VR is designed to make us feel like as if we have left the place we are standing and have been transported somewhere completely different. The more convincing (or immersive) the virtual world is, the more we start believing—or at least feeling as if—we are in the virtual environment.

So how does this happen? Let us imagine that we are going to build an alternate universe for a friend to experience; it needs to be very convincing for us to succeed. If we get it right, then our friend’s brain will be “tricked” into sensing that this universe we have designed feels real, even though she knows, of course, that it is an illusion. If we get it wrong, our friend’s experiences in the alternate universe will fall short of how her brain would perceive (or interpret) things in the real world. She might think the experience is enjoyable, but her brain will not be “tricked” effectively. A third possibility is that we get it really wrong. This last scenario could mean that our friend experiences cybersickness, which is when VR tricks your brain into feelings of motion-sickness (that queasy feeling some people get in a car, plane, or on a boat). In other words, VR is ineffective when the virtual world “behaves” differently than the real one.

How VR “Tricks” Our Brain

To understand how effective VR works, we first need to understand a little about how the brain makes sense of the world around us. Let us stop and think about the senses that allow us to experience the world: vision, hearing, and touch, to name a few. To make sense of the world, the brain needs to first bring in information from sensory organs, such as the eyes, ears, and skin. But bringing in the information only describes sensation—the act of transmitting information from the sense organs to the brain. What happens next is that the brain interprets this information, allowing us to understand what is happening in the environment. The brain’s interpretation of the senses which create our understanding is called perception. For instance, we can see a dog running across the room, hear her bark, and feel her fur brush against our skin—these are sensations that we come to understand and perceive as experiences. The sensations all come together through perception to give us the experience of the dog. It is this interplay of sensation (using vision, hearing, etc.) and perception (our brain’s interpretation of this information) that creates our experience of reality.

How Ingredients in VR Can Cook up a Kind of Reality

There is a lot to keep track of if we are to construct an alternate universe! Because our senses are numerous and complex, we will discuss a simpler example—an alternate universe that we can experience through vision alone. We can still make this place convincing because our brains tend to rely more on vision than on any other sense. Let us imagine that what we create is something like a Martian dune (Figure 1). How do we start with this simple image and get our friend to perceive it as an immersive environment? For starters, our friend needs to be able to experience this two-dimensional image of a Martian dune as though it were as real as the three-dimensional space you are in right now. To make this happen, we will need to begin by building something into our environment called stereoscopic vision.

A typical photograph is a motionless picture viewed from a single perspective. In the real world, however, when we view a scene we can move around and look at things from different angles. In Figure 2, we have Leika the dog, sitting by a chair. We can sometimes see more of the chair than we can see of Leika, depending on our particular vantage point. How much our view of Leika is blocked by the chair depends on where we are standing. It is also important to note that we get a sense of depth in these pictures. In the photos, we can tell that the chair is (usually) closer to us than Leika, because it partially blocks our view of her. Knowing how close or far away things are partly depends upon our having binocular vision. Binocular vision means that our left and right eyes see things from slightly different viewpoints, because they are located on different sides of the face. This means that our brain has to merge together information from these two perspectives. This process is called stereopsis. Seeing in stereo allows our brains to let us know whether an object is close up or farther away. For our friend, stereopsis is what will create the illusion of being in the picture instead of just looking at it.

How to Create a Virtual World

So how do we create stereoscopic vision for our friend’s virtual experience? First, we need to get her a VR headset. VR headsets simulate binocular vision by presenting slightly different images to each eye, giving the illusion that a two-dimensional picture is a three-dimensional environment. Advanced VR headsets can be expensive, but you can make an inexpensive one at home with Google Cardboard and then use a mobile phone as the presentation screen (using apps that split the image into separate views for each eye).

Seeing in stereo is not the only thing we will need, because our friend will want to look around at her surroundings the way we do when we are exploring a new place. So, we need to introduce the idea of head-tracking. Imagine walking into a beautiful cathedral. Chances are you will look up and down, left and right, and even behind you. In the real world, this happens so naturally that we do not even notice. However, in discovering how to create a VR environment, we often need to stop and question things that we usually take for granted.

What would happen if our VR headset always showed the same piece of the picture no matter how far up or down we looked? It would not be terribly convincing! An unconvincing virtual environment is just what we would get if we forgot about the head-tracking component of VR. When our friend looks up or down, the angle of the Martian dune should match the angle her head is pointed. When she turns around, the immersive environment needs to show her the visual information that was previously “behind” her. Head-tracking simply monitors the direction that your head is pointing by using something called an accelerometer, which can sense whether (and in what direction) something is moving. Smartphones have accelerometers built into them by which you are able to enjoy certain types of games that involve tilting your phone. The phone’s accelerometer detects how you are tilting it and then adjusts your movement in the game accordingly.

Virtual Reality Gone Really Wrong

Accelerometers in VR imitate the “accelerometers” we carry around with us in our heads. Any time we move our heads, sense organs in our inner ears—the vestibular system—provide the brain with information about how our head is oriented in space. The vestibular system is crucial for helping us maintain balance, making sure we do not fall down, and letting us know if we are lying down or standing up. Building head-tracking into our friend’s VR headset will make the virtual environment convincing if we get it right. But if it goes wrong, then our friend could get something called cybersickness. Cybersickness happens in VR when a person feels disoriented and nauseous. It is like the kind of motion sickness you might get while trying to read a book in the backseat of a car. Interestingly, however, motion sickness and cybersickness are not quite the same thing. A closer look at the difference will help us understand how VR can be so successful at “tricking” the brain, sometimes in the wrong kinds of ways.

What happens when we get motion sickness? It is the result of the brain receiving two conflicting signals. One signal comes from the eyes and the other from the inner ears (specifically, the parts that sense how your head is tilted in space, not the parts that detect sounds!). If you are trying to read a book in a moving car, your eyes convey to the brain that you are not moving, but your vestibular system registers the movement of the car as it leans into the turns, brakes, and accelerates. Your vestibular system sends your brain information that you are moving. Your brain cannot always quite reconcile these two contrary messages, and sometimes the result is motion sickness. This is partly why staring out the window can help relieving motion sickness. By having your eyes send “we are moving” signals to the brain, there is no longer disconnect between the messages from your eyes and ears, and you start to feel better.

So, how is it possible to feel motion sick in VR when you are not moving at all? Your vestibular system generally is not highly activated in VR, because you are often either standing still or sitting. But, stereoscopic vision and head-tracking give you the illusion of moving. Cybersickness is thought to result from this illusion, also known as vection. You might have experienced a vection illusion if you were ever on a train or bus and thought you were moving forward or backward when it was in fact the vehicle next to you that was moving. This illusion can arise in VR, because when moving through virtual space (using a game controller, for instance), your eyes say, “I am moving,” whereas your vestibular system says, “I am staying still.” Cybersickness can also occur when there is a time delay between when your head moves and when the screen adjusts the perspective of the environment. Some people are particularly prone to motion- and cybersickness, though the exact reason for this difference between individuals is not yet known [1].

Just How Real Does VR Feel?

Let us assume we have got the components of our VR headset correct and that our friend is not prone to cybersickness; we have pulled off a successful virtual environment, but how do we know how real it feels? Presence is an important concept in VR. Presence is used to measure how much a person feels like they are now in the virtual environment, instead of in the physical one. One way to measure presence is by recording a person’s heart rate and other signs of stress. If you get too close to a cliff edge in real life, you will likely experience certain sensations: a faster heartbeat, sweaty palms, and more rapid breathing. Measuring these same symptoms of stress can be also done with people on a virtual cliff edge in a simulated environment. One of the many ways VR is used outside of gaming is actually for the treatment of specific phobias, such as acrophobia (fear of heights). With the careful use of VR by mental health professionals, people who have an intense fear of heights (or other types of phobias) can be treated by a process called systematic desensitization, in which they are able to slowly master their fear in a safe environment [2].

Virtual reality has the potential to allow us to experience things we would likely never encounter in real life. The virtual environment we created for our friend only included the sense of vision. Advanced VR technology, however, incorporates other senses as well. The more of our senses that are correctly incorporated into a VR environment, the more immersive, or true-to-life, it is. The more immersive the VR world is, the more present we feel and the more we lose track of the place where we actually reside.

Conclusion

There is a big difference between learning about something through reading or watching documentaries and actually getting to experience it. Often, we learn about subjects, such as astronomy through textbooks and videos. In the future, however, science class might just include field trips to VR environments, where we get to explore and feel what it might be like to walk around on a Martian dune. Ultimately, this technology “tricks” our brain, making us feel like we are somewhere else by mimicking the perceptual experiences we have in the real world, and convincing us that we are inside our games, or on the surface of a different planet. How will you use this exciting technology?

Glossary

Immersion: ↑ How well virtual reality is able to mimic or simulate the real world as we know it.

Cybersickness: ↑ A feeling of disorientation and/or nausea that can result from the illusion of moving through virtual environments. These unpleasant sensations can also be caused by lagging (or delays) between what your vision expects and what the virtual world presents.

Sensation: ↑ The different ways our body has of bringing us information about the world around us (for example, vision, hearing, touch, and taste), and the act of sending that information to our brain to perceive.

Perception: ↑ The process of our brain interpreting our senses into experiences.

Binocular Vision: ↑ Our left and right eyes are in slightly different positions on our head, and our brain is able to merge these two perspectives together much like looking through binoculars.

Stereopsis: ↑ Seeing in stereo. How our brain combines visual information from our left and right eyes into one single image.

Accelerometer: ↑ A device that can tell whether (and in what direction) something is moving.

Presence: ↑ How convincing we perceive a virtual environment to be.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.