New Discovery Neuroscience and Psychology Published: August 30, 2024

Do Online Reviews Truly Represent the Writer’s Experience?

Abstract

At any given moment, people all over the world are posting and reading online reviews about hotels, restaurants, cars, and more. Reviews influence important decisions and experiences and have financial, personal, and social consequences. Reviews should accurately reflect the writer’s experience so that readers can make more informed decisions. Our research on hundreds of thousands of hotel reviews shows that this is not quite the case. We found that random changes in weather, even days after staying in a hotel, significantly affected the content and rating of the review. In other words, factors unrelated to the experience itself caused the writer, probably due to a change in mood, to write a review that did not represent their actual experience. This can have a big impact on businesses and digital platforms such as shopping sites and social media.

Does Online Information Influence People’s Decision Making?

The internet is full of texts, words, and videos—there is so much information online that it is very difficult to estimate its true amount. About 8.5 billion Google searches are performed and tens of billions of text messages are sent every day—more than the number of people on the planet. It is a whole lot of information, which people write to convey information, feelings, and thoughts to other people.

If people want to find a good hotel for their vacation, buy a new sofa for the living room, or do some history homework, what will they use? Probably the internet. If you heard from your friends that turkeys can blush, how do you know if it is true or just a tall tale? You would probably check online (this fact is true, by the way). A good review of a hotel can convince people to stay there [1], a Wikipedia article can teach people something new, and the information posted on the website of the local community center helps people decide whether to join swimming, dance, or judo classes. Research shows that both children and adults use the internet regularly to find information to help them make important decisions [2].

Information on the Web Affects People and Economies

Think about it: if what we read online influences our decisions, and billions of people go online every day, that means that information on the internet affects the decisions of billions of people daily. On such a large scale, these decisions affect the economy. Although this is not the place to explain the exact meaning of the word “economy”, it is important to understand that the economy involves many of the things that help us survive—the availability of food in the store, fuel for the car, electricity in our homes, doctors willing to treat us when we are sick, and much more. The economy in the West is influenced by the decisions of ordinary people: leaders, banks, buyers, and sellers—all of us. Unfortunately, there are countries where the economy does not function very well, which can lead to poverty and serious problems in daily life. If information on the web influences our decisions, it also affects the economy. If it is important to us that the economy is functioning properly, it is important to understand the effect that online content has on our behaviors and decisions.

This is why online reviews are very interesting to scientists who research human behaviors.

Online Reviews—How Does the Information We Read Online Affect Us?

Online reviews (Figure 1) are a way for people to provide others with useful information about the world, to help others make better decisions. If I am trying to decide which cell phone to buy, I want to know which one is the best in my price range. Before the internet existed (when I was a child), if I wanted to buy a good, affordable speaker (for example), I would have to guess, or hope that someone I knew had already bought the speaker and could tell me if it was any good.

Figure 1 - Example of an online review of a hotel.
  • Figure 1 - Example of an online review of a hotel.
  • You can see that the reviewer gave a rating of 9 out of 10 stars—they really liked the proximity to the sea and the attention of the hotel staff.

Online reviews solve a big part of this problem. Today, when I want to buy a quality speaker (or anything else, for that matter), I can look online for one that fits my needs and has a lot of high-rated reviews. I trust that the people who wrote the reviews are describing the actual experience they had with the product. Online reviews are a way to give me “access” to other people’s thoughts and experiences, so I can make good decisions.

A significant number of human behavior researchers assume that online reviews accurately represent the experiences of the people who write them [3]. After all, why would someone go through all the trouble of writing a review that was not true? Why would somebody deliberately mislead others? We were not sure—so we decided to investigate the matter.

The Weather as a Huge, Real-World Laboratory

To determine if people write reviews that reflect their actual experiences, we examined people’s behavior during an event that had nothing to do with their visit to a hotel: a rainy day… when they are back at home after a vacation.

Imagine you have come home from a vacation. After a few days, as you are about to write a review of the hotel, the sky turns gray, and it begins to rain. Should the rain affect the review you write about the hotel you visited a few days ago? It is known that rainy days can affect people’s moods [4], but there should not be any connection between the weather several days after you come back from the hotel and the experience at the hotel itself, right? This is what is called a natural experiment.

Reviews Affected by Rain

So how did we determine if rain affected the hotel reviews? We studied hundreds of thousands of hotel reviews written by hundreds of thousands of people over 10 years. We found that rainy weather in the area where the writer lived, at the time of writing the review, did affect the review. Before you continue reading, can you guess how the reviews were affected? (Hint: look at Figure 2—how does a rainy day affect your mood?).

Figure 2 - Researchers found that rain and bad weather affect people’s moods.
  • Figure 2 - Researchers found that rain and bad weather affect people’s moods.
  • Rainy weather (compared to sunny weather) puts people in a more negative mood. Which of the two pictures above gives you a more “gloomy” feeling?

We found that when it rained, people tended to give more negative ratings (Was your guess right?). Not only was the average rating lower when it was raining, but the words in the review were less happy and positive. Criticism was longer, more detailed, and more harsh. Why does this happen? Of course, we could not ask hundreds of thousands of people why, but even if we could, I am not sure they would be able to explain it. Fortunately, other studies (which have been done in the laboratory) provide a possible explanation.

Emotions and Mood are Information the Body “Sends” to Itself

So, where does this strange behavior come from? After all, if people had fun at the hotel, why is it that when they sit at home and it is raining outside, they suddenly report a different, more negative experience?

Many studies show that weather can affect mood—bad weather can create a more negative mood. But why should a change in mood affect the review a person writes about the hotel? After all, they have already been to the hotel and the experience was stored in their brains. It turns out that people are a little less level-headed and consistent than you might expect, and they are probably using their moods as information [5].

Let us say I am sitting down to write a review about a hotel in Eilat (Israel) that I recently stayed in. I am trying to remember the experience. Suddenly it starts raining outside, and my mood becomes more negative. This mood “mixes” with my thoughts and influences them. It is as if my body is sending me a signal: “You feel bad because the hotel experience was not that good”—which can cause me to write a more negative review.

Are you having trouble believing this? Try to remember a time when you were angry for some reason—did it change the way you looked at the world?

Why do people use their moods to provide information about the world? Maybe because sometimes this can actually help people in their daily lives. If I suddenly feel bad, maybe it is because I am hungry, thirsty, or sick. It is like my body sending a warning to myself: “Check in on yourself, maybe you need to rest, eat, or drink”.

In the case of online reviews, people’s moods can tip them into writing reviews that may not accurately reflect their experiences.

Does This Mean We Should Not Believe Online Reviews?

According to the research, despite all sorts of problems and biases, most reviews on the web do accurately represent the quality of products and services. There are quite a few studies showing that, overall, reviews are a good thing and provide important information—certainly much better than when I was a kid, when there were no online reviews at all. But when you read a review, try to keep in mind who wrote the review, when it was written, and why it was written—that is, the context in which the review was written. That is always good advice. I also hope that, thanks to this and other articles that find problems with reviews, new methods will be developed to ensure that reviews reflect the actual experiences people have—or that they at least include a note: “This review was written when the writer was in a particularly bad mood!”.

My conclusion is that you have to be careful what you read online. It is important to remember that humans have limitations, and it is not always easy for people to explain what they are really thinking or feeling. Hopefully, over time, we will be able to use technology to fix our limitations—or at least some of them. Such technologies might improve the accuracy of reviews over those that are available today.

Glossary

Economy: How people, companies, and the government of a country use available money and resources (like your pocket money) to create, buy, and sell things they need or want.

Online Reviews: Opinions and comments that users post online about products, services, people, events, companies, and more, to share their experiences and help others make decisions.

Natural Experiment: A natural situation that can be seen as a huge experiment, providing an opportunity to measure cause and effect in the real world as opposed to in the lab, which is sometimes less realistic.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Original Source Article

Brandes, L., and Dover, Y. 2022. Offline context affects online reviews: the effect of post-consumption weather. J. Cons. Res. 49:595–615. doi: 10.1093/jcr/ucac003


References

[1] Chevalier, J. A. and Mayzlin, D. 2006. The effect of word of mouth on sales: online book reviews. J. Market. Res. 43:345–5. doi: 10.1509/jmkr.43.3.345

[2] Liu, X., Lee, D., and Srinivasan, K. 2019. Large-scale cross-category analysis of consumer review content on sales conversion leveraging deep learning. J. Market. Res. 56:918–43. doi: 10.1177/0022243719866690

[3] Wu, C., Che, H., Chan, T. Y., and Li, X. 2015. The economic value of online reviews. Market. Sci. 34:739–54. doi: 10.1287/mksc.2015.0926

[4] Klimstra, T. A., Frijns, T., Keijsers, L., Denissen, J. J., Raaijmakers, Q. A., van Aken, M. A., et al. 2011. Come rain or come shine: individual differences in how weather affects mood. Emotion 11:1495. doi: 10.1037/a0024649

[5] Schwarz, N. 2012. Feelings-as-information theory. Handb. Theor. Soc. Psychol. 1:289–308. doi: 10.4135/9781446249215.n15