Why is it Important to Verify Study Results?

Scientists do research to understand the world better and because they want to apply this knowledge in real-life situations. When a scientist discovers something interesting, it is important that others can confirm, or verify, the study results by repeating the study.

This is important for different reasons: imagine that someone just developed a potion that can make you fly. How cool would that be? If a researcher conducts a study and claims that their potion indeed gives people the ability to fly, you would want to make sure (1) that it works, and (2) that there are no negative side effects before you give it to other people.

If the potion turns out not to work after all, this could have serious consequences: people could get injured if they try to jump off a tree as they attempt to fly. That would be quite a problem, and we would definitely want to avoid this.

There could also be side effects even if the potion works. What if the people who drink the potion get blue spots on their faces? Unless this becomes a new trend, most people would probably prefer their faces without blue spots.

This means that we need a quality check to verify research findings.

How Can Scientists Verify Research Findings?

In the research process, this quality check ideally happens by reproducing a research finding, which means that someone else repeats the study. Reproducibility refers to obtaining the same results when using the same materials and procedures [1]. In the case of the flying potion, this would mean using the same ingredients to create the potion and following the same production steps to see whether it indeed gives people the ability to fly.

It is probably not very likely that someone is currently researching such a potion, but the same ideas apply to actual research. For example, during the Covid-19 pandemic, researchers developed “recipes” for vaccines. These shots aimed to prevent people from becoming seriously ill. Before everyone could be vaccinated, the researchers needed to know how well the vaccine worked and if it had any side effects. As in our potion example, a quality check of the new finding was needed. This quality check was done by many other researchers who repeated the vaccine production process. To do so, they needed to know which ingredients were required and how to combine them.

In an ideal world, research findings should always be reproducible. If another scientist does the same research the same way, they should get the same result. However, sometimes research is not reproducible because, for example, materials or information are missing.

What Happens if Something is Missing?

We can break down what happens if materials or information are missing from the scientific process by thinking about playing with LEGO®. LEGO® has building sets with which you can build things like a truck, for example (Figure 1). When you open the package, you get all the pieces you need (your materials) and a step-by-step guide that explains the steps of how to put the pieces together to build the truck. If you give this set to a friend after you have built the truck yourself, but you accidentally lost a wheel, then your friend will not be able to reproduce the truck. The truck will have only three wheels instead of four, and reproducibility is about obtaining the exact same result using the same materials.

A similar problem arises when there is a mistake or missing information in the guide. If a few steps are not clear or missing because a page is damaged or torn out, for instance, your friend will likely have a hard time building the exact same truck shown on the box. It would be even trickier if the whole guide or the example picture on the box was missing.

This means that it is very difficult, maybe even impossible, to reproduce the same outcome when materials or information on the procedure are missing [2]. This was nicely shown in a recent study in which one group of researchers received all information about a study (i.e., what was studied and how) and the materials. The other group did not receive all the materials. Both groups were asked to reproduce the original study results. The researchers without the materials ended up with different results than the original study more often than the researchers who had all the materials available [3].

In research, we are usually not trying to reproduce a LEGO® truck, but researchers are working on various topics to generate more knowledge and better understand the world. If a researcher finds something interesting, they must make sure that the results can be reproduced. Unfortunately, this is not always the case.

Why is Research not Always Reproducible?

There are multiple reasons why research is not always reproducible. We already discussed that missing materials and missing information on the procedure can cause a researcher to end up with different findings than the initial study. The initial findings might have been true, but it is impossible to check this if crucial information is missing, like when you are missing the guide for the LEGO® truck. This is a very common reason why research cannot be reproduced. Researchers are very busy and sometimes forget or do not have the time to note down all the details of their work. They also know their own work very well, so they do not always understand what kind of information someone else needs to repeat the study. For example, people unfamiliar with the topic might need more detailed instructions.

Another reason is that researchers are human and can make mistakes. Even if they are willing to share all their materials and information, they might make a mistake that causes different study results. Research is quite complex and requires many skills. Throughout the entire process—from deciding what to study to sharing the results—things can and do go wrong. For example, it is very easy to accidentally type an incorrect number when writing down how much of each ingredient is needed. This can happen just as easily as typing the wrong letters when texting a friend. Usually, such mistakes are found and corrected during the research process, but it is important to ensure that no mistakes are overlooked. Fortunately, researchers are aware of this and have come up with some strategies to improve the reproducibility of research [4].

Strategies to Increase Reproducibility of Research

There are many ways to achieve reproducibility, and the exact steps that are necessary depend on the specific type of research. Many of the practices that contribute to reproducibility are referred to as open science practices [5]. Those practices mostly relate to sharing materials, data, and all relevant information about how the study was conducted on a website where everyone can find it. When all this information is available, other people should be able to reproduce the research results.

When researchers find something interesting in their research, they usually describe it by writing an article for a scientific journal (much like this article describes what reproducibility is). A scientific article should contain information on how the study was conducted, with detailed information about the materials and the procedures. Some scientific journals have a checklist to make sure the researcher provides all of the relevant information needed to reproduce the findings. Reviewers—fellow researchers with expertise on the topic of the article—are responsible for checking whether all relevant information is provided, and they can ask for additional information if something is missing (Figure 2). Nowadays, it is also easy for researchers to share their articles publicly on the internet. This allows everyone (not only journal reviewers) to access and verify a study’s findings.

How Can Researchers Avoid Mistakes in Their Own Research?

Earlier, we mentioned that researchers can make mistakes. Some researchers are helping other researchers by developing technical solutions that detect or avoid mistakes [4]. For example, there is software that lets a researcher check if they made a mistake in writing down their findings, and there are tools that keep track of what a researcher has done, to prevent accidental loss of information. Collaborative research, where multiple researchers work together on the same project, can also help reduce such mistakes. In such projects, the researchers double-check each other’s work.

Nowadays, researchers are devoting more and more time and energy to carefully documenting and sharing their work so that other researchers can check it and build upon it. By being very open about how they do their work, researchers make sure that their findings can be reproduced by other researchers. The more often research findings are repeated, the more we can trust them.

Glossary

Research Finding: ↑ The result or outcome of a scientific study (e.g., the observation that, in a test comparing two types of medication, one medication brings down your fever quicker).

Reproducibility: ↑ Getting the same results when a research study is repeated using the same materials and steps—like making a medicine that works the same every time.

Materials: ↑ The “ingredients” needed for a research project, such as the different chemicals needed to make a medication.

Procedures: ↑ The steps in the “recipe” of a research study, like the specific step-by-step manner in which chemicals must be combined to make a medication.

Open Science: ↑ Public sharing of materials, detailed procedures, and data by researchers, much like you might share your homework with a friend so that they can check it or learn from it.

Reviewers: ↑ Researchers who check other researchers’ work to see whether they forget to mention important information or made mistakes, much like your parents might check your homework before you hand it in.

Conflict of Interest

The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

LD was supported by a Starter Grant awarded by the Dutch Ministry of Education, Culture and Science to SV.

AI Tool Statement

The author(s) declared that generative AI was used in the creation of this manuscript. LD declares the use of Microsoft Copilot for grammar and spelling correction.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.