Scientific Replication

This note provides an overview of replication and its roles in science. Because you will perform a replication in the exercise at the end of this guide, you may find this background information valuable. This note is not required to complete the exercises.

One of the goals of social science is to provide information that will help us understand society and influence policy. Evidence from a single study is not usually considered sufficient for accepting a scientific knowledge claim. Results gain credibility when other scientists arrive at similar conclusions in later studies.

Replication and transparency go hand-in-hand. Transparency refers to clarity and oppenness about the data and methods used in the analysis. It is what makes replication possible. In its narrowest sense, a replication of a study follows the original study very closely. Scientists apply the same method to the same data, even as precisely as using the same software code. If successful, the replication produces the same results as the original study. A broader sense of replication includes taking the methods used in the original study and applying them to different data. Or, a replication might test the original hypothesis with improvements to the method.

In each case, the replication is successful when its results match those in the original study. The scientific process of replication serves important purposes:

• Successful replication can increase confidence in the original findings. Scientific consensus accrues as new studies replicate a finding again and again.
• Failures to replicate may reveal flaws in the methods used by the original study. There could be problems in how data were handled, how analyses were conducted, or how results were presented.
• Failures to replicate could also point to differences in context that turn out to be important. For example, we might find out that a pattern of results holds among the U.S. population, but not for European samples.
• The process of replication can be a learning experience. Following the method used by earlier researchers requires careful attention to detail. Scientists can become familiar with a source of data, and students can learn the finer points of conducting research.

This guide is geared toward the last point about learning. The National Crime Victimization Survey (NCVS) is complex. An excellent way to begin using the data is to replicate official figures published by the Bureau of Justice Statistics. After completing the exercise in this guide, you can have confidence in your statistical work when your replicated results match the original reports.

Want to hear more about replication? Check out this National Science Foundation interview with Brian Nosek of the Center for Open Science.