Reproducibility Project

The project was led by the Center for Open Science and its co-founder, Brian Nosek, who started the project in November 2011.

Results

Brian Nosek of University of Virginia and colleagues sought out to replicate 100 different studies, all published in 2008. The project pulled these studies from three different journals, Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory, and Cognition, published in 2008 to see if they could get the same results as the initial findings. 97 of the original studies had significant effects, but of those 97, only 36% of the replications yielded significant findings (p value below 0.05), and the effects were often smaller than those in the original papers. The authors emphasized that the findings reflect a problem that affects all of science and not just psychology, and that there is room to improve reproducibility in psychology.

In 2021, the project showed that of 193 experiments from 53 top papers about cancer published between 2010 and 2012, only 50 experiments from 23 papers could be replicated. Moreover, it showed that the effect sizes of that fraction were 85% smaller on average than the original findings. None of the papers had its experimental protocols fully described and 70% of experiments required asking for key reagents.