The Center for Open Science Announces the Launch of the $1,000,000 Preregistration Challenge

The Preregistration Challenge, a new competition announced today by the Center for Open Science (COS), incentivizes transparent research by awarding 1,000 researchers with $1,000 for publishing the results of their preregistered studies.

“Scientists value objectivity and transparency in the practice of science,” says David Mellor, project manager at the Center for Open Science. “However, we are rewarded at nearly every step for making results both cleaner and more surprising. Preregistration helps keep our motivated reasoning in check when analyzing data and presenting our results.”

Preregistration increases the credibility of hypothesis testing by having scientists specify details of their research plans in advance of conducting a study. Hypothesis testing is a common scientific practice in which a researcher uses data to evaluate an explanation for their topic of study. This is distinct from another important scientific practice - hypothesis generating - in which a researcher might use data to create a possible explanation. Preregistration makes clear when a researcher is doing hypothesis testing versus hypothesis generating research. This is important because they cannot be done easily with the same data without biasing the results. If a researcher looks at a dataset to generate new ideas, and then conducts statistical tests on the same data to evaluate those ideas, he or she is more likely to inadvertently make false discoveries because of ordinary random variation in data. “Researchers may see meaning in randomness and fail to recognize that they were generating new ideas rather than testing those ideas,” says Sara Bowman, Project Manager at COS.

 

“There are so many decisions that take place during the analysis of a typical research project,” says Courtney Soderberg, Statistical and Methodological Consultant at COS, “and seeing the data can affect those decisions, steering the researcher toward more exciting looking results. Preregistration simply requires making some of the analysis decisions upfront, before the data are known, so that reasoning biases cannot affect what analyses and results to take seriously.”

COS increases transparency and reproducibility in research by lowering barriers and increasing incentives to practice open science. This mission led to the creation of the Preregistration Challenge. Researchers will be rewarded for publishing the results of studies preregistered on the Open Science Framework, a free and open source scholarly commons designed to support researchers with tools for project management, sharing, and archiving.

Though required by law for clinical trials, preregistration is virtually unknown in pre-clinical and basic research. The goals of the Preregistration Challenge are to increase awareness of preregistration, educate the research community about its potential, and initiate it as a normal part of the research workflow.

“We’re using cash prizes to motivate this new behavior because it is a new step for a busy group of people. We believe that researchers will find this process valuable once they give it a try, so we are trying to help them get over the initial hurdle to do so,” according to Jolene Esposito, project manager at COS.

In addition to a guided process for preregistration, the Preregistration Challenge offers education and support materials to teach researchers about preregistration and help them through the process. Those interested in the Preregistration Challenge can visit the project page to learn more and initiate the process.

Contact

Center for Open Science 
David Mellor 
David@cos.io 

About the Center for Open Science

The Center for Open Science (COS) is a non-profit technology startup founded in 2013 with a mission to increase openness, integrity, and reproducibility of scientific research. The Center for Open Science has a variety of initiatives to increase the transparency and reproducibility of scientific research. Recently published in the journal Science, the Transparency and Openness Promotion (TOP) Guidelines are a set of standards designed for academic journals and funders to increase the transparency and reproducibility of research. Badges to acknowledge open practices allow journals to signal and reward transparent research. Finally, journals that adopt Registered Reports take preregistration one step further and reduce publication bias by conducting peer review prior to data collection. This improves the likelihood that journals publish the most accurate and methodologically rigorous research, instead of the flashiest findings. Learn more at cos.io and osf.io.