The Prereg Challenge Is Ending. What's Next?

June 12th, 2018, Alexander DeHaven, David Mellor


prereg challenge

The Preregistration Challenge was launched in 2016 as an education campaign designed to initiate a new habit among empirical researchers. Now that the Challenge is in its closing chapter, the final prizes will be awarded in December 2018, it is a good time to reflect on what has been accomplished so far and what is coming next. Since launching this campaign, preregistration has caught fire among researchers. Over 2,200 researchers have entered the Prereg Challenge, with over 30% of entrants submitting more than one preregistration.

Preregistrations have come from every corner of the scientific community. Ecologists, political scientists, linguists, evolutionary biologists, neuroscientists, geneticists and researchers in education and finance have taken the Challenge. There have been partner initiatives to support preregistration within specific communities. And of course, the real credibility revolution has occurred within the psychological sciences. Social-personality and cognitive psychologists know better than most how bias can affect us, and have preregistered in droves.

The Prereg Challenge is a rigorous preregistration process including review.  Other options for preregistration are available on OSF, and growth across all of them has been tremendous.  The figure below from a recent article in the APS Observer highlights this trend.



We learned a tremendous amount about the process of preregistration throughout this campaign, and will be applying those lessons in the future. Preregistration is new and unfamiliar to most researchers. Active guidance through our review process helped many to improve their research plan, and helped us to improve the process and our education materials. Going forward, we remain committed to supporting researchers who want to preregister, either through quick discussions on social media, or through comprehensive statistical and methodological consulting. We also are seeing a growing body of expertise on preregistrations, between those who are evaluating it as a process (as Coosje Veldkamp, https://psyarxiv.com/g8cjq/, and Tom Hardwicke and John Ioannidis https://osf.io/preprints/bitss/fzpcy/ are doing) and those who are creating guides, templates (such as the one created by Kevin McIntyre and Benjamin Le at Project TIER https://osf.io/n39tr/), and other resources to support preregistration (such as principles of secondary data analysis being curated by Sara Weston: https://osf.io/cgw86/).

We are committed to continually improving the registration process to encourage clear and rigorous preregistrations. The most exciting work that is on our development roadmap will be a major improvement to OSF Registries. Right now, OSF Registries helps researchers discover registrations on OSF and other registries such as clinicaltrials.gov. Soon, OSF Registries will enable communities to launch and manage their own registries to establish their own standards and moderate submissions. And because of SHARE, there will be no risk of fragmentation, registrations created in one community will be discoverable by any other. Another improvement on the roadmap for development includes a way to make deviations from preregistered plans more transparent and trackable.

Like all good things, the Preregistration Challenge will end. Papers must be published and award claims submitted by December 31, 2018 to be eligible for a prize. However, this is clearly only the beginning for preregistration more generally.  This flash-in-the-pan campaign helped to raise awareness about registration and provide experience that will help improve the quality of the registration process and the resulting research. Now we are transitioning to fostering the growth and quality of preregistration toward becoming a normal part of the research lifecycle for everyone.  


Recent Blogs

The Content of Open Science

What Second Graders Can Teach Us About Open Science

What's Going on With Reproducibility?

Open Science and the Marketplace of Ideas

3 Things Societies Can Do to Promote Research Integrity

How to Manage and Share Your Open Data

Interview with Prereg Challenge Award Winner Dr. Allison Skinner

Next Steps for Promoting Transparency in Science

Public Goods Infrastructure for Preprints and Innovation in Scholarly Communication

A How-To Guide to Improving the Clarity and Continuity of Your Preregistration

Building a Central Service for Preprints

Three More Reasons to Take the Preregistration Challenge

The Center for Open Science is a Culture Change Technology Company

Preregistration: A Plan, Not a Prison

How can we improve diversity and inclusion in the open science movement?

OSF Fedora Integration, Aussie style!

Replicating a challenging study: it's all about sharing the details.

Some Examples of Publishing the Research That Actually Happened

How Preregistration Helped Improve Our Research: An Interview with Preregistration Challenge Awardees

Are reproducibility and open science starting to matter in tenure and promotion review?

The IRIS Replication Award and Collaboration in the Second Language Research Community

We Should Redefine Statistical Significance

Some Cool New OSF Features

How Open Source Research Tools Can Help Institutions Keep it Simple

OSF Add-ons Help You Maximize Research Data Storage and Accessibility

10 Tips for Making a Great Preregistration

Community-Driven Science: An Interview With EarthArXiv Founders Chris Jackson, Tom Narock and Bruce Caron

A Preregistration Coaching Network

Why are we working so hard to open up science? A personal story.

One Preregistration to Rule Them All?

Using the wiki just got better.

Transparent Definitions and Community Signals: Growth in the Open Science Community

We're Committed to GDPR. Here's How.

Preprints: The What, The Why, The How.

The Prereg Challenge Is Ending. What's Next?

We are Now Registering Preprint DOIs with Crossref

Using OSF in the Lab

Psychology's New Normal

How Open Commenting on Preprints Can Increase Scientific Transparency: An Interview With the Directors of PsyArxiv, SocArxiv, and Marxiv

The Landscape of Open Data Policies

Open Science is a Behavior.

Why pre-registration might be better for your career and well-being

Interview: Randy McCarthy discusses his experiences with publishing his first Registered Report

Towards minimal reporting standards for life scientists

This website relies on cookies to help provide a better user experience. By clicking Accept or continuing to use the site, you agree. For more information, see our Privacy Policy and information on cookie use.