Interview: Dr. Daniel Simons, editor of Advances in Methods and Practices in Psychological Science (AMPPS) on how to “get started” with open science practices

The journal Advances in Methods and Practices in Psychological Science (AMPPS) publishes academic articles on scientific research methods across the psychological sciences. AMPPS is innovative in a number of ways, including its attention to open science practices. This is reflected in its TOP Factor rating, of 25 out of 29 possible points- a rating of how journal policies align with the Transparency and Openness Promotion Guidelines. 

AMPPS requires adherence to high transparency standards including the requirement that data, code, and materials for studies be posted to a trusted third-party repository for public access. AMPPS also encourages submission of Registered Reports, a publishing format that involves a two-step peer review process, with peer review occurring once before data collection, and once after data collection. 

We asked Dr. Daniel Simons, editor of AMPPS and Professor of Psychology at the University of Illinois at Urbana-Champaign, to share his thoughts on the role of journals in advancing open sciences practices, his experience implementing AMPPS’ rigorous standards, and his advice for researchers and editors on advancing a vision of increasingly transparent research in academia.

What part are journals playing in advancing practices toward more transparent research?
As long as traditional journal articles remain the currency of the field, they can incentivize practices that improve research transparency. When journals establish transparent practices as a standard, researchers will adapt what they do to meet those requirements. For that reason, journals need to be careful in establishing requirements that are well grounded and that enhance transparency without inhibiting some forms of rigorous research that might not meet rigid requirements. For example, it's great to establish a requirement that data be made as widely available as is ethically and legally possible, but a blanket requirement that all data be publicly available would prevent publication of research on sensitive data (e.g., data that could be made available in a protected repository but not posted publicly due to identifiers).

What is the journal's motivation for being a leader in the promotion of open science practices in journal publishing?
AMPPS focuses on improving statistical and research practices, and transparency has been a core part of the journal's mission from the start. One aspect of our mission is to make it easier for researchers to adopt new practices that make their own work more rigorous. By setting transparency standards for the journal, we also signal that those are practices we want to encourage across all of psychology.

How do you feel the strong open scholarship standards championed in your policies have impacted the quality of research in your journal?
It's hard to know because we don't have a baseline comparison group. AMPPS is a new journal, and we've adopted those policies from our launch. We don't have a comparison group of articles published before we adopted those standards. Overall, the majority of submissions we receive adhere to high standards for transparency (and those that don't typically don't fare well). In part, that's because our mission focuses on communicating research practices, and the authors who submit to AMPPS typically are familiar with those policies and standards. Overall, I think the papers we've published have been of consistently high quality, but I don't know if that's due to our policies or to the topics covered by the journal. I don't get the sense that there has been any cost to the quality of our publications from adhering to those policies.

What would be your advice to researchers who want to adhere to high open science standards?
Get started. You don't need to adopt all possible practices at once. Pick a few practices and get started. There are great, accessible tutorials on practices like preregistration and data sharing. The first time you adopt a new practice, it will be more labor intensive, but as you develop a workflow, it becomes easier and easier. For example, the first time you develop a comprehensive preregistration plan, it will take time and effort (get others to look it over and make sure they can follow your procedures). But, the second time you do it, you'll have a template to work from.

What would be your advice to editors who want to implement open science practices in their journals?
Think carefully about the types of papers you publish and the policies that would naturally apply to the highest proportion of those papers. If your journal mostly publishes meta-analyses or systematic reviews, for example, you could focus on requiring transparent documentation of the search and inclusion process in a way that makes it reproducible. If you publish standard empirical papers, focusing on making data and materials available to reviewers (as well as preregistration) are natural places to start. Journals need not adopt the most stringent requirements initially—the key is transparency. For example, you could require a statement of whether and where data are available or an explanation for why they aren't available. Signalling to authors that transparency is required and indicating what standards apply in your journal can help to shift practices across the field. Adding easy-to-meet requirements can still have large benefits.

Visit cos.io/top to explore more steps journal editors, funders, and societies can take to implement transparent policies for their communities.