An enhanced TOP Factor journal interface has been released, which provides a more interactive view of journal ratings, increases transparency of the scoring process, and provides features for sharing groups of scored journals.

About TOP 

Policies that guide how science is conducted and reported often overlook key principles that build trust and credibility. Practices that focus on novelty instead of transparency, for example, may be helpful for raising awareness about discoveries, but come at the expense of being able to “check under the hood” and verify their credibility. 

The Transparency and Openness Promotion Guidelines (TOP, Nosek et al 2015) were created by societies, funders, and journal editors to give policymakers tools to implement standards that follow these principles. TOP provides specific, actionable policies that increase data transparency, detailed reporting of study designs and methods, and that incentivize replication of previously reported findings. 

TOP Factor is a measurement of how these policies are implemented at individual journals. Each TOP standard can be implemented in one of three levels of increasing rigor. For example, Data Transparency can (1) require disclosure of data ability, (2) require transparency to the maximum extent permitted by ethical considerations, or (3) verify outcomes through computational reproducibility. Does a journal merely encourage data sharing but not enforce any specific consequence for being opaque? Do they discourage replications to studies that they originally published? These policies do not comply with TOP (i.e. they are “Level 0”), and they diminish the credibility of empirical research. Thus, they result in a low TOP Factor. 

Using TOPFactor.org

The new interface gives users a more interactive view of journal ratings, increases the transparency of the scoring process, provides features for sharing groups of scored journals, and surfaces information about the specific journal’s transparency policies. 

Two features in particular will help proponents of open science policies share TOP Factors. First, each journal has a specific page on TOPFactor.org that can be shared with a URL. For example, the journal Psychological Science, a leader in adopting open science practices, can be directly shared with its unique URL, https://topfactor.org/journals/psychological-science .

Second, groups of journals can be shared from the “All Journals” page based on groupings by discipline, publisher, and by individual TOP standard. For example, this sharable URL shows the data transparency standards of journals that publish research in epidemiology: https://topfactor.org/journals?factor=Data+Transparency&disciplines=Epidemiology Other combinations of policies and disciplines can help refine such filters.

These two features are specifically designed to promote outreach by community members to decision makers and policy makers, so that conversations can occur that focus on open science.

Thanks to the efforts of many collaborators, over 1,400 journals are now included. Soon, we will have an interface that increases the capacity to evaluate and update journals’ policies. This year, we will use the new interface to engage editors on opportunities to improve their transparency policies in line with their progressive peer journals. 

Facts on TOP Factor:

• TOP Factor is a measure that evaluates journal policies that promote open science and reproducibility.
• TOP Factor is scored using a publicly available rubric (https://osf.io/t2yu5/). 
• TOP Factor is an alternative way to assess journal qualities and improves traditional metrics that measure mean citation rates. 
• Journal editors can change their TOP Factor score by changing their author guidelines.

New interface:

Example of a Scored Journal:

TOP Factor Key Performance Indicators Report:

Check out the interview with Dr. Daniel Simons, editor of Advances in Methods and Practices in Psychological Science (AMPPS), scored on TOP Factor on the COS blog.