The following is a guest post from Dalson Figueiredo and Nicole Janz who helped to co-author Seven Reasons Why: A User's Guide to Transparency and Reproducibility, the first guide of its kind outlining reproducibility best practices for government institutions and the political science community in Brazil.
The authors, Nicole Janz of the University of Nottingham, UK and Dalson Figueiredo of Federal University of Pernambuco, Brazil.
The scholarly interest in transparency, reproducibility and replication has been rising recently, with compelling evidence that many scientific studies have failed to be replicated in political science, sociology, economics, medicine and genetics. In short, both the natural and the social sciences are facing a reproducibility crisis. Worse still, there is no agreement as to what transparency, reproducibility and replication mean exactly.
With a grant from the British Academy, we faced the problem of trying to introduce transparency norms to the political science community in Brazil – for scholars and journal editors. After running several workshops, we realized: there’s no simple, short user’s guide for scholars looking at transparency for the first time. Together with colleagues in Brazil, among them our brilliant students, we wrote that guide, in collaboration with our Brazilian colleagues Rodrigo Lins, Amanda Domingos, and Lucas Silva. Here are seven reasons to work and publish reproducibly.
- Replication materials required by journals help to avoid disaster: Data posting reduces potential scientific misconduct in journals and helps authors to compile and review their files carefully before article submission. Evidently, nobody wants to get it wrong, but if your data and code are publicly available, the scholarly community will know in advance that you did your best to promote open science.
- Transparency makes it easier to write papers: Creating replication materials on-the-go will force researchers to plan every step of their work carefully, and document their decisions. This will help with writing the paper later on, especially with writing a good methods section.
- Replication materials can lead to better peer review: When authors give peer reviewers access to replication materials, they can judge the work much more easily, find potential issues, help improve the paper – or simply see for themselves that the analysis is sound.
- Replication materials enable the continuity of academic work: Have you ever tried to pick up an old project and complete it after a longer break? Have you ever tried to write a follow-up article based on your (long ago) PhD? Working transparently helps, in the first instance, authors to work on projects continuously. If you run your data in Excel or by click-and-play software, there is a high probability that you will face problems in following the same steps one week later. Believe us, we have been there more times than that we would like to admit.
- Replication materials help to build scientific reputation: Journals and authors who work toward transparency of research and publications have much more to gain than to lose: a good reputation. Even if an error occurs, they need not be afraid of accusations – all data and code were available, and errors are human. Withholding data would cause much more suspicion in the community than human errors.
- Replication is a powerful tool to learn data analysis: From our teaching experience, we have observed that students are more motivated to work with real data and arriving at the same figures than working with some dull, repetitive homework assignment unrelated to what they are studying.
- Replication materials increase the impact of scholarly work: Most articles are never cited papers – but papers that share data increase their citation count massively.
If we’ve persuaded you to give transparent workflows a try (or you’re a journal editor wanting to support your authors), our full article – co-authored with our Brazilian colleagues Rodrigo Lins, Amanda Domingos, and Lucas Silva – contains the most important tools and initiatives to use, introducing: TOP guidelines for journals, Project TIER with teaching tools, Github and Rmarkdown, OSF data archive, pre-analysis plans and more.
