Symposium: A critical analysis of the scientific reform movement

Psychologists psychologizing scientific psychology: An epistemological reading of the replication crisis

8:05 am PT / 11:05 am ET / 5:05 pm CEST

In this talk, I will discuss the role played by psychologists’ own epistemological thinking for the reform movement. I will use this discussion to make an analytical distinction between three loose groups of psychologists engaged with the reform debate: status-quo psychologists, replication-minded reformers and theory-minded reformers. The differences between these groups will be highlighted by their views of what is the function of scientific literature in either accumulation or systematization of knowledge claims contained within. Considering the way researchers in psychology have engaged with the literature during the 20th century, I will identify two supposedly emergent properties of the literature I call myths: self-correction and self-organization. The goal of the talk is not to ascribe philosophical positions to individual psychologists, but to encourage the extension of methodological and institutional reform to include a deeper historical and philosophical awareness of the methods and institutions being criticized.

The scientific and social implications of implementing Open Science policies and procedures

8:25 am PT / 11:25 am ET / 5:25 pm CEST

It is well acknowledged by now that what Open Science means for the future of science in/and society depends largely on how it is implemented. Per se, there is little novelty attached to the wish to uphold values such as openness, transparency, reproducibility and critical scrutiny as hallmarks of good research practice. What matters are the specifics of how such values are interpreted in relation to specific situations that matters, especially in a global context rife with inequity, highly diverse (and often problematic) incentive systems, various forms of discrimination and a short-term understanding of scientific, political and economic gains. In this talk I reflect on some of the implementation challenges confronted by Open Science and the crucial importance of responsible practices that support researchers across locations, disciplines and career stages.

Metascience as a scientific social movement

8:45 am PT / 11:45 am ET / 5:45 pm CEST

Emerging out of the “reproducibility crisis” in science, metascientists have become central players in debates about research integrity, scholarly communication, and science policy. The goal of this article is to introduce metascience to STS scholars, detail the scientific ideology that is apparent in its articles, strategy statements, and research projects, and discuss its institutional and intellectual future. Put simply, metascience is a scientific social movement that seeks to use the tools of science—especially, quantification and experimentation—to diagnose problems in research practice and improve efficiency. It draws together data scientists, experimental and statistical methodologists, and open science activists into a project with both intellectual and policy dimensions. Metascientists have been remarkably successful at winning grants, motivating news coverage, and changing policies at science agencies, journals, and universities. Moreover, metascience represents the apotheosis of several trends in research practice, scientific communication, and science governance including increased attention to methodological and statistical criticism of scientific practice, the promotion of “open science” by science funders and journals, the growing importance of both preprint and data repositories for scientific communication, and the new prominence of data scientists as research makes a turn toward Big Science.

Formal methodology in scientific reform

9:10 am PT / 12:10 pm ET / 6:10 pm CEST

Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. We show how our formal approach can inform and shape debates about three popular methodological claims.

Open science in the era of informational mystification

9:30 am PT / 12:30 pm ET / 6:30 pm CEST

Over the course of the last couple of decades, the Internet has seemingly transformed from a weird utopian place for geeks to gather, into a grotesque thunder-dome whose steely tentacles extend into the most mundane spheres of daily life, spiking levels of negative affect and socially corrosive spectacle while precisely moderating and quantifying labor for the purposes of capital accumulation. Rather than a singular movement, open science consists of a dynamic, nuanced, and complex set of attempts to figure out where technoscience fits in. In this talk, I try to follow the work of other feminist scholars of science and technology to consider the ethical obligations and political potential of critique in this specific moment of informational mystification. Reflecting on ethnographic fieldwork in open science, I consider how open science infrastructures (both symbolic and material) can potentially connect with and learn from adjacent movements for worker’s rights in other “cognitive” or “immaterial” industries.