Strategy for Culture Change

Behavior change is hard.  Whatever its faults, the status quo is familiar and the warts are known.  The status quo is also easy to maintain. Just do nothing, inertia takes care of everything.  We even have a tendency to defend the status quo. We’d rather believe that the way it is, is the way it should be.

Overcoming the allure of the status quo is helpful but not sufficient to change behavior.  Many behavior change strategies focus on providing reasons to have the motivation to change, and skills and knowledge to have the ability to change.  If I know what to do, want to do it, and believe that I can do it, then I’ll do it, right?

Sometimes.

A focus on the individual’s skills, motivation, and sense of self-efficacy is sometimes sufficient to change behavior. But, the focus on the individual can miss a key barrier to change—the culture.

People are embedded in social and cultural systems.  Those systems shape behavior by [1] communicating norms—this is what people do, this is what people should do, [2] providing incentives—this is what people are rewarded for doing, and [3] imposing policies—this is what you have to do to be part of this system.

Interventions that provide individual motivation and ability to change may be rendered ineffective because of strong norms, incentives, and policies that counter the intended behavior.  Ironically, this can worsen conditions because the individuals have knowledge and motivation to do the new behaviors, but feel powerless against a culture that does not value or promote those behaviors.

When norms, incentives, and policies are misaligned with desired behavior, a much more encompassing strategy is needed to effect culture change.  Here is our strategy to change the research culture to accelerate discovery, solutions, and cures. The same concepts can be applied to other culture change efforts.

Our mission at the Center for Open Science (COS) is to increase openness, integrity, and reproducibility of research. Our behavior change targets are to get researchers to show their work and to share.  In the present culture, standard practice is that research teams operate independently and share their findings in publications that summarize what the team learned.  Usually, publications only share successes, and it isn’t possible to see what other things were tried to evaluate the credibility of the findings that are reported. In our future culture, standard practice would make the process of discovery transparent with researchers registering their plans to make all studies discoverable, and so that initial plans can be compared with the final outcomes and conclusions.  Also, in our future culture, by default, researchers would share the materials, protocols, and data that they produced in the research so that others could confirm, challenge, extend, or reuse the work. All these behavior changes are in service of accelerating science and reducing waste.

Sounds great right?  An advantage for COS’s mission is that the values we promote—transparency, self-skepticism, reproducibility—are nearly universally endorsed by the research community.  Moreover, most researchers have the motivation to produce reproducible findings and be transparent in how they arrived at those findings.  And, many researchers already have knowledge to get started with registration and sharing data and materials.

With a focus on individuals, everything is ready for change—the new behaviors are values aligned, people have motivation to do the behaviors, and many have the ability to start the behavior.  But, a focus on individuals is not sufficient. The research culture maintains a dysfunctional constellation of incentives and policies that are blocking widespread change toward showing work and sharing.  In particular, the currency of research advancement is publication.  Publishing frequently and in prestigious journals is the key to getting a job, keeping a job, and earning the rewards of scholarship such as grants and more publications.  But, not everything gets published. Researchers are rewarded more for obtaining findings that are [1] novel as opposed to incrementally building on or affirming existing findings, [2] positive—showing a treatment is effective or things are related—as opposed to negative—finding things are ineffective or unrelated, and [3] tidy—showing evidence that all fits together neatly.

These are good things, but they don’t happen in science very often because researchers are studying things that they don’t yet understand. Progress can be slow. But, because the reward system is clear and competition for limited positions in science is strong, individuals are faced with compelling cultural incentives that lead to shortcuts, selective reporting, and other behaviors that undermine the credibility of the findings—and none of which support showing your work or sharing.  Researchers have the values for transparency, but are skeptical that the culture rewards the behaviors associated with those values.

Given this cultural context, we will not succeed in changing the research culture by focusing on individuals’ motivation and ability to be open and reproducible with their research.  We must pursue a comprehensive change strategy.

COS’s strategy for culture and behavior change requires five levels of intervention represented by the pyramid above. These levels are progressive, reflecting the fact that successful implementation of higher levels depends on successful implementation of lower levels. Infrastructure is the base of the pyramid making behavior change possible. We maintain the open-source Open Science Framework (OSF) for researchers to be able to show their work and share.  Researchers can register their studies, post their data and materials openly or with protected access for sensitive materials, and share their outcomes at any time to accelerate and ensure communication regardless of whether it will ever be published.  

To facilitate adoption for busy researchers, OSF needs to be easy to use and integrate with researchers' existing workflows.  OSF users can connect popular services together (e.g. GitHub, Dropbox), and the service can be used privately to help with collaborative management among research teams during the research lifecycle.  This way, their private collaborative management work is seamlessly integrated with transitioning to make some or all of that work transparent or accessible to others.

The infrastructure at the base of our culture change pyramid is reinforced by addressing norms, incentives, and policies.  For shifting norms, we are trying to make the desired behaviors visible. For example, when journals adopt badges to acknowledge authors who preregister their studies and share their data and materials, those badges become signals to other researchers reading those articles that others in my field do these behaviors (descriptive norm setting), and the journal and others value these behaviors (prescriptive norm setting).  Visibility of desired behaviors is critical to accelerate adoption among those that are willing but have not yet adopted.

Incentive and policy interventions address the reward systems.  Publication as the currency of advancement is deeply embedded in the research culture.  So our strategy began with nudging the incentives in the publication process to align with values for transparency and reproducibility.  For example, rather than rewarding researchers contingent on the results they obtain (novel, positive, tidy results), we promote a publishing model called Registered Reports in which journals make decisions about publication based on the importance of the research question and the quality of the methodology. The journal evaluates the research before knowing what results will be obtained.  Researchers are rewarded for rigorous research and making clear their precommitments. This retrofits the existing reward structure with a relatively simple intervention that is values-aligned.

By attending to the critical decision points in an existing reward system, sometimes small interventions can have dramatic impacts without the unforeseeable consequences of needing to destroy and reinvent an entire reward system.  We are now pursuing similar incentive and policy changes at the other key decision points—awarding of grants by funders and hiring and promotion by institutions. An organizing framework called the Transparency and Openness Promotion (TOP) Guidelines provides a mechanism for aligning values, incentives, and policies across the major drivers of the research culture—journals, funders, and research institutions.

When behavior change requires culture change, it is essential to consider the structural features of the culture and how they enable and constrain individuals to behave according to their intentions and values.  Successful, normative, incentive, and policy interventions require effective infrastructure that provides easy transitions from how they behave today. Likewise, enacting that behavior change requires sensible incentives and policies that align with the behavioral tools available to individuals.  For widespread embrace, the changing behavior must be visible to the community to stimulate the diffusion of innovation.

In a future post, I will discuss how the pyramid for behavior change connects with the classic and contemporary models of adopting new technologies.