Data from 25+ of MDRC’s Postsecondary RCTs Available via ICPSR

The mission of the Center for Open Science is to “increase openness, integrity, and reproducibility of research.” We want in.

In 2001, MDRC launched the first ever large scale (n ≈ 1,500) randomized controlled trial (RCT) at a community college. Two decades later, MDRC has conducted 30+ large-scale postsecondary RCTs at 45+ unique institutions, including 67,000+ students.

Over the last few years, we (Mike & John) have led a small group of researchers on a project known as The Higher Education Randomized Controlled Trials, or THE-RCT. Do we regret the project name? Sure, sometimes (Mike’s to blame). But mostly we’re at peace with it now. The centerpiece of THE-RCT is a restricted access data file securely housed at ICPSR that contains standardized, de-identified, student-level data from 26 out of the 31 postsecondary RCTs MDRC has conducted since 2001 (we hope to add those last five in a Version 2.0). 

The point of this blog post is to let you know about this resource because we are citation hungry. Kidding (about the citation hungry part, we really do want you to know about the resource). You can gain access to the data here—you’ll need to jump through some important hoops (e.g., get IRB approval, agree that you won’t attempt to re-identify study participants, etc.). There’s also a user’s guide, codebook (thanks, Colin!) and an RCT-level database (thanks, Austin!)—see documentation (we recommend starting with the PI docs)—that you can download immediately.

Mike and colleagues (Rebecca Unterman and Dorota Biedzio) have used THE-RCT to examine “What Happens to Program Effects After the Program Ends?” This 5-pager presents the findings.

Figure 1 (below) summarizes the main finding from that work. Each line presents findings from one of 29 unique programs represented in the THE-RCT database. The y-axis is the estimated effect of the program on cumulative credits earned. The x-axis represents time in years, centered around the final program semester. Negative time values (zero included) are “in-program” semesters. Positive time values are “post-program” semesters, the emphasis of the research. Since most studies have at least one year of post-program follow-up, that time frame is highlighted—that is, the time between the two vertical dashed lines. The upshot: effects on cumulative credits earned are consistently maintained one year post-program. At least with respect to this outcome, higher education interventions don’t have a fade-out problem! 

We think these findings are substantively important (and the above figure is very cool; thanks, Dorota!), and we’re equally excited to have this as a case study that showcases what can be done with THE-RCT’s restricted access file.

We also have plans to use THE-RCT to examine:

• Which program features/components (for example, enhanced advising or increased financial support) are associated with larger improvements in student success?
• What magnitude of improvement in student success can various higher education programs reasonably expect to achieve?
• How well do short-term effects predict longer-term effects?

We hope that this database will inspire new scholarship on these and other topics that will help improve outcomes for low-income, underrepresented, and underprepared students, who have long been a focus of MDRC’s higher education studies. The database may also serve as a rich resource for methodological research (as in this ongoing project) and teaching and learning. Simultaneously, it would be fantastic if this work serves as an impetus to “increase openness, integrity, and reproducibility of research” in higher education. Do you want in?

For more information on using the database, visit ICPSR. If you’re conducting a higher education RCT and want to include it in THE-RCT, please reach out to us (firstname.lastname  @ mdrc.org)!

Acknowledgements: Funding for The Higher Education Randomized Controlled Trials (THE-RCT) project was provided by the Institute of Education Sciences, U.S. Department of Education, through grant R305A190161 to MDRC. Opinions expressed are those of the authors and do not represent views of the Institute or the US Department of Education. The database for THE-RCT was created with additional support from Arnold Ventures.