We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
The gold standard for whom? Schools' experiences participating in a randomised controlled trial.
- Abstract
Background: Randomised controlled trials (RCTs) have long been considered the gold standard in education research. Federal funds are allocated to evaluations that meet What Works Clearinghouse standards; RCT designs are required in order to meet these standards without reservations. Schools seek out interventions that are research based, in other words, interventions that have demonstrated effects in RCTs, creating a cycle that incentivises RCT research. However, RCTs pose tremendous obstacles for school‐based participants. Methods: This paper presents a descriptive case study of an RCT of an adolescent reading intervention implemented in four districts over three years, drawing on interviews and document analysis to illustrate the challenges that participation in an RCT poses for schools. Results: In two participating districts, the random assignment requirement caused major difficulties with recruitment and retention. Of the 15 schools that began implementation in fall of year 1, 11 left the study by fall of year 2. Of those 11, eight explicitly stated that the random assignment requirement was the reason they withdrew. Stakeholders expressed that they felt it was unethical to withhold the intervention from some students. Conclusions: This rate of attrition casts doubt on the external validity of the programme evaluation, as these schools are likely to systematically differ from schools that met the requirements of random assignment. Suggestions are offered for alternative methods that may be used to evaluate promising interventions, in ways that are better aligned with real‐world educational contexts. Highlights: What is already known about this topicRCTs are considered the gold standard in education research, and research funding is distributed accordingly.Promising interventions often fail to replicate under effectiveness conditions. What this paper addsThis paper examines the challenges of participation in an RCT for schools.Student‐level random assignment presents a barrier to school participation in RCTs.High attrition rates cast doubt on the external validity of the findings. Implications for theory, policy and practiceWhen designing intervention research beyond the efficacy stage, feasibility for practitioners and external validity should be given greater weight.Alternative methods for evaluating promising interventions are proposed, including switching replications designs, RCTs comparing interventions with no business as usual control group, regression discontinuity, matched non‐random comparison groups, and improvement science approaches.
- Subjects
RANDOMIZED controlled trials; EDUCATION research; GOLD standard; SCHOOLS; METHODOLOGY
- Publication
Journal of Research in Reading, 2022, Vol 45, Issue 3, p406
- ISSN
0141-0423
- Publication type
Article
- DOI
10.1111/1467-9817.12395