The Impact of Sample Size Misestimations on the Interpretation of ARDS Trials: systematic review and meta-analysis

Background: Indeterminate randomized controlled trials (RCTs) in ARDS may arise from sample size misspecification, leading to abandonment of efficacious therapies. Research Questions: If evidence exists for sample size misspecification in ARDS RCTs, has this led to rejection of potentially beneficial therapies? Does evidence exist for prognostic enrichment in RCTs using mortality as a primary outcome? Study Design and Methods: We identified 150 ARDS RCTs commencing recruitment after the 1994 American European Consensus Conference ARDS definition and published before October 31, 2020. We examined predicted-observed sample size, predicted-observed control event rate (CER), predicted-observed average treatment effect (ATE), and the relationship between observed CER and observed ATE for RCTs with mortality and nonmortality primary outcome measures. To quantify the strength of evidence, we used Bayesian-averaged meta-analysis, trial sequential analysis, and Bayes factors. Results: Only 84 of 150 RCTs (56.0%) reported sample size estimations. In RCTs with mortality as the primary outcome, CER was overestimated in 16 of 28 RCTs (57.1%). To achieve predicted ATE, interventions needed to prevent 40.8% of all deaths, compared with the original prediction of 29.3%. Absolute reduction in mortality ≥ 10% was observed in 5 of 28 RCTs (17.9%) but was predicted in 21 of 28 RCTs (75.0%). For RCTs with mortality as the primary outcome, no association was found between observed CER and observed ATE (pooled OR: β = –0.04; 95% credible interval, –0.18 to 0.09). We identified three interventions that are not currently standard of care with a Bayesian-averaged effect size of > 0.20 and moderate strength of existing evidence: corticosteroids, airway pressure release ventilation, and noninvasive ventilation. Interpretation: Reporting of sample size estimations was inconsistent in ARDS RCTs, and misspecification of CER and ATE was common. Prognostic enrichment strategies in ARDS RCTs based on all-cause mortality are unlikely to be successful. Bayesian methods can be used to prioritize interventions for future effectiveness RCTs.