Education Endowment Foundation:Improving power calculations in educational trials

Improving power calculations in educational trials

Study investigating parameters commonly used for statistical power and sample size calculations.

Authors: Akansha Singh, Germaine Uwimpuhwe, Dimitrios Vallis, Nasima Akhter, Tahani Coolen-Maturi, Jochen Einbeck, Steve Higgins, Martin Culliney, Sean Demack

The aim of this study was to investigate and empirically derive parameters commonly used for statistical power and sample size calculations to better inform future trial design.

Statistical power analysis is a crucial aspect of trial design. It helps researchers determine the appropriate sample size needed to detect meaningful effects and ensures that the study has a high probability of detecting real differences if they are present.

This report mainly provides estimates for parameters such as correlation coefficients between test scores and intra-cluster correlations (ICC) from Early Years Foundation Stage to Key Stage 4. Estimates are derived through four sets of data: i) all English schools (using the whole National Pupil Database – NPD), ii) a random sample of English schools (also derived from the NPD), iii) schools that have participated in an EEF trial (trials available in the EEF Archive) and iv) the EEF schools with NPD data.

The EEF encourages evaluators to incorporate these findings when designing forthcoming trials. The complete analysis is available in the report, and all estimates can be accessed through the provided spreadsheets.