In 2015, the Department for Education asked the EEF to explore the creation of an ‘Education Data Service’ (EDS), similar to the Ministry of Justice Datalab, that could provide a quick and cost-effective means for education organisations to obtain estimates of the impact of their programmes on pupil outcomes based on samples matched on observable background characteristics. An EDS would largely draw on administrative data from the National Pupil Database (NPD) to create the matched estimates of impact, although it would be possible in theory to use data from other sources, such as commercial test providers. This could provide useful early evidence of promise and could help inform decisions about which projects should progress to EEF trials.
In 2017, the EEF commissioned FFT Education Datalab to pilot an EDS. The pilot aimed to test the feasibility of providing such a service, exploring the methodological needs, as well as the practical and strategic implications, of a fully implemented EDS. This was done by evaluating seven programmes and reflecting on the process in order to draw out key challenges and recommendations.
During the pilot, organisations that volunteered to take part could choose whether to publish their impact evaluation results. Three organisations decided to publish results for their programmes, which can be found below:
Magic Breakfast evaluation report
Mathematics Mastery evaluation report
SAM learning evaluation report
The final report for the pilot is published below. Ultimately through the pilot FFT Education Datalab have shown that it is feasible to use the National Pupil Database to calculate non-experimental treatment effects of educational interventions when the following two conditions are met:
- Relevant outcomes are available in the NPD (such as attainment or attendance)
- Programmes, projects and interventions:
- Are aimed at the whole school (or whole cohorts); or
- Have selection mechanisms that can be explicitly modelled using the data in the NPD
The report finds that the future operation will be shaped by three key considerations:
- Arrangements for accessing NPD data
- The level of methodology consistency required
- The number (and type) of evaluations to be conducted each year
- The EEF will explore the scale-up of an Education Data Service in 2021.