Education Endowment Foundation:EEF statement: re-publication of the evaluation of school breakfast clubs

EEF statement: re-publication of the evaluation of school breakfast clubs

Author
EEF
EEF
3 minutes •

Today the Endowment Foundation (EEF) is publishing a revised report of its evaluation of school breakfast clubs, following the notification of an error by the Institute for Fiscal Studies (IFS) in its evaluation. A statement from IFS on this error can be found here.

Many schools in England offer breakfast provision for their pupils, and there are many reasons for doing so. Most importantly, breakfast provision helps ensure that no child has to learn when they’re hungry.

In November 2016, the Education Endowment Foundation (EEF) published an independent evaluation of school breakfast clubs, which found that a model of free, universal, before-school breakfast clubs was a cost-effective way of raising pupil attainment in primary schools

The EEF had appointed the IFS to undertake this evaluation as a randomised controlled trial (RCT). The evaluators’ results were rated by the EEF and the independent peer reviewers as high security’, reflecting the quality, size and design of the study.

In 2019, the IFS reported an error in its evaluation process to the EEF. The mistake resulted from human error in the randomisation code used to allocate schools to intervention and control groups. Because of the error, the evaluation design can no longer be classified as an RCT and has been reclassified as a control group study.

There are two consequences for the results of the study:

  • For Key Stage 1 outcomes, the impact rating as originally reported, of +2 months’ additional progress, is unchanged. The security of the findings has been reduced from high security’ to moderate-to-low security’. 
  • For Key Stage 2 outcomes, the impact rating as originally reported, of +2 months, has been reduced to 0 months. The security of the findings has been reduced from high security’ to moderate-to-high security’. 

The IFS carried out robustness checks on the KS1 outcome result and found that the impact remained, independent of analysis choice. As in the original study, the revised report finds that pupil behaviour, as measured by a teacher survey, improved in breakfast club schools. Attendance at school also improved for children in breakfast club schools, resulting in about 26 fewer half-days of absence per year for a class of 30

Today the EEF is publishing a revised evaluation report, reflecting the changes to the study. The EEF publishes all results, regardless of outcomes, to prevent publication bias

The updated report has been peer reviewed by two independent reviewers, Professor John Jerrim and Dr Alex Sutherland. Their statement is available here

All EEF reports are independently peer-reviewed and the analysis replicated by the EEF’s overarching evaluators at Durham University. In this instance, the EEF, peer-reviewers and Durham colleagues were unable to identify the error in the original report because the randomisation code that was included in the report did not match the code used by the IFS in the randomisation

The revised report still finds a positive impact on KS1 attainment in maths, reading and writing. However, due to the study’s revised security rating, the programme no longer meets the EEF’s Promising Projects criteria, which requires at least moderate-to-high security’ (3÷5 padlock rating). The EEF remains interested in school breakfast clubs and will look for ways to add to the evidence base about them in the future.

This study is still the largest and most robust study of breakfasts in the UK and the findings remain important. Schools considering implementing breakfast clubs should also consider the multiple positive impacts of the approach, including improved pupil behaviour and improved attendance

As always, the EEF recommends that schools carefully consider how to introduce any new programme and monitor its delivery and outcomes to check it is achieving the desired impact. 

Sir Kevan Collins, CEO of the Education Endowment Foundation, said: