1. The evaluation was unable to provide a secure estimate of the programme’s impact on reading or writing attainment, due to the low number of pupils who took part and the problems with testing.
2. The programme was relatively expensive compared to other literacy catch-up approaches delivered in the normal school year.
3. Given the relative cost of the programme, even if the indicative impact which was detected had been secure, as a way of improving academic outcomes other approaches are likely to be more cost-effective.
4. Participating pupils enjoyed the programme and were engaged by the literacy workshops and the enrichment activities.
5. The specific challenges of pupil recruitment, attendance and test completion experienced should be considered prior to undertaking any further evaluations of summer schools.

The evaluation sought to compare the literacy skills of pupils who attended the summer school with similar pupils who did not attend the summer school. The indicative effect sizes detected were 0.24 in writing and 0.21 in reading. This can be envisaged as saying that pupils who participated in the project made approximately three additional months’ progress compared to similar pupils who did not participate. However, the effect sizes were not statistically significant, which means that the differences between the pupils who attended the summer school and those who did not could have been down to chance.

The conclusions which can be drawn from this study are severely limited due to the small number of families who agreed to participate in the trial, and to subsequent problems with testing, detailed further below. The low pupil numbers also meant that it was not possible to conduct a separate analysis of the impact of the programme on students eligible for free school meals.

Though the evaluation was primarily focused on measuring attainment, the observations which took place suggested that most children were engaged by and enjoyed the summer school. Staff were supportive and children were encouraged to think creatively about their writing. The interactive approach in poetry sessions in particular appeared to motivate pupils.

This evaluation was set up as an efficacy trial. Efficacy trials seek to test evaluations in the best possible conditions, but they do not seek to demonstrate that the findings hold at scale in all types of schools.

An individually randomised controlled trial design was employed to compare outcomes for pupils attending the summer school with outcomes for pupils in a ​‘business as usual’ comparison group who did not attend. Eligible and consenting pupils were randomised on an individual basis to either the intervention or comparison group. At the end of the intervention period all trial pupils were asked to complete the Progress in English test developed by GL Assessment, with the primary outcome measure being based on two extended writing tasks. Pupils in the comparison group were offered additional reading and writing support delivered on one Saturday in the autumn term of 2013.

However, the intervention and evaluation suffered a number of substantial setbacks. The initial recruitment target of 250 pupils was not achieved by Discover, with only 124 eligible pupils agreeing to take part in the intervention and evaluation. The length of the summer school and the timing of the recruitment phase made recruitment more difficult: many parents had already booked holidays or made alternative plans for the summer by the time they were invited to participate. In addition, a large proportion of pupils in both the intervention and comparison groups did not complete the tests. These problems made it impossible to detect a statistically significant effect size and increased the possibility of bias as the pupils who sat the tests may have been systematically different from those who did not. As a result, the evaluation findings are insecure.