1. There is no evidence that the Hallé Shine on Manchester programme had an impact on the reading attainment of children in the trial. This finding was consistent across all age groups, for pupils eligible for free school meals, and for all three years of the trials.
2. There is no evidence that the Hallé Shine on Manchester programme had an impact on the attainment of children in mathematics, or attitudes to reading, maths, music, and school of the children in the trial.
3. Attendance of eligible pupils was often low and considered as a barrier to successful implementation. Reasons for low attendance included pupils’ lack of availability to attend the Saturday sessions, variable parental engagement with the programme, and limited time at the beginning of the programme for schools to engage children and parents.
4. The process evaluation revealed a positive picture of involvement and engagement for those pupils who attended the Saturday school activities. Evaluators observed good working relationships between the teachers and pupils, and positive and purposeful learning environments in lessons. All stakeholders felt pupils were making noticeable improvements in behaviour, confidence, and development of social skills.

The findings demonstrated no evidence that the Hallé Shine on Manchester programme had an impact on the primary outcome of reading attainment of children in the trial. This outcome was consistent across all age groups, for both the pilot and main trial. This effect also applied to pupils eligible for free school meals.

No positive effects were observed for the secondary outcome measures of attainment in maths and attitudes to reading, maths, and music. Further exploratory analysis of Key Stage 2 results also showed no signs of positive impacts in reading, maths and writing for Year 5 and 6 pupils in the first two years of the trial.

Pupil recruitment and attendance was often low and a barrier to successful implementation. Reasons for low attendance included pupils’ lack of availability to attend the Saturday sessions, variable parental engagement with the programme, and limited time at the beginning of the programme for schools to engage children and parents. Nevertheless, further analysis that took attendance into account did not alter the overall finding.

In contrast, the process evaluation revealed a positive picture of involvement and engagement for those pupils who attended Saturday school activities. Good working relationships between the teachers and pupils were observed, with positive and purposeful learning environments demonstrated in lessons. Teachers, parents, and musicians all felt pupils were making noticeable improvements in behaviour, confidence, and the development of social skills. Although pupil engagement and participation in lessons were generally very good, more challenging feedback may have helped pupils’ learning.

These results are generally consistent with existing evaluations of out-of-school programmes, showing small or no effects on literacy or maths achievement when robust research methods are adopted. Small positive effects on academic outcomes have previously been observed in other research when qualified teachers were used (as is the case in this evaluation); however, this study did not confirm this finding. Another EEF-funded pilot evaluation of a similar SHINE programme with pupils in Year 7 also found only very small non-statistically significant effects of

Findings from this study have moderate security. The study used randomised controlled trialAn RCT is used evaluate an educational programme by assigning settings to one of two groups: the intervention group, who receive the programme or the control group, who continue with business as usual. This ensures that any differences in outcomes can be confidently attributed to the programme, providing a robust estimate of the impact and contributing to the evidence for what works in improving educational outcomes. designs (RCTs) with random allocation at the level of the year group (i.e. schools identified eligible pupils from two year groups, one of which was randomly selected to participate in the intervention). The trials were efficacy trials, which aim to test whether the intervention can succeed under ideal conditions.

The trials were smaller in size than expected because not as many pupils were recruited as planned and because a significant proportion of pupils did not complete all the tests at the end of the project (15% in Year 1, 22% in Year 2, and 25% in Year 3).

The costs of the intervention were calculated by dividing the grant provided to each school to run the programme for a year by the number of pupils the intervention could accommodate. This cost estimate is £727 per pupil per year.