There is a large body of evidence that tutoring and small-group tuition is effective, particularly when it is targeted at pupils’ specific needs. It can be particularly beneficial for socio-economically disadvantaged pupils. Because of this, the NTP was set up in response to COVID-19 and the subsequent school closures, to help children catch-up with their learning. The EEF commissioned an independent evaluation of the Academic Mentoring Programme (AMP), which had a Quasi-Experimental Design (QED), meaning that schools who took part in the programme were matched to similar schools who didn’t take part in the programme. Teach First conducted their own process evaluation.

Both the delivery and the evaluation of AMP was affected by the ongoing COVID pandemic. Unanticipated partial school closures from January – March 2021 meant that there was greater variation in schools’ deployment of mentors during that period, and delivery was impacted by COVID-19- related absences of mentors and pupils. The evaluation was also impacted. Challenges included recruiting schools to the evaluation and Key Stage 2 assessments being cancelled. While the AMP evaluation initially aimed to assess the impact of the AM programme on pupils in primary and secondary schools, a lack of available assessment data meant that the evaluation focused on Year 11 pupils only, and was substantially narrowed from its original focus. Year 11 GCSE examinations were also replaced with Teacher Assessed Grades (TAGs). Given uncertainty around the use of TAGs as a research outcome measure, the analysis is considered exploratory and therefore has not been awarded a padlock rating. A more detailed summary of the challenges to the evaluation can be read here.

The analysis for Y11 pupils looked at the effect of the AMP on pupils eligible for Pupil Premium funding (PP), as these groups could be identified for both AMP and non-AMP schools. This was done by comparing the progress of all Y11 PP eligible pupils in schools participating in the AMP, to all PP eligible pupils in schools not participating in the AMP. Year 11 pupils eligible for PP in schools that received AMP made, on average, similar progress in English compared to Year 11 pupils eligible for PP in comparison schools, and in maths, Year 11 pupils eligible for PP in schools that received the AMP, made, on average, slightly more progress (equivalent to 1 months’ additional progress) compared to Year 11 pupils eligible for PP in comparison schools. However , only 4% of Y11 pupils eligible for PP were selected for mentoring in maths and 3% in English (this is despite 49% of pupils selected for AMP overall being eligible for PP, and 46% in the analysed Y11 sample). This is because the number of Year 11 PP-eligible pupils receiving mentoring in AMP schools was small as a proportion of all Year 11 PP eligible pupils, and the number of these Year 11 PP eligible pupils receiving mentoring in maths and/​or English (as opposed to other subjects), was smaller still. This means that in the analysis, the number of Year 11 pupils who actually received the AMP in maths and/​or English was very heavily ​‘diluted’ by the number of pupils who did not, and that the vast majority of the pupils included in the analysis did not receive mentoring. The evaluators therefore believe it is unlikely any of the reported differences were due to the AMP.

This means that the evaluation of AMP is unable to conclude, with any certainty, whether or not the AMP had an impact on the English and maths attainment outcomes of those pupils who received it. It was also not possible to do analysis on a subset of schools with higher proportions of all Y11 PP pupils selected for mentoring, due to small sample sizes.

Teach First has conducted their own process evaluation of the Academic Mentoring Programme, which is available here.

• The analysis was based on 318 schools, half of which took part in AMP and half of which were comparison schools.
  
• The programme was targeted at state-maintained primary and secondary schools serving disadvantaged populations. 89% of the schools met Teach First’s priority criteria, which is based on the proportion of children living in income deprived families (IDACI) and whether the school is in an area of chronic and persistent underperformance (AEA). The remaining 11% of Academic Mentors were matched with schools that had an above average proportion of pupils eligible for Pupil Premium.
• Schools had discretion to decide which pupils should access mentoring through AMP, but were encouraged to consider those from socio-economically disadvantaged households, or those whose education had been disproportionately impacted by COVID-19, such as children with Special Educational Needs and Disabilities (SEND) or children with a social worker. 49% pupils who received mentoring were eligible for Pupil Premium or FSM overall (46% in the analysed Y11 sample), and 23% of pupils were identified as having a special educational need or disability.