The Education Endowment Foundation (EEF) led the management of the Tuition Partners (TP) pillar of the National Tutoring Programme (NTP) in 2020/2021, funded as part of the government COVID-19 catch-up package. The TP programme allowed schools to access subsidised tuition from a list of 33 tuition partners, quality-approved by the EEF, to support pupils who have missed out the most as a result of school closures due to the COVID-19 pandemic. The focus was to support disadvantaged pupils, in particular those eligible for Pupil Premium; however, schools had the flexibility to select pupils to take part.
The EEF commissioned the National Foundation for Educational Research (NFER) and the Behavioural Insights Team (BIT) to run nimble randomised controlled trials (RCTs) with approved NTP Year 1 (2020−21) Tuition Partners to evaluate different strategies to reach and engage both schools and pupils in the programme, looking at the most effective ways to get schools signed up and pupils to attend tutoring sessions.
The four approaches were:
1. Testing different types of recruitment emails (performed no better than business as usual)
This RCT explored the impact of two distinctive types of recruitment emails sent on school sign-up to the Tuition Partners programme, provided by EM Tuition. One email included a testimonial from a headteacher on the benefits of tutoring, the other included a summary of the research evidence of the benefits of tutoring. EM Tuition sent recruitment emails during February and March 2021 to 1,949 primary, secondary, and special schools in areas of England where they offered tutoring provision. Schools were randomly allocated to receive one of the two types of email messages.
This project was evaluated by the National Foundation for Educational Research (NFER).
2. Leveraging similarity to improve pupil attendance (found to improve attendance – see below for further detail)
The intervention involved a web-based survey (Snap Survey) in which pupils and tutors were asked about their personal interests, hobbies and preferences. Once completed, both tutors and pupils received instant feedback on their similarities. Tutors also received reminders of their similarities with their pupils for the subsequent five weeks, including some suggested conversation prompts.
The target group were tutors and pupils associated with The Tutor Trust, Manning’s Tutors, Tute and Schools Partnership Tutors: both primary and secondary age pupils participated in the trial. 701 tutors and 8,922 pupils were involved in this trial.
This project was evaluated by the Behavioural Insights Team (BIT).
BIT have developed activities and resources to further support the use of Snap Survey. The survey is available for free here (in response to feedback, there is also an ‘offline’ version, available for download here). You can also find a 2‑page summary, step-by-step guide for tutors, and video explanation.
3. Prioritising tutoring relationships to improve pupil attendance (performed no better than business as usual)
The intervention involved completing a web-based activity in which tutors were encouraged to reflect on a time they connected with a pupil or someone else, before learning a range of relationship-building strategies used by other tutors, and then having an opportunity to develop their own, personal relationship-building strategy. Tutors then received emails after completing the activity reminding them of their personal strategy.
The target group were tutors from Action Tutoring, CoachBright, and The Access Project. All tutors working with the TPs involved during the trial period were eligible for the trial: 1,144 tutors and 2,587 pupils were involved in this trial.
This project was evaluated by the Behavioural Insights Team (BIT).
4. Reminders to improve pupil attendance and engagement (performed no better than business as usual)
The purpose of this trial was to test whether behaviourally informed reminder messages, sent directly to pupils via email, can improve their attendance at tutoring sessions, compared to a standard reminder message also sent via email. The intervention was testing the impact of the specific wording of the reminder, rather than the effectiveness of sending a reminder message at all versus no message. The target group were secondary school pupils (Years 7 – 11) who took tutoring sessions with the NTP Tuition Partner, Pearson, during 10th March – 27th August 2021.
The intervention involved pupils receiving one behaviourally informed reminder message each week. The messages were automatically sent by Bramble, the online tutoring platform used by Pearson, the day before the pupil’s tutoring session. In total, 4,463 pupils were included in the trial.
This project was evaluated by the Behavioural Insights Team (BIT).
1. Leveraging similarity to improve pupil attendance
- Across the four Tuition Partners, pupils in the treatment arm had an attendance rate at tutoring sessions that was 4.2pp (95% CI: 1.5pp, 6.8pp) higher than those in the control group, meaning that the relationship-building activity performed better than the tutoring providers’ ‘business-as-usual’. However, this positive effect was larger for two of the Tuition Partners and negligible for the other two, suggesting that a wider exploration of variation between tutoring organisations and implementation factors is also helpful.
- Pupil Premium pupils in the treatment arm had an attendance rate at tutoring sessions that was 2.6pp (95% CI: ‑0.004pp, 5.6pp) higher than those in the control group, meaning that the relationship-building activity performed better than the tutoring providers’ ‘business-as-usual’. Again, this positive effect was larger for two of the Tuition Partners and negligible for the other two Partners.
- In the implementation and process evaluation (IPE), completion rates were collected to better understand delivery implementation. 68% of tutors were reported to have partially completed the intervention whilst 21% of tutors were reported to have fully completed the intervention.
- Interviews with tutors suggested that the intervention may have been particularly effective for tutors who had less experience.
2. Prioritising tutoring relationships to improve pupil attendance
- Pupils in both arms in the trial had comparable attendance rates at tutoring sessions, meaning that the light-touch training intervention did no better or worse than the TPs’ ‘business-as-usual’.
- Pupil Premium pupils in both arms in the trial had comparable attendance rates at tutoring sessions, meaning that the light-touch training intervention did no better or worse than the TPs’ ‘business-as-usual’.
- 60% of tutors were recorded as completing the materials provided for the intervention.
- Tutors found the activity to be a useful overview of relationship-building strategies and identified barriers that may have affected impact, including: the time between completing the activity and the first tutoring session; the degree of engagement from pupils, and the relevance of the relationship-building strategies for an online context.
3. Reminders to improve pupil attendance and engagement
- Pupils in all three arms in the trial had comparable attendance rates at tutoring sessions, meaning that the behaviourally informed messages did no better or worse than the control message.
- Pupil Premium pupils in all three arms in the trial had comparable attendance rates at tutoring sessions, meaning that the behaviourally informed messages did no better or worse than the control message.
- The proportion of reminder messages opened by pupils was 26%, meaning that a significant number of pupils did not read the body of the reminder message. However, pupils not opening the reminder messages may still have read the subject lines, which varied each week for the treatment conditions, and did incorporate behavioural insights.
- Comparison of the average open rates for the reminder messages across the trial conditions suggests that the behaviourally-informed messages were more likely to be opened by pupils (a difference of approximately 3 percentage points).