Texting Students and Study Supporters (Project Success) was developed by the Behavioural Insights Team (BIT) and aimed to use text messages to improve GCSE English and maths re-sit pass rates by prompting college students to attend classes and exams, engage with study materials and form better study habits, either through direct contact with the learner or through prompting a dialogue with a nominated study supporter, e.g. a family member. Over the course of the academic year, weekly text messages were sent to students (a total of 36 for English or 37 for maths) and/or their study supporters via the BIT Promptable text messaging service.
Using text messages to improve parental engagement and increase outcomes is a well-evidenced approach. The EEF previously funded an evaluation of text messages in secondary school, which found small positive impact on GCSE maths outcomes. This evaluation explored whether the approach could successfully improve the outcomes of students re-sitting GCSE exams in further education settings.
Our trial of Project Success included 3,779 students across 31 further education settings. This independent evaluation found that students who received text messages or had study supporters that received messages did not, on average, have higher pass rates in GCSE re-sits. These results have a high security rating.
There are some important limitations to the evaluation which may explain the different results from the Texting Parents studies that have taken place with younger age groups. Firstly, the measure of impact was whether students passed or failed the GCSE resits. This binary measure means that it is harder to detect the type of small impacts seen in the Texting Parents trial. A second limitation is that students had to opt into the trial – the process evaluation found that these students were already highly motivated. Text messages were less likely to have an impact on students who were already motivated.
The EEF has no further plans for a trial of Texting Study supporters.
- There is no evidence that the Project Success intervention had any impact on the GCSE English or maths re-sit pass rate for further education college students.
- There is no evidence that the Project Success intervention had any impact on the attendance of further education college students re-sitting GCSE English or maths.
- The intervention did not have a differential impact on the GCSE re-sit pass rate by gender or by eligibility for free school meals (at the end of KS4). The subject being re-examined or the number of re-sits being taken also did not lead to differential effects from the intervention.
- The use of mobile phone technology was perceived as a highly appropriate, effective, and low risk means of engaging with the target student cohort, though mobile phone use was less popular among study supporters.
- There were significant limitations to the programme’s ability to engage those who may need it the most as it was the highly motivated students that were more engaged with their studies and with college generally who were more likely to sign up to the intervention.