Texting Students and Study Supporters

Texting Students and Study Supporters (Project Success) was developed by the Behavioural Insights Team (BIT) and aimed to use text messages to improve GCSE English and maths re-sit pass rates by prompting college students to attend classes and exams, engage with study materials and form better study habits, either through direct contact with the learner or through prompting a dialogue with a nominated study supporter, e.g. a family member. Over the course of the academic year, weekly text messages were sent to students (a total of 36 for English or 37 for maths) and/or their study supporters via the BIT Promptable text messaging service.

accessibility

Key Stage 5

Key stage

school

Cross curriculum

Subject

EEF Summary

Using text messages to improve parental engagement and increase outcomes is a well-evidenced approach. The EEF previously funded an evaluation of text messages in secondary school, which found small positive impact on GCSE maths outcomes. This evaluation explored whether the approach could successfully improve the outcomes of students re-sitting GCSE exams in further education settings.

Our trial of Project Success included 3,779 students across 31 further education settings. This independent evaluation found that students who received text messages or had study supporters that received messages did not, on average, have higher pass rates in GCSE re-sits. These results have a high security rating.

There are some important limitations to the evaluation which may explain the different results from the Texting Parents studies that have taken place with younger age groups. Firstly, the measure of impact was whether students passed or failed the GCSE resits. This binary measure means that it is harder to detect the type of small impacts seen in the Texting Parents trial. A second limitation is that students had to opt into the trial – the process evaluation found that these students were already highly motivated. Text messages were less likely to have an impact on students who were already motivated.

The EEF has no further plans for a trial of Texting Study supporters. 

Research Results

Pass GCSE re-sit in English or maths: texts to students

0
Months' Progress
Evidence Strength

Pass GCSE re-sit in English or maths: texts to supporters

0
Months' Progress
Evidence Strength

Pass GCSE re-sit in English or maths: texts to students and study supporters

0
Months' Progress
Evidence Strength

Pass GCSE re-sit in English or maths: ever eligible for FSM at the end of KS4 (sub-sample)

0
Months' Progress

N/A

Were the schools in the trial similar to my school?

  • There were 31 further education colleges included in this trial
  • 30 of the colleges were in urban areas, while one was in a rural area
  • Ofsted ratings were missing for 14 of the 31 colleges
  • Of the 17 colleges that had an Ofsted rating, 13 were rated good and 4 were rated outstanding. 

Could I implement this in my school?

  • Text messages were sent to students and study supporters via BIT’s texting platform Promptable (https://promptable.com)
  • Tutors for participating students attended a workshop, detailing their responsibilities
  • Messages were provided to tutors by BIT, but tutors could tailor messages in line with their own schedule of work.
  • Some barriers to delivery were identified – student concerns about data security and the length of time taken to sign up were mentioned in the process evaluation
account_circle

Volunteers

Delivered by

person

Individuals

Participant group

date_range

1 Year

Intervention length

How much will it cost?

The costs per pupil per year over three years for delivering each of the three arms of Project Success are £13 for text messages to students or study supporters only, or £15 for text messages to both.

£

£15

Cost per pupil

people_outline

English and maths tutors

No. of Teachers/TAs

today

1 Day

Training time per staff member

Evaluation info

Schools

31

Pupils

3,779

Key Stage

Key Stage 5

Start date

March 2017

End date

May 2020

Type of trial

Efficacy Trial

Evaluation Conclusions

  1. There is no evidence that the Project Success intervention had any impact on the GCSE English or maths re-sit pass rate for further education college students.

  2. There is no evidence that the Project Success intervention had any impact on the attendance of further education college students re-sitting GCSE English or maths.

  3. The intervention did not have a differential impact on the GCSE re-sit pass rate by gender or by eligibility for free school meals (at the end of KS4). The subject being re-examined or the number of re-sits being taken also did not lead to differential effects from the intervention.

  4. The use of mobile phone technology was perceived as a highly appropriate, effective, and low risk means of engaging with the target student cohort, though mobile phone use was less popular among study supporters.

  5. There were significant limitations to the programme’s ability to engage those who may need it the most as it was the highly motivated students that were more engaged with their studies and with college generally who were more likely to sign up to the intervention.


  1. Updated: 4th May, 2020

    Printable project summary

    1 MB pdf - EEF-texting-students-and-study-supporters.pdf

  2. Updated: 30th April, 2020

    Evaluation report

    1 MB pdf - Texting_Students_and_Study_Supporters_report_(final).pdf

  3. Updated: 25th September, 2017

    Project Protocol

    534 KB pdf - Post_16-_Texting_students_protocol.pdf

  4. Updated: 15th December, 2017

    Project Protocol (Amended)

    539 KB pdf - Post_16_-_Project_SUCCESS_AMENDED.pdf

  5. Updated: 18th June, 2018

    Statistical Analysis Plan

    497 KB pdf - Post_16_-_Texting_Students_(Project_SUCCESS)_SAP.pdf

Full project description

Project Success was developed by the Behavioural Insights Team (BIT) and aimed to use text messages to improve GCSE English and maths re-sit pass rates by prompting students to attend classes and exams, engage with study materials, and form better study habits, either through direct contact with the learner or through prompting a dialogue with a nominated study supporter such as a family member. The text messages were targeted at further education college students aged between 16 and 18 years and who were re-sitting maths or English. Over the course of the academic year, weekly text messages (a total of 36 for English or 37 for maths) were sent to students or their study supporters (or both) via the BIT Promptable text messaging service. This project was funded by the Education Endowment Foundation (EEF) as part of a joint initiative with J.P. Morgan to explore how to improve outcomes for disadvantaged 16- to 18-year-old students who achieve below a grade 4 in GCSE English or maths.

The evaluation included 3,779 students across 31 further education (FE) colleges in England. The efficacy trial used a four-armed, multi-site, randomised controlled design with individual random assignment to each trial arm. The four trial arms were: student received text messages, study supporter received text messages, both student and study supporter received text messages, and control (no text messages). The efficacy trial investigated the extent to which the receipt of text messages (either by the student, a study supporter, or both) improved students’ college attendance and GCSE maths or English re-sit results. The trial also explored whether any effect of receiving text messages varied according to the student’s gender or whether they had ever been eligible for free school meals. The primary outcome was obtaining a pass grade (4–9) in GCSE English or maths upon re-sitting, with lesson attendance being assessed as a secondary outcome. Where students were re-sitting both English and maths, the subject used for the intervention was randomly selected; the same subject was analysed as the primary outcome.

Alongside the impact evaluation, a mixed-methods implementation and process evaluation (IPE) was carried out. This included: a short, online diagnostic survey of students developed by BIT and NatCen, which was completed as part of the recruitment process; an observation of a tutor workshop; and interviews with college project and subject leads, study supporters, and students. The trial started in September 2017 and concluded in October 2019 with the intervention being delivered throughout the academic year 2017/2018.