Affordable Maths Tuition
Affordable Maths Tuition is a one to one tutoring programme where pupils receive maths tuition over the internet from trained maths graduates in India and Sri Lanka. It is delivered by the organisation Third Space Learning (TSL). Tutors and pupils communicate using video calling and a secure virtual classroom. Before each session, the pupils’ normal classroom teachers are able to select lessons from Third Space Learning’s maths curriculum to target individual learning issues. In this evaluation, the tutoring sessions took place once a week, at the same time each week. The intervention was targeted at Year 6 pupils who were working at Key Stage 2 level 3 or an insecure KS2 level 4, and was delivered over 27 weeks from September 2014 to May 2015 by Third Space Learning (TSL) in an initial testing phase, with support from Nesta.
The impact of the intervention was evaluated using a randomised controlled trial design, involving 64 schools and 600 pupils. Schools were randomised to either receive the intervention or deliver 'business as usual’ teaching, which might have involved intensive one to one support for maths. A process evaluation was undertaken to understand the perceptions of teachers and pupils, assess whether the intervention was delivered as intended, and inform any future development of the intervention. The evaluation should be considered an efficacy trial. Efficacy trials aim to test whether the intervention can succeed under ideal conditions.
The York Trials Unit
Organising your school
The following conclusions summarise the project outcome
The impact evaluation found no evidence that the intervention had an impact on Key Stage 2 maths, compared with ‘business as usual’ teaching and support in Year 6.
Teachers were largely positive about the online tuition, and reported that it appeared to improve pupils’ comprehension, verbal fluency, and confidence in maths.
Schools should consider whether their computer network can support the implementation of an online programme. Teachers were positive about the technical support and user experience of the programme, but some experienced technical challenges in the implementation.
As the online tuition is a ‘talking’ intervention, it appeared to work better when the pupils were spaced out in larger rooms so that the noise from other sessions was less distracting.
Future research could examine whether the programme has an impact on pupils’ comprehension, mathematical capacities, verbal fluency, and confidence in maths, as this was an outcome reported by teachers.
What is the impact?
The impact evaluation found no evidence that the intervention had an impact on the primary outcome of the Key Stage 2 maths test, compared with ‘business as usual’ teaching in Year 6. There was also no evidence that the intervention had an impact on the Key Stage 2 reading test, or a differential impact on pupils who were eligible for free school meals or pupils who took part in more tutoring sessions. The intervention was generally implemented as intended, so the lack of impact does not appear to result from poor fidelity of implementation. The process evaluation suggested several potential explanations for the lack of impact observed:
- Some pupils complained they were interrupted when working as the tutor could not see that they were still tackling a problem, or being pushed too hard when they did not understand. This may have been due to the lack of face-to-face contact with the tutors.
- Short-lived technical issues at the beginning of the project.
- Control group activity, as high-stakes testing takes place in Year 6 and we might expect control schools to have employed other forms of intensive tutoring. Five schools in the control group reported implementing one to one tuition using face-to-face tutors, and this may have diluted the effect slightly.
The process evaluation also described many positive aspects to the intervention. Schools were largely positive about the online tuition, and confident that it was beneficial for their pupils in terms of improved comprehension, verbal fluency and confidence in maths. Pupils were also generally positive about the impact of the intervention on their own maths capabilities. Teachers commended the programme for its clarity and simplicity, good content and objectives linked to the curriculum. Delivery of the intervention was well supported by TSL who provided good technical and on-site support.
This evaluation was undertaken when Third Space Learning was in a relatively early stage of its development, and the findings should be considered in this context. TSL is committed to developing the programme and has already instituted many of the improvements that this report recommends.
|Group||Effect Size (95% confidence interval)||Estimated months' progress||Security rating||Cost rating|
|Affordable Maths vs control group||-0.03 (-0.35 to 0.28)||-1 months progress|
|Affordable Maths vs. control group (FSM)||-0.08 (-1.23 to 0.74)|
|Since this report was published, the conversion from effect size into months of additional progress has been slightly revised. If these results were reported using the new conversion, the overall result would be reported as 0 months of additional progress rather than -1.|
How secure is the finding?
This evaluation had moderate security. It was a well-designed randomised trial, and relatively few pupils were lost to the analysis due to issues such as moving school. Two padlocks were removed from the rating because the trial was only designed to reliably detect an impact of four months’ progress or more. The trial was designed in this way because Affordable Maths Tuition is an intensive intervention, and would require a larger impact to be cost-effective.
How much does it cost?
The programme cost £378 per pupil for 27 weeks of tuition. There was some time required at the beginning of the year to set up the intervention. On average, teachers spent 10 minutes per pupil to create their academic profile and account. At the beginning of the programme, it took on average 90 minutes to set up and test the local computers for a class. Each week, it took teachers about 25 minutes to select the lesson for the group. Supervising the sessions required 45 minutes of a teaching assistant’s time.
The one-to-one tuition strand of the Teaching and Learning Toolkit concludes that one-to-one interventions are often effective, but can be costly because they involve an adult providing intensive individual support to just one pupil. The EEF funded an evaluation of Affordable Maths Tuition to investigate whether online tuition could provide a cost-effective alternative to traditional face-to-face approaches. However, the evaluation did not provide any evidence that Affordable Maths Tuition had an impact on pupils’ attainment in Key Stage 2 maths.
One-to-one tuition is still a promising area for further research, and the EEF will continue to test different approaches. The full list of one-to-one tuition projects funded by the EEF is available here (link).
Schools should consider piloting small group tuition as a potentially more cost effective alternative to one-to-one tuition. The Teaching and Learning Toolkit compares the evidence for one-to-one tuition against the evidence for small-group tuition. In some cases one-to-one tuition has led to greater improvement, while in other cases tuition in groups of two or three has been equally or even more effective compared to one-to-one.