One of the challenging aspects of conducting honest, robust evaluations is that they often suggest – honestly and robustly! – that the programmes and practices you intuitively favour are not as effective as you imagined. This perhaps should not come as a surprise.
Getting positive results in randomised controlled trials (RCTs) conducted at scale in real world conditions is difficult, illustrated by the fact that only 12% of the RCTs commissioned by the Institute of Education Sciences (a US equivalent to the EEF) have shown positive effects. Whilst some of these findings may be due to poor implementation rather than poor interventions, it nevertheless points to a sobering scenario where, despite our best intentions, many of our efforts to improve teaching and learning are not having the impact we expect.
It is because of these trends that I think the latest evaluation findings from the EEF deserve such attention. The two projects that reported this week, Nuffield Early Language Intervention and REACH, both involve Teaching Assistants (TAs) delivering high-quality, targeted support to pupils who were struggling with their literacy (either one-to-one or in small groups). Both interventions produced over three additional months progress to pupils’ learning, compared to existing practice.
Whilst these results are encouraging in themselves, taken in the context of our previous evaluations this means that all six projects involving TA-led literacy/numeracy interventions have shown positive impacts on pupil’s learning, typically adding around three to four additional months progress (see table below). Encouragingly, there are signs that these interventions disproportionality benefit low attaining and pupils eligible for Free School Meals, and so could be effective approaches to ‘narrow the gap’.
Having worked in communicating the findings from research for over ten years, this is by far the most consistent, positive set of results I have seen for real-world RCTs, which I imagine will be difficult to repeat. So why might this be the case and what are the implications? As is so often the case, the devil is in the detail.
Crucially (and you can’t emphasise this enough!), the positive effects observed in these projects only occur when TAs work in structured settings with high-quality support and training. When TAs are deployed in more informal, unsupported instructional roles, they can impact negatively on pupils’ learning outcomes. This was one of the key findings from the large-scale Deployment and Impact of Support Staff (DISS) study, which looked at typical ways in which TAs were being used in UK schools, and the impact they were having on pupil attainment. The research team observed that the majority of TAs spent most of their time working in a direct, but informal, instructional role with pupils on a small group and one-to-one basis (both inside and outside of the classroom). Shockingly, the amount of time TAs had in this role was negatively correlated with pupil progress; a particularly striking finding given that schools spend nearly £5bn employing TAs each year, with more TAs than teachers now working in primary and nursery schools.
So what makes the difference here? Clearly, it is the amount and type of training, coaching and support provided by the school i.e. the content and delivery of the intervention, not whether an intervention is being used. In this sense, structured evidence-based programmes provide an excellent means of aiding high-quality delivery.
Last year, EEF published its first guidance report for schools, Making best use of Teaching Assistants, which I would recommend as a starting point for anyone looking to review and reframe the use of TAs in their school or classroom. Three of the seven recommendations cover the use of TAs in delivering out-of-class, targeted interventions in detail (see image below). To help you act on this evidence we have created a range of supplementary resources, such as an ‘interventions healthcheck’, and a Red Amber Green (RAG) self-assessment tool to monitor current practice against the guidance.
Overall, we suggest schools should adopt one or two well-chosen, and well-implemented, TA-led interventions, judiciously used to complement and extend class-based teaching and learning. Intervention sessions should be brief (<30mins), regular and sustained, with clear objectives and expectations. Teachers should have a good awareness of the structure and coverage of the programmes, and help pupils make connections between their learning in interventions and the wider curriculum. Headteachers need to ensure there is space and time for these interactions to occur.
Over the next few years the EEF will continue to evaluate both new TA-led interventions, as well as re-evaluate projects that have shown promising effects, at greater scale. We want to capture the long-term impacts of these approaches, as well as better understand how they can be integrated into general classroom teaching and learning.
Of course, there are no guarantees in either research or practice, and only schools themselves can work out what is most suitable for their particular context. Nevertheless, the emerging evidence around high-quality, TA-led interventions suggest they should be a serious consideration for anyone looking to improve the learning for lower-attaining pupils.
Jonathan Sharples, Senior Researcher at the EEF, explains why last week’s evaluation reports deserve such attention. This first appeared on Schoolsweek.co.uk.