EEF Blog: A lasting impact - 6 lessons from the evaluation of ABRA
The EEF's Thomas Martell and Prof. Jonathan Sharples on six lessons to draw from the updated independent evaluation of our trial of ABRA...
This week, we've published an addendum report (starts at p.105) to the independent evaluation of the ABRA project, originally published in November 2016. ABRA is a 20-week online literacy programme based around a series of texts delivered by a teaching assistant to small groups of pupils.
That original evaluation, conducted for the EEF by the London School of Economics, showed that children who took part in the programme were better at reading (+3 months’ additional progress) than their peers in the comparison group at the end of Year 1.
The latest findings indicate that, on average, those children who participated in the programme were continuing to do better than their comparison-group peers a year after the intervention finished (as measured by Key Stage 1 SATS).
These findings add to the growing international evidence that ABRA is an effective way to improve literacy outcomes for young children. This will, we hope, encourage still more primary schools to sign up to take part in our larger effectiveness trial that is currently recruiting.
Here are six lessons that this project exemplifies:
1. Use a balanced approach that develops word recognition and language comprehension
The Improving Literacy in Key Stage 1 guidance report emphasises the need for a balanced approach to teaching reading that develops word recognition and language comprehension. This is something the ABRA project aimed to achieve. The idea of a balanced approach to reading is summarised in the Simple View of Reading, which can also be used to identify need and prioritise teaching.
2. Reading fluency should (probably) be given more focus
In addition to phonics and reading comprehension, the ABRA programme aims to develop reading fluency. Fluency is not included in the Simple View of Reading, and it has often been highlighted as a neglected goal of reading. From our experience running workshops based on our guidance reports and talking to hundreds of teachers, it seems that many teachers lack confidence about what reading fluency is and how it can be developed (quick explainer here). The findings of this project are consistent with the wider evidence that explicitly developing reading fluency is beneficial.
3. Improving reading also improves writing
The ABRA project focuses on reading, yet the analysis also finds an improvement in children’s writing. Perhaps this is not surprising: any professional writer would likely tell you that reading helps to improve writing – and this view is supported by the findings of a recent meta-analysis – yet reading and writing are sometimes thought to be very distinct. Perhaps there are benefits to recognising the close relationship between reading and writing.
4. Interventions work well when they are short, sharp, and regular
The EEF has published seven independent evaluations of teaching assistants (TAs) delivering high-quality, targeted support to pupils who were struggling with their literacy (either one-to-one or in small groups). Six of these have shown positive impacts on pupils’ learning. A consistent feature of these interventions are that they are brief, regular and sustained, offering targeted support to pupils in a way that supplements the work of the teacher, and minimises their time away from the classroom.
5. Teaching assistant-led interventions benefit from focused training and high-quality support
Effective teaching assistant-led literacy interventions typically involve TAs working in structured settings with high-quality support and training. When TAs are deployed in more informal, unsupported instructional roles, they can impact negatively on pupils’ learning outcomes. ABRA exemplifies the high-quality initial training and ongoing support for TAs that is typically needed to improve pupil outcomes.
6. Teaching assistants can have a lasting impact on children’s attainment
Although there is good supporting evidence for teaching assistant-led literacy interventions, there is limited evidence on the long-term impacts of such approaches. Encouragingly, the finding in this updated evaluation of ABRA shows that the short-term benefits of TA-led interventions can lead to longer-term impacts, as measured by national assessments.
This trial tested both the original online version of ABRA and a non-digital, ‘pen and paper’ version using the same content. The original analysis and the further analysis reporting here found larger effects for the non-digital version of ABRA.
It is possible that the difference is due to how the two interventions were implemented. For instance, the short, sharp nature of the sessions meant that a relatively large amount of time was lost logging into the software compared to the non-digital version. The EEF’s new trial of ABRA, this time being independently evaluated by the York Trials Unit, will allow a better understanding of any difference in effects.
You can find out more about the trial of ABRA that is currently recruiting here.