Our Head of Policy, Jonathan Kay, examines the findings from our most recent evaluation of Accelerated Reader.
Today, we have published results from our latest evaluation of the Accelerated Reader programme in KS2. Accelerated Reader is one of the most popular reading programmes in the English education system, and our previous trial of the programme found a promising result for pupils transitioning to secondary school
However, the result from this most recent trial is, unfortunately, less encouraging
The result for the cohort of pupils who first received the programme in Year 5 shows no positive impact overall or for pupils eligible for free school meals. A second cohort, who were in Year 4 at the start of the programme, was also tested (results of which can be found here) with no positive impact overall. This result included a small positive impact for children eligible for free school meals – but with too much statistical uncertainty to confidently establish efficacy
With any disappointing result, it is important to try and understand why the impact was limited, and to consider the consequences for schools using Accelerated Reader.
A null result from one of our trials can often be attributed to the challenges associated with implementing a new educational programme from scratch. New approaches or interventions are often difficult to quickly embed. In order to monitor this, every EEF evaluation includes a process evaluation that examines how a programme is received in schools
In the case of Accelerated Reader, however, the trial found that there was a high-level of implementation of AR during year one of the trial and a fairly high-level of implementation during year two based on the fidelity measure developed for this evaluation. This suggests that schools were able to use the programme as intended
Although implementation overall was described as high, it is important to note that the ideal engaged reading time of between 15 – 30 minutes a day was only achieved by between 38% to 43% of pupils in first year of the trial, and between 30% to 42% of pupils in the second year.
Compared to what?
In any trial, the absolute progress of pupils is not measured – instead the progress of pupils receiving an intervention is compared with that of pupils in similar schools who are not. As such, another explanation for the null result might be the activity of the schools that were used as comparison
The process evaluation indicates that many of these control schools were allocating dedicated reading time comparable to the schools delivering Accelerated Reader, as well as delivering other intensive literacy interventions.
This is particularly important in interpreting this result – the trial did not compare something with nothing, and therefore the finding is not that Accelerated Reader does not work. Rather, the trial showed that the intervention was no more (or less) effective than the interventions used by comparison schools preparing for KS2 exams. If schools were to just stop using Accelerated Reader, without implementing alternative reading interventions, it is likely that this would have a negative impact on pupil outcomes.
Accelerated Reader at the transition?
The main difference, however, between this trial of Accelerated Reader and the last, is the context in which they took place. The first trial focused on readers at the secondary transition who were below age-expected levels in reading and measured impact with GL Assessment NGRT. Meanwhile, this most recent trial focused on reading during KS2 and collected data from KS2 SATs. The recent study did deliver the same NGRT reading test with Year 4 pupils but unfortunately ceiling effects were found and so the data could not be used.
Using SATs data has pros and cons. Their position in the system makes them a motivating point for schools to work towards within a trial, but this may also compromise their usefulness in terms of understanding overall reading levels than tools like NGRT that are not used for accountability.
Furthermore, the move from the secondary transition to primary school only delivery might make more of a difference than the test used to establish impact. Part of the mechanism by which Accelerated Reader is thought to have an impact is through the selection of texts that provide an appropriate level of challenge to readers. This might be particularly beneficial at the transition where teachers are less familiar with readers.
The evidence of promise from the first trial means that the EEF will continue to list Accelerated Reader as a promising programme for use during the transition between primary and secondary school. As with any intervention, it is crucial that schools consider the nuance within findings from educational trials, as well as the particulars of their own context, when implementing a new approach