1. Due to the weak control group and problems with recruitment and retention, this trial has not added much to the existing evidence on the impact of Response to Intervention.
2. The work of the developers and the results of the process evaluation suggest that the approach is feasible and welcome in schools.
3. Increasing the time between training and completion would enhance the chances of the intervention having an impact.
4. Running the intervention at the very end of Year 6 in preparation for Year 7 made it necessarily brief, and put it in competition with too many other end-of-school activities. This meant that some schools reported delivering very few sessions in practice, compared to previous studies which involved 20 or more hours of tuition.

The overall impact is based on the New Group Reading Test post-test, which sought to compare the outcomes of pupils who were supported using the RTI approach to other similar pupils who were not. The first had an estimated effect size of +0.19 . This headline result suggests that this could programme have an impact equivalent to 3 months of additional progress in one year.

However, this result is based on a trial which should be considered ​‘spoilt’ due to the level of school dropout post-allocation, the number of schools which did not carry out post-testing especially in the control group, and confusion over which pupils were eligible for the intervention. Therefore this effect size should not be taken as indicative of impact.

Even further caution must be taken when reading the results for specific sub-groups of pupils such as boys only and pupils eligible for free school meals. These show larger effect sizes but these results are based on very small numbers, and are therefore not deemed secure.

The process evaluation suggested that the intervention would have been more effective had it been started at the beginning of the year and run over a longer period of time, rather than as a catch-up intervention in the busy period at the end of Year 6. Some schools were only able to implement a small number of sessions, which made it impossible to implement, monitor and adjust the intervention as intended.

The approach was popular with teachers and pupils, and appears likely to have had positive benefits on wider outcomes such as confidence and self-esteem.

The evaluation was set up as an efficacy trial to test the impact of RTI as delivered with the developers, CUREE and AFA3As, leading the training and overseeing the provision of the intervention. Efficacy trials seek to test evaluations in the best possible conditions to see if they hold promise, but they do not seek to demonstrate that the findings hold at scale in all types of schools.

The findings are based on a randomised controlled trial using a simple waiting list design. All volunteer schools would receive the RTI intervention; half would be randomly allocated to the immediate intervention group and half would receive the intervention the following school year.

A total of 61 schools agreed to take part and were randomised to treatment (30) or waiting-list control (31). All Year 6 pupils completed a pre-test before the allocation of schools to treatment or control, and before identification of the pupils eligible or suitable for intervention in each school. Schools were required to identify their eligible Year 6 pupils by using the results of the pre-test, teacher judgement, and the RTI diagnostic tool. The identification of these pupils in both treatment and control schools, and the sharing of this information with the evaluator, was important in establishing a valid comparison group prior to the start of the intervention.

Ideally, schools would have identified eligible pupils prior to randomisation in order to ensure that the pupils in the control group were identified under the same conditions and timing as those in the intervention group. However, though this sequencing was originally proposed by the evaluator, it was agreed that pupil identification could take place after randomisation due to the short timeframe in which the intervention was to be delivered. This would not have compromised the trial if the eligible control pupils had been identified early, and this data shared with the evaluator. In practice, however, the evaluator did not receive data on the eligible control pupils until after the delivery of the intervention. The timing and unreliability of this data meant that the findings of the evaluation were substantially weakened.

The security of the findings was further undermined during the trial by the withdrawal of 12 schools between randomisation and post-test.

Due to the combination of problems noted above the trial was irrevocably damaged and consequently the findings are deemed too insecure to assess the impact of RTI as a catch-up intervention.

Results were also analysed using a second measure, the Progress in English test. In contrast to the New Group Reading Test, this test suggested that the approach may have a negative impact on outcomes (an effect size of ‑0.09). These mixed results make interpretation of the results more challenging and highlights the sensitivity of outcomes to what is tested.

Though prior research, mostly from the US, suggests that RTI is an intervention with some promise, this evaluation has not answered the question as to whether the approach can be effective in English schools.