1. The evaluation was unable to provide a secure estimate of the programme’s impact on literacy, primarily due to the large number of schools that dropped out of the trial.
2. It would have been beneficial to undertake further programme development, including software testing, prior to trialling the programme in English schools.
3. A number of schools perceived the programme as having some promise and worked with the developers and Success for All to overcome initial difficulties with the software.
4. The programme had been adapted as a catch-up literacy programme for Year 6, but the process evaluation suggested that it may be better targeted at a younger age group.
5. To assess the efficacy of approaches that combine collaborative learning with computer-assisted technology to improve reading comprehension, further evaluation would be required.

• On average, pupils who received Tutoring with Alphie showed a small positive improvement in reading comprehension, compared to pupils in the comparison group. However, because many schools failed to provide data at the end of the programme, it is not possible it attribute the observed positive impact to the programme itself.
• The software program was an adapted version of a previous tutoring program but substantial technical problems were experienced with the implementation of the new program, particularly in the early stages.
• The technical problems had a major impact on the overall evaluation. No schools managed to implement the programme fully as intended, and a number of schools withdrew from the programme completely due to frustration with the problems experienced with the software. Some schools also experienced problems with the computerised assessments used to measure pupil progress. Only 6 out of the original 21 schools completed the planned tests.
• Many of the difficulties with the software were resolved over time and a number of schools who persisted with the programme perceived it as beneficial to the children’s reading. However, they also felt that the current content of Tutoring with Alphie may be better targeted at younger readers than pupils in Year 6 and that this would also provide an opportunity for the school to see potential benefits through improved SATs results.
• Previous research on digital technologies has suggested that overall digital technology can lead to a positive effect on learning outcomes. However, there are also indications that the effectiveness of digital learning technologies depends less on the technology itself, but rather the way in which it is used and how it is combined with different pedagogical approaches.
• There is evidence from a previous trial in the United States to suggest that the approach underpinning Tutoring with Alphie which combines digital technology with cooperative learning can lead to improvements in learning. However, this was based on a younger group of struggling readers than those considered in the current evaluation.
• There remain relatively few rigorous evaluations that look at the combined effect of cooperative learning and computer-assisted instruction on reading comprehension. This makes it difficult to establish the efficacy of this approach for pupils of different ages and further research in this area would be beneficial to the existing knowledge base.

Findings from this trial have very low security. The trial was set up as a randomised controlled trial, which aimed to compare the progress of pupils who received the programme to similar pupils who did not. The trial was classified as an efficacy trial, meaning that it sought to test whether the intervention can work under ideal or developer-led conditions in ten or more schools. However, a very large number of participating schools dropped out of the project, which substantially reduced the security of the resulting impact estimates.

In total, results from 72 pupils were assessed, compared to 248 pupils who began the project. Fifteen of 21 schools dropped out of the project. The high drop-out rate makes it hard to attribute any improvement to the programme, rather than to chance. It is also possible that the level of drop-out may have reduced the accuracy of the estimate in other ways. For example, if those who dropped out of the programme were on average less engaged, the estimate could overstate the impact of the approach.