Tutoring with Alphie

Tutoring with Alphie is a computer-assisted programme that aims to improve the literacy skills of struggling readers. The programme combines elements of collaborative (or ‘cooperative’) learning, computer-assisted instruction and small group support. Participating pupils are grouped in pairs and follow a series of activities that seek to improve reading comprehension and fluency of expression. Pupils are supervised by a tutor, most commonly a teaching assistant.

The programme was designed by the charity Success for All and was adapted from a programme which was available in the United States for younger struggling readers (6–7 years).

This project sought to assess the impact of the programme on Year 6 pupils who were at risk of not achieving Level 4a and above in reading at the end of Key Stage 2. 72 pupils from six schools across England completed the trial. Pupils followed the programme on a daily basis for 30 minutes each day over a period of six weeks. Typically four to eight pupils, split into pairs with similar levels of prior attainment, would participate at the same time.

The study was funded by the Education Endowment Foundation as one of 24 projects in a themed round on literacy catch-up at the primary–secondary transition.

Key Conclusions

The following conclusions summarise the project outcome

  1. The evaluation was unable to provide a secure estimate of the programme’s impact on literacy, primarily due to the large number of schools that dropped out of the trial.

  2. It would have been beneficial to undertake further programme development, including software testing, prior to trialling the programme in English schools.

  3. A number of schools perceived the programme as having some promise and worked with the developers and Success for All to overcome initial difficulties with the software.

  4. The programme had been adapted as a catch-up literacy programme for Year 6, but the process evaluation suggested that it may be better targeted at a younger age group.

  5. To assess the efficacy of approaches that combine collaborative learning with computer-assisted technology to improve reading comprehension, further evaluation would be required.

What is the impact?

  • On average, pupils who received Tutoring with Alphie showed a small positive improvement in reading comprehension, compared to pupils in the comparison group. However, because many schools failed to provide data at the end of the programme, it is not possible it attribute the observed positive impact to the programme itself.
  • The software program was an adapted version of a previous tutoring program but substantial technical problems were experienced with the implementation of the new program, particularly in the early stages.
  • The technical problems had a major impact on the overall evaluation. No schools managed to implement the programme fully as intended, and a number of schools withdrew from the programme completely due to frustration with the problems experienced with the software. Some schools also experienced problems with the computerised assessments used to measure pupil progress. Only 6 out of the original 21 schools completed the planned tests.
  • Many of the difficulties with the software were resolved over time and a number of schools who persisted with the programme perceived it as beneficial to the children’s reading. However, they also felt that the current content of Tutoring with Alphie may be better targeted at younger readers than pupils in Year 6 and that this would also provide an opportunity for the school to see potential benefits through improved SATs results.
  • Previous research on digital technologies has suggested that overall digital technology can lead to a positive effect on learning outcomes. However, there are also indications that the effectiveness of digital learning technologies depends less on the technology itself, but rather the way in which it is used and how it is combined with different pedagogical approaches.
  • There is evidence from a previous trial in the United States to suggest that the approach underpinning Tutoring with Alphie which combines digital technology with cooperative learning can lead to improvements in learning. However, this was based on a younger group of struggling readers than those considered in the current evaluation.
  • There remain relatively few rigorous evaluations that look at the combined effect of cooperative learning and computer-assisted instruction on reading comprehension. This makes it difficult to establish the efficacy of this approach for pupils of different ages and further research in this area would be beneficial to the existing knowledge base. 
GROUPEFFECT SIZEESTIMATED MONTHS' PROGRESSEVIDENCE STRENGTHCOST
Tutoring with Alphie versus comparison+0.11+1 month

How secure is the finding?

Findings from this trial have very low security. The trial was set up as a randomised controlled trial, which aimed to compare the progress of pupils who received the programme to similar pupils who did not. The trial was classified as an efficacy trial, meaning that it sought to test whether the intervention can work under ideal or developer-led conditions in ten or more schools. However, a very large number of participating schools dropped out of the project, which substantially reduced the security of the resulting impact estimates.

In total, results from 72 pupils were assessed, compared to 248 pupils who began the project. Fifteen of 21 schools dropped out of the project. The high drop-out rate makes it hard to attribute any improvement to the programme, rather than to chance. It is also possible that the level of drop-out may have reduced the accuracy of the estimate in other ways. For example, if those who dropped out of the programme were on average less engaged, the estimate could overstate the impact of the approach.

How much does it cost?

The cost of the approach as delivered in the trial is estimated at £582 per pupil. This estimate is based on a teaching assistant supporting six students per school. The estimate includes 3.5 days of externally provided training and support from Success for All (UK) delivered on-site (£3,030), website licence fee (£300) and an initial start-up pack, including a programme manual and four electronic key pads (£160) to cater for six pupils and an adult tutor in a group. It does not include direct salary costs, supply cover for training or costs associated with the provision of computers. Given that a large proportion of the cost incurred is related to staff training, the per pupil cost to schools could be substantially reduced if a greater number of pupils followed the programme and were supported by the same teaching assistant, or if a larger number of teaching assistants attended training at the same time.