Education Endowment Foundation:Quest – trial

Quest – trial

Success for All
Implementation costThe cost estimates in the Toolkits are based on the average cost of delivering the intervention.
Evidence strengthThis rating provides an overall estimate of the robustness of the evidence, to help support professional decision-making in schools.
Impact (months)The impact measure shows the number of additional months of progress made, on average, by children and young people who received the intervention, compared to similar children and young people who did not.
0
months
Project info

Independent Evaluator

Queen's University Belfast logo
Queen’s University Belfast
Collaborative learning in literacy at transition.
Schools: 30 Grant: £572,832
Key Stage: 3 Duration: 2 year(s) 7 month(s) Type of Trial: Efficacy level evidence
Completed January 2015

The project

Quest is a whole-year group approach to teaching English in Key Stage 3. Pupils are grouped according to their current level of reading comprehension, typically with smaller classes for the lowest attaining groups. Key components of the programme include: an emphasis on collaborative (or ​‘co-operative’) learning; a requirement that participating teachers follow a consistent ​‘cycle of instruction’; and the use of formative assessment in every lesson. Pupil progress is reviewed every eight weeks, with results used as the basis for re-grouping the class.

The programme was designed by the charity Success for All and had been adapted from an existing programme available in the United States called Reading Edge.

This project sought to assess the impact of Quest on Year 7 pupils’ reading comprehension and was supplemented by a process evaluation that assessed programme implementation. Six schools followed the programme, and a further 13 participated in the evaluation. All participating teachers received two days training and extensive resources to be used as part of the approach, including lesson plans, graded texts and ​‘digitexts’ (interactive texts). The programme was designed to be delivered in daily 60-minute lessons by all Year 7 English teachers, and run over the full school year.

The study was funded by the Education Endowment Foundation as one of 24 projects in a themed round on literacy catch-up at the primary-secondary transition.

Outcome/​Group
ImpactThe size of the difference between pupils in this trial and other pupils
SecurityHow confident are we in this result?
Quest vs. comparison
0
Months' progress
(Effect size -0.04)

Key Conclusions

The following conclusions summarise the project outcome
  1. The evaluation was unable to provide a secure estimate of the impact of Quest on reading comprehension outcomes among Year 7 pupils, primarily due to a high level of drop-out from the trial.
  2. Few, if any, schools implemented the programme as designed, suggesting that substantial adaptation may be required if the approach is to gain wider adoption in English schools.
  3. Some of the main barriers to successful implementation included: the difficulty in covering the expected material in a single school lesson; a perception of an insufficient focus on writing activities; and its adoption as a whole-year group intervention.
  4. Many schools were positive about the range of resources provided by the programme and the co-operative learning aspects of the programme.
  5. Though it is not possible to draw a conclusive statement about the impact of the programme, on average pupils who received the programme made less progress than those who did not.

What is the impact?

  • On average, pupils who received the programme made less progress than those who did not. However, due to the high dropout rate and the small sample size, it was not possible to draw a conclusive statement about the impact of the programme.
  • The level of implementation of the programme in Quest schools was low, which may have diluted any potential impact on reading comprehension.
  • Conversely, if those who dropped out of the programme were on average less engaged, or implemented the programme with less fidelity, the estimate could overstate the impact of the approach.
  • The programme did not appear to have differential effects according to gender, free school meal eligibility, or prior reading attainment.
  • The process evaluation highlighted a number of issues that both undermined teacher confidence in the programme and impacted upon the extent of implementation.
  • The perceived weaknesses of Quest according to teachers were: that it was overly prescriptive; that the lesson plans contained too much material to be covered in a standard 50 or 60-minute school period; and that it contained insufficient writing opportunities.
  • Positive feedback was provided by the teachers in relation to the range of books and digital texts provided for pupils, although these could be improved for ​‘lower ability’ sets.
  • On the whole, teachers were also positive about Quest’s co-operative learning structure (particularly the teamwork and opportunities for discussion).

How secure is the finding?

Findings from this trial have low security. The trial was set up as a randomised controlled trial that aimed to compare the progress of pupils who received the programme to similar pupils who did not. The trial was classified as an ​‘efficacy trial’, meaning that it sought to test whether the intervention could work under ideal or developer-led conditions in ten or more schools. However, a large number of participating schools dropped out of the project which substantially reduced the security of the resulting impact estimates. In addition, there was some imbalance between the intervention and comparison groups in terms of the proportion of pupils eligible for free school meals and their gender.

There were high levels of attrition and missing data, particularly among the intervention group (from which 4 out of 13 schools withdrew, compared to two comparison schools). The high drop-out rate makes it hard to attribute any difference between intervention and comparison schools to the programme, rather than to chance. It is also possible that the level of drop-out may have reduced the accuracy of the estimate in other ways. For example, if those who dropped out of the programme were on average less engaged, the estimate could overstate the impact of the approach.

How much does it cost?

The cost of the approach as delivered in the trial is estimated at £161 per pupil. This estimate is based on a per-school cost of £24,171 and an average of 150 pupils per year group. The cost estimate includes costs for training, ongoing support for teachers, all programme materials and resources. It does not include the salary costs of participating teachers, or of any supply cover required to permit teachers to attend training.

Downloads