The Literacy Intervention Toolkit (LIT) programme aims to improve the reading ability of children in Year 7 who scored below Level 4 at the end of primary school using a method known as reciprocal teaching. Reciprocal teaching methods encourage children to ‘become the teacher’. They are taught how to apply four comprehension strategies: summarising, clarifying, questioning, and predicting. These strategies enable children to check that they understand the content of the material they are reading and can make inferences based on what they have read.
The LIT programme is tightly structured, providing training to staff as well as a set of detailed lesson plans on the use of reciprocal teaching to deliver basic instruction in literacy. However, the method of delivery is not particularly prescriptive. It can be used to teach whole classes or small groups, can be delivered by teachers or teaching assistants, and can be offered in addition to or instead of regular English classes. In this evaluation, children typically received 3–4 hours of LIT tuition per week for eight months, mostly delivered in small groups.
The programme was devised by the Hackney Learning Trust, who delivered the training sessions for staff and developed a detailed set of lesson plans for them to follow.
The primary outcome was reading ability as assessed by scores on the Access Reading Test (ART).
The LIT programme was tested using a randomized control trial (RCT). 41 schools were recruited, with 22 schools randomly allocated to a treatment group and 19 schools to a control group.
A programme which aims to improve the reading comprehension of struggling Year 7 pupils.
The Institute for Fiscal Studies
Developing effective learners
Staff deployment & development
Language and literacy
The following conclusions summarise the project outcome
This evaluation cannot conclude with certainty what impact the LIT programme had on reading ability for those pupils who received the intervention.
This evaluation did not find evidence of a significant impact on reading ability at the year group level (comparing all Year 7 pupils in treatment schools with similar Year 7 pupils in control schools).
Teachers felt that the programme facilitated ‘healthy debate’ within the classroom, increased confidence in pupils who struggled with core literacy skills, and promoted independent learning. However, they did not feel it worked well with children with underlying cognitive issues requiring intensive vocabulary support.
Feedback from those who used the programme suggested groups of 5–6 children led by qualified teachers or teaching assistants with experience of delivering literacy interventions worked best.
What is the impact?
The primary objective of the independent evaluation was to test whether the LIT programme had a significant impact on the literacy skills of the approximately 15% of Year 7 children who were eligible to take part (those with the poorest literacy skills). Unfortunately, it was not possible to do this because the characteristics of pupils in treatment and control schools were too different to yield an unbiased estimate of the impact of the programme. However, it was possible to compare all Year 7 pupils in treatment schools with similar Year 7 pupils in control schools, regardless of whether they received the LIT programme. This captures the effect of the LIT programme on the whole year group.
These results suggest that the reading scores of Year 7 pupils in schools running the LIT programme rose 0.09 standard deviations more over the course of the year than those of similar Year 7 pupils in control schools. This effect size of 0.09 is equivalent to 1 month’s additional progress in reading. However, statistical tests indicate that this estimate is not ‘significant’, i.e., we cannot be sure that the difference in scores is not due to chance. It is important to note that this is the average effect across all pupils in Year 7, i.e., across the 15% of pupils who received the LIT programme as well as the 85% of pupils who did not. It is possible that the effect on those who received it was much higher.
Recommendations for further study
In addition to a further study to estimate the impact of the LIT programme on pupils who received it, it would be useful to explore in more detail which elements of the programme are vital to its success. The LIT programme was delivered in a variety of ways across schools in the evaluation, with variation in programme length, number of pupils per group, qualifications and experience of staff delivering the programme and whether they were specialists in English, and whether the LIT programme was delivered in addition to or instead of English or literacy lessons. Unfortunately it was not possible to separately identify the contributions of these different elements to the overall programme impact.
What did schools think of the programme?
The reciprocal teaching approach on which the LIT programme is based was seen as having a number of strengths: it encourages teamwork and debate, gives lower attaining pupils confidence, and is fun. But it was not felt to be suitable for the lowest ability pupils who required intensive support. Teachers felt that the positive benefits of using the LIT programme arose from the reciprocal teaching approach, the fact that teaching occurred in small specialised groups, the engaging programme content and quality of resources, and the assessment and pupil feedback element. Several schools reported that they would continue to use the programme at the end of the intervention.
|Group||No. of pupils||Effect size (95% confidence interval)||Estimated months' progress||Evidence strength||Cost|
|All pupils||4,413||0.09 (-0.04, 0.22)||+1|
How secure is the finding?
The intention was to estimate the impact of the LIT programme by comparing the subset of pupils in treatment and control schools who were eligible to receive the programme with similar pupils in control schools who were not. However, the randomization was undertaken at school level and hence does not guarantee that the characteristics of a subset of pupils in treatment and control schools are similar. In addition, a small number of schools dropped out of the programme, and not all pupils sat the ART post-test, which meant that the original randomization was compromised. Consequently, we had to try to ‘balance’ the characteristics of pupils in treatment and control schools, to ensure we were comparing like with like.
This was a challenging exercise, as the characteristics of pupils in treatment and control schools were very different. We identified pupils in treatment and control schools who were as similar as possible using a number of characteristics we deemed important for pupil progress such as prior attainment, gender, ethnicity, eligibility for free school meals, school type and composition.
Unfortunately it was not possible to find a group of pupils in control schools who ‘looked’ similar enough to pupils who were eligible to receive the LIT intervention across all of the dimensions that we thought mattered for progress. This meant that the original intention of estimating the impact of the LIT programme on the approximately 15% of Year 7 children in treatment schools who were eligible to receive it was not possible. Instead, the final analysis compared the reading scores of the 2,889 Year 7 pupils in 19 treatment schools – 660 of whom were eligible to receive the LIT programme and 2,229 of whom were not – with the 1,524 Year 7 pupils across 15 control schools who provided useable data at the start and end of the trial.
If we instead thought that only prior attainment (and specifically Key Stage 2 scores) mattered for pupil progress then we could estimate the impact of the LIT programme on pupils who were eligible to receive it, by comparing those pupils in treatment and control schools with similar Key Stage 2 scores. Doing so resulted in an estimate of the impact of the LIT programme on those who are eligible to receive it (but not necessarily receiving it) of 0.13 standard deviations. However, this estimate is not statistically different from zero, and is highly likely to be biased by the fact that the pupils that we are comparing differ in other ways that matter for their progress. We would therefore strongly caution against regarding this as a robust estimate of the effectiveness of the LIT programme on those who received it.
To view the project's evaluation protocol click here.
How much does it cost?
The cost of training and detailed materials for the programme is £3000 per school. This price includes all of the planning and pupil resources, grammar and punctuation booklets for teachers and pupils, a handbook, on-site training for up to 20 members of staff, and follow-up support via email or phone.
The other major costs are likely to be staff time: this will vary depending on whether a teacher or teaching assistant delivers the programme, whether the lessons are given in addition to or in place of English lessons, what the staff member would have been doing instead of teaching the LIT programme, and the number of pupils treated.