Chatterbooks is an extracurricular reading initiative that aims to increase a child’s motivation to read by providing schools with tools and resources to encourage reading for pleasure. The intervention developed for this trial consisted of nine weekly one hour sessions where the pupils read and discussed an age-appropriate book. The programme was delivered by trained graduates to Year 7 pupils who had not reached a secure Level 4 in English at the end of Key Stage 2.
Chatterbooks was developed by the Reading Agency as an extracurricular activity, organised in local libraries on Saturday mornings by volunteers. The sessions are intended for six to twelve-year-olds and attendance is voluntary. For the purposes of this evaluation, the Chatterbooks programme was altered significantly to test its impact in a more structured and formalised school environment. A variation called Chatterbooks Plus was delivered alongside the Chatterbooks programme. In this intervention, fifteen minutes of a sixty minute session were replaced with dialogic reading where children read aloud and were offered explicit prompts.
The target population for this evaluation was pupils in secondary schools in an area of the Midlands accessible from Coventry University. The programme was delivered by trained graduates at Coventry University who had received training from the Reading Agency and Professor Clare Wood (for the dialogic reading component). Intervention delivery took place from April to June 2013.
Chatterbooks aims to encourage reading for pleasure. It is assumed that this translates into an improvement in reading ability. This evaluation set out to measure what impact the scheme had on reading ability as measured by the GL Assessment New Group Reading Test.
The evaluation was funded by the Education Endowment Foundation as one of 23 projects focused on literacy catch-up at the primary-secondary transition. It was one of four programmes funded with a particular focus on reading comprehension.
Testing the impact of two library-based approaches which encourage children to read for pleasure.
Language and literacy
The following conclusions summarise the project outcome
There was no evidence of impact of either Chatterbooks or Chatterbooks Plus on reading ability.
The process evaluation indicates that schools encountered problems with timetabling, planning and resourcing, which make the intervention unsuitable in its current format.
If implementing this intervention with low-achieving pupils, it would be important to find a way of increasing motivation to mitigate behavioural issues.
The dialogic reading component of Chatterbooks Plus could be expanded and trialled separately.
To make Chatterbooks more suitable for a school setting, it would be important to consider how sessions that focus on reading for pleasure can be developed to demonstrate impact on reading ability.
What is the impact?
There was no evidence of impact of either Chatterbooks or Chatterbooks Plus on pupils’ reading ability either immediately after the interventions or at a three-month follow-up. The headline findings suggest that Chatterbooks and Chatterbooks Plus had a slightly negative impact on reading ability with effect sizes of -0.14 and -0.01 respectively. However, they are not statistically significant, suggesting that the difference in outcomes between the control and intervention group occurred by chance. Consequently, the effect sizes are indistinguishable from zero. The results for children on free school meals in both interventions should be interpreted in the same way.
From the observations and interviews with staff in the process evaluation, a lack of engagement and poor behaviour spoiled the delivery of some sessions. This may have contributed to the lack of impact in the intervention.
Further analysis suggests that the dialogic reading component of Chatterbooks Plus was possibly too limited to see an effect.
|GROUP||NUMBER OF PUPILS||EFFECT SIZE (95% CONFIDENCE INTERVAL)||ESTIMATED MONTHS’ PROGRESS||IS THIS FINDING STATISTICALLY SIGNIFICANT?||EVIDENCE STRENGTH|
|Chatterbooks||159||-0.14 (-0.31, 0.03)||-2||No|
|Chatterbooks Plus||150||-0.01 (-0.18, 0.16)||No|
|Chatterbooks (FSM pupils)||37||-0.30 (-0.63, 0.03)||-4||No||N/A|
|Chatterbooks Plus (FSM pupils)||40||-0.09 (-0.39, 0.21)||-1||No||N/A|
How secure is the finding?
Impact was assessed though a three-arm pupil-randomised controlled trial in twelve schools. 577 Year 7 pupils were randomised either to receive Chatterbooks or Chatterbooks Plus, or to a waitlist control group. Although there was attrition of 21% by the final analysis, there was no evidence that this led to bias or a loss of power that might impinge on the security of the findings.
There had been no formal assessment of the impact of Chatterbooks prior to this trial although some case study work has been carried out in a small number of primary schools. Chatterbooks Plus is a new form of the intervention and has not been trialled in any form. However, there is some evidence from the United States that the dialogic reading element has an impact on younger readers in terms of improvement in oral language. It is important to note that both variants evaluated in this trial were changed significantly from the established Chatterbooks programme that has been running since 2001 as an extracurricular offer.
As the present study represents the first formal evaluation of both programmes and was closely managed by the deliverer, it can be regarded as an efficacy trial. Efficacy trials seek to test evaluations in the best possible conditions to see if they hold promise. They do not indicate the extent to which the intervention will be effective in all schools since the participating schools are selected from one area, and the programme is delivered by the developers.
The primary outcome was reading ability as assessed by scores from the GL Assessment New Group Reading Test (NGRT). The secondary outcomes were the two NGRT subscales: sentence completion and passage comprehension.
All data was collected by a team from Coventry University who were involved in the Chatterbooks project, but not involved in carrying out the Chatterbooks sessions. Every effort was made to ensure that test administration was blind in every school although complete blindness, as if delivered externally, cannot be guaranteed. The test marking was carried out by GL Assessment and was therefore blind.
Analysis was completed on an ‘intention to treat’ basis, reflecting the reality of how interventions are delivered in practice.
To view the projects protocol click here.
How much does it cost?
Chatterbooks training costs £1,000 per session for 20 people. A ‘Chatterpack’ for each child costs £5. For a school that trains 20 people and provides the Chatterbooks programme to 150 pupils, the cost will be between £10 and £20 per pupil depending on the amount of supply cover used. The lowest figure in this estimate is based on a non-teaching member of staff delivering the intervention and therefore not requiring cover.