Education Endowment Foundation:Voice 21: Improving Oracy – second pilot

Voice 21: Improving Oracy – second pilot

The 21 Trust
Project info

Independent Evaluator

Alpha Plus logo
Alpha Plus
An oracy framework that helps students develop their speaking and listening skills.
Schools: 12 Grant: £170,000
Key Stage: 3 Duration: 2 year(s) 8 month(s) Type of Trial: Pilot Study
Completed June 2018

The Voice 21 Oracy Improvement Programme supports schools to develop pupils’ use of speech to express their thoughts and communicate effectively. Participating schools were asked to devote one hour a week of lesson time to developing spoken language, and received materials and training in oracy-based approaches

School 21 is a free school that has been widely praised for its approach to oracy. The EEF funded this pilot of the Voice 21 programme to test the feasibility of the programme and its evidence of promise, and to assess the reliability of the oracy assessment measure developed by School 21 in collaboration with the University of Cambridge as part of a previous EEF-funded project.

The findings of the pilot suggest that the Voice 21 approach is promising. The programme was well received by teachers and all school staff reported some improvements to pupils’ oracy skills. Teachers also felt that the programme could be implemented in most schools.

Although teachers were not confident that the observed improvements to oracy skills would have an immediate impact on attainment, some felt that there could be longer-term academic benefits. This pilot did not collect any quantitative data on academic outcomes.

The oracy assessment measure, tested as part of the pilot, had limited reliability, and should be adapted or replaced for any future evaluations. Given the evidence of promise for the programme, the EEF will explore options for conducting an efficacy trial to test the programme’s impact on academic outcomes.

  1. Teachers reported that pupils’ oracy skills improved as a result of the pilot; assessment results also showed that pupils’ oracy skills improved. However, as there was no comparison group, it is not possible to say whether these changes would have happened anyway.
  2. Many schools were beginning to develop a whole-school oracy culture by the end of the programme, but felt that only limited change was achievable in one year and when focusing on only one year group.
  3. Most teachers were positive about the programme and agreed that it would work in most schools with minimal adjustments.
  4. The Voice 21 oracy assessment measure used in the pilot did not provide sufficiently reliable data. A revised or alternative impact measure would be needed for a trial.
  5. Delivery was not uniform across schools, or within schools, in part due to an initial lack of clarity about which elements of the programme were mandatory and which optional.The core components of the programme would need to be clearly articulated at the outset of a trial while maintaining the flexibility in delivery that was popular with teachers.
Question
Finding
Comment

Is there evidence to support the theory of change?

Yes, but limited.

All school staff reported some improvement to pupils’ oracy. Oracy was also measured using the School 21 assessment and was found to have improved. However, given the limited reliability of the assessment, and the lack of a comparison group, we cannot conclude from these results that the programme improved oracy. The pilot did not measure impact on academic attainment.

Was the approach feasible?

Yes.

The programme was well received across the pilot. Teachers felt it could be implemented in most school contexts, given the necessary support from senior leadership.

Is the approach ready to be evaluated in a trial?

Yes, with some changes.

A clear definition of the programme’s core components is needed before it can go to trial. More work should also be done to improve the oracy assessment measure so that it produces reliable data. Alternatively, another suitable attainment measure could be selected.