Voice 21: Oracy Curriculum, Culture and Assessment Toolkit – first pilot

1. The Oracy Skills Framework provides a useful tool for schools wishing to review and develop their approach to oracy. The associated Oracy Assessment Toolkit provides teachers with a tool that can be used diagnostically and to track students’ progress in developing oracy skills.
2. The Oracy Curriculum, Culture and Assessment Toolkit as implemented in School 21 appears to provide a sound foundation for the development of oracy skills, with particular strengths in supporting persuasive talk and talk for presentational purposes and in formal contexts. This multi-stranded approach may demand some fundamental shifts in approach for any new school adopting it, including allocating dedicated curriculum time, cultural changes and whole staff training.
3. Further refinement of the curriculum and associated resources is required to highlight the role, nature and development of exploratory talk and to ensure diverse opportunities for oracy, formal and non-formal, are provided. The supporting website requires development to provide an effective professional development resource for other schools.
4. It was not possible at this stage of development to provide a valid measurement of impact. Piloting in other schools and further research would be required prior to a randomised control trial of the intervention to establish a stronger evidence base on the impact on oracy skills and attainment across subjects. Research is also required on how the intervention is interpreted in other schools, and on the opportunities and barriers that arise when implementing these approaches at other sites.

Oracy Skills Framework – The Oracy Skills Framework provided an appropriate and effective structure to support the design, review and refinement of School 21’s Year 7 oracy curriculum. It effectively underpinned the development of an assessment tool which supports diagnostic and formative assessment in oracy, and tracking of students’ progress.

Year 7 oracy curriculum – The curriculum appears to have been well designed to provide sound foundations for the development of oracy skills with particular strengths in supporting persuasive talk and talk for presentational purposes as well as in formal contexts. A number of elements remain under development and a strengthening of the cognitive strand, including provision for exploratory talk, is required. The curriculum also needs to emphasise the need to address oracy within a diverse range of informal as well as formal contexts and explore the appropriateness of talk to context.

Oracy in every lesson – The commitment to promote oracy across the curriculum, and in every lesson, has ensured that School 21 staff and students possess and utilise a shared language for oracy and are familiar with a range of approaches for organising, promoting and reviewing talk. Students have opportunities to use talk within a diverse range of motivating contexts and for different purposes. However, further consideration of how to engage students in exploratory talk and how this supports learning across the curriculum is required.

Whole school oracy culture – The multi-stranded approach to embedding oracy across School 21 has been effective in generating commitment from students and staff.

Oracy Assessment Toolkit – The Oracy Assessment Toolkit provides a useful tool for measuring students’ oracy skills along with a range of support materials which if used appropriately, following training, should enhance the reliability of the tool.

Indicative impact – Findings from the pre and post Ravens Progressive Matrices tests, administered by the evaluators, showed no impact of the intervention on student’s non-verbal reasoning skills. However, tests undertaken by the University of Cambridge using the prototype Oracy Assessment Toolkit developed for this project provided a tentative indication that the intervention had a positive impact on students’ oracy skills. As explained below neither of these tests were able to provide conclusive evidence of the impact of the intervention.

The evaluation was undertaken alongside the development of the Oracy Curriculum, Culture and Assessment Toolkit and is therefore primarily formative, with the main purpose of supporting the further development of the project. These formative findings are informed by a combination of quantitative and qualitative data collection including: observations of oracy teaching, teaching of other subjects and whole school oracy activity; interviews and focus groups with the School 21 and University of Cambridge project teams, School 21 senior leaders, oracy teachers, teachers of other subjects, and students; and a review of the Voice 21 website and other documentary evidence.

It was not appropriate at this initial stage of development to attempt to provide a robust measurement of impact. Two indicative quantitative measures of impact were captured:

1. The Raven’s Progressive Matrices Test – a non-verbal reasoning test. Pre and post test results at the beginning and end of the academic year for 20 Year 7 students from School 21 were compared with 18 Year 7 students from a comparison school with similar characteristics.
2. The prototype Oracy Assessment Toolkit created by the University of Cambridge. This was used with 12 Year 7 students in School 21 and 12 Year 7 students in a comparator school on a pre and post basis spanning five months within the academic year.

A number of factors severely limit the confidence that can be placed in the measurement of impact that was possible during this project. These include: the lack of an existing standardised test that accurately measures the oracy skills that the project was setting out to develop; impact measurement being undertaken at the same time that the project was being developed; and the project only being implemented in one school.