IRIS Connect: Developing classroom dialogue and feedback through collective video reflection

IRIS is designed to improve primary school teachers’ use of dialogue and feedback through using video technology for collaborative teacher development with a view to improving academic outcomes for pupils. It is based around a video technology system (IRIS Connect) which enables teachers to record, edit, and comment on teaching and learning.

In this pilot, the project comprised six ‘film club’ events each lasting two hours: three in which teachers reviewed lesson clips from other schools, two in which they reviewed each other’s lessons, and one film club in which they reviewed their experience. The project aimed to create long-term, whole-school change, embedding the use of dialogue and feedback in school culture. This evaluation, however, focused specifically on the impact of IRIS Connect on the teachers attending film clubs, as stated in the original protocol agreed prior to the project starting.

Schools were free to choose which teachers participated in the trial, but were encouraged to focus on Year 5 pupils. Teachers worked collaboratively in the ‘film clubs’ to review lesson clips from other schools and to plan, teach, record, and review their own lessons using the IRIS Connect online platform. The project was designed and supported by IRIS Connect and Whole Education with advice from academics at the Universities of Cambridge and Leeds, and additional content from Routledge.

This pilot project evaluated (i) how teachers implemented the intervention, and (ii) the change in teachers’ thinking and practice. Teachers participating in the project were compared with other teachers in their schools who did not participate. The project lasted for seven months from January to July 2016. The first three clubs in each school were held between January and April and the second three between May and July. Twelve schools were initially recruited for the project, eleven of which participated.

Key Conclusions

The following conclusions summarise the project outcome

  1. The overwhelming majority of teachers who responded to the survey believed that the intervention was a good use of their time and had improved their practice.

  2. During the seven-month pilot, three of the eleven participating schools chose not to engage with the second half of the project in which teachers videoed their own lessons.

  3. The intervention demanded a substantial proportion of teachers’ development time. A further trial should provide schools with sufficient time to build this into their annual plans and to embed the intervention in their plans for school improvement.

  4. Ten of the eleven participating schools reported that they would continue to use IRIS Connect after the pilot had ended.

  5. The training materials are well developed, clearly understood by teachers, and ready for trial.

What is the impact?

On the basis of a range of evidence from videos of lessons, interviews, and ‘before and after’ surveys, this evaluation has found: moderate evidence of change in school climate, strong evidence that film clubs promote discussion of teaching and learning, moderate evidence of change in teachers’ thinking and moderate evidence of changes in practice.

All participating schools successfully implemented the first half of the intervention, which involved groups of teachers discussing video clips provided through the online platform. Three schools did not implement the second half of the intervention, which involved teachers videoing and discussing their own lessons. A large majority of teachers responding to an end-of-project survey were positive about the value of the project to them, and all but one of the teachers acting as a ‘school champion’ reported that their school would be continuing to use IRIS Connect along the lines of this project.

The intervention seems to operate in two ways: (i) by helping teachers to identify possible improvements in practice, and (ii) by helping teachers to become more comfortable with discussing their classroom work with others. The second of these processes takes longer to embed and this carries implications for the length of an efficacy trial.

The intervention is well defined and could be replicated in the form used in this pilot. Some refinements could help teachers and schools to overcome some of the challenges they faced, such as: strengthening the online guidance to emphasise the value of teachers openly discussing the impact of the programme on their own practice; emphasising the value of discussion that focuses on why a particular example of teaching might be engaging pupils in a particular way that is fostering their learning; highlighting for teachers how the online platform can be used to identify and comment on moments in a lesson that exemplify an aspect of teaching deemed critical to learning; and encouraging discussion of these moments between schools. Many of these recommendations have been implemented by Whole Education in the time since the pilot.

QuestionFindingComment
Is there evidence to support the theory of change?Moderate (with strong formative findings)Participating teachers believed the intervention improved their practice. This change in practice is supported by evidence collected through surveys and analysis of videoed lessons. No data on pupil outcomes was collected as part of this pilot.
Was the approach feasible?ModerateAlthough the intervention demands teachers’ time, teachers felt it was worth it and the participating schools planned to continue after the project. Three of the eleven schools did not fully implement the programme within the seven months of this pilot.
Is the approach ready to be evaluated in a trial?YesThe online platform and the materials used in this project are ready for systematic delivery across a larger number of schools.

How secure is the finding?

A process evaluation examined the implementation of the intervention and its impact on the thinking and practice of teachers in the pilot as compared with teachers in the same schools who did not participate. The evidence used in the evaluation included usage of data from the online platform, lesson videos, reports and videos of film clubs, focus groups, interviews, and surveys of teachers before and after the project, including a comparison group of teachers who did not take part in the pilot.

How much does it cost?

Schools who buy a licence for the IRIS Connect technology receive training on how to use the package and how they might embed its use in their plans for school improvement. They also receive on-demand telephone support for technical problems. These services are provided as part of the contract.

The cost of the hardware and licence varies depending on the hardware used. In this pilot the cost per pupil for three years ranged from £6.96 to £11.40. On the basis of feedback provided during and at the end of the project we have made a very rough estimate of required teacher time of between 25 hours and 50 hours.

EEF commentary

IRIS Connect is a video technology package that supports teachers to review, reflect on, and develop their practice. This pilot focused on primary teachers’ use of dialogue and feedback. Teachers first used the technology to review videos of best practice in other schools, and then recorded their own lessons to review with peers.

Schools are increasingly turning to technology to deliver improvement, but existing research suggests that the impact of digital technology products varies widely. Technology interventions appear to be most effective when they support changes in the nature of teaching and learning rather than, for example, delivering the same content in a different format. We funded IRIS because it was specifically designed to improve outcomes by developing teaching skills.

The vast majority of teachers in this pilot believed it improved their practice, and all but one of the schools said they would continue to use IRIS Connect. However, three of the eleven participating schools did not finish the full project in the expected time.

The EEF will explore the potential for developing the approach further, alongside other teacher development projects reporting later this year.