EEF Blog: The significance of “no significance”

Jonathan Kay reflects on last week's evaluation reports.

Recent Findings

Last week the EEF published four evaluation reports. There was a lot of excitement over the “Improving Writing Quality” programme, and the nine months of additional progress that has been achieved for very little cost. At the same time as this report was published, three other evaluations were released. While the “Improving Writing Quality” report showed a major improvement for students, the other three reports did not. They were, however, also valuable.

An important part of the research funded by the EEF is examining practices that are currently prevalent in schools but have not been evaluated. In the same way that we may sometimes be surprised by the success of some new interventions, we may also be surprised by current practices that turn out not to have the positive effects that we hope they will. In order for an evidence based approach to education to be effective, school leaders will need to listen openly to the evidence, even when it goes against current beliefs.

Changing Practice

It is important to emphasise that a report which shows no significance does not mean that an approach should necessarily be abandoned in its entirety. It may mean that more research needs to be done, or it may mean that the approach needs to be improved. Above all, it may mean that careful thought is needed by schools considering adopting the approach, or spending the Pupil Premium in this way.

One example of this is research on teaching assistants. Initially the research contained in the Teaching and Learning Toolkit suggested that, on average, teaching assistants had no impact on the attainment of students. However, research funded by the EEF, and others, showed that TAs could have a positive impact when helping small groups or using structured interventions. This highlighted that care was needed in terms of providing support and training for TAs; ensuring that teachers do not reduce input to students supported by TAs; and identifying carefully the areas where TAs can support learning. This example shows why all educational practitioners should welcome evidence that challenges the status quo. When a teaching practice that many see as valuable is called into question by research, we can focus on trying to identify how improvements can be made.

Hard Decisions

There will, however, be some practices that are shown by research to not be effective. This will often be difficult. It will often challenge approaches led by people that truly care about education. A move towards an evidence based practice will require school leaders to embrace these results and make changes to existing practices. In a world of limited resources, spending money on approaches that do nothing to benefit students harms their education. The EEF is proud to be a “What Works” centre; “What doesn’t work” is just as important.

Jonathan Kay is a member of the EEF Team