Why do programmes teachers believe are beneficial not always convert into improved results for their pupils? The EEF’s head of programmes Eleanor Stringer explores…
Those are the words of an experienced teacher, a subject coordinator, who participated in a recent EEF trial.*
I’m glad that they enjoyed the training, and I hope that it led to improved teaching in their school. But the results from our trial of the programme they were delivering suggest that – overall, on average – attainment outcomes for pupils in the half of schools who received it were no different to those in the other half of schools which didn’t receive this inspirational training.
The feedback provided by this teacher is similar to that found in the majority of the 100+ EEF-funded projects which have so far completed. Yet the rigorous independent evaluations we fund find positive impacts on attainment outcomes in only around one-quarter of EEF trials
What’s with the mismatch? Why do programmes teachers feel are beneficial not always convert into improved results for their pupils?
People often like exciting new things, the ‘novelty value’ of trying out something different
There could be any number of explanations. Teachers may genuinely learn lots from the CPD on offer in many of our funded programmes. Or they may be saying particularly positive things to be polite (though our insistence that evaluators are wholly independent of the project deliverer aims to overcome this). Or our trials may be failing to detect genuine improvements in learning, improvements that are not converted into the attainment outcomes we’re measuring.
I have a particular hypothesis. People often like exciting new things – the ‘novelty value’ of trying out something different – and especially if they know it’s being tested because it’s a programme which may boost attainment.
But I think change – real change, which is positive and endures – only comes about through challenging existing practice and a concerted focus on changing old behaviours and embedding new ones. And this can be where the ‘novelty value’ wears off because that kind of change can be difficult and uncomfortable. It is sometimes said that “learning happens when pupils have to think hard”; perhaps the equivalent in professional development is being properly challenged.
I think change – real change, which is positive and endures – only comes about through challenging existing practice
To make the point in a different way, here’s a counterexample.
The independent evaluation of our trial of Embedding Formative Assessment, based around ongoing professional development about a cornerstone of good teaching – formative assessment – found positive impacts on GCSE outcomes. Though relatively modest in size, it’s very rare to find programmes which boost grades in Key Stage 4, so, for us, it was an exciting finding!
However, that feeling wasn’t one necessarily shared by all the teachers when delivering the programme. Some felt that it was repetitive, that it wasn’t telling them anything new – “I would imagine every school will have developed those strategies already”.
Given how important assessment is, there is always room for teachers to discuss their practice and make improvements, however marginal. And this study suggests doing so improves results
But it is not necessarily going to inspire teachers in the way that a snazzy new teaching approach might. Instead, it’s going to require regular reflection on whether teachers really know what understanding your pupils have about what you’re teaching, and what you can do to support them in getting better at applying this knowledge. Such reflection and focus can be challenging.
The most promising programmes may not always be those that teachers find the most enjoyable
Another counter-example is one of our most promising early years programmes, Nuffield Early Language Intervention (NELI).
Built on high-quality research about how to support oral language development in young children, the programme trains teaching assistants to deliver a carefully designed, manualised intervention for pupils identified as most likely to be benefit.
The independent evaluation from our first trial found that, while schools saw the benefit, they also felt that the programme was “onerous and resource-intensive”, and “too time consuming”. Yet it yielded positive impact on pupils’ language skills which we found were being sustained six months after the programme had concluded. So while the programme may be hard work to do properly it appears to be well worth it.
True, I’ve picked examples at either end of the spectrum here. There are many evaluation reports of EEF trials in which teacher feedback was ambivalent and consistent with the lack of impact found on attainment outcomes. And, indeed, others with positive impacts that teachers liked immensely.
But it’s worth bearing in mind; the most promising programmes – the ones most likely to improve outcomes, but which, in turn, require time and sustained effort – may not always be those that teachers find the most enjoyable.
* I’ve not named the trial, because I don’t mean to pick on it, and there may be other reasons why we didn’t see an impact – it’s just an example.