What immediate actions should countries wanting to embrace ‘what works’ consider to increase the supply of evidence and facilitate its adoption by policymakers and practitioners? That’s the question the EEF’s deputy chief executive James Turner explores in this blog, adapted from his presentation to this week’s Innovation in Evidence conference in Canada.
The Education Endowment Foundation – the UK’s What Works Centre for Education – is seven years old. One major lesson from that time is the importance of context, whether that be in the implementation of individual projects or in the overall ‘evidence ecosystem’ that it is appropriate to create in one circumstance compared to another.
Nonetheless, the ‘what works’ movement is founded on the principle that learning from what has been found to be effective elsewhere is a good place to start in giving an idea maximum chance of success, even if that is in a different school, hospital, region, country or continent. So, based on our work to date, there are three things that stand out for me in terms of useful next steps as Canada thinks about its own journey:
1. Institutions are important anchors.
To set the evidence bar high and to ensure consistency, comparability and accessibility in outputs, you need a convening force.
The step-change in the production of high quality evidence for English schools – which has seen the number of randomised controlled trials (RCTs) burgeoning from pretty much zero to 150 in a few years – has only been possible because of a partnership between schools, academics and the not-for-profit sector. But to instigate reform, to set the evidence bar high and to ensure consistency, comparability and accessibility in outputs, you need a convening force, and institution whose sole mission is to make these things happen and drive through change.
That institution need not be entirely new – the EEF is itself a creation of two charitable foundations and the UK government – but it needs to be independent of vested interests and objective in its view. Establishing with absolute clarity who the flag-bearers of the movement will be, and ensuring they have credibility with practitioners, decision-makers and the host of partners they will need to engage, is a must.
2. Harvest what is already out there.
The ‘what works’ approach is often conflated with the desire to do something entirely new. Teachers in England are highly cynical of the latest fads and fatigued by constant change. And it takes time to generate new, high-quality evidence as well as confidence to resist the urge for ‘quick and dirty’ results.
Focus new evidence-generation work on promising approaches that are already in the system.
So, start with assimilating and rating the existing research and – crucially – summarising the findings in a way that is genuinely useful to practitioners. That provides huge ‘bang for buck’ in terms of impact.
Focus new evidence-generation work on promising approaches that are already in the system. There is a pressing need to know whether they work or not; and if they do, scaling up an initiative that has proved its mettle in the cut and thrust of, say, a busy classroom, will be easier than an academic fancy. Of course, there is an important place for innovation – but focused on disciplined innovation, based on evidence and sound theory.
3. Establish genuine political buy-in from the outset.
The ‘what works’ movement has gained great traction in the UK government over the last five or so years. New centres have been formed and there are examples of evidence shifting significant sums of government spending for the better. But with popularity comes the danger of superficiality: the tendency to pepper press releases and strategy documents with ‘evidence’ and ‘what works’ rhetoric, without any real commitment to decisions being made differently.
A final word of warning: Don’t let the best be the enemy of the good.
Focus initially on a few key areas where there is an appetite to think radically and where there is the political cover for a genuinely evidence-informed approach. Success here will be an important way to gain traction for the approach elsewhere.
Finally
A final word of warning: “Don’t let the best be the enemy of the good”. Setting the evidence bar high and pushing up the quality of evaluations is at the heart of what we are all aiming to do. But as important is that the work is relevant and timely for those it is meant to influence, if it is to result in any real-world change. So, be clear where the non-negotiables are and where pragmatism is necessary.
It is rare for any academic study to conclude that further research is not needed; but at some point we need to make the call and start making decisions