Stephen Tall, Development Director at the EEF, looks at the ethics of conducting Randomised Controlled Trials in our work in schools.
The ethics of research has been put under the spotlight following controversy this week over Facebook’s participation in a study examining the impact its news feed could have on users’ emotions. The paper, ‘Experimental evidence of massive-scale emotional contagion through social networks’, was published by academics from Cornell and the University of California in the Proceedings of the National Academy of Scienceon 17 June
The study set out to ‘manipulate’ (its word) the newsfeeds of 689,003 Facebook users, increasing either the amount of positive or negative content they saw to test its impact on their emotions. The paper’s finding is that those exposed to more negative content responded in turn by posting more negative content:
When positive expressions were reduced, people produced fewer positive posts and more negative posts; when negative expressions were reduced, the opposite pattern occurred. These results indicate that emotions expressed by others on Facebook influence our own emotions, constituting experimental evidence for massive-scale contagion via social networks.It isn’t too surprising why this research has sparked concern. The Guardian, for example, worried: ‘If Facebook can tweak our emotions and make us vote, what else can it do?’
David Hunter, Associate Professor of Medical Ethics at Flinders University, has posted a good and reasoned assessment of the furore at The Conversation. He looks at the two major issues raised about Facebook’s participation, namely data privacy, and the extent to which participants in the study were offered the chance to give informed consent. He argues Facebook acted properly in the former, but not in the latter (and indeed the company has now apologised)
The issue of ‘informed consent’ is an important one for us here at the EEF. In our first three years’ grant-making, we have awarded funding for 87 different trials (74 of which are Randomised Controlled Trials) working in some 2,400 schools and involving more than 500,000 pupils. How do we balance our aim of generating high-quality evidence of what works in raising the attainment of disadvantaged pupils with the need to maintain the highest possible ethical standards in our work?
Start with the evidence we have already of what works…
Unlike the Facebook study, the EEF has no interest in testing hypotheses of negative interventions. All the 87 grants we have awarded to date are based on evidence that what is being trialled will raise the attainment of the pupils involved, and that it will make a particular difference for pupils from low-income backgrounds. The Sutton Trust-EEF Teaching and Learning Toolkit, which summarises more than 10,000 research reports, is our starting point for identifying those approaches most likely to work. For example, the evidence in the Toolkit is that ‘feedback’ can deliver high impact for low-cost – so we are funding eight projects that will give us a much better understanding of what that might look like in the classroom
High-quality, independent – and ethical – evaluation is crucial
Though we back only those projects we think have the best evidence of promise that they will raise attainment and narrow the gap, it is inevitable that not all will work out as well as we hope. We appoint independent evaluators to make sure that neither we (as the funders) nor the delivery organisation (as the grantee) are conflicted. The EEF requires our evaluators to apply for ethical clearance following their organisation’s own internal policies and procedures, and for all funded trials to work within our own ethical guidelines – for instance, that ‘The research’s purpose, method, its intended use and any possible risks must be clearly explained to participating schools and, where appropriate, parents and students’. All EEF evaluation reports will be published openly, no matter what their findings. Finding out what doesn’t work (or at least what doesn’t work well enough yet) is, as Jonathan Kay noted here, as important as finding out what does work
With innovation comes responsibility
Our use of Randomised Controlled Trials (recruiting a group of schools and randomly allocating them to one of two groups, the ‘treatment group’ or the ‘control group’) to evaluate EEF-funded trials is regarded by some as unfair. Surely all children should benefit from the projects we’re funding? However, as our DIY Evaluation Guide – written to help schools to trial new approaches to teaching and learning – points out in its section addressing the ethics of testing:
It is important to explain that an evaluation of a new possible method is taking place, not an evaluation of a better method as if we knew it was better, we’d be doing it already. In addition, it is a general principle of research that participants should give informed consent to take part. However, schools routinely innovate, try out new approaches and informally evaluate them all the time. You should use your own judgement and usual process when it comes to deciding whether to gain consent for children to take part in either the intervention or the testing.This gets to the heart of the EEF approach. Our role is to support schools testing new ways of boosting the attainment of their pupils, especially the most disadvantaged. But this comes with two important professional responsibilities. First, that this should be ‘disciplined innovation’, innovation that builds on what we already understand from existing evidence. And secondly, that these new approaches are robustly evaluated so we find out if our good intentions are matched by pupils’ progress.