Measurements of impact – even those found in the EEF Toolkit – are based on averages, and caution must always be exercised in the interpretation of data presented with averages. The evidence of the positive impact of feedback interventions (+8 months) is what’s called an ‘indicative effect’: this means that it is an indication of the average impact found in the many studies used to reach the finding about feedback’s effect. In some examples of good-quality research, findings are positive, in others there was no impact. In yet others, a negative impact is seen.
When using assessment data in school to measure impact, understanding the spread of data is key. If, for instance, most children in a school achieve scores that are expected for their age and stage, but a small group of them achieve much lower than expected, these low values may draw the average down and give the impression of overall underachievement. So good practice when looking at any average is to try to understand how it is composed (what the spread of scores which contribute to it looks like).