EEF publishes new evaluation reports

The Education Endowment Foundation (EEF) has published independent evaluations of further projects today, bringing the total number of complete EEF-funded projects to date to 141.

These include the reports from a randomised controlled trial (RCT), as well as an update and an addendum to two previously published trials; in addition, there are independent evaluations of two pilot studies.

ASCENTS 121 Support for Science

This programme, co-funded with the Wellcome Trust, aimed to improve GCSE science attainment through weekly one-to-one academic mentoring sessions, delivered by STEM undergraduates outside of school hours under the supervision of a teacher.

Led by the University of Lincoln, ASCENTS 121 was delivered across 46 schools to 845 science pupils in Year 11, all of whom were eligible for free school meals and predicted to achieve a level 3-5 in GCSE science.

The independent evaluation, undertaken by NatCen, was unable to ascertain its impact on GCSE outcomes owing to the cancellation of exams in summer 2020. However, the implementation and process evaluation found that the intervention was well-received by teachers, mentors and mentees, and was implemented as intended.

The full evaluation report, together with the EEF’s commentary on the findings, is available here.

Same Day Intervention (SDI)

The RCT of this programme, developed and delivered by a partnership between Yorkshire and the Humber Maths Hub and Outwood Institute of Education, aimed to improve maths attainment for Year 5 pupils, while also narrowing the gap between lower and higher attaining pupils, through use of the mastery approach.

The EEF published the independent evaluation by NatCen in November 2020. This found that pupils who received SDI made, on average, 0 months’ additional progress compared to those in the control group of schools. This result had a moderate-to-high security rating of 3 ‘padlocks’ on the EEF’s scale.

Subsequent analysis, published today, has added to our understanding of this result. This suggests that poor implementation may have contributed to the overall lack of impact. Pupils in schools which implemented SDI as the programme developer intended made, on average, +2 months’ additional progress in mathematics. However, it should be noted there may be other differences in these schools which means this finding is less secure than the primary analysis.

The full updated evaluation report, together with the EEF’s commentary on the findings, is available here.

Achievement for All (AfA)

The EEF has also published an addendum to our previous independent evaluation of Achievement for All. The findings are consistent with the previous results, which found that AfA had a negative impact equivalent to -2 months’ progress on all pupils in Years 4 and 5 and also a negative impact on the target pupils for the primary outcome of reading attainment.

The full evaluation report and addendum, together with the EEF’s commentary on the findings, are available here.

Direct Instruction in Key Stage 3: Connecting Maths Concepts (DI KS3 CMC)

189 pupils in Years 7-9 from eight schools took part in this pilot study of DI KS3 CMC, a programme that aims to enable pupils to master key concepts through opportunities to practise and review their understanding.

Based on the Direct Instruction model of teaching, which emphasises a sequenced curriculum, systematic teaching, and mastery, this project was delivered by the Midland Academies Trust (MAT) and National Institute For Direct Instruction (NIFDI). Pupils received DI KS3 CMC three to five times a week over 15 weeks in place of usual mathematics lessons.

The independent evaluation, undertaken by NFER to explore the promise and feasibility of the programme to support pupils with low prior attainment in mathematics, reports mixed results. Some elements of DI KS3 CMC were found to be effective, such as the sequenced structure of the programme, clear explanations of mathematics concepts, feedback, mastery, and the motivation system. However, other elements were viewed as less effective, such as the lack of differentiated content, choral responding, and repetition. Overall, the DI KS3 CMC programme is not ready for trial.

The full evaluation report, together with the EEF’s commentary on the findings, is available here.

International School Exchanges Programme (ISEP)

This programme, led by the British Council and funded by the Department for Education, aimed to enable 11-19 year olds from state-funded schools and colleges to take part in an international experience.

All trips in this pilot study were due to take place between July 2019 and May 2020. As a result of the Covid-19 pandemic, the last international visit took place in March 2020, which meant that only 42 out of 142 planned exchanges took place, involving 38 schools and colleges.

The independent evaluation of ISEP by Kantar used a mixed-methods approach, gathering evidence from in-depth qualitative interviews with teachers and pupils. This found teachers and pupils reported an increased intercultural outlook, confidence with, and tolerance of, other cultures, and pupil resilience. Schools and colleges also reported that ISEP had increased their capability to deliver international exchanges in the future and they appreciated its unique elements – such as requiring exchanges to include a proportion of disadvantaged pupils, significant interaction with partner schools and a curriculum focus with the potential to increase appeal for prospective pupils.

The full evaluation report, together with the EEF’s commentary on the findings, is available here.