Eedi is a mathematics homework setting platform, designed to facilitate teachers’ ability to identify student mathematics misconceptions through diagnostic multiple choice questions. Each topic in a teacher’s scheme of work has two associated 10-question assignments that are set as homework, three weeks apart. Eedi aims to save time teachers would ordinarily spend marking homework and also has a parental update function.
The EEF is committed to building the evidence base around teachers’ marking: this project was considered an interesting test of an alternative to time-consuming written marking and there is solid rationale for teachers to have access to information about their students’ misconceptions. In addition, the programme used lessons from the successful EEF Texting Parents project to communicate to parents. Previous EEF trials examining formative assessment interventions have found mixed results.
This project and its evaluation were affected by the 2020 partial school closures caused by the COVID-19 pandemic, and the cancellation of GCSE examinations in 2020. As a result, the evaluators were not able to use GCSEs in order to estimate the impact of the project on maths attainment as planned.
The evaluation combines quantitative and qualitative methods to establish the extent to which the intervention was implemented and gain an understanding of how teachers and students experienced using Eedi.
Most teachers receiving the intervention reported a reduction in their homework-related workload; however, quizzes were not set as frequently as intended and the process evaluation (the teacher survey and request for case study visits) also had a low level of compliance.
There were also reports (from teachers and students) of students guessing their Eedi homework and students who were eligible for free school meals were less likely to start or complete an Eedi quiz compared to their non-FSM counterparts.
The EEF has no current plans for a further trial of Eedi.
- Due to Covid-19, the primary outcome for this evaluation was not collected and so no measure of impact on maths attainment is reported. Key conclusions are based on qualitative data from the implementation and process evaluation. There was some evidence that Eedi reduced teachers’ homework-related workload as reported in approximately half of teachers’ responses in school visits and survey questions. Teachers responding to the survey in intervention schools noted an average reduction in their workload of 28 minutes per week, compared with teachers responding in control schools.
- Further exploratory analysis showed that students who were eligible for free school meals were less likely to start or complete an Eedi quiz compared with students who were not eligible. The parents of students who were eligible for free school meals were also less likely to log into the Eedi platform compared with parents of children who were not eligible.
- Teachers did not set the Eedi homework quizzes as frequently as intended. Students were set an average of 25 quizzes in Year 10 and four quizzes in Year 11, compared with expected figures of approximately 80 quizzes in Year 10 and 52 in Year 11. The low dosage limited the ability of Eedi to deliver its intended benefit as a formative assessment tool.
- Reports from teacher surveys and from students (in four of seven focus groups) suggested that students were guessing answers to their Eedi homework due to the difficulty of questions, the desire to complete homework quickly, and a lack of engagement. Teachers reported finding it useful when students completed the reasoning box to document their workings; however, students reported not using this feature.
- The parental update function was enabled for 36% of students in Year 10 and 11% of students in Year 11. Of the teachers who responded to the survey, only 1% in Year 10 and 3% in Year 11 reported that they had noticed a major positive change in parental awareness or engagement with their child’s maths learning; 43% of Year 10 teachers and 31% of Year 11 teachers who responded to the survey reported a minor positive change.