Digital Feedback in Primary Maths

Digital Feedback in Primary Maths aimed to improve the feedback provided by primary school teachers using a tablet application called Explain Everything, diagnostic assessments and training on effective feedback. In this trial, it was expected to increase the maths attainment of pupils in Years 4 and 5.

Digital Feedback in Primary Maths

St Margaret’s Primary School, Kyra Teaching Scho

Testing the use of diagnostic assessments and digitally recorded verbal feedback in maths for primary school pupils

Cost
Evidence Strength
Impact (months)
0
add close
accessibility

Key Stage 2

Key stage

exposure

Maths

Subject

EEF Summary

The EEF Toolkit reports that feedback is a high-impact teaching approach. However, there is little evidence for the extensive written marking that often forms the backbone of schools’ formal feedback to pupils. The EEF funded this trial to test a way teachers can provide pupils with more frequent verbal feedback. A small short-term trial of the approach had previously found some evidence of promise. 

Our trial involved 34 schools, 108 classes, and 2,564 pupils. The independent evaluation found no evidence that the programme had a positive impact on the measure of attainment chosen for the trial, the maths attainment of pupils in Years 4 and 5. 

This result is rated as moderate-to-low security: 2 out of 5 on the EEF padlock scale. This is mainly because of the large numbers of pupils who were not included in the testing. 

The EEF has no plans for a further trial of this programme, but will continue to consider other projects which aim to provide effective feedback using digital technology. The EEF will publish a guidance report on effective feedback in 2020; our guidance report on using digital technology effectively is available here.

Research Results

Were the schools in the trial similar to my school?

  • 34 schools were involved in the trial
  • They were all state primary schools from Lincolnshire, North Lincolnshire, Nottinghamshire, and the Humber
  • Approximately 30% of the pupils involved were eligible for free school meals

Could I implement this in my school?

Participating teachers received a one-day training session at the start of the programme and were assigned to a Professional Learning Community (PLC) of 8 to 12 teachers from different schools. PLCs were expected to meet once a month to discuss the implementation of the intervention.

One teacher in each participating school was designated as a Research Lead, responsible for providing coaching to their colleagues using the programme. Research Leads received a half-day training session on their coaching role. It was intended that Research Leads would also meet once per term to receive additional training.

Teachers struggled to implement the digital aspects of the programme as intended. Supporting teachers to use the digital aspects effectively would require a significant investment of time and effort.

account_circle

Teachers

Delivered by

group_add

Whole Class

Participant group

date_range

1 Year

Intervention length

How much will it cost?

The estimated average cost of Digital Feedback in Primary Maths for one school for one year was around £840, or £15 per pupil per year when averaged over 3 years.

£

£15

Cost per pupil

people_outline

Variable

No. of Teachers/TAs

today

9 Days

Training time per staff member

Evaluation info

Schools

34

Pupils

2,564

Key Stage

Key Stage 2

Start date

October 2016

End date

September 2019

Type of trial

Efficacy Trial

Evaluation Conclusions

  1. There is no evidence that Digital Feedback in Primary Maths had an impact on pupils’ maths outcomes. This result has a moderate to low security rating.

  2. There is no evidence that the programme has an impact on the maths outcomes of pupils eligible for free school meals.

  3. The programme, as designed, was not easily implemented by teachers and schools. There was evidence that teachers and Research Leads sometimes did not attend training, have coaching sessions, or provide regular digitalfeedback in the classroom.

  4. Both the diagnostic assessment and training in effective feedback were reported as more useful by teachers than the use of the app to provide digital feedback. Evidence from surveys, PLC observations, and an interview indicated that the diagnostic assessments were eventually being used as a global classroom assessment tool.

  5. There were significant challenges with identifying which pupils were in the treatment group and which were in the control. This meant that 431 pupils could not be included in the main analysis. This presented a small threat to the security of the trial.


  1. Updated: 26th September, 2019

    Printable project summary

    1 MB pdf - EEF-digital-feedback-in-primary-maths.pdf

  2. Updated: 26th September, 2019

    Evaluation Report

    3 MB pdf - Digital_Feedback.pdf

  3. Updated: 12th June, 2017

    Evaluation Protocol

    1 MB pdf - Round_10_-_Kyra.pdf

  4. Updated: 18th June, 2018

    Statistical Analysis Plan

    519 KB pdf - KYRA_SAP_18.06.01_FINAL.pdf

  5. Updated: 11th September, 2018

    Evaluation Protocol (amended)

    1 MB pdf - KYRA_Protocol_2018.08.20_Protocol_Update.pdf

  6. Updated: 10th October, 2018

    Statistical Analysis Plan (amended)

    743 KB pdf - Kyra_SAP_2018.08.20_UPDATE.pdf

Full project description

The Digital Feedback in Primary Maths programme aimed to improve the feedback provided by primary school teachers using a tablet application called Explain Everything, diagnostic assessments, and training on effective feedback. In this trial, it was expected to increase the maths attainment of pupils in Years 4 and 5. The programme was developed and delivered by KYRA Teaching School Alliance.

Instead of providing written comments, teachers used the app to record video of themselves providing verbal feedback on pupils’ work or record themselves annotating photographs of pupils’ work. Pupils were able to access and replay the feedback videos using tablets available in class. Teachers could address common misconceptions by making recorded feedback available to all pupils in the class. The delivery team also provided teachers with diagnostic assessments to assess pupils’ understanding of a topic. These assessments were very short, consisting of only three or four questions. It was suggested that teachers use both the digital feedback and diagnostic assessments two or three times a week.

The delivery team supported teachers to adopt these approaches by providing initial training, setting up Professional Learning Communities (PLCs), and arranging coaching. Participating teachers attended a one-day training session on the principles of effective feedback and on using the app. Teachers were assigned a PLC of 8 to 12 teachers from different schools who were expected to meet once a month to discuss the implementation of the intervention. One teacher in each participating school was designated as a Research Lead, responsible for providing coaching to their colleagues using the programme. The project team provided Research Leads with a half-day training session on their coaching role. It was intended that Research Leads would also meet once per term to receive additional training.

The impact of Digital Feedback on maths was estimated using a randomised controlled trial involving 34 schools, 108 classes, and 2,564 pupils. Classes were randomised to either receive the intervention or to a control group which carried on with business-as-usual teaching. Maths attainment was measured using ACER’s Essential Learning Metric and the level of disaffection with school was measured as a secondary outcome. A process evaluation used surveys, interviews, observations of training and PLC sessions, and analysis of feedback videos, coaching records and diagnostic assessments to examine how the programme was implemented. The delivery of the programme started in September 2017 and concluded in June 2018.