Education Endowment Foundation:Increasing Pupil Motivation

Increasing Pupil Motivation

University of Bristol
Implementation costThe cost estimates in the Toolkits are based on the average cost of delivering the intervention.
Evidence strengthThis rating provides an overall estimate of the robustness of the evidence, to help support professional decision-making in schools.
Impact (months)The impact measure shows the number of additional months of progress made, on average, by children and young people who received the intervention, compared to similar children and young people who did not.
+1
months
Project info

Independent Evaluator

The Institute for Fiscal Studies logo
The Institute for Fiscal Studies
A programme which aims to increase pupil effort in Key Stage 4.
Schools: 84 Grant: £1.1m
Key Stage: 4 Duration: 3 year(s) Type of Trial: Effectiveness Trial
Completed January 2015

The project

‘Increasing Pupil Motivation’ was designed to improve attainment at GCSE by providing incentives to increase pupil effort in Year 11. Two schemes for incentivising pupil effort were implemented. The first provided a financial incentive, where pupils were told they had £80 at the beginning of each half-term. Money was deducted if they did not reach the threshold in four measures of effort: attendance, behaviour, classwork and homework. The second provided an incentive of a trip or event. Pupils were allocated a certain number of tickets at the start of term and lost them if they failed to meet targets on the same set of four effort thresholds. Pupils that retained enough ​‘tickets’ were rewarded with an event, chosen by pupils in the year group at the start of the school term.

Pupil effort was monitored by the schools involved in the intervention, but the design and development of the incentive schemes was undertaken by the project team at the University of Bristol in the first four half-terms of the 2012/13 academic year.

The target population was relatively deprived schools, classified by schools with an average pupil IDACI (Income Deprivation Affecting Children Index)score in the highest 10% in England. 279 eligible schools were invited to participate by the project team, with 84 schools indicating an initial willingness to participate. 63 schools agreed to participate in the intervention after an initial training event in July 2012; they then formed the final set of experimental schools.

Outcome/​Group
ImpactThe size of the difference between pupils in this trial and other pupils
SecurityHow confident are we in this result?
GCSE Maths
+1
Months' progress
(Effect size 0.04)
GCSE English
0
Months' progress
(Effect size 0.02)

Key Conclusions

The following conclusions summarise the project outcome
  1. Event Incentives – There is no evidence of a significant positive impact of event incentives on GCSE attainment in Maths, English or Science.
  2. Financial Incentives – There is no evidence of a significant positive impact of financial incentives on GCSE attainment in Maths, English or Science.
  3. There is a statistically significant improvement in classwork effort across English, Maths and Science for the financial incentive treatment. There is no evidence of impact on behaviour, attendance or homework effort. This may suggest that even when there is a marked improvement in effort in classwork, this does not translate into higher GCSE attainment.
  4. There was a positive impact of both the event and financial incentives on GCSE Maths for pupils with low levels of prior attainment – equivalent to about one quarter of a GCSE grade in Maths, although this is not statistically significant for the financial incentive treatment.
  5. Schools found it difficult to organise and pay for events before they knew how many pupils were likely to meet their targets. Schools should also consider the cost of monitoring and providing feedback about pupil effort.
  6. Further research should explore the level of incentive required to induce pupil effort, and the long-term impact of such schemes. Further research might also be needed to see if there are any adverse effects if schools just decided to incentivise one subject (e.g. Maths) or just one group of pupils (e.g. those with low levels of prior attainment). Additionally, future studies should explore why incentives appear to change classwork effort but do not necessarily translate into higher attainment. The relationship between improved pupil effort and its impact on attainment should be examined in greater detail.

What is the impact?

The estimated effects of the financial and event incentives are shown in Tables 1 and 2 for GCSE Maths and English, respectively (results for Science are shown in the main report). There is no secure evidence from this evaluation that financial incentives have an impact on overall pupil attainment in GCSE Maths or English. The small positive effect sizes detected – 0.04 and 0.02 in Maths and English respectively – are not statistically significant from zero. There is also no evidence of a statistically significant effect on GCSE Science. This evaluation can rule out medium to large effects on pupil attainment.

There is some suggestion that when pupils were offered the incentive of an event or trip, there was a small positive impact on Maths attainment at GCSE, but this is not statistically significant from zero. The effect size of 0.08 is the equivalent of approximately one month’s progress or one sixth of a GCSE grade. There was no evidence of a statistically significant effect on GCSE English or Science though.

There is evidence to suggest that the impact of the event incentive treatment is larger in Maths for pupils with low levels of prior attainment at Key Stage 2. The estimated impact for this incentive treatment is 0.13, significant at the 5% level, which is the equivalent of approximately two months’ progress, or one quarter of a grade in GCSE Maths. For English, the estimated effects of both sets of incentive for pupils with low prior attainment were also positive, but not statistically significant. For Science, the effects were closer to zero.

A secondary outcome measure was pupil effort. For the financial incentive, there was a positive and statistically significant improvement in classwork in English, Maths and Science (at the 5% level). For the event incentive, there was also a positive impact although this was not statistically significant. Across the other measures of effort, there was no secure evidence of a positive impact. Combined with the results on attainment, this suggests that improvements in classwork is the main area where effort has improved as a result of the incentive schemes, but there is only some evidence to suggest this translates into an effect on attainment in the case of Maths. One possible explanation is that classwork effort in Maths translates into higher GCSE attainment than classwork effort in English or Science.

There was no process evaluation commissioned as part of this independent evaluation.

How secure is the finding?

Previous research has focused on the provision of incentives that are awarded directly for test scores. Most studies found positive effects on test scores, but these positive findings are not universal. Relatively few studies have examined the effect of rewarding pupils specifically for effort tasks, rather than test score performance. One study by Fryer (2011) examined the effect of paying pupils to read books and finds little positive effect on test scores. However, as the author acknowledged, this study had some limitations. Therefore, there is currently no clear finding in the literature surrounding the effects of direct incentives on pupil performance and very little work that examines the effect of incentives for pupil effort rather than performance.

The evaluation was designed as a cluster randomised controlled trial, with randomisation at the school level, and two treatment groups of schools and one control group of schools. In one treatment group, Year 11 pupils received financial rewards for the successful completion of effort tasks. In the second treatment group Year 11 pupils received a non-financial reward in the form of a trip or event for effort tasks.

The evaluation was run as an effectiveness trial. Effectiveness trials seek to test if an intervention will work under typical conditions across a number of schools in different settings.

The design of the trial is sound. However, there are two limitations that affect the interpretation of the findings. First, the incentives were combined with a system of feedback in both treatment groups – pupils were informed of their performance and how close they were to achieving their targets. It is therefore not possible to estimate the effects of providing incentives and providing feedback separately; we must instead estimate the combined effect of the provision of incentives and feedback on pupil effort and attainment.

Second, a number of control schools (18 out of 33) did not provide effort data to the project team, which implies that pupil effort was not monitored in these schools. This subset of control schools therefore differed from treatment schools in the provision of incentives for effort, feedback and monitoring of pupil effort. To mitigate this potential problem, a robustness check was conducted to estimate the effect of both the financial and event incentive treatment relative to the subset of control schools that monitored pupil effort (those that submitted effort data to the project team). The estimated impact of each incentive treatment is not lower when we drop controls who did not submit effort, though it is higher, suggesting that schools who dropped out may be different in unobservable ways.

It is also important to acknowledge that in this trial the cost of the incentives was met by the project budget, and not directly from school budgets, and it was teachers who recorded pupils’ effort. It is therefore possible that teachers were more generous in allowing pupils to reach thresholds because they knew the money was not coming directly from school budgets. However, this is likely to be the case only in marginal decisions.

How much does it cost?

The cost of the financial incentives represented a maximum of £320 per pupil (if all targets for effort were met). The average cost per pupil was approximately £225. The cost of the event incentive treatment was the budget given to schools for organising an event at the end of each term. The budget allocated by the project team was about £80 per pupil to cover both terms. In addition to this, all schools were given an additional £2,000 to cover the expected cost of monitoring pupils’ effort, although the true cost of monitoring pupils was not calculated.

One important caveat to these figures is that this intervention focused on one particular level of incentives. Schools could in principle choose to offer lower or higher levels of incentives for pupil effort and the cost would naturally change as a result. However, more evidence would be needed in order to calculate the likely effect of offering lower or higher levels of incentives.

Schools should also consider the cost in staff time and effort organising and delivering the chosen incentive. For example, the project team reported some difficulties in paying the financial rewards in general. The project team also reported that the organisation of events at the end of term was operationally complex as deposits often had to be paid, even before schools knew how many pupils were likely to attend.

Downloads