Sci-napse: Engaging the Brain’s Reward System
Sci-napse aims to improve pupils’ science knowledge using quizzes in lessons. Teachers use an online quizzing tool called Q-fire to deliver quiz questions interspersed with usual classroom activity. This trial evaluation tested two versions of the programme – a ‘test-based’ version, in which fixed points were allocated for correct answers; and a second, ‘game-based’ version, in which pupils were allocated varying awards.
Testing the impact of game-based rewards in secondary school science classes.
The Institute for Effective Education
Feedback & monitoring pupil progress
There is some evidence from neuroscience that the motivation associated with rewards can influence the rate at which pupils learn – and that this impact might be increased through the use of uncertain rewards. The EEF and Wellcome funded this evaluation to see if these neuroscientific findings would translate to improved pupil outcomes in the classroom.
The independent evaluation did not find a positive impact on science test scores for Key Stage 3 pupils. However, this result compares all classes of teachers that were offered the programme to a control group of classes where teachers were not offered the programme – meaning that teachers who did not fully implement the approach contribute to the overall result. This is standard in all EEF evaluations, as it is critical to find out whether a programme is likely to be used by teachers in their classrooms. This is important to bear in mind in the case of Sci-napse because only 54% of teachers in the test-based approach and 29% of the teachers in the game-based classes met the minimum requirements of the intervention. wherein those classes which did meet the minimum requirements there was some tentative evidence of a positive impact. Some of the reasons given by teachers for this poor programme implementation include difficulties with the Q-fire quizzing tool and the challenge of fitting six quiz questions into a lesson.
This trial shows the importance of implementation, both for schools and programme developers. Despite the theoretical evidence to support the intervention, proper implementation is crucial if programmes are to have a positive impact on pupil outcomes. The EEF is focused on improving the implementation of approaches in schools, and has published a guidance report to support best practice. The EEF has no further plans to trial the Sci-napse programme.
Children in both test- and game-based Sci-napse classes made a small amount less progress than pupils in the comparison classes. The difference in scores is not “statistically significant”. This means that the statistical evidence does not meet the threshold set by the evaluator to conclude that the true impact was not zero. This finding has moderate to high security.
Only 54% of teachers in the test-based Sci-napse classes and 29% of in the game-based classes met the minimum requirements of the interventions. This meant using questions in some, most or all of their lessons – and, for the game-based intervention, making use of escalating points and the wheel of fortune.
Exploratory analysis found that the test-based Sci-napse classes that met the minimum requirements of the intervention made an average of three additional months progress. A small positive impact was found in the game-based classes that met the minimum requirements. The small number of pupils included in these analyses means that they should be treated with caution.
Teachers and pupils reported that Sci-napse was valuable as a revision tool but indicated that it was difficult to fit six questions into each lesson and that there were challenges with the technology – particularly around adding their own questions.
Full project description
Sci-napse is a programme developed by researchers at the University of Bristol and Manchester Metropolitan University, which aims to improve the science knowledge of Year 8 pupils through using quizzes in science lessons. An online quizzing tool, called Q-Fire, provides teachers with banks of questions, which can be added to or adapted to fit with lesson plans. Teachers ask the questions, interspersed with usual classroom activity. Teachers were given training in the use of Q-Fire, either in person or using online professional development materials. This trial tested two versions of Sci-napse, against a business as usual control:
- Test-based: Small teams of pupils (3 or 4) are allocated fixed points for answering questions correctly throughout the lesson.
- Game-based: Small teams of pupils (3 or 4) are allocated varying awards. Points for correct answers increase as the lesson progresses and there are opportunities for extra points through bonus rounds or random chance games, such as a “wheel of fortune” that gave teams the chance to double or lose their points.
The evaluation was a three-armed randomised controlled efficacy trial with allocation at the class level within 44 schools. An implementation and process evaluation collected data using classroom observations, teacher and pupil focus groups, interviews with the programme developers, and an online teacher survey. The project took place between September 2016 and July 2017. The evaluation was funded by Wellcome and the Education Endowment Foundation as part of the Education and Neuroscience funding round.