A pilot to evaluate Thinking Mathematically, a 20-week programme that explicitly teaches pupils how to tackle mathematical problems via an evidence-informed approach that aims to increase teachers’ use of modelling, heuristics and discussion to improve pupils’ metacognitive skills and resilience in answering problem-solving questions.
A programme that trains teachers to help Year 7 pupils confidently tackle unfamiliar mathematical problems.
In 20 weekly problem-solving sessions, teachers model their own thinking using a ‘think aloud’ method and appropriate worked examples.
Teachers provide pupils with support by using a checklist of best practice and creating ‘debrief’ opportunities for pupils to discuss their methods and thought processes.
Thinking Mathematically went through the EEF’s Early-stage Programme Development process. This supports organisations to design, develop and deliver programmes that tackle educational disadvantage. As part of this process, the approach has been implemented in 21 settings and was shown to be acceptable and feasible.
While many elements of the programme have already been tested, the EEF were interested in funding this pilot to test the programme’s scalability in a larger number of schools, via an independent evaluation.
Thinking Mathematically was developed from key findings in the EEF’s Guidance Report on Improving Mathematics in Key Stages 2 and 3. In particular, its recommendations on teaching strategies to solve problems. It also aligns with wider evidence on mathematical problem-solving.
The programme was evaluated by Oxford MeasurEd as a pilot. The evaluation was a mixed-methods implementation and process evaluationAn IPE is used to understand how and why an intervention has (or has not) been successful. Data is analysed to explore programme quality, reach, adaptation and differentiation, as well as setting fidelity and responsiveness to the trial design. to explore how schools implement the programme and perceive it.
Findings from the pilot indicate that the programme has some evidence of promise, but that some causal pathways are weaker than others.
Although the response rate to the post-delivery surveys was low, most respondents reported improved problem-solving for pupils as a positive outcome of the programme. Pupils and teachers perceived problem-solving to be different to ‘business as usual’ teaching, and pupils expressed being motivated to engage with the programme. However, teachers reported that pupil engagement was not consistently high and was shaped by how accessible and relevant pupils found the problems.
Post-delivery survey data indicate promising levels of sustainability. Nearly three-quarters of teachers reported intending to sustain key practices, including encouraging pupils to discuss and compare approaches (the debrief) and modelling their thinking through ‘think aloud’. However, it is important to note that the self-selecting nature of the respondents means these survey results may not represent the views of all pilot participants.
While training took place as planned and cascading took place in most schools, the programme was not consistently delivered as intended.
There was variation regarding session duration and coverage. Classroom sessions were delivered regularly, but teachers did not consistently adhere to the core components – there was inconsistent usage of programme materials and strategies.
Teachers praised the classroom resources for their clarity but highlighted that some questions were less suitable to pupils with special educational needs and disabilities (SEND) or English as an additional language (EAL).
Ultimately, the programme is partially manualised and would benefit from further piloting before being evaluated for a trial. Comprehensive and high-quality materials exist but require additional guidance and a clearer articulation of core components before delivering the programme to a larger number of schools. This should include a more explicit distinction between non-negotiable and flexible elements.