Let’s Think Secondary Science
Let’s Think Secondary Science (LTSS) aims to develop students’ scientific reasoning by teaching them scientific principles such as categorisation, probability, and experimentation. LTSS challenges students’ thinking, develops their metacognitive skills, and encourages cooperative learning. Let’s Think Forum (a registered charity led by a group of academics, teachers, and consultants) created LTSS by adapting a programme called Cognitive Acceleration through Science Education (CASE). Let’s Think Forum’s adaptations to CASE included reducing the number of lessons from 30 to 19, reducing the number of scientific principles that are taught, rewriting lesson plans, and providing additional resources like PowerPoint slides and video tutorials. In this trial, Let’s Think Forum provided one day of training and three support sessions per year to the science teachers who would be teaching LTSS. These teachers then delivered the lessons to a cohort of Year 7 students instead of their usual science lessons. LTSS was delivered over two years, so this cohort of pupils continued to receive the programme in Year 8.
LTSS was evaluated using a randomised controlled trial with over 8000 students in 53 schools. Schools were randomly allocated to deliver either the programme or their ‘business as usual’ science teaching. It should be considered an effectiveness trial, as it aimed to test a scalable intervention under realistic conditions in a large number of schools. The primary outcome measure was an age-appropriate science test based on a Key Stage 3 SATs paper, and the secondary measures were English and maths tests provided by GL Assessment. The process evaluation consisted of lesson observations, surveys, and interviews with staff, and surveys and focus groups with students. The trial started in September 2013 and ended in July 2015.
Testing a cognitive development programme for science.
The Institute for Effective Education
Developing effective learners
Staff deployment & development
The following conclusions summarise the project outcome
This evaluation provided no evidence that Let’s Think Secondary Science improved the science attainment of students by the end of Year 8.
Students who received LTSS did worse than the control group on the English and maths assessments, but this result could have occurred by chance and we are not able to conclude that it was caused by the programme.
Many schools did not implement the programme as intended by the developer. In many schools, individual teachers delivered fewer than the full programme of 19 lessons and senior leaders were less engaged than prescribed by the programme.
Although most teachers were providing opportunities for students to work collaboratively, there was some evidence that more support to help teachers promote effective small group discussions would be welcomed.
Previous evaluations of CASE have suggested that it had longer-term impacts on academic attainment. Future research could examine whether LTSS also has a long-term impact by examining the GCSE results of the pupils involved in this evaluation.
What is the impact?
This evaluation provided no evidence that Let’s Think Secondary Science had an impact on science attainment. Students who received LTSS did worse than the control group on the English and maths assessments, and girls who received LTSS did slightly better than the control group in the science assessments, but these results could have occurred by chance and it is not possible to conclude that they were caused by the programme. However, they could provide useful areas of focus for future evaluations of LTSS. There was no evidence that the programme had a differential impact on pupils according to their eligibility for free school meals (FSM), or prior attainment in Key Stage 2 (KS2) maths and reading.
The process evaluation suggested several possible reasons why the programme did not have an impact. Schools often did not implement the programme as intended. Although just over half the teachers had taught at least 13 of the 19 different LTSS lessons, a quarter of them had taught 6 or fewer. This did not necessarily mean that students would not receive all 19 lessons, since the survey was completed before the end of the year and also the teachers could have been sharing delivery with a colleague. However, when teachers did teach an LTSS lesson, they generally appeared to do so as the programme developers intended. A member of each school’s leadership team was supposed to oversee the school’s implementation of the programme, but senior leaders appeared to be involved to a much lesser degree than expected, if they were involved at all. LTSS encouraged teachers to deliver each lesson first to the year group above the trial cohort, so that they would have the opportunity to practise delivery. Only two in five of the teachers reported that they did this very or quite often, and a quarter reported that they never did. Teachers also reported that it was challenging to fit all the lesson material, which was designed for hour-long lessons, into the 50-minute slots that many schools used.
Although the large amount of group work was popular with most students, some teachers complained about it leading to disruptive behaviour. This suggests that some teachers needed more support to be able to encourage more productive small group discussion. Some teachers felt that LTSS was less accessible for lower-attaining students. However, it should be noted that the impact evaluation did not find any evidence that lower-attaining students who received the intervention performed worse than similar pupils in the control group. Student reaction was mixed, with many enjoying the discursive and collaborative nature of LTSS, but some students held negative views about the repetitiveness of lessons and working in groups. Overall, most teachers enjoyed the challenge of LTSS, were positive about the practical activities, and recognised improvements in their questioning techniques.
Previous evaluation of CASE also failed to detect an impact on science attainment immediately after the intervention finished at the end of Year 8. However, there is evidence that CASE had a long-term impact on attainment, as students who received CASE outperformed the control group in their GCSE science, maths, and English language exams. Future research could examine whether LTSS also has a long-term impact by examining the GCSE results of the pupils involved in this evaluation.
|Group||Effect size||Estimated months’ progress||Security rating||Cost|
|LTSS versus control group||-0.02 (-0.09 to 0.04)||-1|
|LTSS FSM versus control group||-0.03 (-0.18 to 0.07)||-1|
|Since this report was published, the conversion from effect size into months of additional progress has been slightly revised. If these results were reported using the new conversion, all results would be reported as 0 months of additional progress rather than -1.|
How secure is the finding?
Overall, the findings from this evaluation are judged to be of moderate security. It was a large and well-designed randomised controlled trial. At the beginning of the trial, the schools and pupils who received the intervention were similar to the schools and pupils in the comparison group. Two padlocks were removed from the rating because 26% of the pupils did not complete all the required tests and were not included in the final analysis. 11% of schools dropped out of the study. There were no other major threats to the security of the trial.
How much does it cost?
The overall cost per school for the two-year programme was £4,490. The average cost per student is estimated as £3.99 per year across three years. Cost information was collected from the project delivery team. The main costs related to the training, with much lower amounts for lesson resources and equipment. LTSS involved some burden of extra time, which the delivery team estimated at five hours per year for teachers (mainly planning) and two hours per year for a technician.
Let’s Think Secondary Science (LTTS) is based on a programme called Cognitive Acceleration through Science Education (CASE). CASE has been the subject of several previous evaluations, which have suggested that it has an impact on academic attainment and scientific reasoning. LTSS was intended to be a shorter version of CASE, with updated lesson plans and additional resources such as video tutorials. These changes were based on information about how teachers had used CASE over many years. The EEF funded this evaluation because it was more rigorous than previous evaluations of CASE and it was an opportunity to test the changes to the original programme that were included in LTSS.
This evaluation provided no evidence that LTSS improved the science attainment of students by the end of the programme. Previous evaluations of CASE suggest that it had a long-term impact on academic attainment. It would therefore be valuable for schools to know the impact of LTSS on longer-term outcomes as well. The EEF will be measuring the impact on GCSE results once they are available for the participating students.
The EEF previously funded an evaluation of Thinking, Talking, Doing Science (TDTS), a similar programme for primary age pupils which also aims to improve pupils’ scientific reasoning. TDTS was found to have a positive impact and the EEF is currently recruiting to an effectiveness trial of this approach.