Matched comparison groups

A teacher in a primary school wishes to understand the impact of a new reading scheme on Year 4 reading ability over the year and decides to compare this year with last year’s cohort.

  1. Ensure both groups have taken the same baseline test. This year’s results will be compared with last year’s on the same reading test, since children are routinely tested at the start and end of every year. The test taken at the end of Y3 is the pre-test for both groups.
  2. Rank on pre-test. All the scores on this test for both groups are pasted into adjacent columns in a spreadsheet with current Year 4s coloured blue and the previous Year 4s coloured red. All the scores from both groups are sorted in order.
  3. Identify matches. The teacher matches each current Year 4 pupil’s test score (blue) with a close score for a pupil in last year’s group (red). She decides beforehand that pupils must be within two marks to be a match. By doing this, pairs of matched pupils are created.
  4. Check the groups are balanced. The teacher checks that the pairs are balanced. The number of pairs where blue is higher than red should be is the same as the number the other way round. Some students’ scores do not match any from the previous so the teacher excludes these pupils from the analysis. The teacher also checks that the pairs are balanced on other characteristics, so they are removed from the evaluation. Importantly, the teacher ends up with two balanced groups, each with 16 children in, creating 16 matched pairs.
  5. Compare outcomes. The intervention is implemented over the next year, followed by the post-test. She is then able to look at the average scores for the intervention and matched control group, and observe the average differences in score between the two groups of matched pupils. The teacher compares the average scores and calculates the effect on attainment (see Stage 3).