Constructing your comparison group.

Random allocation is the most robust way to establish a comparison group, but it is not always practical to run a trial in this way. For example, if you want to evaluate the impact of a new sanctions and rewards system, it would not be possible (or ethical) to give only half the school access to rewards for good behaviour or effort. In this case, a matched comparison group is more suitable.

There are three broad categories for establishing a comparison group:

  1. Random allocation
  2. Matching
  3. Simple comparison

Random allocation

Random allocation involves one 'population' of pupils being allocated randomly into an intervention group, where they will receive something new (such as a new style of teaching, or a specific intervention to improve numeracy skills for example), and a control group, where they will continue receiving their education as usual. The population may a specific group of pupils (e.g. Pupil Premium), a year group, or a subject group. The most important thing to remember with random allocation is that once pupils have been assigned to a group they cannot change. If you have pupils change groups this will alter the randomness of the random allocation and it will no longer be a valid randomised controlled trial. Randomising pupils is a relatively straight forward process involving three steps. Click here for a step by step guide or watch a short guide here:

How to randomise pupils

A Matched Comparison Group

Creating a matched comparison group involves identifying similar pupils to the pupils in your intervention group who do not receive the intervention. The matched pupils may be taken from a different school or from a different cohort of pupils (e.g. last year's year 11). It is important to match pupils on important characteristics that affect the progress of pupils. For example, finding a matched control group might include looking for pupils with similar characteristics in:

  • prior attainment
  • currently working at grades
  • gender
  • SEND needs

Matched comparison groups are useful when you cannot allocate pupils randomly into groups. However, they are a lot less reliable than randomised controlled trials so they are a much weaker option. Click here for a step-by-step guide to matching pupils

Simple Comparison

A simple way of evaluating impact is to compare pupils who have received the approach you are evaluating to pupils who have not received the approach. The comparison group might be current pupils not receiving the intervention, or previous year group.

Unlike matching, you do not need demographic data for this group, but this means it is less accurate than matching. For example, you may choose to compare your current Year 10 cohort to last year’s Year 10 cohort. Your current cohort may have significantly more SEN and EAL pupils than the previous year’s cohort and may end up scoring lower on your chosen measure. A simple comparison would lead you to conclude that your approach had no effect when in fact it may be that this year’s cohort had a larger number of SEN and EAL pupils. Click here for a step by step guide to simple comparison. 

lightbulb_outline

Your comparison

For an interactive guide to choosing the design of your comparison group, start the Decision Tree in the righthand box

show hint help_outline