Planning and recording what happened
Prior to implementing the intervention, it is useful to write down exactly what you intend to do (e.g. how long will the intervention be delivered for?; how many times a week will it take place?; what training or preparation will teachers receive?). This will ensure that if the intervention is successful then you will know exactly what it was you did to make it work. Without this step we could end up making claims about the impact (or lack of impact) of something that was not actually implemented.
Despite this step, often the intervention may not be delivered exactly as you intended. Teachers may change it, select from it, improve it or just fail to do it properly (e.g. the plan might have been to deliver an intervention daily, but in reality it may have been delivered only once a week). It is also useful to record exactly what was actually delivered for a number of reasons. In cases where the intervention does not appear to be effective, you will be able to check whether this was because it didn’t actually take place as intended. In cases where the intervention is effective, you may have learned something new (e.g. weekly mentoring might be effective as you had hoped daily mentoring would be).
The DIY Guide is primarily about impact evaluation - understanding whether or not an intervention has had an impact on attainment. However, in addition to impact evaluation, process evaluation can be used alongside to understand how the intervention was delivered on the ground, including:
- Was it delivered as intended?
- What are the staff and pupils' perceptions of the approach?
- What has worked well and what has worked not so well?
Information from the process evaluations will enable you to understand how the intervention might be improved and whether is is practical to roll it out. There are various types of quantitative and qualitative data you could collect in an impact evaluation. Find out more about process evaluations here.