Education Endowment Foundation:Guest Blog: Top 7 Tips for Researchers Working with Schools

Dr Emily Williams and Jo Atkinson – Research Fellows at the University of Leeds – lead the intervention delivery arm of the EEF-funded randomised controlled trial (RCT), ​‘Helping Handwriting SHINE’. Their experiences in delivering a school-based RCT are shared here in the form of tips. 

These tips have been developed in collaboration with Professor Roisin Corcoran from the University of Nottingham, leader of numerous RCTs in educational settings across the world, and Charlotte Clowes, Principal of Wilmslow Academy, who acted as an independent consultant throughout the Handwriting RCT.

The evidence-based revolution in education is underway. Practitioners, researchers, policymakers and funders want to better understand ​‘what works’ to improve education. Randomised controlled trials make up 80% of the 200 evaluations funded by the Education Endowment Foundation (EEF) [1]. They can sometimes seem off-puttingly complex to run in schools. However, the EEF has now run trials in over 13,500 schools, well over half of all the schools in England. As a result, a lot of knowledge about how to deliver RCTs well has been gained along the way. In 2018, Dr Anneka Dawson and EEF colleagues [2] proposed solutions to five challenges inherent in educational RCTs. These insights proved of key importance, so we thought it useful to build on their work. We therefore present seven tips to help build great relationships with participating schools.

Our experiences in building mutual trust between schools and researchers comes from working on the intervention delivery of a teacher-led intervention, with an independent evaluator. However, we believe our recommendations will be useful to any researchers interested in optimising their working relationships with schools when involved in an RCT in education.

1. Develop seamless signposting

Arrange a formal introduction with the members of the evaluation team at the start of the project, even if just through emails or a conference call. 

Find out who will be managing the school-directed inbox as you will be forwarding questions from schools to them at times. Provide teaching staff with the name, phone number, email information and address for the contact person responsible for leading the evaluation.

In particular, know who you should contact if you receive an evaluation-related question at the training events and a representative from the evaluation team is not present. Have their phone number handy so that you can call during a break and provide the answer as soon as possible. Being able to have someone to point a school to or a quick route to finding an answer really helps build trust with the schools, and trust is key when retaining schools in trials.

2. Support the evaluation team

Remind and encourage teachers to complete any fidelity assessments or focus groups as these aren’t ​‘optional extras’

School-based intervention programmes are not always implemented as designed. When implementing complex interventions in the ​‘real world’, variability both within and between schools is to be expected, and thus the independent evaluator will usually design and conduct both an impact and process evaluation. The study protocol should detail the approach that will be used to capture implementation fidelity, including the logic model, the key components that will be assessed, and when each will be measured. High-quality impact and implementation studies are essential to provide information about why, when, and how interventions work. To answer all of these questions it is essential to support teachers in providing information on implementation. This information helps the evaluation team better understand implementation and contextualise the impact of the intervention. The regular contact could also reduce the likelihood of schools lapsing in their intervention delivery.

3. Make it personal

Take advantage of software programmes that make it easier to manage communications

For example, Microsoft Word’s Mail Merge function (usually found under the ​‘Mailings’ tab) will allow you to type out an email (or letter) and insert certain fields (e.g. name, year group) from a spreadsheet or database such as Excel. Personalising emails can help to build your relationship with the teaching staff and increase the chance of a response. You can also add other mail merge fields, such as, ​“It was nice to meet you on your training day on [date]” or use some ​“IF” statements to include a few different sentences depending on year group or whether they are a member of Senior Leadership. You can find an introduction to Mail Merge here. In addition to building the relationship, including names may increase feelings of responsibility, accountability, and inclusion.

4. Consider alternative lines of communication

Diversify your means of communicating with schools for the best chance of providing easy access to information and sustaining interest

While email may be the standard form of communication, it is sometimes helpful to use alternative (and more informal) ways to maintain engagement with teaching staff. If you choose to use social media, then have this set this up ahead of time ready to provide teaching staff with details at the training. Sticking to platforms already used by the teaching staff (e.g. Twitter and Facebook) may achieve the best outcomes, rather than asking teachers to join something unfamiliar that they may be less likely to interact with (e.g. Slack, Groups.io, or Reddit). For example, Professor Roisin Corcoran’s team for the EEF-funded ​‘Glasses in Classes’ project at the University of Nottingham send out regular newsletters to stakeholders using Microsoft Sway. Whatever the platform, if you’re communicating about specific aspects of the intervention, make sure to keep this private.

5. Consult with service/​intervention users

Consider hiring a member of Senior Leadership from a school not recruited into your RCT as a consultant. 

The ​‘Helping Handwriting SHINE’ (HHS) steering group included a Primary School Principal (Charlotte Clowes) whose input into the training materials was invaluable. When training teaching staff, commenting on whether the intervention has been implemented in a school previously can help them judge if the intervention is likely feasible in their workplace. Our training, for example, included short videos of (RCT-independent) teachers using the HHS approach with their students. The more feasible an intervention is judged as, the less likely schools may be to withdraw from the trial.

6. Provide high-quality professional development (PD) and coaching

Structure professional development with on-going coaching wisely to get to know the teaching staff, their schools and allow for networking opportunities. 

Pitch PD in an appropriate style. When creating PD materials, strike a balance between including academically interesting information and writing for a lay audience, in terms of the level of detail and tone. Include a range of learning activities (e.g., practicals, problem-solving, or team-based learning) to make the PD as enjoyable and useful as possible. Meanwhile, remember that PD must stay relevant and remain focused on developing participants’ own competence. Augment PD with ongoing one-to-one coaching. Teachers and school leaders must receive high-quality continuous professional development to improve their practice and competence in their school context. Make the most of the time the teaching staff are offering, and ensure they feel they have had the best possible ​‘value’ for their investment of time.

7. Ask the evaluators to attend and present at training events

Ask a representative from the evaluation team to attend in a formal capacity, including presenting evaluation-related information to the teaching staff and stakeholders, and being on hand to answer any questions directly

The evaluation team should attend a sample of training events to assess the quality of delivery and observe the engagement of the teaching staff. This suggestion will also make it easier for participants to know who to contact with a given query. This would remedy Dawson and colleagues’ (2018, p. 299) observation that ​“When developers, evaluators and separate recruiters are involved in a project, there is potential for schools to become confused about who they need to work with and when”

Conclusion

When aiming for positive collaborative relationships with schools, key issues appear to be; recognising teaching staff’s role as co-experts, respecting their time, and being as approachable as possible. While some of these tips may seem trivial (e.g. personalising email greetings) any sensible action that increases engagement with, or fidelity to, the intervention is worthwhile in an RCT. The most effective approaches don’t just aim to get ​‘buy-in’ from stakeholders, they also include high-quality continuous professional development. Effective social and emotional learning interventions [3], for example, involve approaches that go well beyond classroom teaching and learning to promote positive school climate, ethos, and culture. Thus, it is crucial to keep the upskilling opportunities for teaching staff at the front of your mind when designing the intervention and evaluation.

We have thoroughly enjoyed the experience of working with teachers, school staff and parents through the course of RCTs, and have been impressed by the work we have seen and heard about in schools. Without fail, we have met inspirational people at all of our training events. We hope these recommendations will continue to be discussed and developed with the aim of best supporting teachers and researchers in their practice.