The Education Endowment Foundation (EEF) funds high-potential interventions that aim to improve attainment and related outcomes for disadvantaged children and young people between the ages of 2 and 19. The majority of EEF projects are conducted in education settings, particularly in schools, but an increasing number occur in early years and 16 – 19 providers. Each project is evaluated by a team of independent evaluators, allowing us to build our understanding of what works in improving outcomes for children and young people. The EEF funds a range of evaluations with a focus on pilots, Randomised Controlled Trials (RCTs) and quasi-experimental designs to understand the impact of interventions alongside an implementation and process evaluation to understand the elements of successful delivery. Each project that is funded by the EEF is evaluated independently by a team selected from the EEF’s panel of evaluators following a process of open competition. All our reports are published on our website irrespective of results. For trials, the security of the findings is rated according to prespecifiedcriteria, to help our main audience of school leaders and policy makers interpret the results.
Since its launch in 2011, the EEF has spent over £100 million on evaluations that have reached over13,000 schools and settings. Many EEF evaluations use data from the National Pupil Database to estimate the impact of interventions. Where this is not possible, collecting outcome data from education settings poses significant risks to our evaluations, as attrition, missing data, poor validity and/or reliability can have a significant impact on the security of the findings. In order to help independent evaluators manage these risks, we have invited expressions of interest from organisations that wish to facilitate data collection on EEF evaluations and can demonstrate:
- prior experience in collecting high-quality data from children and schools, early years settings, and post-16 providers (collectively, setting), particularly administering attainment tests,
- prior experience undertaking the following activities: recruitment of appropriate surveyors/data collectors and supervisors, providing training, capturing data/data entry, quality assurance processes and data cleaning
- capacity to collect data from large numbers of settings and children over relatively short time periods, with a high response rate,
- ability to work closely and collaboratively with evaluators and delivery teams, and
- a transparent costing model that provides value for money.
These offers received from selected interested organisations are reproduced on the EEF website here under the following terms and conditions.
1. Members of the EEF data collection panel:
a. are responsible for the accuracy of the information shared on the EEF website. The EEF will only publish information it has received from members of the panel, with appropriate permissions. It is the responsibility of each organisation to ensure the information is accurate and up-to-date, including but not limited to services provided, costs, staffing and contact details. Organisations are responsible for ensuring the information remains accurate and up to date, and for notifying us when any changes are needed. The EEF will also request updates on a regular basis to help maintain accuracy.
b. will be informative and transparent about their expertise. To facilitate informed decision making, we encourage members of the data collection panel to specify the areas in which they have expertise and the areas in which they would be interested in developing expertise. This may include participant ages, educational stages, academic subjects, assessment types and data collection methods. Evaluators are responsible for identifying data collection instruments for each evaluation, so members of the data collection panel need to be willing to use data collection instruments proposed by the evaluator they are working with. Members of the data collection panel should be transparent with evaluator organisations about their expertise to administer the proposed data collection instrument, including whether the panel member is able to provide suitably qualified staff, and whether staff will need training on the data collection instrument to be provided by the evaluator or whether this can be delivered in house.
c. will be informative and transparent about costs. Information published on EEF’s website should provide sufficient detail to enable evaluators to make informed decisions about which members of the data collection panel to approach for a particular evaluation. Information about any potential discounts should be provided, as well as whether the costs are inclusive of VAT. Particularly helpful would be day rates and unit costs that can be easily translated to different evaluation contexts, based on specific assumptions – for example, cost per setting of delivering an assessment and how much that would vary based on:
- assessment length (e.g., 30/ 45/ 60 minutes)
- number of settings (e.g., 50/ 100/ 200)
- type of setting (e.g., early years setting/primary/secondary/16 – 19 provider)
- mode of delivery (e.g. online / paper/1:1 administration)
- administrator level of expertise
Evaluators are expected to have identified subcontractors to support with data collection when they submit their proposals to the EEF. Evaluators may therefore approach members of the data collection panel at bidding stage to ask them to partner on their proposals and will work with panel members to develop an accurate budget for the proposal stage. Members of the data collection panel should be transparent about the assumptions behind their budgets and ensure they obtain the necessary information from evaluators to enable them to budget accurately. If awarded a project, panel members should be prepared to revise their budgets during the project set up period. During this period, meetings are held between the delivery team, the evaluator organisation and the EEF to discuss and agree the evaluation design. Often, some aspects of the evaluation design may change during this period such as the number of settings, the number of children assessed per setting, or the regions that are targeted. Panel members will need to revise their budgets based on this new information. Because data collection costs are the key cost driver of EEF’s evaluations, the EEF may request detailed breakdowns of data collection budgets and assumptions so data collection panel members need to be prepared to provide these.
d. will be transparent about any further sub-contracting involved. Panel members will specify which of their services will be provided by in-house staff and which will be offered via additional third parties (e.g., data collection, data entry, transcription, marking). Sufficient detail should be provided to enable evaluators to make informed decisions and cost their proposals accordingly.
2. Members of the EEF evaluation panel:
a. are responsible for recommending valid outcome measures suitable for the evaluation. The EEF expects its independent evaluators to make their own informed decisions about the most suitable outcome measures and data collection procedures, in line with the EEF mission, recommended best practice for evaluation, delivery model or constraints and any co-funder conditions. These decisions will be made in collaboration with the EEF and where applicable, the intervention delivery team, with the independent evaluator being ultimately responsible for the decision. If feasible and appropriate, evaluators are encouraged to use measures from our Attainment Measures,early years and essentialskills SPECTRUM databases, developed on the basis of systematic reviews we have commissioned. In preparing their initial proposal, evaluators may draw on information about the data collection panel published on the EEF website to identify potential subcontractors. Evaluators should name subcontractors on their proposals and provide detailed, accurate budgets for data collection activities, including outlining the assumptions. Evaluators should ensure they provide members of the data collection panel with detailed information to enable them to prepare accurate budgets. Evaluators should work closely with their subcontractors during the project set up phase to support their subcontractors to update their budgets accurately.
b. are responsible for sub-contracting third parties for data collection. Members of the data collection panel will be identified on EEF’s website with details of the data collection services they can provide. If they wish to, evaluators will approach members of the data collection panel directly to make preliminary arrangements at the proposal writing stage. If successful in their application to the EEF, evaluators will make their own contractual arrangements with the members of the data collection panel. Data collection costs will be included in the evaluation budget specified in a grant agreement between the EEF and the independent evaluator. The EEF will not have any contractual or quality assurance obligations in relation to members of the data collection panel. EEF evaluators are not obliged to use an external organisation for testing or data collection and, if they choose to, are not obliged to choose one from the EEF data collection panel. However, evaluators will be expected to justify their choice of testing procedures and administrators, and ensure these provide good value for money.
c. are responsible for quality assurance. Independent evaluators will make their own informed decisions about suitable testing procedures and providers, ensuring they are satisfied of the level of quality offered. Evaluators are responsible for ensuring data collection instruments and procedures are valid and reliable, and that all measures are taken to quality assure data collection and prevent attrition and missing data.
d. will ensure data collection for EEF evaluations complies with current data protection and safeguarding legislation. This includes ensuring any third parties contributing to testing and data collection have appropriate data protection and safeguarding provisions in place that are in line with the terms and conditions of the EEF grant agreement.
3. The EEF:
- publishes information shared by the data collection panel for information only;
- takes this information at face value and does not verify its accuracy;
- does not endorse the services or products offered by the data collection panel;
- endeavours to amend the offers of the data collection panel within seven working days of being notified in writing;
- reserves the right not to publish on its website any information considered unsuitable;
- is not responsible for the recruitment, working practices, or conditions of test administrators engaged by evaluators or their subcontractors, although the EEF expects evaluators to ensure test administrators are engaged in a fair and responsible manner.