Babcock Research (formerly VT Research) is undertaking a 3 year impact evaluation of an intervention in schools (Improving STEM Careers Awareness – design and implementation of a pilot careers programme) led by Sheffield Hallam University. The evaluation measures the impact through a 3-year baseline/tracking evaluation. The evaluation includes teachers; parents/carers; and students. Babcock Research is designing/conducting the evaluation and disseminating the interim findings to the wider project team as a formative approach, informing the ongoing development of resources. We run research seminars; provide baseline and tracking results; write policy and literature reviews.

The Improving STEM Careers Awareness project encompasses piloting various strands of activity in ‘test-bed’ schools, including: a STEM curriculum-related careers resource; CPD for teachers/trainee teachers; IAG professionals’ support resources; an economic well-being resource; work placements; mentors and role model resources.

The pilot requires impact evaluation specifically to measure the impact/outcomes of the interventions upon direct beneficiaries (Yr9, Yr10 and Yr11 learners in the test-bed schools) and indirect beneficiaries (teachers, parents/carers). The evaluation will examine whether outcomes differ according to the primary project focus (activity strand) in each school.

Initial quantitative surveys provide a baseline from which to track STEM careers awareness over the course of the Programme (two further research phases). This approach enables detailed examination of the effect on beneficiaries as implementation embeds. This approach is supported by the willingness of schools to engage ENTIRE cohorts of learners (3 year groups) and all relevant staff (STEM teachers and careers practitioners) in evaluation. The test-bed schools were selected to be representative e.g. demography; socio-economic structure; school types (Academies; Technology Colleges; Faith Schools etc) giving our 100% sample robustness for recommendations. Secondary data analysis to provide ethical (and non-burdensome) control analysis. Literature, policy and STEM intervention reviews to inform evaluation design and analysis (e.g net impact).Our Project Manager is a member of the wider STEM project team as added value, playing an active role in project meetings and conferences. The wider team uses online approaches to facilitate communication using a project/team portal, to share information, post messages and upload key reports and other outputs.

Thus far, we have completed five project reports. A literature review into factors influencing STEM career choice and examination of other STEM interventions; three cohort-focussed quantitative baseline survey reports; an Executive Summary/Recommendations. The literature review drew upon a wide variety of sources of published research papers and reports. Three quantitative survey reports (student; parent/carer; teacher) combined frequency data with cross-tabulation by key groups. Statistical analyses were also performed on data with robust bases.