The Training and Development Agency for Schools (TDA) contracts specialist careers advisers to work from home, providing tailored advice to potential teacher trainees. Mystery client evaluation was required to monitor adherence to processes and potential client outcomes, during 2005/6 and 2008

The TDA required mystery client research to contribute to the overall evaluation of their RCA programme: to monitor the quality of information, advice and guidance provided by contractors (RCAs); to ensure that the RCA programme is value for money and to recommend changes to the national RCA programme. Additionally TDA wanted to evaluate consistency of service by method of contact and demographic profile/preferred teaching subject.

The client briefs for both projects (2005/6 and 2008) prescribed the methodology and sample sizes. As added value, we created a specialist mystery shopping team of careers advisers to provide peer-review of RCAs. For the second project, teachers joined the mystery shopping team, enhancing client-perspective. Professional researchers provided constant methodological guidance.

Our research process fully replicated real client interactions from our detailed analysis of the client’s management information. All aspects of the process were evaluated, including registration with the TDA for information and referral to an RCA (part of a wider corporate information process), and appointment setting and interactions with an RCA. This provided the opportunity to assess how the RCA initiative was working and for which types of enquirer the current processes were most/least effective.

Quantitative results were recorded alongside qualitative assessments and comments.

Formative feedback and interim reporting were used by TDA for early -identification of any underperformance and/or mis-interpretation of operational guidance.

Final reporting from the first project provided individual performance information which contributed to re-contracting decisions for individual RCAs; prompted team-wide personal development days on electronic communication techniques (due to identification of poor process adherence and low client satisfaction with email exchanges); and re-drafting of contracts and guidance for RCAs. After each project a ‘round-table’ senior management feedback session explored the implications of the research findings for the operational and strategic future of the programme, focussing on whether the programme impact on potential teaching trainees justified its not insignificant cost.

Additionally, a presentation of anonymised findings and recommendations on process and communication styles was made to the team of RCAs. This session provided insight into client perceptions, highlighting that clients were unaware of the prescribed processes and parameters that RCAs work within. It also allowed for the sharing of strengths/good practice between individual contractors (who mainly work independently).