External evaluation of STEM Learning regional programme (2016-19)
1. Summary
STEM Learning Ltd. is seeking an evaluator for Stage 4 of the network of Science Learning Partnerships Sept 2016 – March 2019. The Programme is funded by the Department for Education to provide high quality science continuing professional development (CPD) for primary and secondary schools and FE colleges in England.
The evaluation is intended to establish how successful the programme, is at improving the quality of science teaching and driving forward school‐led science‐specific CPD in an effective and sustainable way particularly after the introduction of the new management and delivery elements in 2016.
2. Background
STEM Learning exists to achieve a world‐leading education for all young people in science, technology, engineering and mathematics (STEM), where every young person is taught by knowledgeable, enthusiastic teachers throughout their science education. We share the Department for Education (DfE)’s mission to improve the standard of science teaching in primary, secondary and further education. We have a proven track record that demonstrates that, via our networked and school‐led delivery of subject‐specific Continuing Professional Development (CPD) for teachers and technicians, we have a measurable and positive impact on the quality of science teaching and upon science learning outcomes for young people.
Central to achieving this vision is our established national network of 46 Science Learning Partnerships (SLPs), which bring together local expertise in teaching and learning in science, facilitate excellent CPD and provide bespoke school‐to‐school support. SLPs provide easily accessible, high quality, responsive and varied methods of support across the whole of England, ensuring that all teachers and technicians can easily access career‐long science‐specific CPD within a reasonable day’s travel.
Science learning partnerships are led mainly by local Teaching School Alliances, schools and colleges with excellence in science and also by higher education institutions and other local partners with cutting‐edge expertise in science. The network also complements the work of the National STEM Learning Centre based at the University of York and a growing provision of online CPD and educational resources for STEM subjects.
The current phase of the regional programme also includes an intensive two year programme of support targeted at those schools not currently offering Triple Science or those schools with persistently low uptake. This is a new element to the regional programme as previously (2012-16) support for the teaching of triple sciences was delivered within a separate programme run by STEM Learning. Each September from 2016 to 2019, SLPs will begin work with a new cohort of eligible schools.
The overall programme outcomes and measures including:
· an increase in reach in the numbers of teachers and support staff accessing support
· an increase in the reach across primary and secondary schools and post16 institutions including a mix of newly engaged and re-engaged schools/colleges
· an increase in the number of students being taught triple Science GCSE in supported schools – two years plus after the start of the project.
· an increase in proportion of pupils in supported schools entered for triple science (two years plus after the start of the project.)
· an increase in the uptake of STEM A levels including
o increase in the percentage of girls taking STEM A levels two plus years after the start of the project
o a percentage change in the number of girls progressing to STEM A Levels[1] in supported schools with at least 10 percentage points increase above the percentage change in students from state-funded, non-network supported schools
· the provision of effective, targeted support for schools and colleges where performance in regard to national priorities is not strong or CPD is not impacting on student outcomes
· further support of school-based SLPs in their development as autonomous and self-sustaining high-quality local centres of science-specific CPD and expertise in science teaching.
· improvements in pupil attainment
· evidence of the programme’s ability to complement and draw on provision from other programmes and services provided by STEM Learning and its partners (e.g. STEM Ambassadors, STEM Insight)
A list of evaluation KPIs can be found in Appendix 1
3. Purposes of the evaluation
The purposes of the evaluation are
· To establish how successful the programme is at facilitating the development and implementation of high quality school‐led science‐specific CPD, which meets national priorities and local need, and which enables eligible schools to use this CPD to improve the quality of science teaching in effective and sustainable way.
· To evaluate the success of the programme in delivering additional, focussed, support to priority schools including those not currently providing triple science or those with persistently low uptake.
· To provide further recommendations on how to improve management, delivery and impact of the programme particularly in the development of school-based SLPs as autonomous and self-sustaining high-quality local centres of science-specific CPD with expertise in science teaching.
4. Methodology
The evaluation will be expected to use data and method triangulation and combine qualitative and quantitative methodologies. We particularly welcome the inclusion of innovative methodologies to address the problem of declining rates of response to school and teacher evaluation surveys. School visits and case studies must form part of the methodology. We also expect the evaluator to use STEM Learning data which includes national pupil and school performance data and data on teacher and school uptake of our programmes.
The focus of the evaluation will be guided by the delivery targets and key performance indicators (KPIs) agreed with the Department for Education (Appendix 1). The evaluation should complement programme monitoring information as well as evaluation and impact data, which STEM Learning collects internally.
Your tender response should set out a detailed explanation of the methodology you propose to use for this evaluation; including evaluation models that you feel are particularly suitable for a programme of this kind. Proposals should indicate which aspects of the criteria in Appendix 1 will be covered by the planned methodology.
A previous evaluation report covering 2014 - 2016 phase of the programme can be found here
The final evaluation report for the Triple Science Support programme can be accessed here
5. Intellectual Property Rights
All intellectual property arising from any work will be vested in STEM Learning.
6. Deliverables and Audience:
The main deliverables need to be clear, succinct progress and milestone reports that set out the key findings and make recommendations for improving the programme.
The principal audiences for the reports will be STEM Learning’s management group and the national SLP network Steering Group. These groups will consider the reports so that lessons can be learned and implemented as work progresses. Reports will also be shared with the programme’s funders, the Department of Education.
7. Timeline:
Activity / MilestoneOpen, competitive ITT issued. Clarification questions responded to. / October 2016
Independent evaluator appointed and evaluation started / November 2016
Meetings with STEM Learning Programme team updating on the progress / Initially monthly until February 2017, then if appropriate, termly throughout contract lifetime
Interim reports from evaluator / March 2017, Sept 2017, March 2018 and Sept 2018
Final report / April 2019
8. Costing and pricing
Proposals should include a full budget plan for the whole costs including any anticipated travel, expenses and VAT. All costs should be clear and transparent, clarifying the number of days of each individual working on the activity that they provide. Decisions will be based on perceived value for money, rather than the lowest cost proposal. Proposals should not exceed £70,000 for the contract period (Oct 2016 – April 2019)
9. Tender content
Tenders should show, as a minimum:
· how the aims and actions of the evaluation will be addressed to ensure effective and ethical completion of the evaluation across England;
· the timeline for development of instruments, data collection, analysis and reporting;
· the roles, responsibilities and brief CVs of personnel in the research team;
· a breakdown of costs by staff and non-staff categories, office costs, travel and subsistence costs.
· risk assessment and mitigating actions
· contact details of two referees
Where possible links to recently published reports should be included
8. Submission and format of proposals
The deadline for receipt of submissions is 12.00 noon on 3rd November 2016. Late submissions will not be accepted. All submissions will be acknowledged with a notification of receipt.
An electronic copy of the proposal should be received in PDF format by this deadline. This is an electronic only submission process; therefore all documentation must be submitted in PDF format as a single zipped folder if the size of the submission is greater than 10Mb.
Proposals should be no longer than 10 sides of A4 (excluding CVs of people involved) and typed in Arial size 12 font. If the proposal exceeds this limit evaluators will be advised to disregard any information outside the limit.
Proposals with the subject line “STEM Learning Evaluation Submission” should be emailed to:
STEM Learning reserves the right not to award a contract.
A decision will be made as soon as possible after receiving the tenders, with a face-to-face clarification meeting if needed.
9. References
Tenderers should indicate the names of two current or recent customers for whom similar evaluations have been carried out and who would be prepared to act as referees.
11. Selection criteria
Proposals will be evaluated against the following criteria which are not listed in order of importance:
· compliance with, and understanding of, requirements outlined in this tender document
· understanding of the educational landscape, STEM education and the background and context of the STEM Learning Network and it’s relationship to the National STEM learning Centre and other STEM learning programmes
· effectiveness of the proposed methodology for undertaking the evaluation
· clarity and feasibility of the programme of work, work plans and risk assessment
· relevant experience and expertise of a programme team which I allocated to this project long term
· ability to meet the required timetable
· management of any existing or potential conflict of interest identified by the bidder
· value for money.
12. Enquiries
These instructions are designed to ensure that all tenders are given equal and fair consideration. It is important therefore that you provide all the information asked for in the format and order specified.
Please contact Samantha Martin if you need any further clarification in providing the information requested. Pre-tender negotiations are not allowed. If you have any questions relating to the requirements then please contact Samantha Martin as above. We will aim to respond to any queries within 5 working days. If we feel that the query relates to an issue which all prospective bidders should be made aware we reserve the right to communicate our response to all those who have requested tender documentation.
If you consider any point you raise to be confidential to your own proposal please make this clear. Please note the last date for the submission of any queries is noon 27th October 2016
Appendix 1
01 April 2018 – 31 March 2019 Evaluation KPIsIncrease in the number of students being taught Triple Science GCSE in supported schools - two years plus after the start of the project.
Increase in proportion of pupils in supported schools entered for triple science (two plus years after the start of the project.)
Primary: Quality of teaching, measured by self-evaluation of enhanced subject and pedagogical knowledge and understanding; and greater enthusiasm and confidence in the classroom
Secondary: Quality of teaching, measured by self-evaluation of enhanced subject and pedagogical knowledge and understanding; and greater enthusiasm and confidence in the classroom
Improvements in pupil attainment measured by teacher assessment of pupil outcomes
Uptake of STEM A levels percentage of girls two plus years after the start of the project. Percentage change in the number of girls progressing to STEM A Levels in supported schools. 10 percentage points above the percentage change in students from state-funded, non-network supported schools.
(STEM A levels: maths, further maths, biology, chemistry, physics and computing)
Increase in percentage of school led SLPs
[1] STEM A levels: maths, further maths, biology, chemistry, physics and computing