Greater Manchester Early Years Shared Outcomes Framework
2August 2013
TO: Early Years Steering Group
FROM: Early Years Data Group
CONTACT OFFICER:David Morris, New Economy
1INTRODUCTION
Greater Manchester is in the process of moving towards a new way of delivering services and support for all children aged 0-5. Access to robust and up-to-date data and evidence will be crucial to the success of this new delivery model (NDM).
Since the Early Years (EY) City Region pilots of 2010, repeated efforts have been made to reach a GM-level agreement on how we measure the impact of our EY work. In 2012, as part of the Early Years business case development process, 30 different pieces of data and information (or KPIs) were identified as potentially being relevant to the NDM. These were placed within a draft shared outcomes framework.[1]The Early Years Data Group was established to refine and finalise this initial work. The group includes representatives from all 10 GM localities (both policy and performance and Children’s Services staff) and the NHS.[2]
The group’s ambition is two-fold:
- For GM to be able to robustly, regularly and usefully demonstrate at GM and locality level the impact that the NDM is having upon children, families, the services that they access and the funding that is called upon to deliver these services. This demonstration would come in the form of regular reports, showing GM and where possible locality/cohort scores for the agreed outcomes and the direction of travel in that outcome over the quarter and over the previous 12 months; and
- For GM to have a system or systems in place that enable the proportionate gathering, analysing and sharing of child-level data between officers and agencies so that children and families receive the support they need to progress. If successfully realised, this could mean officers having access to charts and graphs plotting the progress of an individual child along the 8 stages of the NDM and giving them the ability to drilldown to understand the reasons behind changes in progress and the referrals which were made in responses to such changes. On the following page we illustrate what such an output could look like:
1
Figure 1: Illustration of possible output from NDM individual child tracking system
1
In light of this ambition, we are working towards two objectives:
- To identify the KPIs which will need to be reported under a shared outcomes framework in order for GM to have:
- A population-level picture of the impact of the Early Years NDM;
- A functional Early Years investment agreement; and
- The child-level information needed for specific parts of the NDM to function.
- To make recommendations on how to overcome the technical (e.g. hardware and software), cultural (e.g. resourcing and staff training) and legal (e.g. data sharing legislation) barriers that currently block the sharing of these KPIs.
This report details our progress to date against these objectives.
2POPULATION-LEVEL KPIsfor inclusion in the shared outcomes framework
The 30 KPIs proposed in 2012 were all population-level indicators, i.e. they give a snapshot of conditions within a particular cohort but do not explain the factors that have given rise to these conditions. Each KPI has been objectively and robustly assessed using a three-stage methodology[3]:
- At stage 1, all group members scored each of the 30 KPIs based upon their performance against 8 criteria. This provided initial opinions on which of the draft KPIs seemed well-defined, unique, and relevant to the NDM;
- For stage 2, group members with detailed knowledge of a specific KPI were asked to undertake further research on this KPI and to rate the KPI in terms of its relevance and data availability. The findings from this stage were combined with those from stage 1 to identify 10 KPIs that the group felt should be proposed for the final shared outcomes framework; and
- Stage 3 involved a GM-wide online consultation, to which 77 responses were received, offering views on the 10 KPIs identified at stage 2 and giving respondents the opportunity to suggest alternative KPIs to be included in the final outcomes framework. Feedback from the consultation was considered by the group, the final shared outcomes framework altered accordingly, and presented in this report.
At each stage we have applied the principles that the final shared outcomes framework should be brief, should cover the varied factors that enable a child to be ready to learn, and should gather data that is of use to families and public agencies alike.
The shared outcomes framework proposed below includes both KPIs indicating a need for NDM intervention and KPIs showing the impact of the NDM’s work. We have not separated the two categories because several of the proposed KPIs sit in both categories or perform different roles for different agencies – for example, a child welfare KPI could indicate that a family needs additional support or that the NDM is better at intervening to take a child out of harm. For each proposed KPI we give a short case for its inclusion. Later on in the section we discuss those KPIs which were considered for inclusion but ultimately rejected.
2.1Population-level KPIs for which regular collection at the GM and/or locality level is already undertaken or will be easy to achieve
2.1.1Children achieving a ‘Good level of development’ at the end of the EYFS
The percentage of children defined as having reached a ‘good level of development’ at the end of the EYFS is the headline measure of the NDM’s success. This is a universal and statutory measure and data is already collected and reported by all localities; collation of these reports into a single GM figure would not be a major undertaking. The Data Group is aware that there is currently a national consultation on changes to pupil assessments in primary school that may have an impact on the statutory requirement to collect data and is advising on the GM response to the consultation.
2.1.2Breastfeeding rates at 6-8 weeks
We found lots of robust evidence of the link between breastfeeding and positive outcomes for children and mothers andadvice in relation to breastfeeding will be central to the initial stages of the NDM. We feel that it makes more sense to record sustained breastfeeding rates than breastfeeding initiation rates given thata third of mothers stop breastfeeding soon after birth.
The percentage of infants being totally or partially breastfed at 6-8 weeks is a piece of data that is supposed to be submitted to DOH on a quarterly basis. However, despite being a longstanding public health indicator, reporting is not robust across GM. The group proposes that NHS England be asked to work with providers to ensure reporting is improved.
2.1.3Hospital emergency admissions caused by unintentional and deliberate injuries in children
The group views this KPI as an important indicator of child health and potentially of where further NDM intervention is required. By helping to improve parenting skills, by identifying vulnerable children earlier, and by better linking up health and children’s services, the NDM should generate a reduction in hospital emergency admissions.
There was strong support from consultees for an indicator relating to the hospitalisation of young children. Public Health England has confirmed that the indicator will be included in the new Public Health Outcomes Framework, with clearer guidance to medical staff on how to code admissions. One minor challenge will relate to identifying 0-4 year olds within the broader dataset; this is something that GM is already addressing with CCGs via the investment agreement modelling work.
2.1.4Child welfare
Those children in GM who are referred to social care services represent our most vulnerable cohort and there is evidence showing a correlation between child welfare concerns and subsequent poor educational attainment. The NDM could influence this metric in either direction: helping to improve standards of parenting should see this KPI fall, whilst increased assessments and joint-working between agencies could see this KPI rise.
In terms of collecting the KPI, the Children in Need census is an annual submission which records all children who are referred to children’s social care services; this cohort includes those who are in care, those who are supported in their family and those subject to a child protection plan. A sub-group will be formed to define whether this KPI covers all three categories of children or a subset. Once the definition is confirmed, calculating a single GM number can begin.
2.1.5Domestic abuse
Exposure to domestic abuse is known to have significant impact upon child outcomes and is often cited in serious case reviews. The NDM will result in increased communication between midwifery and other EY services, which should result in increased awareness of those families where we have a domestic abuse concern.
Incidents of domestic abuse are universally collected by midwifery from all women at multiple points in the antenatal/neonatal pathway. There was strong support for a KPI related to this topic but concern over the logistics of collecting the data and requests for further detail on the support pathway that would sit behind the identification of an incident of domestic abuse. The Data Group will work with the recently formed Domestic Abuse group to address both issues.
2.1.6Childcare offer
A more integrated EY system for GM will be better able to plan childcare provision, link the funding of this provision to improvements in quality, and help parents to take-up childcare opportunities.
Statutory childcare sufficiency assessments are available for all GM localities but are more useful from a service provider’s rather than a family’s perspective. The Data Group therefore proposes working with the day care group to develop a hybrid KPI that shows the rate of uptake of early education places within Ofsted-rated good or outstanding childcare settings. This new KPI would tell us about both the quality of childcare in GM and the take-up of quality childcare.
Recommendation – the Steering Group is asked to agree that the 6 population-level KPIs listed above be included in the shared outcomes framework and that work begins to establish a baseline position for each KPI
2.2 Population-level KPIs for which regular collection at the GM and/or locality level will require further work
2.2.1Children living in poverty
There is a clear link between childhood deprivation and educational attainment, with the impact of deprivation upon a child being seen in the first few years of life. Having data on the numbers of children living in poverty will help GM to develop and target capacity within the NDM; however we would not expect the activity of the NDM to have a major direct influence on childhood deprivation given the importance of factors such as rates of economic growth and welfare policies upon this outcome. These additional factors are out of GM’s control.
The old national indicator definition (the percentage of children living in families in receipt of out of work benefits or in receipt of tax credits where their reported income is less than 60% of median income) is felt to have its problems. In its place proxies such as Housing Benefit or Council Tax Benefit data have been proposed to us along with a new bespoke measure such as the numbers of households where a parent is in work. A sub-group will be formed to look at the pros and cons of these alternatives and to recommend a KPI(s) and the steps by which a GM metric(s) is/are to be created.The sub-group will work with the Worklessness and Skills PSR theme in undertaking this research.
2.2.2 Secure attachment
Secure attachment not only stimulates brain growth but affects personality development and lifelong ability to form stable relationships. Improving GM’s parent-child attachment scores should lead to longer term savings from higher end reactive services. The NDM, through its support for breastfeeding and use of universal and targeted parenting interventions should help to improve parent-child attachment.
The group and consultees feel that GM should collect an indicator relating to the quality of parent-child relationships but there is no widely used or understood way of doing this at present. A sub-group will quickly review the pros and cons of different measurement tools. If a suitable tool is not identified, it may be possible to build up a GM-level indicator of secure attachment via the individual child level assessment activity (e.g. from interrogation of responses to Ages and Stages questionnaires).
2.2.3Home Learning Environment
If we know those children at risk due of lower attainment due to growing up in a poor home learning environment well before they start school the NDM can help to make a great difference. A strong home learning environment has been found to be one of the strongest influencers of later attainment – stronger even than family income or socio-economic class.
The group and consultees feel that GM should collect an indicator relating to the quality of home learning environments. The limitations of the existing EPPE Home Learning Index have been pointed out to us. Data Group members will support officers in the 8 stage assessment group who are looking at how a GM HLI could be created for earlier and more widespread use.
Recommendation – the Steering Group is asked to agree that the 3 population-level KPIs listed above be included in the shared outcomes framework subject to
GM-suitable tools and approaches being identified
2.3. Population-level KPIs which were considered for inclusion in the shared outcomes framework but ultimately excluded
As noted, the group inherited 30 potential population-level KPIs from the 2012 work. Some of these, such as ‘levels of parental self-efficacy’ and ‘self-reported measures of parental well-being’, were rejected due to a lack of available data and/or robust measurement technique. Others such as ‘numbers of lone parents’ and ‘family over-crowding’ were discounted because we found no/little evidence of how these linked to educational attainment. In a few instances we decided that combining KPIs into a composite would be best (e.g. the proposed childcare offer KPI combines the existing ‘childcare sufficiency’ and ‘childcare entitlement’ take-up measures).
Discussion was had regarding the large number of child health and development KPIs originally proposed. For some such as the ‘rate of smoking at time of delivery’ we found no evidence that this linked to school readiness. For others such as ‘population vaccination coverage’, ‘obesity levels at reception year’ and ‘prevalence of tooth decay in 5 years olds’ the group felt that high scores were not necessarily an indication of parental neglect or that they posed a serious risk to school readiness when compared to the 9 indicators proposed in the previous section. This is not to say that Public Health colleagues will not find wider value in continuing to compile such datasets.
During the consultation there was support for the addition of KPIs relating to child speech and language skills and maternal mental health. The former issue will be picked up via individual child Ages and Stages checks and through referrals to additional support services (e.g. Every Child a Talker); the latter is partly covered in the secure attachment measure proposed above but will also be collected via the standard midwife checks at stages 1 and 2 of the NDM. Thus, for both it will be possible to produce a GM-level indicator if so needed at a later date.
2.4.Mapping the shared outcomes framework back to the EY business plan
In the 2012 EY business case, three pre-requisites to children being ready to learn were identified. These were:
- Improved family health and well-being;
- Improved family economic well-being; and
- Improved parenting skills.
The Data Group has mapped the proposed population-level KPIs to confirm that the shared outcomes framework will cover all three pre-requisites. This mapping is shown below:
Figure 2: Aligning the proposed KPIs to the EY NDM objectives
The proposed population-level KPIs will provide GM with a balance of information across the three NDM sub-objectives. Family health and wellbeing measures (which dominated the draft framework developed in 2012) will remain central to the shared outcomes framework but will be complemented by three new indicators of parenting skills and childcare quality. The work of the sub-group to identify a children living in poverty KPI(s) will be crucial in providing information relevant to the improved economic wellbeing sub-objective.
3Identifying INVESTMENT AGREEMENT KPIsfor inclusion in the shared outcomes framework
GM officers, supported by Social Finance, are in the process of identifying which EY outcomes will be tracked in order to trigger payments into or draw-downs from a GM-wide EY investment agreement. This will involve identifying those EY outcomes for which GM has a clear understanding of the costs they entail for agencies plus discussions with budget holders to understand how changes in KPIs will influence their future investment decisions.
The work is due to be completed in September/October. Once it has completed, we will work with the investment agreement team to ensure that those KPIs included in the final investment agreement are also included in the shared outcomes framework.
4Identifying child-level DATA for inclusion in the shared outcomes framework
Collecting the suite of KPIs listed above will give a regular indication of improvement, and the financial impact of that improvement, to be considered against our starting point of 40% of GM children being judged as not school ready; however, these KPIs will not in themselves enable measurement of the success of individual interventions.
Each GM locality is developing plans to adopt the New Delivery Model or aspects of it. A child-level dataset is required as a matter of urgency for use in localities that are preparing to roll out the full model in a ward and work is underway to draft this. In addition to this plans are being progressed to outline pathways from universal assessments into targeted assessments, interventions and specialist services. The Data Group will work with the localities and the pathway groups to seek answers to questions such as what data and information is (or is not) currently collected by each specific assessment tool or intervention, who collects this information, in what format, how that data is stored, and legal issues relating to information sharing.This dataset will include information that relates to the monetisable outcomes within the PSR programme.