MEASUREMENT FRAMEWORK FOR TESTING THE 8 PILLAR MODEL – updated July 2014, following LEARNING SESSION 2 on 26 June 2014
This paper sets out the draft measurement framework for testing a range of approaches to providing better integrated care and support, based on the ‘8 pillars’ model of community support (see commitment 3 of the dementia strategy). The paper builds on an initial measurement discussion paper, a webinar on 11th March 2014, Learning Session 2 on 26 June and subsequent comments from 8 pillar test sites. The vision for this work, the aim, guidance on baseline data and proposed measures are provided. The proposed data collection tool is attached as a separate excel document (appendix 1: 8 Pillar Data Reporting Template).
Vision: person-centred, safe and co-ordinated care for people with dementia and their carersAim: To improve the care and experience of people with dementia and their carers by January 2016, through testing and evaluating a range of approaches to providing better integrated care and support using 8 pillars model
Our theory for testing: having a Dementia Practice Co-ordinator and 8 pillar model, will enable better person centred co-ordinated and safe care, leading to improved experience for the person with dementia, their carers and staff.
Why measure?
Measurement will be vital if we are to demonstrate that we are making a difference to people’s experience and care and will support us with making future recommendations about the 8 pillar approach to inform the next Dementia Strategy in 2016. Measurement will tell us:
§ how well the process is performing
§ whether we have reached our aim
§ how much variation is in our data/process
§ the impact of small changes
§ whether the changes we have made have resulted in improvement, and
§ whether changes have been sustained.
Baseline Data
In order to show improvement over time, it will be essential for the test sites to gather baseline data. This will provide valuable information on the current pathway and experience of people with dementia, carers and staff. There was agreement on the webex on 11th March that sites should track 20 pathways (20 individuals with dementia), at the beginning, middle and end of the project. This would allow the project to show complexities in the system, including information sharing, crisis points, cost of service provision and interactions. This could be achieved through case note review of people who have been allocated a dementia practice co-ordinator compared to a random sample of case notes for people who are not part of the test site work but who have similar needs (this could be a retrospective sample before the DPC role was introduced) . This should be complemented with interviews with services users, carers and staff to gather experience of the current and inform ideas for improvement. There are lots of things we could measure in this work so we have to choose the vital few measures that will demonstrate improvement over time. Test sites will have an opportunity to share their baseline measurement on a planned webex on 14 August and as part of the flash report update at Learning Session 3 (17 September 2014).
National Measures
In Scotland we collect national benchmarking data and SMOR1 and SMRO4 data, which will be relevant to this work, including:
· Rates of unplanned hospital admissions for people with dementia to both acute and psychiatric facilities· Rates of unplanned hospital admissions for people with dementia by source (eg home/care home)
· Destination on discharge for individuals admitted to acute hospital with dementia
· Length of stay for people with dementia (psychiatric unit and acute)
Due to the small numbers of people who will be allocated a dementia practice co-ordinator as part of testing the 8 pillars, this may not have a significant impact on the overall benchmarking data. There will also be other factors which will impact upon these outcomes including post diagnostic support HEAT target improvement activity. As such, we require to introduce some specific measures for the 8 pillar testing in order to demonstrate specific improvements.
Measurement specific to 8 Pillar Improvement
OUTCOME MEASURES:Improved Experience
· people with dementia
· carers
· Staff
‘What matters to me’ – what is important to the person with dementia is central to the personalised plan
Carers have a right to receive an assessment of their needs (NICE guidelines)
Staff have necessary skills and support (Promoting Excellence)
People with dementia are living in their place of choice
Carers feel supported / Improved Safety
· Reduced emergency call-outs
· Reduced A&E attendance
· Reduced emergency admissions
People with dementia feel safe / Improved Co-ordination
· People with dementia and their carer have access to the right support at the right time
(as per personalised outcome plan)
The person with dementia’s needs are responded to
PROCESS MEASURES:
· Person with dementia has a personalised outcome plan
· Carer of the person with dementia has a personalised outcome plan
STRUCTURAL MEASURES:
· Hours of input from a variety of professions/services and costs
· Understanding the range of roles, grades and sectors of different dementia practice co-ordinators
BALANCING MEASURES: Reduction in falls. Care giver stress. Other balancing measures may emerge as testing develops.
Sample size
Test sites will identify people with dementia to be allocated a Dementia Practice Co-ordinator and 8 pillar model, in line with Alzheimer Scotland 8 pillar model and dementia practice co-ordinator guidance. The sample size should account for sample attrition (eg through death), ie the number of people selected for the initial sample should be proportionately bigger than the anticipated sample size at the end of the improvement programme. By the end of the programme there should be a minimum of 20 cases per test site.
Methods for Gathering and Reporting Data to Support Outcomes
8 Pillar Data Recording Template
A data recording template has been developed (see appendix 1: 8 Pillar Data Reporting Template) for reporting quantitative data. Test sites will begin to gather data using the template from 1st September 2014 and will submit this every 3 months to Scottish Government. Information to populate this template will be gathered through case note review.
The data reporting template will enable us to capture the following information:
o Stage of illness when a DPC has been allocated
o Wait time from identification of need to the first contact with the DPC
o Whether the person with dementia has a personalised outcome plan
o Whether the carer has a personalised outcome plan
o Number of crises – number of A&E attendances, number of emergency admissions, total days spent in hospital, number of GP contacts. We appreciate that people with dementia may have a multiple co-morbidities which will necessitate multiple GP appointments and increased admissions and length of stay in hospital. As such, these numbers may not decrease with the input from a dementia practice co-ordinator and 8 pillar model, and so qualitative information will provide a fuller picture on how crises have been averted and/or managed.
o Which pillars are being accessed – this should be relevant to the person’s needs/personalised outcome plan and as such we are interested in whether there are pillars which are more frequently accessed, ie we are not advocating that everyone has to have all pillars. Information to understand how many steps people go through to get support, or how many people they have to contact, with/without a DPC would also be helpful information to collect locally to supplement the data recording template.
Qualitative Data
In addition to the data recording template, it will be necessary to capture qualitative data to demonstrate improved experience for people with dementia, their carers and staff, and to show improved safety and co-ordination. A workshop is being arranged for 18 September 2014 on ‘Managing Qualitative Data’ to support 8 pillar test site representatives to build on their knowledge and experience of managing qualitative data, to share their current approaches and to support the management of this data through testing the 8 pillar model. Test sites will have the opportunity to continue to share their qualitative results at each learning session and through case study and pathway discussion.
Outcomes / Qualitative MethodsImproved experience / Personal outcomes tool (eg Talking Points)
Carer satisfaction through existing available tools
Carer Journey Tool (North Lanarkshire)
Interviews with carers
Carer survey
Focus group with staff
Staff questionnaire on staff experience and description of structure around the DPC training, supervision and reflection
Improved safety / Case note review
Interviews with patients, families, carers and staff to understand how crises were averted and/or managed
Risk assessment tools – understanding levels of risk acceptable to the person and their carer
Stories and case studies to demonstrate indicators of partnership working arrangements, processes and shared responsibility
Improved co-ordination / Working with the individual cases, how multiagency interventions were accessed, the experience of working across a range of professions and services, and impact of those interventions, understand how crises were averted and/or managed.
Stories of partnership working arrangements, processes, mandates and credentials and shared responsibility.
Case note review to understand how many steps people go through to get support, or how many people they have to contact, with/without a Dementia Practice Co-ordinator.
Activity Tracker Tool
It is recommended that test sites complete activity tracking for four weeks every 6 months (three times over the course of the project) to provide activity of the Dementia Practice Co-ordinator across the other 7 pillars and the impact on their existing role at key points in the project.
Scottish Government currently have a mental health activity tracker tool which can be revised for 8 pillars. This tool offers a platform with which staff information on allocated time can be recorded. It would be possible to add respective staff pay band and travel time/costs. (Nils Michael and David Scott currently looking to revise this with a view to testing this with one DPC before rolling this out to the other test sites. Further information will be available on the webex on 14 August.)
Quality of Life Outcomes
There is an opportunity through this work to explore the measurement of quality of life outcomes. For example EQ-5D health questionnaire which provides a simple descriptive profile and a single index value of health status, applicable to a wide range of health conditions and treatments. Changes in index outcomes can be used to construct measures of Quality Adjusted Life Years (QALYs). This is being further explored by economic colleagues within Scottish Government and we can provide further information on this at the webex on 14 August.
Costs and benefits
For the purposes of evaluation, it will be important to capture costs and benefits of this work. The activity tracker will provide useful information in this regard.
Other costs which could be monitored include:
· A training needs analysis for staff involved with 8 pillars and an estimate of training costs
· Start-up/infrastructure costs of implementing the 8 pillars model
· Equipment costs where appropriate
· An estimate of associated transport costs (this could be incorporated in the activity tracker tool)
Benefits (cost reduction/cost avoidance)
· Emergency admissions, A&E attendance, total days in hospital, and care home admissions are being recorded in the data recording template. This information can be used to draw conclusions about potential avoided costs from the differential between control and intervention groups.
· The number of GP visits is also being recorded in the data recording template. If possible, it should be distinguished between necessary GP visits and preventable visits. It is also noted that people may have multiple comorbidities which necessitate frequent GP attendance which is unavoidable.
Next steps
· Measurement Webex on 14 August 2014 to report on usability of data collection tool, share activity tracker tool, quality of life outcomes and share any baseline data gathered by the test sites
· Learning Session 3 – 17 September 2014, test sites will provide flash report update including any baseline data gathered
· Managing Qualitative Data workshop – 18 September 2014
Please contact or one of the NDCIP team including your JIT contact (Douglas, Eileen or David) for further information or to discuss.
File Name: 20140723 Measurement Framework following Learning Session 2 of 8 Pillars 26 June 2014 / Version: v1 / Date: 23 July 2014Produced by: Michelle Miller, 8 pillar national team and 8 pillar test sites, Nils Michaels & David Scott / Page: 1 of 5 / Review Date: 14 Aug 2014