Educating for Patient Safety

Educating for Patient Safety

Lead Trainers: Mark Johnston & Emma Levy

Evaluator: Mudenda Munkombwe (Revisions: Mark Johnston/Emma Levy)

Introduction - Since 2009 NHS Education for Scotland has delivered bespoke training delivery support to NHS territorial boards to support their efforts in the education of staff primarily involved in the delivery of the Scottish Patient Safety Programme and their efforts to make care safer.NHS Borders asked for our support in the delivery of a workshop to take place in November 2012. The key aim communicated to NES was a desire to re-energise patient safety in NHS Borders. The workshop was designed in consultation with the host board around core themes of patient safety; the use of in-board expertise to deliver sessions was encouraged. A programme of the day is at appendix 2 to this report. The workshop was delivered in partnership with NHS Borders colleagues and colleagues from other health boards such as Healthcare Improvement Scotland (HIS).

The learning outcomes for the workshop were;

Outline the objectives of the Scottish Patient Safety Programme (SPSP)
Describe the current status of patient safety within NHS Borders
Explain what ‘Person Centred Care’ means
Explain the ‘Model for Improvement’ and its use in improving patient safety
Name key components of data interpretation in relation to patient safety
Apply knowledge of the model for improvement to design a Plan, Do, Study, Act test of change, including selection of appropriate measures.

The following report has been written to provide intelligence on participants’ reaction, satisfaction, and engagement with the workshop, which could contribute to a return on investment impact assessment. The report also aims to highlight issues around capability and capacity building for patient safety that the board may wish to investigate further. In addition, possible improvements for the future delivery of such workshops are identified; these may be of interest to a range of stakeholders

Mark Johnston

May 2013.

Executive Summary

Overall the workshop was rated positively by respondents, with 57% of respondents rating it 4 or 5 (on a 1-5 scale), and the remaining 43% rating it 3. The number of delegates who attended was 17.

76% of respondents declared they would recommend the workshop. Comprehensive evaluation data is at appendix 1 of this report.

Questions designed to illicit ‘reaction, satisfaction and engagement’ from participants were included in the evaluation form with respondents agreeing or strongly agreeing with the following questions as a percentage.

Relevant to my professional practice / 81%
Increased my understanding / 75%
Met my learning needs / 67%
Confident I can apply my learning to my job / 57%
Committed to apply learning to my job / 57%
Would you recommend this workshop to a colleague? / No / 2 / 12% / Yes / 13 / 76% / Not recorded / 2 / 12% / 17

Diversity of patient safety subject matter: A high proportion of respondents at 81% report the workshop as relevant to their professional practice, suggesting patient safety is clearly understood as integral to their work. However 33% state the workshop did not meet their learning needs. This may be a reflection of the diversity of patient safety subject matter and the need to ensure audience requirements are clearly understood.

Needs analysis of training requirements: With the training delivery recorded on average as very good (4) or excellent (5) for 88% of respondentsacross four relevant questions, (Appendix 1, Questions 14, 15, 16 and 17) the low percentages recorded for the last two questions, (Appendix 1, Questions 10 and 11) may be a reflection of audience targeting, work pressures impacting on delivery of improvement/patient safety work, culture or management issues? I recommend consideration is given to needs analysis to ensure the correct mix of subjects and audience. Also a survey may be useful for the board to understand attitudes to patient safety. NES can help with this if no in-board support is available.

Subject mix and depth of content:The insufficient time given to complex subject areas such as data interpretation comes out highly on qualitative analysis. Fewer subjects to allow for increased time on data interpretation and/or a lower level of content may have delivered higher levels of satisfaction from those participants that attended with a desire to improve their knowledge and confidence around data and its interpretation.For future workshops, either increased training time or a range of subjects delivered at regular intervals over time would appear to be of benefit.

Learning points for NES:The training team acknowledge the lessons for future delivery they take from this report, in particular; pre-workshop needs analysis, advising the board on audience targeting and suitable level of content to meet the learning needs of the audience.

Main report

Thematic analysis of the evaluation forms received from respondents reveal the following qualitative statements in answer to three questions.

Benefits of Training to the respondents

Information about outline of the SPSP
Learning how to apply, design the whole PDSA cycle
Learning about driver diagrams
Learning about the model for improvement
How to interpret data collected for improvement and graphs
Problem solving techniques using statistical process control method

Respondents’ Planned Implementation of Learning

More patient involvement in decision making to improve patient care.
Implementing small changes to create a uniform pathway which will make care safer
Improve communication within services and across services, e.g. staff working in secondary care understanding systems which link them to primary care services (how they make their appointments); better communication of patient information
Improve team working, planning and prioritising
Use PDSA technique for continuous improvement of services
Promote collaborative working

NES Continued Support to Participants/Health-Boards (HBs)

Simplified information packs to supporttrainers in HBs with their local training
Promoting networking with other patient safety trainers
Access to training NES presentations
Support on data gathering and interpretation skills
Encouraging health boards to have protected time for learning

Impact of learning on Respondents’ self assessment of knowledge and skill of the learning outcomes

At the beginning of the workshop participants were asked to self assess their knowledge and skill level against the learning outcomes on a scale of one to five, with one being low and five being high. The exercise was then repeated at the end of the workshop.Based on a standard benchmark for learning outcomes of 80%, the data was then collated to produce a percentage score of those recording scores of four or five before the training and those recording scores of four or five after the training. The percentage shift was then calculated. The data is at appendix 1 of this report and a summary of findings follows:

Outline the Objectives of the SPSP

At 65.95%, this learning outcome showed the highest shift in self-reported knowledge and skill from before to after the workshop. The key aim communicated to NES was a desire to re-energise patient safety in NHS Borders. From the pre-workshop evaluation data, this audience were unclear as to the objectives of the Scottish Patient Safety Programme.Senior management and SPSP leads may wish to consider some needs analysis work is carried out across NHS Borders to determine if this is reflective of the wider workforce or an isolated pocket. Further awareness level training may be necessary for the wider workforce.

Describe current status of Patient Safety within NHS Borders.

No delegate reported knowledge and skill in respect of this learning outcome above three on the five point scale prior to the workshop.Only 44% of respondents recorded a four or five point score on the five point scale following the workshop, suggesting that at least half of the respondents still felt unable to articulate patient safety into the local context.Senior management and SPSP leads may wish to consider that a communication and education strategy is developed for NHS Borders that includes patient safety as a core component to raise its profile amongst the workforce.

Explain what “Person Centred Care” means

At 38%, this learning outcome showed the highest level of self-reported knowledge and skill prior to the workshop. At 83% post workshop, this learning outcome had the highest level of reported four and five on the five point scale. This suggests that the training achieved its objectives. As one of the three NHS Scotland Quality Ambitions, senior management and SPSP leads may wish to consider that this subject be included in future workshops either as a standalone subject or as a theme running through other subjects.

Explain the 'Model of Improvement' and its use in improving patient safety

This session reported the lowest shift of the six learning outcomes. I recommend further follow up training for respondents to ensure the methodology used in the SPSP, the model for improvement is understood. The Quality Improvement (QI) Hub has recently developed e-learning resources to enhance learning of QI methodologies; these can be accessed on-line at the QI Hub website[1]. Future workshops may benefit from participants accessing these resources before attending. In addition Dr Paul Bowie et al on behalf of the NES Multi-Disciplinary Patient Safety Group has published a study that identifies and summarises the implementation and contextual factors that may influence the effectiveness of Plan, Do, Study, Act (PDSA) cyclesthe study can be accessed from the NES homepage[2]. Future training in PDSA will look to incorporate the recommendations of this study and future revisions.

Name key components of data interpretation in relation to patient safety

The training team feel that although respondents report increased knowledge,with a 43.95%, shift, greater time was needed to cement knowledge for the participants. Qualitative statements included a desire to ‘slow down’ and ‘more time on data’. In addition some respondents commented that they did not see the relevance to patient safety and that it was more for managers as opposed to nursing and clinical staff. Other respondents however, rated this section highly and commented on its usefulness in the qualitative comments.

For multi-disciplinary audiences I would recommend a lower level of detail unless a pre-workshop needs analysis reveals a desire for high level data interpretation training. Following this event, Emma Levy of NES in partnership has developed a workshop (currently in pilot phase) called Skills for Improvement (Measurement) which allows for greater time on this important subject.Until it is available I recommend that participants are encouraged to access the resources on the QI hub highlighted. If additional training is required in data interpretation for NHS Borders, a bespoke session can be arranged to ensure focus and learning.

Apply knowledge of the model for improvement to develop a PDSA test of change

At 55% this learning outcome showed the second highest shift on the five point scale. Qualitative feedback suggested that this session was highly valued by respondents but that greater time would have been welcomed. It seems that they may not be able to articulate the role of the Model for Improvement in Patient Safety, but they do think they can apply their knowledge of it to instigate changesto their work processes.

Conclusions and Recommended Improvements

Workshop Duration: The complexity of the workshop content requires this training to be delivered over a longer period of time, e.g. two days

Targeting of audience: Some respondents reported that data interpretation may be most suitable and practical for managers rather than clinicians. Comments such as this suggest barriers to be overcome in terms of participants understanding the importance of data interpretation in improvement and patient safety work. Pre-workshop assessment to tease out attitudes would have been useful in understanding the local challenges.

SPSP Presentation: This was too high level for the participants with the speed of presentation delivery highlighted as a negative aspect. Thepresenter should slow pace of talking and content should be practical for participants.

Statistics & Data Interpretation: This was a complex topic requiring more time and identification of a teaching method which will help participants understand the subject.Future workshops will need to clearly differentiate between the requirement for low and high level content and tailor the delivery accordingly.

Choice of Venue: There appeared to be a problem with ventilation and/or access to drinking water at this venue. Ensure that there is access to necessities such as drinking water or good ventilation (Health & Safety).

Next steps.

NES is cascading a survey to elicit workforce needs in respect of patient safety education. NHS Borders SPSP programme manager (Julia Scott) will be a recipient of this survey for onward cascade to colleagues. This survey or a shorter version of it may be suitable in determining the educational needs of NHS Borders staff and how best NES can offer support.

The NES Training Development Support Unit (TDSU) has a portfolio of generic skills courses that have for the last ten years been delivered to doctors in training across NHS Scotland. Some of these courses may be useful in complementing and enhancing learning from patient safety education, of particular interest for this audience I believe would be Persuasion skills, Human Factors and Leadership and Safety Culture. These courses and others will become available for booking via NES Portal during 2013/2014 and can be found on the NES Patient Safety training page[3]. Local training expertise will be able to access training packs to deliver these sessions in-house in due course.

Any future training for NHS Borders would benefit from additional time spent on delivery and careful identification of the audience and subject matter to ensure the maximum benefit to the board.

Mark Johnston

May 2013

Evaluation report for NHS Borders Bespoke Patient Safety Workshop 23rd November 20121

Appendix 1: Evaluation Data

Learning Outcomes – Self assessed average score of all respondents knowledge/skill / % of respondents rating 4,5 pre-Workshop / 4, 5 post-Workshop / % shift
Outline the Objectives of the Scottish Patient Safety Programme / 0.5% / 66% / 65.95%
Describe current status of Patient Safety within NHS Borders / 0% / 44% / 44%
Explain what “Person Centred Care” means / 38% / 83% / 45%
Explain the 'Model of Improvement' and its use in improving patient safety / 11% / 50% / 39%
Name key components of data interpretation in relation to patient safety / 0.05% / 44% / 43.95%
Apply knowledge of the model for improvement to develop a PDSA test of change / 11% / 66% / 55%
Reaction, satisfaction and engagement data
How would you rate the following: / Strongly Disagree / Strongly Agree / Not recorded / Total
1 / 2 / 3 / 4 / 5 / 0
5 / Workshop was enjoyable / 0 / 0% / 0 / 0% / 0 / 0% / 0 / 0% / 0 / 0% / 16 / 100% / 16
6 / Relevant to my professional practice / 0 / 0% / 0 / 0% / 3 / 19% / 9 / 56% / 4 / 25% / 0 / 0% / 16
7 / Increased my understanding / 0 / 0% / 1 / 6% / 3 / 19% / 8 / 50% / 4 / 25% / 0 / 0% / 16
8 / Understood the training objectives / 0 / 0% / 0 / 0% / 6 / 38% / 5 / 31% / 5 / 31% / 0 / 0% / 16
9 / Met my learning needs / 0 / 0% / 2 / 13% / 3 / 20% / 6 / 40% / 4 / 27% / 0 / 0% / 15
10 / Confident I can apply my learning to my job / 0 / 0% / 2 / 13% / 5 / 31% / 7 / 44% / 2 / 13% / 0 / 0% / 16
11 / Committed to apply learning to my job / 0 / 0% / 2 / 13% / 5 / 31% / 6 / 38% / 3 / 19% / 0 / 0% / 16
12 / Sufficient opportunity for interaction & discussion / 0 / 0% / 2 / 13% / 3 / 19% / 6 / 38% / 5 / 31% / 0 / 0% / 16
13 / Comfortable with pace of training delivery / 0 / 0% / 4 / 25% / 5 / 31% / 4 / 25% / 3 / 19% / 0 / 0% / 16
How would you rate the following: / Poor / Excellent / Not recorded / Total
1 / 2 / 3 / 4 / 5 / 0
14 / The presenting skills of:
Lead Trainer / 0 / 0% / 0 / 0% / 1 / 6% / 10 / 63% / 5 / 31% / 0 / 0% / 16
15 / The knowledge of:
Lead Trainer / 0 / 0% / 0 / 0% / 0 / 0% / 8 / 50% / 8 / 50% / 0 / 0% / 16
16 / Quality of practical exercises/activities / 0 / 0% / 1 / 6% / 2 / 13% / 8 / 50% / 5 / 31% / 0 / 0% / 16
17 / The training resources / 1 / 6% / 0 / 0% / 2 / 12% / 8 / 47% / 5 / 29% / 1 / 6% / 17
18 / Would you recommend this Workshopto a colleague / No / 2 / 12% / Yes / 13 / 76% / Not recorded / 2 / 12% / 17
Poor / Excellent / Not recorded / Total
1 / 2 / 3 / 4 / 5 / 0
Overall this Workshop was / 0 / 0% / 0 / 0% / 6 / 43% / 3 / 21% / 5 / 36% / 0 / 0% / 14

Appendix 2: Event Timetable

NHS Borders Patient Safety Event /
23rd November 2012 / The Tryst, BordersGeneralHospital, Melrose. / 2012
TIMING / SESSION TITLE / METHOD / DESCRIPTION
0900 - 0915 / Introductions and pre-Workshop evaluation
0915 - 0945 / Person Centred Care
Shaun Maher / Presentation Discussion / One of the three elements of NHS Scotland’s Quality Strategy, a view from the front line of what it means in practice.
0945 - 1015 / SPSP National Perspective
Jane Ross / Presentation Discussion / National Facilitator for the Scottish Patient Safety Programme, Jane Ross provides an update and overview of the programme, its successes and future challenges.
1015 – 1115 / Leadership & Culture
Dr Jonathan Kirk / Presentation Discussion / NHS Borders, Associate Medical Director, Dr Jonathan Kirk gives his take on leadership, culture and overcoming the barriers to improvement implementation.
Tea / Coffee/ Networking (1115 – 1130)
Mark Johnston and Emma Levy – Training and Research Officers, NHS Education for Scotland
1130 - 1245 / Preparing for Improvement / Presentation / Discussion / Activity / A refresher of the thinking behind improvement for patient safety. Understand and create driver diagrams, prioritise your improvement effort and understand how to identify and select appropriate measures.
Lunch & Networking (1245 – 1330)
1330 – 1445 / Understanding data / Presentation / Discussion / Activity / Interpreting run charts and understanding variation within a system. Understand the importance of starting small with your improvement efforts.
Tea / Coffee/ Networking (1445 – 1500)
1500 - 1600 / Planning a test of change / Discussion / Activity / Bringing together all the themes of the day to plan your first test of change for an improvement project in your workplace.
1600 - 1610 / Evaluation of the day / Summary of main learning points and a chance for final questions. Post-Workshop evaluation to be completed with a view to improvement of the Workshop.
CLOSE (1610)

Evaluation report for NHS Borders Bespoke Patient Safety Workshop 23rd November 20121

[1]

[2]Healthcare Improvement and Rapid PDSA Cycles of Change: A Realist Synthesis of the Literature

[3]