Pilot Study Into the Use of the NHS Direct Knee Arthritis Decision Aid

Pilot Study into the Use of the NHS Direct Knee Arthritis Decision Aid

August 2011

1.0 Executive Summary

Patients showed a definite desire to access more information about their treatment and were keen to use the decision aid
38% of patients were not able to access a web-based tool
80% of patients who refused to use the tool refused because of internet based design, rather than the concept of a PDA
For those that used it, in general the decision aid was easy to use and met patients’ expectations.
Did not address needs of some patient groups – e.g. younger patients who had tried traditional conservative measures.
Only surgical option discussed was total knee replacement – no information about uni-compartmental knee arthroplasty or high tibial osteotomy
It takes significantly longer for patients to go through the information in the decision aid than the 15-30mins the site suggests
It helped patients prepare for the clinical consultation by making them better informed, ensuring they had considered the options and thinking about the questions that were pertinent to them
In general it took no extra consultation time and reduced consultation times in some cases
The decision aid added value to the clinical consultation process as the clinician was able to dedicate more time to patient-specific questions, rather than explaining the basics of all the options
The PDA did not answer all patient questions and reinforced the need for a discussion with a professional
There would be minimal training or preparation required to enable other clinicians to assess patients who had accessed the PDA, although more specific patient questions could necessitate a more specialised clinician
Further research into the impact of the PDA on decisional conflict and decisional regret compared to a control group would be valuable.

2.0 Background

Patients increasingly wish to be more involved in decisions about their care and treatment[1]. In recognition of this, the 2010 Equity and Excellence White Paper placed great emphasis on patient involvement, with the mantra “No decision about me without me”[2].

Patient Decision Aids (PDAs) have become a useful tool in shared decision making, helping to guide the clinician-patient relationship towards the most appropriate treatment decision. PDAs are tools, delivered through a variety of media, which help patients reach a decision about their treatment. They are most commonly used when there is no clinical evidence that one treatment is better than another and patients aren’t sure which option will be best for them[3].

The Department of Health is keen to explore the possibilities for using PDAs as part of a shared decision making process and accordingly has developed a number of web based tools accessed through the NHS Direct website. One of these tools is the knee arthritis decision aid, which is openly available on the website.

PDAs are considered to be of particular benefit for discretionary surgery, such as knee surgery, where there is no clear evidence that one intervention will lead to improved outcomes over another. However, whilst there is well documented evidence for the benefits of PDAs in other care pathways[4], their use in orthopaedic surgery and supporting evidence is relatively new.

3.0 The Study and The Tool

The Pilot Study

As a relatively new tool, the Nuffield Orthopaedic Centre was commissioned to undertake a pilot study into the use of the knee arthritis decision aid, based on around 20 patients.

Given the small number of participants the study did not seek to measure the impact of the tool on patient outcomes or changes in conversion rates. Instead, the study focused on the user-friendliness of the tool, it’s effectiveness in helping patients reach a decision and the impact of the tool on clinical consultation.

The Knee Arthritis Decision Aid

The tool is for people with arthritis of the knee who have pain, stiffness, limited mobility or other symptoms that may make everyday activities difficult. Its purpose is to help people decide which treatment for knee arthritis is right for them.

It is an online tool with written information about knee arthritis and various treatment options with videos from clinicians and patients.

The decision aid allows patients to create a personal record of their journey by building a summary of the pages visited and the answers to all the questions asked. At various points in the site patients can watch films of real-life experiences from people in their situation.

The site states it takesbetween 15 and 30 minutes to work through depending on how much patients want to read and how many videos they view.

Knee Arthritis Decision Aid – Final Report Page 1 of 22

4.0 Method

Patients were identified from referral letters received in the tier 2 triage service.
An ESP physiotherapist who was familiar with the PDAwas responsible for identifying suitable patients and for seeing these patients in clinic assessment.
Other than discussing any questions raised by the PDA the clinic appointment was conducted in the same was as a usual triage assessment.

Data collection

The pilot study contained a small number of patients so it was not appropriate to attempt to study the efficacy of the PDA in terms of health outcomes or changes in the treatment choice reached. Therefore, the study was focused on assessing four key areas:

Measuring the Usefulness, Satisfaction and Ease of Use of the PDA – measured through part 1 of pre-consultation questionnaire (this questionnaire was developed based on standard methods for assessing web based tools)
How useful the PDA was in preparing the patient for their appointment – measured through part 2 of pre-consultation questionnaire (this was based on questionnaires used by Ottawa Hospital Research Institute in decision aid studies)
How well this assisted them in the clinical consultation – measured in follow-up questionnaire and clinician questionnaire
How they felt about the decision reached after the consultation – measured in follow-up questionnaire (this was based on the decisional conflict scale questionnaire used by Ottawa Hospital Research Institute in decision aid studies).

The questionnaires are appended.

At the end of the study period an interview was conducted with the clinician involved in the project to understand the impact of the tool on the clinical consultation.

Analysis

The numerical data from the questionnaire responses were analysed to identify minimum, maximum and mean values.

The decisional conflict questionnaire responses were scored following the scoring method used by the Cochrane systematic review of trials of patient decision aids. Items are given a score value of:

0 = strongly agree, 1 = agree, 2 = neither agree nor disagree, 3 = disagree, 4 = strongly disagree. The responses for each answer are summed and converted to a total score out of 100. Scores range from 0 (no decisional conflict) to 100 (extremely high decisional conflict) on the decisional conflict scale.

Text comments were analysed to identify common themes.

Knee Arthritis Decision Aid – Final Report Page 1 of 22

5.0 Results

Participant demographics

Number identified as suitable / Average age / Number who used the tool / Average age (of those who used the PDA) / Max age (of those who used the PDA) / Min age (of those who used the PDA)
Male / 25 (48%) / 70 / 10 (53%) / 67.1 / 80 / 52
Female / 27 (52%) / 70 / 9 (47%) / 67.1 / 81 / 60
Total / 52 / 70 / 19 / 67.1

52 patients were identified as suitable for the study
Of these 27 (52%) agreed to take part
The main reason for those that refused was lack of access to the internet (80%) with the remainder either not contactable or unable to attend
Of those that agreed to participate 19 (83%) had accessed the tool prior to consultation
The average age of those who used the PDA (67.1) was slightly lower than the sample average (70), although there was quite a range from 52 to 81.

Knee Arthritis Decision Aid – Final Report Page 1 of 22

Patient pre-consultation questionnaire results

Knee Arthritis Decision Aid – Final Report Page 1 of 22

Patient Comments:

Consistency of information:
“The issue of jogging post-surgery - can't do it / or will shorten the life of the new knee or both? What are the alternatives?”
“Found it difficult to access the videos so have not looked at those. I think that a full knee replacement was mentioned but not a partial knee replacement.”

Ease of use:
“A link straight from the email would be useful”
“Not everyone is computer literate. I am not! My husband uses a computer but is not sufficiently proficient to deal with this questionnaire.”

Ambiguity of questions:
“I found the following question ambiguous, "during the past 4 weeks have you had little interest or pleasure in doing things" I have real interest in doing things but sometimes due to my knee problem not so much pleasure.”

Inability to answer patient questions / frustration:
“I used the internet for these questions but found it difficult to accept. I've done everything to ease this pain. I do a fair degree of walking. It gets intolerable to carry on. I try to lose weight but this is difficult because I've had extensive surgery”

Reinforces need for expert opinion:
“The key issue remains that I do not know how to interpret the x-ray (objective) - the professional's job. I understand my objective desire - to run. But I don't understand between my pain (both subjective and objective) and the objective condition. It did help though.”

Clinician questionnaire results

Clinician Comments

Ease of use:

Patient reported difficulty with access to PDA and not being 'PC literate' struggled a little.
This patient struggled with using the computer system. Despite reading the printouts was really unsure about what she wanted to do.

Informed consultation/decision:

Patient remained aware of the implications and indications of knee replacement at assessment
Patient categorically decided against knee replacement after watching PDA. Decided symptoms weren't severe enough and fearful of potential risks and outcomes
This lady has had previous consultation with an orthopaedic surgeon in Bath who suggested she would need a TKR. After reviewing the PDA she did say that she hadn't appreciated she might not be able to kneel following surgery. She was not aware her range o
Patient very positive about the PDA and influence on his decision-making process
Patient reported a definite increasing of awareness of his options and had made his decision based on the PDA prior to consultation

Inability to answer patient questions / frustration:
This gentleman is a social scientist and therefore an expert in questionnaire design. He did have specific questions that the tool didn't cover (i.e. can you run after a replacement) & felt it took 45min - 1hr to go through whereas the tool designers think it should take 15-30mins

Knee Arthritis Decision Aid – Final Report Page 1 of 22

Patient post-consultation questionnaire results

Although not many patients completed the post-consultation questionnaire (perhaps due to having to access it online), those that did indicate a relatively low level of decisional conflict
Scores lower than 25 are associated with implementing decisions. 4 out of 7 scored lower than 25.

Scores higher than 37.5 are associated with decisional delay or feeling unsure about implementation. No patients scored higher than 37.5

Knee Arthritis Decision Aid – Final Report Page 1 of 22

Discussion, implications and recommendations

Access and ease of use:

The idea of the tool was clearly popular with patients. Everyone who was contacted was interested in using the tool and those who were unable to take part was only due to a lack of internet access.
However, this is a pertinent point, particularly given the demographics of the intended audience (OA knee patients). 38% of identified patients were not able to access the tool and of those that did several raised concerns about computer literacy and ability to access the tool.
It would be worth developing additional media for sharing the information, such as DVDs or information leaflets.
A number of patients identified that the tool took longer to go through in full (up to 1 hour) than the site suggests (15-30 minutes).
Also, the site states that you can save your progress and return to it later. However, this is not possible – the site does not remember previously entered answers. This issue was raised with NHS Direct but has not yet been resolved.

Usefulness and satisfaction

The questionnaire results suggest that overall patients were very satisfied with the tool with the majority of respondents agreeing or strongly agreeing with the questions about usefulness, satisfaction and ease of use
One patient strongly disagreed that it met their needs, with another strongly disagreeing that they could use it without written instructions (this may well link to computer literacy)
Some of the comments suggest that the tool could go into more depth around what some of the limitations may be post-surgery as well as exploring uni and total knee replacement. However, it may be that patient-specific expectations post-surgery are better discussed in the consultation. Indeed, the clinician felt that all of the patients were much more informed about the options in general which freed up more of the consultation time for patient-specific questions
There was also some frustration that the tool left questions unanswered or highlighted the patient’s lack of expertise. These issues may be difficult to overcome, but certainly could be mitigated by setting patient expectations about the tool – i.e. that there may well be questions that come out of it and that these can be discussed in the clinical consultation.

Assistance in decision making

Although there is no control to measure what people’s decisional conflict would be without the tool and the numbers responding to the decisional conflict questionnaire were low, the results do suggest that the PDA was useful in helping people recognise that a decision needs to be made, informing people about the pros and cons of each choice and preparing them for the discussion with the clinician
100% of people agreed or strongly agreed that they knew which options were available to them and the benefits and risks of each option
100% of people also agreed or strongly agreed that they have enough advice to make a choice and that they are clear which choice is best for them
No patients scored within the range associated with decisional delay or feeling unsure about implementation. This could be significant, as studies have suggested that even relatively small increases in decisional conflict can increase the likelihood of blaming a doctor for bad outcomes. In a study into prostate cancer testing decisional conflict was an independent predictor of blame, separate from other predictors such as knowledge scores and age of the patient.[5]
The clinician stated that for some, the tool seemed to have helped them reach a decision about treatment even before the consultation, and there were several patients who realised some of the significant impacts of surgery that they had not otherwise considered.
Where the clinician noted that the questionnaire had raised problems, these were related to the ability to access the system, the time it took and the difficulty in reaching a decision despite all the information.

Impact on assessment

With the majority of patients (84%) the assessment took no additional time and the clinician felt that in many cases it made the consultation process quicker.
There were only 3 patients for which the tool assisted the clinician in the clinical assessment or decision, however the clinician agreed that the majority of patients were better informed allowing more of the consultation to be dedicated to patient-specific questions rather than informing them of their options.
The clinician did not note any change in patient expectations in terms of the treatment they may receive and there was no sense of patients feeling that they were being pressured to make their own decision.
The clinician felt that the preparation for clinicians to be able to discuss the PDA with patients would be relatively simple. They would just need to familiarise themselves with the site and the types of information provided. Potentially, non-knee specialists in a multidisciplinary service might be asked more challenging questions by patients (for example specific capabilities post-surgery), but this would be no different to a patient who had been well-informed by other sources).
The clinician felt the tool was accessed at the most appropriate stage in the pathway – i.e. before a patient had been directed on a surgical or non-surgical treatment pathway.

Limitations and recommendations for further study

The study makes no claims about the impact of the PDA on patient outcomes. It focuses solely on the usability and functionality of the tool. It is recommended that a larger study is undertaken to compare the decisional conflict and decision regret of patients against a control group post-consultation.
The study does not investigate the relative benefits of the tool compared to other media for delivering patient information. It is recommended that any further study would compare the user acceptance and decisional support of the tool against similar information delivered through written literature, audio or DVDs. This would be particularly beneficial given the relatively low uptake of the internet-based tool because of access.
The study did not investigate the differential impact of the tool on different population groups. Previous studies[6] have shown that PDAs can impact on the racial disparities in perceived outcome of knee replacement surgery, so it would be valuable to explore that further. It would also be worth measuring the value of the tool to people who do not speak English as a first language or who are not fully literate.
The study did not investigate what other information patients may desire. For example, the PDA does not give any specific data on the outcomes of the different treatment options and some patients might wish to see this.

Knee Arthritis Decision Aid – Final Report Page 1 of 22