Additional Online Information

Applying Cognitive Debriefing to Pre-Test Patient-Reported Outcomes in Older People with Multiple Sclerosis

Michelle Ploughman1,2*, Mark Austin1, Mark Stefanelli3, Marshall Godwin2

1Rehabilitation Program, Eastern Health Authority, St. John’s, Newfoundland and Labrador, Canada

2Primary Health Research Unit, Faculty of Medicine, Memorial University, St. John’s. Newfoundland and Labrador, Canada

3Department of Neurology, Faculty of Medicine, Memorial University, St. John’s, Newfoundland and Labrador, Canada

Patient-Reported Outcome Measures

Frenchay Activities Index (FAI)

The Frenchay Activities Index (FAI) is a measure of instrumental activities of daily living (iADL), assessing a broad range of activities associated with everyday life including domestic chores, social activities, work and leisure [1]. The frequency with which each of the 15 activities is undertaken over the past 3 or 6 months (depending on the nature of the activity) is assigned a score of 0-3 for a summed score from 0 – 45. The FAI can be used in adults as a mailed questionnaire with excellent test-retest reliability [2]and also correlates well with face-to-face interview scores [3]. The FAI has been mainly used in measuring iADL after stroke however it has been used in large MS population studies [4, 5] even though the measure has yet to be validated in people with MS.

The FAI required the most format modification initially; specifically navigation was changed from ‘top to right to bottom’ to simply ‘top to bottom’. Gridlines were removed and answer keys were re-printed below each item. The scales and categories as well as reference periods changed throughout the measure causing difficulty for some respondents. To re-orientate respondents, key points were underlined. Using the FAI guide [1], we provided examples to clarify terms such as ‘washing up,’ ‘D-I-Y,’ ‘gainful work,’ and to delineate ‘social outings’ versus ‘travel outings’ and ‘heavy housework’ versus ‘light housework’. The term ‘gainful work’ was perceived as ambiguous and stimulated the most discussion especially around if gainful work was paid work or not. We added examples to indicate that gainful work was work that could be paid such as child care, volunteer work and paid work.

Barthel Index (BI)

The Barthel Index (BI) measures function and mobility in activities of daily living (ADL) in ten areas: feeding, bathing, grooming, dressing, bowel control, bladder control, toileting, transfers, ambulation and stair climbing [6]. It is widely used to measure functional disability in people with stroke and other neuromuscular or musculoskeletal disorders. Items are rated in terms of whether individuals can perform activities independently, with some assistance, or are dependent. The index yields a total score out of 100 - the higher the score, the greater the degree of functional independence. The BI is typically assessed by a health professional, although it has been shown to be reliable whether administered by telephone, in-person or by mail [7]. Subjects required the least amount of help in completing the BI. Simple instructions were developed directing respondents to choose the lower score if they were unsure of which response category to choose. The term ‘feeding’ was changed to ‘eating’ to reflect a patient-oriented approach. Metric conversion (45m) was added for the walking distance of 150 feet.

Simple Lifestyle Indicator Questionnaire (SLIQ)

The SLIQ was developed to measure the individual components of lifestyle that affect cardiovascular disease [8]. It measures aspects of one’s health; diet, exercise, alcohol consumption, smoking rates, and life stress. Initial content and face validity was completed by the developers with adequate test-retest reliability, internal consistency and external validity when compared to scorer’s blinded assessment of 136 people attending a family practice clinic.

In the SLIQ, respondents had the most difficulty classifying their level of physical activity into mild, moderate and vigorous categories. They tended to use the exercise examples (4-5 per category) as a checklist and if they did not participate in any of the suggested activities, marked the item as ‘0’. We used the Borg Perceived Level of Exertion Scale to describe each level of exercise [9] (eg. moderate exercise: physical activity which causes you to feel slightly out of breath) then used examples from respondents in each category (reduced to 3 examples per category). In addition, the questionnaire specified that exercise should be for 30 minutes duration which caused confusion for some respondents. In the instructions, we added that exercise had to take place for periods of at least 30 minutes duration per session not including breaks.

EuroQol EQ-5D

The EuroQol EQ-5D measures 5 dimensions of health: mobility, self-care, daily activities, pain/discomfort, and anxiety/depression [10]. The EQ-5D also measures one’s perceived level of present overall health through the use of a scale that ranges from 1-10. Data from the EQ-5D can be used to identify 243 health states and provides a single index value for health status. Fisk and colleagues report that in a clinic registry population of people with MS, EQ-5D demonstrated good test-retest reliability and construct validity although 42.9% of people with moderate to severe disability skipped the mobility item [11].

The EQ-5D measures one’s perceived level of present overall health using a visual analogue scale. This scale required more assistance to complete than any other measurement in the study. Subjects are asked to draw a line from a box (oriented on the bottom of the page) to a point on the scale (to the right of the page) which best represented their health. Subjects did not read instructions and simply circled a number on the scale or drew a line extending from the bottom of the scale to the selected point. After a number of iterations, instructions were simplified and the scale was aligned so that respondents drew a line from the box to a point on the line below it.

Most wheelchair users wished to skip the mobility item in the EQ-5D because they did not feel that the categories applied to them. After consultation with respondents, the second mobility category was modified as below.

I have no problems in walking about.

I have some problems in walking about or I use a wheelchair.

I am confined to bed

Personal Resource Questionnaire 2000 (PRQ2000)

The PRQ2000 measures one’s level of social support [12]. It consists of 15 short statements which the subject then agrees or disagrees based on a seven-point Likert scale; higher scores indicating higher support (eg .I belong to a group in which I feel important 1-Strongly Disagree to 7-Strongly Agree). PRQ2000 demonstrates adequate content, predictive and construct validity [12] and mailed versions have been used in community-living women with chronic health conditions [13] and adults with MS [14].

Other than reformatting so that the answer key was positioned under each question item, the PRQ2000 caused few problems for respondents. One subject noted that use of the word “love” implied that friends were not a source of social support. Despite examples broadening the concept of ‘work’, some respondents felt that emphasis was placed on employment (e.g. People let me know that I do well at my work [job, homemaking]).

Respondent Burden

Once modifications were made, the survey, consisting of the five questionnaires, took about 20 to 30 minutes to complete. Almost all respondents took more time and had more frustration with items requiring tabulation. The most troublesome PROs requiring calculation and therefore more cognitive effort were the FAI and the SLIQ. The FAI, in particular, in addition to requiring tabulation, also required the respondent to recognize scale changes in the instrument. Some subjects required re-orientation to the scale change. e.g. note the scale change below between subsequent items in the FAI.

Washing up (dishes, kitchen counters):

  • 0 = never
  • 1 = less than once a week
  • 2 = 1-2 times a week
  • 3 = most days

Washing clothes:

  • 0 = never
  • 1 = 1-2 times in three months
  • 2 = 3-12 times in three months
  • 3 = at least weekly

References

1.Wade, D.T., Legh-Smith, J., Langton Hewer, R. (1985) Social activities after stroke: measurement and natural history using the Frenchay Activities Index. International Rehabilitation Medicine, 7(4), 176-81.

2.Turnbull, J.C., Kersten, P., Habib, M., McLellan, L., Mulee, M.A., George, S. (2000) Validation of the Frenchay Activities Index in a general population aged 16 years and older. Archives of Physical Medicine & Rehabilitation. 81(8), 1034-1038.

3.Carter, J., Mant, F., Mant, J., Wade, D., Winner, S. (1997) Comparison of postal version of the Frenchay Activities Index with interviewer-administered version for use in people with stroke. Clinical Rehabilitation. 11(2), 131-8.

4.Einarsson, U., Gottberg, K., Fredrikson, S., von Koch, L., Holmqvist, L.W. (2006) Activities of daily living and social activities in people with multiple sclerosis in Stockholm County. Clinical Rehabilitation, 20(6), 543-51.

5. Ytterberg, C., Johansson, S., Gottberg, K., Holmqvist, L.W., van Koch, L. (2008) Perceived needs and satisfaction with care in people with multiple sclerosis: a two-year prospective study. BMC Neurology, 8, 36.

6.Mahoney, F.I., Barthel, D.W. (1965) Functional Evaluation: The Barthel Index. Maryland State Medical Journal, 14, 61-5.

7.Yeo, D.,Faleiro, R., Lincoln, N.B. (1995) Barthel ADL index: A comparison of administration methods. Clinical Rehabilitation. 9(1), 34-39.

8.Godwin, M., Streight, S., Dyachuk, E., van den Hooven, E.C., Ploemacher, J., Seguin, R., Cuthbertson, S. (2008) Testing the Simple Lifestyle Indicator Questionnaire: Initial psychometric study. Canadian Family Physician. 54(1), 76-7.

9.Borg, G.A.V. (1998) Borg's Perceived Exertion and Pain Scales. Human Kinetics.Champaign, IL

10.Brazier, J., N. Jones, Kind, P. (1993) Testing the validity of the Euroqol and comparing it with the SF-36 health survey questionnaire. Quality of Life Research. 2(3), 169-80.

11.Fisk, J.D., Brown, M.G., Sketris, I.S., Metz, L.M., Murray, T.J., Stadnyk, K.J. (2005) A comparison of health utility measures for the evaluation of multiple sclerosis treatments. Journal of Neurology Neurosurgery and Psychiatry, 76(1), 58-63.

12.Weinert, C., Brandt, P.A. (1987) Measuring social support with the Personal Resource Questionnaire. Western Journal of Nursing Research. 9(4), 589-602.

13.Weinert, C., Cudney, S., Hill, W.G. (2008) Rural women, technology, and self-management of chronic illness. Canadian Journal of Nursing Research, 40(3), 114-134.

14.Long, K.A., Weinert, C. (1992) Descriptions and perceptions of health among rural and urban adults with multiple sclerosis. Research in Nursing & Health, 15(5), 335-342.

1