BSRM Philip Nichols Prize Submission 2016

1

BSRM Philip Nichols prize submission 2016

The UK Functional Assessment Measure (UK FIM+FAM): Psychometric Evaluation in Patients Undergoing Specialist Rehabilitation following a Stroke from the National UK Clinical Dataset

Author

Dr Meenakshi Nayar

NIHR Academic Clinical Fellow in Rehabilitation Medicine (ST6)

Royal National Orthopaedic Hospital, Brockley Hill, Stanmore, United Kingdom

Co-Authors

Professor Lynne Turner-Stokes (Supervisor)

Regional Hyper Acute Rehabilitation Unit (RHRU), Northwick Park Hospital, London, United Kingdom

Dr Roxana Vanderstay

Department of Palliative Care Policy and Rehabilitation, Kings College London, Faculty of Life Sciences and Medicine, London, United Kingdom

Professor Richard J. Siegert

School of Public Health and Psychosocial Studies and School of Rehabilitation and Occupational Studies, Auckland University of Technology, Northcote, Auckland, New Zealand

Abstract

Background

The UK Functional Assessment Measure (UKFIM+FAM) is the principal outcome measure for the UK Rehabilitation Outcomes Collaborative (UKROC) national database for specialist rehabilitation. Previously validated in a mixed neurorehabilitation cohort, this study is the first to explore its psychometric properties in a stroke population, and compare left and right hemispheric strokes (LHS vs RHS).

Methods

We analysed in-patient episode data from 62 specialist rehabilitation units collated through the UKROC database 2010–2013. Complete data were analysed for 1,539 stroke patients (LHS: 588, RHS: 566 with clear localisation). For factor analysis, admission and discharge data were pooled and randomised into two equivalent samples; the first for exploratory factor analysis (EFA) using principal components analysis, and the second for confirmatory factor analysis (CFA). Responsiveness for each subject (change from admission to discharge) was examined using paired t-tests and differences between LHS and RHS for the entire group were examined using non-paired t-tests.

Results

EFA showed a strong general factor accounting for >48% of the total variance. A three-factor solution comprising motor, communication and psychosocial subscales, accounting for >69% total variance, provided acceptable fit statistics on CFA (Root Mean Square Error of Approximation was 0.08 and Comparative Fit Index/ Tucker Lewis Index 0.922/0.907). All three subscales showed significant improvement between admission and discharge (p<0.001) with moderate effect sizes (>0.5). Total scores between LHS and RHS were not significantly different. However, LHS showed significantly higher motor scores (Mean 5.7, 95%CI 2.7, 8.6 p<0.001), while LHS had significantly lower cognitive scores, primarily in the communication domain (-6.8 95%CI -7.7, -5.8 p<0.001).

Conclusion

The UK FIM+FAM has a three-factor structure in stroke, similar to the general neurorehabilitation population. It is responsive to change during in-patient rehabilitation, and distinguishes between LHS and RHS. This tool extends stroke outcome measurement beyond physical disability to include cognitive, communication and psychosocial function.

Philip Nicols Prize entry Dec 2016

The UK Functional Assessment Measure (UK FIM+FAM): Psychometric Evaluation in Patients Undergoing Specialist Rehabilitation following a Stroke from the National UK Clinical Dataset

Dr Meenakshi Nayar (ST6) - ACF in Rehabilitation Medicine

The latest 2016 stroke statistics show that there are over 152,000 strokes each year[1,2]. It is the leading cause of complex disability with over half of all stroke survivors being left with a disability [3]. A large proportion of patients with complex disability resulting from strokes are admitted into specialist rehabilitation units.Stroke patients are also a diverse and heterogeneous group. Clinical syndromes such as language difficulties tend to be associated with left hemispheric strokes, while right hemispheric strokes have been linked with neglect [4] and impairments in integrative and interpretive aspects of cognition [5]. These disabilities can have a substantial negative impact on the independence of patients.All specialist rehabilitation units in the UK collect UK Functional Assessment Measure (UK FIM+FAM) as part of the minimum standardised data for the national clinical dataset UKROC[6].

This outcome measure was chosen as previous work found that that a large number of specialist rehabilitation units in the UK were already using it as part of routine clinical practice [7]. It is widely used for its comprehensive coverage of motor, cognitive and psychosocial function [8]. At an individual level the measure provides valid and reliable information about a person’s requirements for assistance with essential tasks of daily living, and on a group level it can be used to measure and compare outcomes across different practices and populations [8]. Consequently, it is important to understand the metric properties in the population in which it is being used.

The study described in this article was done to explore the psychometric properties in a purely stroke population, and compare left and right hemispheric strokes (LHS vs RHS). The project presented here has been published in PLOS one Jan 2016 - Validation of UK FIM+FAM in a stroke population. Does the UK FIM+FAM demonstrate differences in functional abilitybetween left and right hemisphere stroke patients?.This submission for the Philip Nichols Prize describes and expands on those findings.

Chronological systematic assimilation of existing literature on the development and validation of the UK FIM+FAM:

To set the scene, Iconducted abrief systematic review of the literature to lay out the background to the development of the UK FIM+FAM. The search strategy used is outlined below and the search results (Appendix 1) can be found on the attached appendices.

Literature search strategy

AND

AND

It is important to note that much of the existing literature was focused on the application of FIM+FAM in traumatic brain injury. All the articles that came up in the literature search have been included below. It shows the chronological systematic assimilation of existing literature on the development and validation of the UK FIM+FAM:

1. 1980's - The FIM is an 18-item ordinal measure of disability which includes 13 motor items and 5 cognitive items. It was developed in the 1980s by a national task force in the United States (US) and is now one of the most commonly used generic outcome measures in rehabilitation. Its psychometric properties have been very thoroughly evaluated in the world literature [10-14]

1. Mid 1990's -The UK version of the FAM was developed by the United Kingdom FIM+FAM users in collaboration with the US originators, to translate it into UK-English and address the known subjectivity and inconsistency of some items [9].

1. Late 1990's - UK FIM+FAM was initially conceptualised for use with traumatic or diffuse brain injury but FAM items are widely applicable in other neurological conditions, including spinal cord injury and progressive neurological conditions [7]

1. 2000 - The UK FIM+FAM has gained in popularity over the last decade overtaking the US version. It has been adopted in the UKROC database as the principal outcome measure for specialist rehabilitation [15]

1. 2010+ ongoing The UK FIM+FAM is being explored as an outcome measure for rehabilitation in other countries, including Europe(notablySpain),South America, Australasia, Iran and Japan[16-18]

1. 2013 - The UK FIM + FAM was found to be a valid, reliable scale of functional independence, which is responsive to changes occurring in a general inpatient neurorehabilitation population.[19]

1. 2014 Functional outcomes of a national cohort of patients with Guillan Barre syndrome and other polyneuropathies showed that these patients had both physical and cognitive disabilities that are amenable to change and picked up on the UK FIM+FAM[20]

1. 2015 - 'Validity and reliability of the Japanese version of the FIM + FAM in patients with cerebrovascular accident' showed the scale had good internal consistency, intra-rater reliability and criterion validity between the full scales and other outcome measures such as Barthel index, National Institutes of Health Stroke Scale, modified Rankin Scale[21]

1. 2015-2016 - This project - Validation of UK FIM+FAM in a stroke population.

The psychometric properties of the 30 item UK FIM+FAM have been evaluated in a general neurorehabilitation cohort in 2013 using both Classical theory (CTT) and and non-parametric Item Response Theory (IRT) methods[19].This analysis demonstrated two distinct domains - motor (16 items) and cognitive (14 items) the latter dividing into a 5-item communicative and 9-item psychosocial component. This yielded an overall factor structure of three subscales (physical, communication and psychosocial), each with a Cronbach’s alpha >0.90.

As many of the cognitive and psychosocial items are relevant to the stroke population, a questionfrequentlyaskedoftheUKROChelplineiswhethertheUKFIM+FAMhasbeenspecificallyvalidatedforuseinstrokeornot.Itwasthereforepertinenttoexplore

a)whetheritspsychometric and scaling properties are the same in stroke patients as in traumatic brain injury andgeneralneurorehabilitationpopulations

b)ifitisresponsivetothechangesthatoccurduringinpatientrehabilitation

c)whetherornotitidentifies(inabroadsense)thedifferences in cognitive, communicative and psychosocial function

Aims and objectives

This project presents the first formal evaluationofthesepsychometricpropertiesoftheUKFIM+FAMinastrokepopulation.

The analysis is presented in two parts, with the following objectives

In Part1:

a) To examine the factor structure (dimensionality and internal consistency) of the UK FIM+FAMinpatientswithcomplexdisabilitiesundergoinginpatientspecialistrehabilitation following astroke

b) To determine its responsiveness to change in functional independence betweenadmission and discharge for this patientpopulation

In Part 2:

a) ToexaminetheextenttowhichtheUKFIM+FAMidentifiedtheanticipateddifferences infunctionalabilitiesbetweenleftandrighthemispherestrokepatients.

Methods

Design, Participants and Setting

Acohortanalysisofanationalmulti-centresampleofleftandrighthemisphere stroke patients who were admitted to inpatient specialist rehabilitation programmes in theUKduringa3-yearperiodbetweenMay2010andApril2013.

In the UK, the majority of stroke patients will make a good recovery with the support of their local (Level 3) stroke rehabilitation services. A smaller number of patients have more complexneedsthatrequireexpertise,equipmentandfacilitiesofadistrict(Level2)ortertiary (Level1)specialistrehabilitationcentre.Typically,theseservicestakeaselectedpopulationof mainly younger stroke patients with a mixture of physical, cognitive, communicative and/or psychosocial difficulties. Detailed criteria for admission to such services are available on the BritishSocietyofRehabilitationMedicinewebsite[22].Outcomeevaluationinthisgroupmust takeaccountofthefullrangeofdisabilities,ratherthanjustphysicalfunction.

The UK Rehabilitation Outcomes Collaborative (UKROC) provides the national clinical database for specialist rehabilitation in the UK. Established in 2010 with funding from the UK National Institute for Health Research (Programme grant RP-PG-0407-10185), the UKROC database collates information on needs, inputs and outcomes of all the case episodes of inpatient specialist rehabilitation of those admitted to specialist rehabilitation (Levels 1 and 2) services in England. Other UK centres participate on a voluntary basis. A national training programme is in place to ensure that clinical teams are trained in the use of the UKROC tools and outcome measures.

The dataset consists of demographic information and process data, together with a hierarchical system of outcome measurement that includes the Barthel Index (at the simplest level), the FIM and the UK FIM+FAM (at the most detailed level) [23], rated on admission and discharge.Atthestartofdatacollection,servicescouldchoosewhichofthesemeasurestoreport as an outcome measure, depending on the time that clinicians were willing/able to spend collectingthedata[23].SinceApril2013,however,reportingoftheUKFIM+FAMhasbeenmandatoryforalllevel1and2specialistrehabilitationservicesinEngland[19].

For the purpose of this analysis, we extracted all the case episodes for stroke patients in whomafullsetofUKFIM+FAMdatawascollectedonbothadmissionanddischarge.

• Sub- arachnoidhaemorrhagewasexcludedbecauseitoftencausesadiffuseinjuryandpatternof deficit,whichisatypicalwithintheusualstrokepopulation.
• TheUKROCdatasetincludesa field for primary localisation of brain injury, including left and right hemisphere, as well as bilateral, frontal brainstem and diffuse. This localisation is recorded by the treating clinical teams.
• Thedatasetdoesnotincludeinformationonneuroimaging,sowecannotexcludethe possibilitythatsomeofthelocalisationdatawasmisreported.

ExtracteddataweretransferredtoMicrosoftExcelforcleaning,andthenanalysedusingthe IBMStatisticalPackageforSocialSciences(SPSSversion21).

Measures—The UK FIM+FAM

The UK FIM+FAM consists of 30 items [9]. Each item is rated on seven levels with a score rangingfrom1—‘Totaldependence’to7—‘Completeindependence’.Nineitemsaddressbasic self-careincludingbladderandbowelmanagement;sevenitemsaddresstransfersandmobility; six items address communication, and nine items address cognitive and psychosocial function.

Scores are rated by the multidisciplinary team, according to the published scoring manual, within10daysofadmissionandwithinthelast7daysbeforedischargefromtherehabilitation programme. Rating takes approximately 20–30 minutes depending on the complexity of the caseandtheexperienceoftheteam.FurtherdetailregardingdevelopmentoftheUKversionis detailed elsewhere [9], and specific information on scoring (including the scoring manual for theUKFAMitems)maybefoundonourwebsite[24].

Analysis

The UK FIM+FAM generates ordinal data and there is continued debate about the approachto statistical analysis in this context. Some authors favour techniques based on Item Response Theory such as Rasch analysis [25] whilst others support initial evaluation using traditional psychometricapproachesbasedonClassicalTestTheory,suchasfactoranalysis[19,26].Even though they are based on parametric assumptions, principal components and factor analysis are widely used in this context and have generally been considered appropriate for the initial stage of exploring and describing the relationships among a large set of variables, even where assumptionsofnormalitymaynotstrictlyhold[27].Inthispaperwepresentedatraditionalpsychometricanalysis.

Psychometric analysis

Factor analysis is a useful tool for investigating variable relationships for complex concepts such as disability [19].. It allows the investigation of concepts that are not easily measured by collapsing a large number of variables into a few interpretable underlying factors. The key concept of factor analysis is that multiple observed variables have similar patterns of responses because they are all associated with a latent ( not directly measured) variable [26].

We debated carefully whether to use parametric or non-parametric statistical analysis. According to Altman and Bland 2009, rank methods are sometimes useful, but parametric methodsaregenerallypreferableastheyprovideestimatesandCIsandgeneralisetomorecomplexanalyses,especiallywheredatamayhavemanypossiblevalues(ie,long-ordinaldata)and samplesarelarge[28].Factoranalysisalreadyusesparametricassumptions,andforourprimary analysis,wethereforeusedparametrictechniques(ttests)forsubscaleanalyseswhere‘long ordinal’data(range28to96points)approximatedtoanormal distribution. (For completeness, an equivalent analysis using non-parametric methods can be found in the published plos one paperwhich shows that bothmethodsgavesimilarresults.)

Non-parametrictechniqueswereusedinanyevent foritem-levelanalysesthatinvolved‘shortordinal’data(range7points)thatweretypically skewed,andsowouldnotfulfilltheassumptionsofparametrictechniques.Toallowformultiple tests,thethresholdforsignificanceoftwo-sidedPvalueswastakenas0.05/numberoftests.

Part 1 analysis: psychometric evaluation.

Overall dimensionality, internal consistency and responsiveness were examined for the whole stroke population (n = 1539).

Dimensionality:FactorstructureoftheUKFIM+FAMwasexaminedfirstwithanexploratory factor analysis (EFA), and then with a confirmatory factor analysis (CFA) on a second sample.

In order to providetwosamplesthatrepresentedthefullrangeofthescale,admissionanddischargedata were first pooled and then randomly divided into approximately equal samples using the randomsampleselectionfunctionwithinSPSS.

After establishing that the two samples were broadly equivalent in terms of demographics andtotalUKFIM+FAMscores,EFAwasconductedonthefirstsampleusingaprincipalcomponentsanalysis(PCA)withVarimaxrotation.TheKeyserMyer-OlkintestandBartlett’sTest ofSphericitywereusedtoensurethatthecorrelationmatrixwassuitableforfactoranalysis.

Thedecisionastothenumberoffactorstorotatewasbasedonconsiderationofthenumberof factorswithEigenvalues>1.5andvisualinspectionofthescreeplot.Thesearewellestablished methods that usually provide clear, interpretable solutions and allow direct comparison with the results of both the previous factor analyses of the UK FIM+FAM [19, 29, 30].

CFAwasconductedonthesecondsampleusingtheAMOSsoftware.AMOSisavisualstatisticalsoftwarespecificallyusedforconfirmatoryfactoranalysis.AMOSstandsforAnalysisof MomentStructures[31].Thequalityofthemodelfitwasassessedwithfiveindices:

(i)chi-square, (ii) p value>0.5, (iii) chi-square/df, (iv) Root Mean Square Error of Approximation (RMSEA)and(v)CFI/TLI.RMSEAofbetween0.08to0.10providesamediocrefitandbelow0.08 shows a good fit. Comparative fit index/ Tucker-Lewis index CFI/TLI values range from0.00 to 1.00 for the last three indices, best fit is 0.90 or higher values [32].

Internalconsistency:Internalconsistencyinthetotalscaleandresultingsubscaleswas assessed using Cronbach’sAlpha.

Responsiveness:Theresponsiveness(changebetweenadmissionanddischarge)wasevaluated using the group comparison at both subscale- and item-level. Significance of change withineachsubscalewastestedforusingpairedt-tests.Cohen’sEffectSizewasalsocalculated as the mean score difference between admission and discharge divided by the standard deviationofadmissionscore.Item-leveldifferencesweretestedusingWilcoxonsignedranktest.

Part2analysis:Comparisonofleftandrighthemispherestrokefunctionalcharacteristics.

In the second part of our analysis, episodes were extracted for which a patient’s left or righthemispherelocalisationhadbeenclearlyidentifiedbytheratingteam(n=1154).Between-groupdifferencesinUKFIM+FAMsubscalescoreswereevaluatedatbothsubscale and itemlevel. Unpaired T tests were used to compare subscales and Mann-Whitney tests wereusedtocompareitemleveldata.

Ethics

The UKROC database collates de-identified data as part of routine clinical practice and the programme registered as a Payment by Results Improvement Project. The analysis of this routinely-collecteddataisclassedasserviceevaluation,whichdoesnotrequireresearchethicspermission in theUK.

Results

The data selection and cleaning process is summarised in figure 1

A total of 1768 stroke episodes were identified from units (n = 68) that routinely recorded the UK FIM+FAM for stroke patients during the data collection period. Of these, 1539 (87%) had complete UK FIM+FAM scores on both admission and discharge.

Fig 1.Flowchart of data extraction.

Of a total stroke population of 1768), 1539 had complete UK FIM+FAM data and were included in the part 1 psychometric analysis; 1154 had been classified as left hemisphere (n = 588) or right hemisphere (n = 566) strokes and were included in the part 2 analysis.

Table 1 shows the demographics for a) the total stroke population (n = 1768), b) the analysedstrokesamplewithcompleteUKFIM+FAMdata(n=1539),c)thoseinwhichtheclinicalteamshadspecifiedthestrokelocationaslefthemisphere(n=588)orrighthemisphere (n=566).Nosignificantdifferenceswerefoundbetweenanyofthegroups,suggestingthatthe varioussubgroupsarereasonablyrepresentativeofthewholestrokesample.

In order to providetwosamplesthatrepresentedthefullrangeofthescale,admissionanddischarge UK FIM+FAM scores were first pooled and then randomly divided into approximately equal samples using the randomsampleselectionfunctionwithinSPSS. Sample A was made up of 1550 UK FIM+FAM ratings while sample B of 1528. There were no significant between group differences in age, gender ratio, length of stay or total UK FIM+FAM scores confirming that the randomisation process had delivered two approximately equal groups.

To rule out ceiling and floor effects we have looked into the percentage of patients with FAM total scores at the ceiling (score 210) and floor (score 30). It was found that 2 patients (0.17% of total sample) had the maximum score of 210 and 2 patients (0.17% of total sample) with the minimum score of 30. This was further looked into by comparing left and right hemispheric strokes to see if these patients were skewed to one hemisphere, it was found that in left hemispheric patients there was 2 out of the total of 588 patients with the max score (0.3%) and there was 1 patient with the minimum score (0.17%). In the Right Hemispheric strokes there were no patients with max scores and 1 patient out of 566 patients (0.18%) with minimum score. These numbers are negligible and have led us to conclude that there was not a considerable ceiling and floor effect on the UKFIM+FAM in this sample.