Decision Making in CEA Committees, 2013-2014

Project Title

Decision making in CEA committees, 2013-2014

Names of Principal Applicant and Co-Applicants (please also include the names of the institutions and email addresses)
Professor Paula Hyde, University of Manchester
Professor Mark Exworthy, University of Birmingham
Pamela McDonald-Kuhne, Kingston University

Introduction

• Clinical Excellence Awards (CEAs) were introduced in 1948 as part of the settlement between the government and medical profession to ensure that the latter participated in the NHS.

•Now, 61% of eligible doctors receive an award. The scheme costs over £500M pa.

•The CLEAR team previously undertook an analysis of the submissions to the DDRB consultation on reform to CEAs. Further research is planned.

•This study involved an empirical study of the ways in which clinical performance is defined, interpreted and implemented in the committees allocating CEAs to medical consultants.

The work followed a preliminary documentary analysis (funded by RHUL School of Management) of responses to the consultation on the future of CEAs.

Report of Activities

Progress is reported against the four research objectives:

1. To identify themes from extant evidence relating to medical power

This objective was fully achieved. Themes were identified and formed the outline document which structured our observations of committees (Broadly, themes were; Format/content, Excellence/performance standards, purpose, process/decision-making). This analytic framework was reworked and subdivided in the light of analysis of observation data. For examples, ‘board format and content’ included; formal structure (e.g ‘lay’ members who are insiders, managers’ contributions), lay input and dominant voices and ‘define excellence’ included; over and above, attribution (how content of form can be verified, claims for individual contribution), subjectivity, words that work (positive and negative e.g. caring=good, locally good=bad) as alternatives to open praise/criticism and fair e.g. time served

1. To observe a sample of CEA committees, focusing on a priori themes

This objective was completed. We aimed to observe 18 committees and we observed 15 committees and had discussions with representatives of a further three organisations. We observed 15 committee meetings (1 national committee, 1 Royal College, 7 regional committees including ACCEA and final meetings and 6 local trust committees). Detailed discussions were also undertaken with representatives of two Royal Colleges and a further local trust. The observations were geographically spread and covered a range of trust types so local and regional observations took place across England (London, Midlands, North West and South East) in a variety of trusts (Large city teaching hospitals, urban acute hospitals, community and mental health trusts) and enabled us to gain an in-depth understanding of how CEA committees function in practice.

1. To disseminate findings to the case study sites (for triangulation), to the Department of Health (national CEA committees) and to wider academic audiences in HR and public health management

This objective is partially complete. Feedback sessions with case sites began on 24th October and will culminate in a presentation to ACCEA (the national body overseeing Clinical Excellence Awards) later in the spring. In addition, we have presented a preliminary paper to the Organisational Behaviour in Health Care conference in Copenhagen (April 2014). We have submitted a paper summarising the findings from this study to the Public Management and Governance stream of the British Academy of Management Conference in 2015 entitled “‘We are learning how to describe excellence’: Medical power in the allocation of CEAs”.

1. To prepare a larger grant application for a national funding body.

An application for a larger grant (£346,990) was submitted to the National Institute for Health Research, Health Service and Delivery Researcher-led stream on 5th February 2015. The applicants are Prof. Paula Hyde, Prof. Mark Exworthy and Pamela McDonald-Kuhne. The project builds on the preliminary work funded by the BAM researcher development grant and would progress research in this area by focusing on medical power and pay through the operation of CEAs as well as considering local variations for providing CEAs. The proposal details a 36 month study beginning in April 2016. A decision as to whether the application has been shortlisted is expected April-May 2015.

Planned Activities for the Future

1. Publications. We intend to present the paper ‘We are learning to describe excellence’: Medical power in the allocation of CEAs to the Public Management and Governance stream of the BAM conference, 2015. This paper will be developed for submission to British Journal of Management. We expect to develop a further empirical paper out of this research once the detailed analysis of all observations has been completed.

1. Further research. We await a decision from the NIHR about funding of a follow-up study (see above). We propose a study as follows;

Building on two previous exploratory studies, the aim of this research is to explore the effectiveness of local CEAs (employer-based awards) and recent locally determined alternative incentives for doctors. It will investigate the part that these incentives (including recent alternative financial and non-financial rewards) play in contributing to the performance of local NHS Trusts. The research plan involves a detailed examination of the national context for CEAs including 4 questions on an IPSOS-MORI omnibus survey to establish public attitudes to doctors pay. (We have consulted IPSOS about this). A further national element to the study is a survey of strategic and operational representatives from all English NHS Trusts about the role of CEAs in recruiting and retaining medical staff and in contributing to organisational performance. The national study will be followed by a series of eight detailed case studies examining strategic, professional and motivational issues in purposively selected contrasting case sites. Following analysis of all data, validation workshops and feedback meetings at each trust will be used to develop implications from the findings. These findings will indicate how pay for performance works with consultants and whether other forms of incentives are needed to reflect changing NHS objectives, workforce shortages and changing work patterns. The study as a whole will be guided by an advisory group and a series of academic and practitioner dissemination activities are planned. This project builds upon three conceptual strands of research: 1. strategic human resource management (S-HRM), 2. Motivation and 3.Professionalism.

i.S-HRM considers how HR practices affect organisational outcomes but less attention has been paid to the impact on employees. Hyde et al examined HR practices affecting the NHS `employee deal.’ This project extends that research by examining such `deals’ for consultants as Trusts adopt more active strategies.

ii.Motivation. Numerous studies of pay-for-performance (P4P) have examined their organisational impacts. While individual P4P for doctors has focused on GPs (through QOF), few studies have examined the effects of P4P on the extrinsic and/or intrinsic motivation of consultants.

iii.Professionalism. Self-determination theory suggests minimal impact if consultants feel ownership of CEAs and dominate CEA committees. However, medical autonomy and work performance has been challenged by managerial strategies. New forms of professionalism might be able to balance autonomy and accountability.

1. Impact We have been invited to present our findings to the national body responsible for CEAs – ACCEA in spring (date tbc). CEAs are under review and the findings from this research may affect future arrangements for performance related pay.

Were the Research Aims and Objectives met?

Yes. See ‘report of activities’ above for a full account of activity against aims and objectives. There remains further analysis of the empirical findings to do and we intend to continue this work in the coming year.

Limitations or challenges encountered

The main limitation we faced arose because most CEA meetings occur late in the year and we had a considerable wait before we could commence with the study. However, we were able to take advantage of the late running of some decision meetings from the previous year and so were able to observe awards over two different awards rounds. In addition, we had to put in considerable work to secure access and this involved more than one visit to many of the sites. We were able to cover the cost of this within the allocated budget.

Analysis of Methods

Following a rapid review of the literature and the identification of key themes; excellence/performance standards, format/content, purpose, process/decision-making) CEA committees were contacted and negotiations for access began. The aim of this study was to explore the medical profession’s power in shaping, interpreting and implementing CEAs through decision-making of CEA committees. In particular, this study sought to examine CEAs as a recognition scheme with financial implications and as a tool of performance management. Specifically, it sought to identify themes from extant evidence relating to medical power, pay and incentives, and to observe a sample of CEA committees, focusing on a priori themes (identified above).

An observational methodology allowed for examination of decision-making machinery at the final CEA award making meetings and at the regional level we were able to observe the ACCEA recommendations meeting and the final decision meetings. By observing committees at national (1), Royal College (1), regional (7) and local (6) levels we were able to observe the contrasting patterns of medical power in distinctly different types of health-care organisations. We observed CEA meetings and engaged in a series of conversations with awards’ administrators to familiarize ourselves with the organisations’ processes and policies in advance of the observations. All documentation circulated to the potential applicants was also made available to the researchers.

Data collection comprised observation of the final meeting of the local, regional, national and Royal College CEA committee meetings in which the available funds for awards were allocated. In addition, we were able to observe the ACCEA recommendation meetings that preceded final regional decision meetings in three cases. The observations lasted between 90 minutes and two and a half hours. We observed the ways in which committees were organized and run, the composition of the membership, how “excellent” performance was designated, (defined as “over and above contractual requirements” in the 5 specific domains used for assessment), and how medical, lay and managerial interests were addressed during the meeting. We used an `observational pro forma’ of anticipated topics for committee discussion (based on preliminary analysis of previous minutes and a priori themes from the literature). This pro forma ensured consistent data collection procedures across the three observers.

Extensive contemporaneous field-notes were made by the observer(s). These were later written in full and anonymised. All transcripts were then to coded and compared between researchers to ensure internal validity. This ‘framework approach’ brought together a priori and emergent themes (Spencer and Ritchie, 1994; Gale et al, 2013). This comprised: transcription, familiarisation, coding, developing analytical framework, applying framework, charting the data into the framework matrix, and interpreting the data. No NHS ethics approval was required since only NHS staff and lay committee members were observed. Approval was, however, sought and secured from Durham University Research Ethics Committee. All data was anonymised to preserve confidentiality.

This study comes with a number of limitations. The researchers were unable to determine the nature or extent of conversations between committee members outside of the formal meeting. As a result, we were not fully able to ascertain if certain trade-offs or deals for support had been negotiated in advance. However, it was clear during the observation that some members were meeting for the first time so this may have been a minor concern. Equally, like the committee, we were not able to corroborate evidence of `excellence’ presented by the doctors. Indeed, counter veiling factors were also problematic. We could not `observe’ doctors who had not applied for a CEA or those who had applied and were unsuccessful. Moreover, this lack of corroboration extends to inimical aspects of medical culture. For example, doctors who do not `fit’ into established medical culture may be ostracised from CEAs, despite the appearance of due process. The presence of observers was not totally unusual as several meetings were also observed by a member of the local negotiating committee (the body which negotiates between senior doctors and the organisation).

Analysis of Results

The data were gathered across local, regional and national award committees from March 2014 to February 2015. The main findings fell under six broad areas summarised below.

Board format and content. When convening a board it was not always clear that the membership was 50% medical, 25% lay and 25% management. Not least because of blurred roles. There was general unhappiness and uncertainty about the effectiveness and fairness of the scoring system. External sources of authority were frequently drawn upon and within meetings the medical direct or senior medic was frequently deferred to. Regional and national meetings had more consistency of membership (and therefore, perhaps, more expertise to draw upon).

Defining excellence ‘comparing apples and oranges’. Excellence is framed by 5 domains; high quality service, service development, leadership, research and innovation and teaching and training. There was some difficulty in comparing across these domains and some suggested difficulty in demonstrating excellence in particular medical specialities e.g. psychiatry and anaesthetics. It was argued that certain positions such as academic doctors were being rewarded for standard components of their job descriptions.

Purpose of CEAs. There was considerable difficulty in establishing what was ‘over and above’. Discussions concerned recruitment and retention and an extended career ladder. In addition, awards appeared to sustain relationships between clinicians and managers.

Process/decision making. There were problems with scoring and the appearance of objectivity. Different approaches were used to try to compensate for differences between scorers. Questions were raised about equity and the perception that it is easier to get awards in some geographic areas. Standards of governance varied greatly.

Uncertain future. There was much discussion about the uncertain future of the awards and the difficulty of gaining additional pay when others were having their pay cut. There was a quoted halving of national awards but no evidence of significant reductions in awards so pressures may have transferred to local committees. There was much myth and rumour in the absence of fact. The potential for appeals also affected decision making.

Differences beween committees. There were competing pressures between types of organisational committees. There was some perception that it was easier to get awards from local committees in particular areas of England. Governance and feedback mechanisms remained varied across local and national awards.

The findings are still in development as further analysis of all observation notes continues. The above provides a summary of some key elements.

Impact of Research to the wider business and management academic community

The recruitment and retention of a skilled and motivated consultant workforce is essential to the effective functioning of the NHS. Performance related pay for consultants takes the form of Clinical Excellence Awards (CEAs). They were established in 1948 at the inception of the NHS and are paid annually to over 60% of senior doctors and medical academics. Despite reforms CEAs have not been subject to any significant assessment in terms of their effects on motivation and performance. It is unclear from the literature and other extant evidence what role the CEAs play in securing a skilled and motivated consultant workforce and whether new forms of incentives are needed to reflect changing workforce, work patterns, shortage areas and to achieve NHS objectives.

Also, see impact section (point 3) of ‘planned future activities’ above.

Further research opportunities the project has highlighted

Further research opportunities include a detailed study submitted to NIHR as described under point 2. ‘planned future activites’ above.

References and Bibliography – 2 pages maximum

• Gabe, J. and Calnan, M. (eds) (2009) The new sociology of the health service. London; Routledge
• Ritchie, J. Spencer, L. (1994) Qualitative data analysis for applied policy research. In Bryman, A. and Burgess, R.G. Analysing Qualitative Data pp173-194