Partners for Health Report

Kate Mackenzie Davey and Richard Giordano

16 June, 2007


Summary

The Partners for Health programme was established with three main aims. These were

1.  Partnership: working together to learn, encouraging participatory learning and reflective practice with the partners

2.  Grant management: managing funds to ensure the efficient delivery of a service in three key areas of health

3.  Evaluation: creating and sharing knowledge for the King’s Fund and wider community in order to influence policy

The programme is entering its final stages and while we cannot draw final conclusions at this stage there are some preliminary indications of the success and limitations of aspects of the programme that can be shared and considered in planning the future developments.

The programme has been effective in building close Partnerships with the funded organizations. Grant management has worked well through these Partnerships. However, while in many cases organizations have reflected on how they can make a difference to the service users and have appreciated the opportunity to consider the barriers to success in a supportive environment, the knowledge creation and evaluation has been more complex and difficult to achieve that anticipated.

The successes of the project appear to be due to the attempts by the Partner’s for Health team to make aims and decision processes explicit, the openness of the interactions in grant management and the close challenging, but supportive relationships between individual members of the team and the Partners. The difficulties have been due to contradictions between the different demands of the programme, resulting in a number of paradoxical demands and conflicts over use of limited resources, especially service delivery versus critical reflection.


Context

The aim of Partners for Health was to change the contract between funders and funded in two ways. First, the programme aimed to build a supportive relationship between funder and funded partner, encouraging individual learning and so increasing the funded organization’s capacity for development. This approach drew on previous work within the King’s Fund on management development and learning. Secondly, the programme aimed to move from standard measures of efficiency of funded partners, to working with them to evaluate the effectiveness of their approach. This approach aimed to create and capture knowledge of what did and did not work in practice and to share this knowledge with other funding bodies to build up an evidence base to support practice.

------

Evidence based practice and practice based evidence

Evidence based practice has become increasingly important for medicine, social care and organizational change. The focus on evidence is driven by greater concern about budgets and a focus on effectiveness rather than efficiency.

There are two broad approaches to knowledge. The first is to focus on the interpretation and use of existing material, either in the form of a meta-analysis of quantitative studies or, more commonly, a review of existing research literature. The second is to focus on creating new knowledge from existing practice through systematic evaluation and action research. These have been contrasted as evidence based practice and practice based evidence (Staler, 2006)

There is a general consensus on the importance of evaluation, whether in social care, medicine or organizational change. However, evidence for sound practice in evaluation and the development of a reliable knowledge base is weak in all areas.

------

Funders in areas of health and social care see their role primarily as supporting and monitoring the efficiency of the project. The focus tends to be on the implementation rather than the outcome of an intervention. The project is likely to be evaluated on criteria such as efficiency or number of clients seen rather than on more fundamental questions of impact of the intervention.

Partners for Health aims to build on the unique position of funders to nurture a knowledge base from their monitoring and development work with projects. Researchers from Management and Organizational Psychology with experience in knowledge management, organizational change and evaluation were brought in to facilitate this process.

The Partners for Health Programme

Partners for Health is developing an innovative approach to funding projects. The key aims are spelt out in the website.

“ We want to become a more active partner by developing new ways of working across the four health areas identified. Development support, with a special focus on evaluation is a key element of the Partners for Health in London programme.

Our expectation is that applicants will apply to the partners for Health programme because their desire to learn, influence and improve health is as strong as ours.”

(Partners for Health website)

The emphasis on being an “active partner” signals fundamental changes to the traditional funder–project relationship. The final paragraph, in bold, specifies the expectations of the desires of the projects. This is not just to help but specifically ‘to learn, influence and improve health’ and reinforces the idea of partnership through the emphasising that these desires should be shared with the funder.

While this appears straightforward, research suggests power relations between a funding and a funded organization may be a real barrier to the creation and, more importantly, the sharing of knowledge. It raised a number of paradoxes for the relationship between funders and the projects and for the identity of the funders. Two key tensions that the team struggled to resolve were, first, the nature of the relationship between the King’s Fund and the projects; and secondly, the nature of, and relations between, evaluation, development, learning and knowledge.

The Partners for Health Team

This section of the report will examine the difficulties faced by the funding team in addressing these aims, the methods used to try and address them and reflections on the effectiveness of these methods.

The team were aware that this was a radical change in traditional funding relationships where the evaluation tended to be in terms of level of practical activity rather than contribution to knowledge. While there was considerable excitement about the project, the team were themselves struggling to make sense of the change in grant management style. It was a change in culture.

There were two key paradoxes confronted by the team. Many of the terms were understood differently by members of the team and the particular foci for the tension were:

·  on the role of the team in managing contradictions between partnership and evaluation in its judgmental sense

·  on the definition and ownership of knowledge

There are links between these two issues that centre around the notion of power. The question that was constantly examined was the extent to which the role of members of the Partners for health team was about nurturing, supporting and developing project workers and the extent to which it concerned challenging them to produce persuasive evaluations of their work.

The term evaluation has negative, absolute judgemental associations that contradict the more social and individual aspects of partnership. This conflict is fundamental to many arguments about the nature of learning as disciplined acquisition of information as opposed to individual exploration and development.


Evaluation

The ideas of evaluation were introduced through an evaluation workshop for grant applicants. The aim of the workshop was to make the process by which grants were awarded explicit rather than based on custom and practice. The model of evaluation adopted was Pawson and Tilley’s realistic evaluation. They appeared to avoid the impossibility of carrying out controlled experiments in practice and the relativity of stakeholder based approaches.

The approach was to guide partners to making their theories explicit. When applicants outlined the service they intended to offer they were encouraged to consider how it would work, what contextual variables could be important and what outcomes might be observed to demonstrate that it had been effective. While this initially appeared to be a simple and practical approach to evaluation it proved immensely complex in application. The discussion of context, mechanism and outcome was not helpful for designing evaluation. However, in some cases it led to high levels of reflection about the nature of the planned interventions. These were exciting and testing for the individuals involved but were often lost in the return to work and the delivery of the service.

The difficulties were both in the complexity of evaluation, the novelty of being asked to consider such questions and the threat of questioning a dearly held belief about how best to help those in need. The use of realistic evaluation was also confusing for those who already had some understanding of evaluation, whether from a medical (controlled trial) or a social (stakeholder analysis) approach. Asking why an intervention might be expected to make a difference and how they could tell when it was successful was undermining for those who dedicated themselves to helping others as best they could.

In order to help clarify the mechanisms by which the intervention could work applicants were introduced to a User Pathway. The aim here was to consider how the service would be experienced by a user, what aspects of context may help or impede them and what outcomes they may hope for.

After the first year it became apparent that the language of the realistic evaluation model, as rigorously applied, was unhelpful and it was dropped. The workshops changed and the evaluation plans were developed with individual guidance and support from a member of the team.

The Partners still struggled to make progress with the evaluation. In many cases it was clear that they were attempting to guess what was wanted in order to release the funding and get on with their customary practice. While reflection and challenge was stimulating, it interfered with the daily pressures to deliver a service.

The evaluation plan was reviewed and in an attempt to move away from the disciplinary tones of evaluation was re-christened a learning plan. One difficulty that the partners struggled with was that while the Partners for Health team were interested in testing theory and exploring what did not work, the Partners themselves were more concerned to justify and defend their practice and very much did not want to share any failure either with the funder or with other partners.

Throughout the programme the Partners for Health team met to reflect on their own practice and the development of the project. The evaluation of Partner’s for Health was formative and iterative and involved developing new tools and adapting existing methods.

The nature of the relationship.

Identity: Partner or judge?

The role of an ‘active partner’ implies a change from a more disciplinary relationship to one of greater involvement and equality. The change in relationship is reinforced by the emphasis on ‘new ways of working’. However, there are tensions in the levels of equality likely between a funder offering substantial but limited support over three years and the longer term values of projects. The identity of team members was being re-constructed in relation to the partners. These difficulties and re-negotiations of role are associated with change in organization but may be especially salient in the not for profit area. Even more fundamentally, the funder wants to learn from failure while the funded want to exemplify the value of their intervention in order to continue to be seen as a good project.

While there was a powerful espoused ideal of learning and developing knowledge there was also an awareness that the partners most wanted to know what the funder wanted in order to be able to deliver. There was a concern about how far there was a learning focus and how far this was seen as paying lip service to please the funder. As one of the team asked, ” Do they buy into the learning process?” There was a rather rueful recognition of the feedback from one of the partners after the evaluation workshop which was that it was “A fabulous opportunity to learn how a funder thinks”. As one of the team observed, “They say they have a good time, but slip into old habits.” While the Partners for Health team were convinced of the radical change in the funder relationship and the benefits to all parties, the traditional funder relationship is not so easily undermined. Whatever the rhetoric, the team still had the responsibility to approve grant applications, and once approved, to clear payments. The shared interests implied by partnership may not be visible on a day to day basis and may be viewed more cynically by the funded partner (Cobb & Rubin, 2006; Craig & Manthorpe, 1999).

One of the questions the team raised was. “What is our aim? Change in evaluation or partnership with projects?” While clearly giving and with-holding funding are key aims for grant management, the image of Partners for Health made this especially salient. The difficulties of criticising the partners were felt strongly. One concern was that the King’s Fund culture was one of openness and courtesy that fitted the partnership approach and engendered happy and relaxed relationships. Evaluation changed the whole tone and was seen as having “dissipated the energy”. There was a sense of unease about challenging or criticising the partners’ evaluation proposals. The relationship shifted from Partner to “Big bad funder”. No matter how positive a change may appear, for most organisations routines are much less effort to maintain. Achieving a change from monitoring service delivery to evaluation would take a shift in the relationship. Using the frame of partnership to transmit clear goals and values presents people with something of a paradox (Sanyal, 2006)