Report to the Tshepong Management Committee

On a case study demonstration exploring the

Marriage of Philanthropy of Community (PoC) and the Most Significant Change (MSC), Monitoring and Evaluation Technique


November 2007-11-12

The Community Grantmaking and Social Investment Programme

Centre for Leadership and Public Values

GraduateSchool of Business

University of Cape Town

1

Table of Contents

1

1.Introduction

2.Focus

3.Feed back and Give back

4.Strategic Issues

Client selection

Secondary services

Networking, leverage and linkages

Volunteer well-being

5.Monitoring and Evaluation

6.Promotion and fundraising

Annexure 1: Significant Change Stories

Annexure 2: Filtering of patient's stories

Annexure 3: Matrix of Tshepong's services

Annexure 4: Tshepong's help circuits

Annexure 5: Tshepong's caseload by diagnostic category (September 2007)

1

1.Introduction

The Community Grantmaking and Social Investment Programme (CGSI), at the Graduate School of Business,University of Cape Town entered into a memorandum of understanding with DOCKDA in an effort pilot test new tools and techniques that could improve the practice of community grantmaking and community driven development.

DOCKDAhad an interest in design, monitoring and evaluation and the Most Significant Change approach (MSC) appealed to them ( They saw potential value for their organisation as well as their partners. The MSCtechnique collects and sorts stories about change in order to gain insight into the effect of development projects. See Box 1 below for an outline.

CGSI was interested to test the union between MSC and philanthropy of community (PoC). (See monograph the Poor Philanthropist: how and why the poor help each other (www. gsb.uct.ac.za) PoC refers to the norms and traditions of self help and mutual assistance that exist in poor communities and are an essential feature of how people meet their needs, cope, survive and advance in adversity. See Box 2 for PoC terminology.

DOCKDA introduced Tshepong to CGSI as a site for the demonstration case. The demonstration occurred over three months in late 2007: September (3 day design and planning workshop); October (5 day social action research); and November (2 days of feedback and report back). A core team emerged. It included: Susan Wilkinson Maposa – CGSI Director, Rebecca Freeth–an organisational development practitioner, Strategy works, Tish Haynes – Director of DOCKDA; Gwen Mashope & Georgina Links from DOCKDA's Kimberleyoffice and three home based care workers (HBC) and Tshepong management committee members of namely: Tebogo Molao, Nombulelo Mopeli and Doreen Nokwane. In addition, the full complement of HBC workers participated at various stages either contributing their stories and or accessing those of their clients. The Management Committee also gave of their time and insight providing valuable information on the history, evaluation, strategy and vision of Tshepong.

We would like to acknowledge the time and contribution made by Tshepong towards this Most Significant Change study. It has been a privilege for us to have the opportunity to work with everyone, and to participate in an open, frank way of sharing information. This speaks very highly to what Tshepong is doing.

Box 1
About MSC - A way of monitoring impact
Most Significant Change, an approach that gathers and selects stakeholder stories about change, was identified as the monitoring and evaluation approach to use for several reasons. It is:
  1. Appropriate where there are no base lines or indicators;
  2. Considered to be within reach of CBOs and NGO that are new to, or dealing with emergent M & E systems;
  3. A way of engaging communities in a process they are most likely to feel comfortable with: storytelling.
MSC is a novel approach to monitoring and evaluation. It gathers stories from ordinary people about the changes they have experienced and highlights the most significant of these changes. Rick Davies and Jessica Dart, who developed and tested this tool, have produced simple guides to its use, making it more accessible than most monitoring tools. It is intended to supplement other, more quantitative, approaches to monitoring, not to replace them.
In its pure form, MSC comprises nine steps:
1. Getting started: establishing champions and getting familiar with the approach
2. Establishing ‘domains of change’
3. Defining the reporting period
4. Collecting stories of change
5. Reviewing the stories
6. Providing stakeholders with regular feedback about the review process
7. Setting in place a process to verify the stories if necessary
8. Conducting secondary analysis of the stories
9. Revising the MSC process
These steps can be clustered into four more familiar action–learning phases:
Preparation (steps 1-3),
Implementation (steps 4-6)
Analysis (steps 7 & 8) and
Learning (step 9)
We adapted the MSC tool to tell us about PoC. This new tool we called PMSC.

2.Focus

In this demonstration case, CGSI wanted to know more about the relationship between external support (philanthropy for community – or PfC) and PoC (self help and mutual assistance). Furthermore we wanted to test whether the story telling method would work and whether step 2 of the MSC approach (domains of change) could be crafted to shed light on changes in local helping behaviour as a result of PfC.

Our work was guided by a central question: Does a more formal intervention (such as Tshepong's intervention to assist the terminally ill) strengthens, distort or deplete, existing and informal help within a community?

Box 2
About Philanthropy of Community
(PoC)
PoC = the relationship of “helping each other” that exists among and between people who are poor.
PfC = the relationship of help and transfer of resources from those who have more to those who have less. The source of help is usually formal and is usually, but not always, external to the community it helps.
Little is known about the relationship between PoC and PfC. We undertook the demonstration study with Tshepong to find out more about this relationship. Specifically, whether philanthropy for community strengthens or depletes philanthropy of community. And we wanted to know whether MSC could produce this information.

During the pilot PMSC story telling exercise with a group of caregivers, the three Tshepong members of the PMSC team observed that:

  • People felt free to speak. This replaced the tendency towards parallel discussions – one in the room and gossip outside.
  • The stories got people worked up – people were continuing to talk about their experiences in the cloakroom afterwards.
  • It took courage to tell some of the stories.
  • The way the exercise was introduced and unfolded made people focus. They could have told lots of peripheral stories but didn't.
  • You understand someone much better if you hear their story.
  • It unites us more than before. Usually men tell men other stories and we say that women are gossipers. We became gossipers too!
  • The stories motivated and uplifted us.
  • As caregivers, we understand each other much better from listening to each other. You think you are alone in your troubles as a carer, but hearing others' troubles encouraged me.
  • It's helping us assess and identify problems, closing gaps we never usually close.

3.Feed back and Give back

The findings of this demonstration case, as well as the methodology, tools, guidelines and learning are documented elsewhere and available to Tshepong. By June 2008, they will be compiled along with other demonstration cases into a practice relevant guideline on the use of a PoC lens for improved practice.

Products include:

  1. Introduction to PMSC
  2. PMSC Quick-Start Guide
  3. Demonstration PMSC Quick-Start
  4. PMSC Training Tool: Facilitator's Guide
  5. Demonstration Report

Products 1, 2 and 4 have been reproduced for the Tshepong Management Committee to inform future application of the MSC approach should Tshepong choose to take this route.

This report is an endeavour to give back to Tshepong by distilling out of the full report; key elements which we hope will be useful to the Management Committee. This includes the following:

  • A summary of the key findings and a crafted exercise of self reflection and ranking for the management committee to further explore the strategic implications of the MSC project.
  • A distillation of what the home based care workers told us they experienced through participating in this training and demonstration case.
  • A set of stories and vignettes which we hope that Tshepong will find useful for further internal reflection and as material to communicate the value of the work to funders or through the media.

4.Strategic Issues

Please see annexure 1 for details of stories told by Tshepong clients and their households. We have included a ranking exercise so that the Management Committee can continue to work with these stories and pull out areas of most significance. These could give a starting point for further M & E. In addition, the process surfaced three key impacts or changes for further reflection and consideration. They are summarised in Box 3.

Box3
Most Significant Changes- As seen by Tshepong Caregivers
Caregivers identified the following three most significant changes out of all the stories of significant change they and their colleagues had told:
  • Patients are more educated about their health.
  • Patients are participating more actively in their health care. This behaviour change is spreading to families and the community.
  • HIV+ patients are more open about their status – which implies that Tshepong's services impact positively on stigma.

Through the many discussions that followed storytelling by Tshepong Management Committee members, caregivers and clients, the following issues came up:

Client selection

Currently, it appears that being ill and living in specific catchments or residential areas are the main criteria to access Tshepong services. Increasingdemand on Tshepong's limited resources could present an opportunity to revisit eligibility criteria – for example informed by type of illness, degree of vulnerability or gaps not being filled by other service providers. In short, no organisation can be everything to all people. There are many variables to consider, including:

  • Who is Tshepong best positioned to service?
  • Which others in the helping network are better positioned to service?
  • What are the rights and entitlements of people seeking Tshepong's service and what should people be expected to do for themselves (i.e. their responsibility)?
  • The compliance criteria and requirements for participation in government projects etc.

Secondary services

At the Tshepong Management Committee, members had a frank conversation about the newer services - the soup kitchen and vegetable garden. Some members worried that it detracted from the core business of home based care. We appreciated the transparency with which this tension was expressed.

We understand HBC to be the core business of Tshepong and the soup kitchen and vegetable garden to be supplementary services. The stories told by patients and home based carers did not feature the soup kitchen and vegetable garden. The home based carers raised questions about why so few patients volunteer in the food garden. This would be a good opportunity to stop and assess the contribution and future of these interventions. Some of the issues to bear in mind are:

What is the trade-off between HBC working in the soup kitchen and vegetable garden?

Are there alternative ways in which Tshepong could tap into other services in the network? By doing this indirectly, it might be a good opportunity for Tshepong to demonstrate a hand-in-hand approach. What sustains these projects?

An M & E focus on impact could help to shed light on this strategic programme decision. There are at least three options for consideration: To continue; but in a different way; or to ensure that people still access resources, but via facilitated access to other service providers.

Networking, leverage and linkages

The capacity of an organisation is in part about its ability to connect with other resources - to link with, and influence, other role players. Annexure 4 details the help network that Tshepong contributes to and draws on the interests of better services to its clients. We hope that this information helps Tshepong to reflect on these linkages as part of their capacity. Further consideration of this map by the Management Committee could highlight critical relationships and focus on how best to strengthen them to address program obstacles or barriers to client satisfaction.

Two issues stand out from the patient's assessment of the service:

-suspension of food packages as well as e-pap

-need to access identify cards and social grants

These issues imply the importance of effective relationships with the departments of Social Development and Home Affairs. It might be useful to think about how Tshepong can most strategically add value to these relationships to smooth out bottlenecks to clients servicing and hence satisfaction. Is their value skilling committee members up to strengthen their capacity to understand policy, regulatory and bureaucratic frameworks, and how to engage and negotiate most effectively with government at the local level?

Volunteer well-being

We learned that there are concerns about volunteer stress and burn out by both DOCKDA and Tshepong. Acknowledging this concern, DOCKDA supported a retreat on caring for caregivers. In addition to a focus on stresses (low remuneration, transport difficulties and client “grief”,) a support strategy could tap into, leverage and reward, the more positive value that HBC workers receive. How could MSC contribute to this?

Story collection affirmed HBC’s and the contribution they make. Please find attached these stories in Annexure 1 and the filtering exercise conducted afterwards in Annexure 2. There is an opportunity to feed these stories back to all Tshepong staff as a form of acknowledgement. These stories could also inform a more formalised way of recognising their hard work.

Two home based carers who were not trained in the tool but who sat in on a morning of story telling by patients, reflected:

  • When you help people, you feel you're not recognised. Listening to the stories, I felt very happy. I [learned that I] give them that hope and trust.
  • The whole time I was walking up and down to my clients. Today, the client told me I had made a big difference in her life.

5.Monitoring and Evaluation

Tshepong maintains an impressive monthly record of quantitative data on service delivery that responds to the management information requirements of the Department of Social Development. This demonstration case offers some preliminary considerations for expanded monitoring systems that consider not only the delivery of services but also their impact. This case study offers some useful client characteristics and issues to consider that could inform a more robust M E system. These potential impact and tracking categories and issues have emerged from the lived reality of Tshepong client and volunteer stories. As such they have the potential to offer up points of change as well as indicators of success that emerge from below as opposed to above – that performance metric’s prescribed by donor needs or requirements for upward accountability. Indeed the two may share commonalities or be complementary.

Tracking impact on the client group

First, recipients of Tshepong’s support are not a homogenous group; rather two significant categories are evident and distinguish users:

  • type of illness
  • level of vulnerability

With respect to type of illness there are two broad categories – HIV/ Aids a stigma related illness and others including TB, cancer etc.Annexure 5 provides a health profile of clients on Tshepong's HBC caseload in September 2007.

Regarding livelihood and vulnerability some clients are extremely poor and marginal while other are better off. A rapid distinguishing factor appeared to be whether there was a wage earner in the household or not.

When combined these two client characteristics offer a quadrants of disaggregates or cuts that could potentially offera useful way to begin to appreciate the finer grain of client impact.

HIV/ Aids higher vulnerability / HIV/ Aids & lower vulnerability
Other illness higher vulnerability / Other illnesses lower vulnerability.

Tracking relevant issues / dimensions of change

The stories shed light on a number of issues that could be important points of change or impact to track. They play out at various levels:

Client / ender user

  • HIV/Aids related stigma
  • Co-operation as a principle of help
  • Level of client dependency on service providers and/or exercise of own agency &self help

Volunteers

  • HBC worker empowerment and leadership growth (disaggregated by gender)

Institutional

  • Capacity to convene and broker client services (referrals, indirect servicing etc)

Linking client characteristics and relevant impact issues

An M E system would link client cuts with specific issues as relevant and useful. Several tables are offered below and are illustrative of various possible impacts to monitor.

To illustrate if one wanted to monitor the impact of service delivery on client’s considerations could include the following variables or combinations:

Client/ Issue / Changes in level of stigma / Changes in dependency/ reliance / Changes in Co-operation
HIV+
Not HIV+
Vulnerable
Less vulnerable

Alternatively if the interest was at the volunteer and institutional level, considerations could take into account the following variables:

Clients/ Issue / Levels of morale burnout / Skills and capacity / Strategic direction and programmes
HBCs- volunteers
Office holders
Management Committee

6.Promotion and fundraising

As a small, locally based CBO, profiling is a major concern. We hope that some of the materials provided liberally in the annexure will be of use and feed into your ongoing fundraising and promotion efforts.

Please find the information, which we hope, you can cut and paste and use for other purposes.

Annexure 1: Client stories

Annexure 2: Filtering client stories

Annexure 3: Matrix of Tshepong's services