Mango’s Guide to Financial Management for NGOs

Further Reading – Accountability & Measuring Performance

November 2005

Introduction

This note on “Further Reading – Accountability & Measuring Performance” is part of Mango’s guide to financial management for NGOs (available at

1Ebrahim, Alnoor, NGOs and Organizational Change, Cambridge 2003

2Edwards, Michael and Hulme, David, Non-Govermental Organisations – Performance and Accountability, Earthscan 1995

3Hofmann, Charles-Antoine et al, Measuring the Impact of Humanitarian Aid - A Review of Current Practice, ODI, Humanitarian Practice Group Report #17, June 2004

4Patta Scott-Villiers, The struggle for organisational change: how the ActionAid Accountability, Learning and Planning System emerged, Development in Practice, Vol 12, Numbers 3 & 4, 2002

5Wallace, Tina and Chapman, Jennifer, Some Realities Behind the Rhetoric of Downward Accountability, INTRAC 2003

Note: This is a very short selection of a few books from a large literature. Many other books and papers discuss the same issues.

1Ebrahim, Alnoor, NGOs and Organizational Change, Cambridge 2003

Ebrahim provides a realistic, well-informed analysis of the relationships between two NGOs in rural western India and donors in North America. Based on extensive field research, he recognises how NGOs work in a complex web of relationships, and how they present information in different ways (and sometimes withhold it) to different audiences.

Ebrahim demonstrates how NGOs and donors depend on each other: NGOs gain funds by providing information on “successful” projects, thereby conveying a positive reputation on their donors. NGOs have incentives not to write anything negative in their reports, because it may damage the flow of funds. Donors have incentives not to press for further information, because it may damage their reputation.

This exchange leads to structured interactions between NGOs and donors that favour short term and easily measurable activities at the expense of longer-term processes of social and political change. Unfortunately, it is a stable system, that reinforces itself without promoting high quality work in the field. This is a critical, system-wide weakness.

Ebrahim also discusses the negative effects of results-based monitoring systems, such as Logical Framework Analysis. He notes that they take a large amount of time, have a negative influence on how NGO staff think about their work, and do not provide NGO staff with information that is practically useful in their work with local communities.

2Edwards, Michael and Hulme, David, Non-Governmental Organisations – Performance and Accountability, Earthscan 1995

This comprehensive book provided a state-of-the-art review of the issues of performance and accountability in 1995, based on a wide range of case studies and the substantial field experience of the authors. It is still highly relevant today.

The book recognises the complexity of what NGOs are trying to achieve, and how difficult it is to measure their performance, which is almost always a matter of judgement and interpretation. But, written largely by NGO insiders, it is also highly critical of NGOs. It points out that there is increasing evidence that NGOs do not perform as effectively as had been assumed – or as effectively as they claim.

It describes how NGOs operate with very little accountability, and how “Northern NGO survival has been almost completely delinked from performance”, because they appear to be “under little obligation to tell the truth to their supporters”.

The book provides evidence that relationships with donors can distort accountability upwards and overemphasise linear approaches to performance-measurement, with damaging effects on the ability of NGOs to be effective catalysts for social change. It also provides evidence that downward accountability is generally weak.

The editors emphasise the importance of the quality of the relationships between NGOs, donors and governments – and of negotiation as a necessary step in improving accountability between different actors.

The editors conclude: “Improving performance-assessment and accountability is not an ‘optional extra’ for NGOs; it is central to their continued existence as independent organisations with a mission to pursue. However, judged by the meagre level of resources devoted to organisational development in these areas and the reluctance of NGOs to innovate, an independent observer would have to conclude that most were not seriously concerned about accountability at all.”

“NGOs have spent too much time criticising others and not enough time ‘putting their own house in order’.”

There is little evidence that the sector as a whole has changed much since this conclusion was written.

3Hofmann, Charles-Antoine et al, Measuring the Impact of Humanitarian Aid - A Review of Current Practice, ODI, Humanitarian Practice Group Report #17, June 2004

Available from:

This report investigates current practice in measuring and analysing the impact of humanitarian assistance. It notes that the level of scrutiny over how money is spent has increased in recent years, but that this has yet to “translate into clear improvements in the measurement of impact. … Assessment of impact is, in fact, consistently poor.”

The report recognises inherent reasons why measuring impact is difficult, including the difficulty of establishing causality, challenging operating environments, a lack of shared objectives and the fact that a great deal of humanitarian activity is preventative (for instance, trying to reduce excess mortality). In addition, external data (such as population figures) may not be available, and agencies may be working under pressure, with limited resources.

It continues to discuss different approaches to defining and measuring impact before, during and after interventions. The report recognises that agencies use a mixture of process indicators and outcome indicators.

The report comments that “it is too early to say whether the introduction of results-based management systems in humanitarian organisations will significantly improve the measurement and analysis of impact” – but that this has not happened yet. It calls for greater investment in skills and capacities to improve impact assessment.

The study concludes with a series of recommendations for improving impact analysis, including relating narrow focused evaluations to broader processes of change and engagement with affected populations and a call for greater clarity about the objectives of humanitarian assistance.

4Patta Scott-Villiers, The struggle for organisational change: how the ActionAid Accountability, Learning and Planning System emerged, Development in Practice, Vol 12, Numbers 3 & 4, 2002

Scott-Villiers describes how ActionAid, one of the UK’s largest international NGOs, has attempted to revolutionise its internal systems in recognition of the failure of bureaucratic approaches to accountability.

The paper honestly recognises weaknesses over a period of 10 years of growth, when the organisation grew from spending £20m a year to £50m. ActionAid’s previous internal system was “an upward accountability system” that requested “regular information from the field up through the hierarchy to the Trustees”. This came to be at odds to ActionAid’s community-centred approach to development: “Since the sponsors and other donors, on whose money everyone relied, were the ones to be kept happy first, the poor would never have real influence.”

The old system led to a situation where “there was a tendency for much to be written by fieldworkers that was not used, many decisions to be made by management that were avoided in the field and much energy expended which might have been better spent. People said they felt they were being pulled in two directions. Participation was increasing stress and mendacity rather than creating influence among the poor and marginalised. Bureaucracy was winning.”

Alongside this frustration, a number of factors drove fundamental change: a new strategy, a highly critical external review of the entire organisation, and visionary leadership.

The new system, introduced in 2000 and known as the “Accountability, Learning and Planning System” (ALPS), is a set of principles rather than rules or rigid procedures. It puts downward accountability first, requiring staff at all levels to spend time in face-to-face discussion with partners, the poor and one another – reflecting on the issues they are facing together. It sets out a highly decentralised way of working, explicitly cutting away unnecessary bureaucracy and requiring staff to take more responsibility for their own work.

ALPS is unproven, and may not succeed. But it is a radically new approach to issues that almost all large NGOs and donors face.

5Wallace, Tina and Chapman, Jennifer, Some Realities Behind the Rhetoric of Downward Accountability, INTRAC 2003

Now published in: “Earle, Lucy (ed), Creativity and Constraint, INTRAC 2004”.

This is a preliminary report from a study on the policies and procedures surrounding aid disbursement. Findings show that there are strong incentives to focus on upward accountability at the expense of downward accountability. NGOs also have an incentive to over-promise what they can do.

Tightening reporting requirements by donors (focused on measurable impact) and in particular the increased use of logframes hinder meaningful development practice. They are passed on to local partners and squeeze the development of different forms of civil society.

These practices are corrosive within NGOs. They take a very large amount of time and energy – and they encourage secrecy. Senior managers and board members from the private sector tend to share donors’ focus on measurable indicators.

© Mango 2005Page 1 of 3

Charity No.: 1081406