“Democracy Audits and Governmental Indicators”
American Political Science Association (APSA) Conference
Goldman School of the University of California
Berkeley, October 30-31, 2009
Assessing Democracy: Donor Rhetoric and Country-led Processess
Massimo Tommasoli
International IDEA ()
336 East 45th Street, 14 Floor, New York – NY 10017
phone +1 212 286 1084 – cellphone +1 718 737 1275
DRAFT – NOT FOR QUOTATION
1. Assessing democracy: The State of Democracy (SoD) methodology
2. Donors’ governance discourses
3. Evolving donor approaches to supporting reform
4. Donor discourses on governance indicators and assessments
5. The SoD assessment framework: Key elements
6. Applying the SoD methodology
7. Lessons learned
8. The State of Democracy approach and donor discourses
References
Appendix 1: Overview of governance assessment tools and methodologies used by aid agencies
Appendix 2: List of search questions in International IDEA’s State of Democracy assessment framework
Executive Summary
The paper addresses some of the main methodological issues arising from the use of democracy assessment frameworks in donor agencies, and the role of democracy assessments in the definition of aid strategies.
Drawing on the experience of the State of Democracy (SoD) approach developed by International IDEA, as well as on democracy assessments tools and methodologies developed by OECD/Development Assistance Committee (DAC) member states, the paper takes into account the inconsistencies between frameworks used at the same time for different purposes, such as: (a) comparing trends at global and regional levels; (b) fostering/facilitating in-country dialogue on reforms; (c) shaping donor policies on democracy assistance as well as broader aid strategies.
Though only the third purpose is explicitly about donor-driven processes, the paper will consider the role of donors with respect to the other two purposes as well. In fact, donors use democracy assessment tools for three main aims: (a) drafting aid programs on "good governance" and democracy building themes; (b) mainstreaming democracy building concerns in country assistance strategies; (c) defining aid allocations based on an assessment of democracy trends, alongside other selectivity criteria. The paper also addresses donors’ quest for “governance” indicators capable of capturing some political dimensions of development, and the methodological challenges faced in the search of relevant and comparable indicators.
The paper analyses in particular International IDEA’s democracy assessment framework, a tool for citizens self-assessment that raises public awareness about democracy, addresses popular concerns about democracy in a systematic fashion, contributes to public debate, and provides an instrument for the assessment of reforms.The framework is based on the idea that the citizens of the democracy that is being assessed should carry out the democracy assessment to provide local ownership and the political will for democratic reform.
The democracy framework is based on two fundamental democratic principles of popular control over public decision-making and decision makers, and equality of respect and voice between citizens in the exercise of that control. These fundamental principles are realised through seven mediating values, including participation, authorisation, representation, accountability, transparency, responsiveness, and solidarity.The framework has over ninety ‘search questions’ that are answered through the collection and analysis of qualitative and quantitative information across four main pillars of assessment.
The democracy assessment framework has been applied in over twenty countries including old, new, and restored democracies and has led in certain instances to the institutionalisation of democratic reform.
Application of the SoD methodology has confirmed that a clear statement of democratic values and principles upfront is critical for the credibility of an assessment tool. Its greatest strength is the local ownership of the tool, a vital element for the legitimacy of the process. The search questions, recently revised in a new practical guide, as currently formulated in the framework do not sufficiently direct attention to the distinctive traditions or culture of a country. There is need to refine the framework so that it is able to capture a dynamic as opposed to a static picture of the political system. This could be done through use of satellite indicators.
Through the SoD project IDEA set itself the ambitious objective to produce a global state of democracy report. That objective has not been attained. The SoD project has however made tangible and significant contribution to democracy dialogue and democratization through the development of a flexible democracy assessment tool that continues to gain currency in democracy building initiatives.
1. Assessing democracy: The State of Democracy (SoD) methodology
There is a long tradition of defining, measuring, and comparing democracy that dates back to Aristotle’s effort to classify ‘good’ and ‘corrupt’ forms of government. Contemporary efforts have built on this tradition through defining, measuring, and assessing democracy, democratic performance and the “quality” of democracy. Such efforts serve five main functions: (1) description, (2) classification, (3) hypothesis testing, (4) prediction and policy advice, and (5) advocacy and reform.
These functions have been variously pursued by scholars and practitioners in the fields of international relations and foreign policy, human rights and social justice, development cooperation and aid conditionality to name a few. These functions are not and cannot be mutually exclusive, but progressive and cumulative. It is not possible to pursue strategies for advocacy and reform without having carried out or reflected on predictions, dominant explanatory hypotheses and empirical analysis, attempts at classification, and rich contextual description of different democratic experiences. But despite their progressive and cumulative nature, description, classification, and hypothesis-testing are most often associated with academic analysis of the genesis, survival, and performance of democracy, while policy advice, advocacy, and reform activities are most often associated with international governmental and non-governmental organisations.
The growing optimism about democratic change that characterized the 90’s also gave rise to a new need to assess the quality and achievements of democracy. Such optimism has also driven the search for universal measures of democracy. There is a growing realization that democracy is not a linear process – one that moves from tyranny to open and plural societies. Democracy remains a hotly contested and multi-dimensional concept. The reality on the ground is that democratic transitions can be blocked, precluded, flawed or stunted for a whole range of reasons. It is therefore not an easy task to develop an assessment methodology that response to the challenges associated with democracy, can be applied to any country in the world and be responsive to the diverse conceptual underpinnings of democracy.
The International Institute for Democracy Assistance (IDEA) developed a State of Democracy (SoD) methodology that was intended to assess the state of democracy in the world. The project has been fairly successful to the extent that IDEA has made available in the public domain an assessment tool that is robust enough to be applied to different political environments and flexible enough to be adapted to a myriad of purposes. The process of developing the methodology and applying it has raised significant challenges regarding the whole enterprise of assessing democracy. More importantly however, IDEA has been able to harvest lessons learned that are being used to refine the methodology as well as enriching the discourse and debates about assessing democracy.
IDEA’s State of Democracy Methodology expanded the UK Democratic Audit’s work to make it universal in application. The framework, agreed on by an international panel of experts, is designed to measure the condition of democracy in countries from all regions of the world. It is based on the assumption that democratization is a process that is never completed; the idea of democracy is a common one and that the best people to assess its progress are a country’s own citizens. The SoD seeks qualitative answers to a set of search questions complemented by quantitative data where appropriate. It keeps the different aspects of a country’s democratic life separate on the basis that some may give greater concern than others and that they cannot simply be aggregated together. The framework is premised on a definition of democracy that emphasizes popular control of political decision making and political equality.
The methodology has been applied in different situations and for different purposes. Soon after the development of the framework IDEA pilot tested the methodology in eight countries from all continents. Subsequently the methodology has been used for different purposes that include educational and training, advocacy and dialogue, monitoring progress regarding democratization and good governance. It has also been adapted and modified to meet specific needs such as assessing democracy at a local level and responding to country and culture specific needs. Its geographical scope has been wide, traveling from Latin America to South and Central Asia, from Australia, North America and Europe to Africa.
A good starting point is a discussion and clearer understanding of the notion of assessing democracy. In the past twelve or so years assessing the state or quality of democracy has been an area of growth. Assessments are carried out for varied purposes. Quite often assessments are carried out in order to ascertain progress in democratic achievements, to examine correlations between democracy and economic conditions and to identify likely recipients of development aid.
The discourse about assessing democracy has been dominated by the search for universally acceptable democracy indicators. Subsets of this debate have been about the appropriate use of quantitative measures and qualitative judgments in measuring democratic progress. At a conceptual level the issue is further complicated by the lack of agreement about the concept of democracy. Put another way, different elements of democracy can be emphasized at any given time depending on a country’s history, culture and socio-economic development.
The need for credible assessment methodologies has been largely driven by the democracy promotion industry led by Western bilateral donors and international multilateral organizations. The sense that modern democracies are the only game in town has also fueled the need to have universally accepted frameworks for assessing democracy. The end of Cold War and the disintegration of communist Eastern Europe precipitated the internal loss of legitimacy of military regimes, single party systems and other forms of authoritarian rule. Western European and North American donors have invested heavily in democracy promotion initiatives aimed at consolidating the democratic gains of the early nineties. The euphoria that accompanied the dramatic democratic transitions of the late eighties to early nineties has waned bringing with it the need to objectively assess the level of democratic progress.
Democracy has remained a contested concept. More recently democracy has faced serious challenges associated with threats posed by terrorism and the West’s response to terrorism. Some of the gains made regarding protection of basic rights such as freedoms of association, assembly, expression and prohibitions against torture, inhuman and degrading treatment have been seriously compromised. While the velvet and orange revolutions in former communist East European states have given optimism to democratic consolidation, democratic progress remains fragile at best and faltering in most new democracies. Most newly established democracies; especially in Africa and Latin America are faced with the challenges of growing inequalities and poverty. Serious questions have begun to be asked about the instrumental value of democracy in a world that is characterized by such global inequalities. The search for home grown models of democratic governance makes the task of developing common frameworks for assessing democracy that much more difficult. The context that prevailed in the 90’s, in the aftermath of the fall of the Berlin Wall and the end of apartheid, has significantly changed. Democratic transitions and consolidation are proving to be fairly complex processes whose progress cannot be captured by simplified indices and frameworks. Yet the need to assess persists and continues to drive the development of “universal indicators”.
A major difficulty is the tension between received notions of democracy and indigenous ideological advances. This is the problem of universal vs. the particular. The former has a homogenizing tendency while the latter has the risk of being reduced to exceptionalism. Although the principles may be universal their routes of reaching the public imagination would be country specific. There is a need to strike a balance between the minimalist and maximalist conceptions of democracy.
The definitional and conceptual challenges that continue to be associated with the concepts of democracy and democratization are also reflected in the different approaches that have been developed over time to try and assess democracy.
David Beetham (2004) describes three main types of democracy assessments. The first is the comparative and quantitative assessment of democracy by social scientists whose main purpose is scientific – to identify a causal link between democracy and various economic variables. Such assessments aggregate quantitative indicators to assess co-relation with economic performance in the form of a democracy index. The second which is also comparative and quantitative are the league tables of human rights and democracy. This approach assigns an overall score for a country’s performance to chart its position relative to others. While the purposes for this type of assessment are often not spelt out they are used for a number of reasons that include as a guide to potential investors, a criterion for aid distribution or as a challenge to countries to improve their performance. The limitations of quantitative methodologies have been widely debated. While acknowledging that there are areas of political life where quantification is both appropriate and necessary (e.g. voter turn-out, gender participation rates, etc), quantification conveys an illusory impression of objectivity and precision to what are essentially qualitative judgments. A third type of democracy assessment is what is usually conducted by international and government aid agencies in order to develop strategies for providing development assistance. While the last approach seeks qualitative answers to the state of democracy of a given country it shares with the first two the fact that they are often externally driven approaches.
2. Donors’ governance discourses
Between 1995 and 2000, at the same time when IDEA’s Handbook on Democracy Assessment was developed, donors sought indicators capable of measuring progress in the achievement of a set of internationally agreed objectives, the Millennium Development Goals (MDGs) set in the 2000 Millennium Summit. Clear and explicitly measurable goals for international development by 2015 selected from the numerous commitments given at major UN global conferences and summits during the early 1990s, the eight MDGs are: (i) eradicate extreme poverty and hunger; (ii) achieve universal primary education; (iii) promote gender equality and empower women; (iv) reduce child mortality; (v) improve maternal health; (vi) combat HIV/AIDS, malaria and other diseases; (vii) ensure environmental sustainability; (viii) develop a global partnership for development. The MDGs embrace 18 specific targets. Donor and partner countries rely upon a set of 48 indicators for monitoring progress on the achievement of individual goals and targets.
The same period witnessed also the emergence and consolidation of donor discourses on good governance. As Landman and Häusermann pointed out, good governance is a systematized concept of governance, developed from a “generic reference to the overall set of relations with the public sphere to one that includes an expanding set of normative dimensions” (Landman and Häusermann, 2003:86).
Broadly defined, governance is “the exercise of economic, political and administrative authority to manage a country’s affairs at all levels. It comprises the mechanisms, processes and institutions through which citizens and groups articulate their interests, exercise their legal rights, meet their obligations and mediate their differences” (UNDP, 1997). Initially seen as “the manner in which power is exercised in the management of a country’s economic and social resources” (World Bank, 1992), the concept of governance has subsequently been related to human development: “Advancing human development requires governance that is democratic in both form and substance - for the people and by the people” (UNDP, 2002). The main normative dimensions of this concept relate to predictable, open, transparent and enlightened policy-making, the presence of a bureaucracy imbued with professional ethos, an executive arm accountable for its actions, and strong civil society institutions participating in public affairs. The role of the private sector is also increasingly underlined. Good governance requires that all these actors behave under the rule of law.
Prevailing donor discourses stress the close linkages between participatory development, democratization, good governance and human rights. In its 1993 Orientations on this subject, the OECD Development Assistance Committee (DAC) identified four areas of concern for good governance: the rule of law, public sector management, controlling corruption, and reducing excessive military expenditure. Following the global UN conferences and summits of the first half of the nineties, the links between open, democratic and accountable systems of governance and respect for human rights, and the ability to achieve sustained economic and social development has been increasingly recognized, stating that “participatory development and good governance must be central concerns in the allocation and design of development assistance” (DAC, 1995).
Despite the priority assigned to building capacity for good governance as a crucial element of an enabling environment for poverty reduction and sustainable development (UNDP, 2002), and though MDG 8 included a reference to “better governance”, the MDGs failed to embody a goal on democracy and governance issues. In the process that led to the definition of the MDGs, donor agencies could not find an agreement on a set of indicators, suitable for capturing some of the political dimensions of development, and at the same time consistent with the prevailing donor discourse good governance.