CONCEPT NOTE

Making schools accountable: What works in the developing world?

I.Objectives

School systems in most developing countries largely fail to deliver quality primary education to poor children. Over 100 million primary age children either never enter or fail to complete primary school, and studies show that even children who do complete may be functionally illiterate and innumerate. In middle income countries, a substantial share of the middle class opts out of public education, and even among poor communities in low-income countries, there is demand for private schooling as an alternative to dismal or non-functional government schools. Policymakers across the developing world are concerned that their education systems inadequately prepare graduates for globalized economic competition.

Efforts to improve education in both the developed and developing world typically focus on providing more inputs to schools—expanding spending along existing allocation patterns. But substantial evidence has accumulated to show that increased funding is not sufficient for improved learning outcomes. Incremental funds may be allocated to inputs that have weak impacts on student learning. Hanushek (2003) has shown that a tripling of real education spending per student since 1960 in the United States has been absorbed by higher teacher salaries and lower class sizes and has had no measurable impact on either student numbers or average student learning levels. Teachers and other educationpersonnel (which typically represent 75% or more of education spending) may be poorly motivated to perform. Chaudhuryet al (2005) documented the significant extent of teacher absenteeism in developing countries; across 9 different countries, on any given day, from 11-30% of teachers were absent from their posts. System actors face incentives for private capture of system resources, or other diversions away from intended final uses. Public expenditure tracking studies of grants intended for schools in a number of different countries have documented sometimes significant leakage, or diversion from intended uses (for examples see Reinikka and Svensson 2004; Das et all 2004, and World Bank 2004).

The 2004 World Development Report Making Services Work for Poor People argued that the underlying cause of such failures in basic service delivery in developing countries is weak accountability relationships between the state, service providers and the citizens and clients they serve. In the education sector, efforts in both developed and developing countries to strengthen these accountability relationships through system reforms have been numerous. However, designs have varied considerably and there has been very little rigorous evaluation of impact. The objective of this report is to analyze the global experience with three of the most common types of accountability-promoting reforms in basic education: school based management; information provision to empower school stakeholders; and teacher contracting and incentive reforms.

These three types of reforms are commonly supported in World Bank operations and there is substantial demand from client countries for advice on the design of policies in these areas. Thisreport will draw on a new wave of impact evaluations of interventions to strengthen accountability which share carefully defined counterfactuals, credible identification strategies, longitudinal and panel data, and usually randomized design. Several of these have been supported by the World Bank, and the BNPP Trust Fund in particular. Synthesizing the existing and newly generated evidence base—from both developed and developing countries—the reportwill provide practical guidance for policymakers on key issues in the design of effective reforms to promote more accountable and effective service delivery in basic education.

Specifically, the primary objectives of reportare to

  1. Review the theoretical and empirical evidence on interventions to enhance accountability, with a particular emphasis on newer developments in the literature.
  2. Describerecent rigorous impact evaluations of those interventions.
  3. Draw out policy implications: in particular on when such accountability enhancing interventions are likely to result in improved service delivery, how much impact improved service delivery can be expected to have on student learning and other key education outcomes, and which design features of these interventions are critical for success.

There are a range of approaches to improving accountability. The report will focus on three broad types whichare often advocated in education sectors in developing countries:

  1. School-based management—the decentralization of school-level decision-making to school-level agents and away from centralized ones. The goals of programs vary, though they typically involve (i) increasing the participation of parents and communities in schools, (ii) empowering principals and teachers, (iii) building local level capacity, (iv) improving quality and efficiency of schooling, thus raising student achievement levels, and (v) by devolution of authority creating accountability mechanisms for site-based actors in order to improve the transparency of processes.
  2. Information for accountability—the generation of information about inputs, outputs and outcomes and its dissemination to local-level stakeholders. Again, the specific goals of the interventions vary, but they typically aim to (i) allow parents and students optimally match to their preferred school, (ii) enable parents and students to use information to lobby their school for performance enhancing changes to school administration and management, (iii) enable parents and students to use information to lobby their local governments for school for performance enhancing policies, and (iv) enable school administrators with objective criterion as to which schools are performing better, and which worse.
  3. Reforms of teacher contracting and pay. There are two common features of these reform efforts: (i) decentralized teacher contracting, to adapt civil-service standards to local conditions and permit local hiring—thereby potentially increasingschool-level accountability; (ii) teacher incentives—the linking of teacher pay (and other non-pecuniary rewards) to measures of effort and performance.

II.Organization of the report

The report will be organized around the three broad approaches to improvingaccountability.

-Overview(15 pages)

Similar to a World Development Report overview, this chapter will be written to be circulated as a stand-alone piece, with all major evidence, conclusions, and policy implications reflected.

-Chapter 1: Framework for accountability in education service delivery (15 pages)

Building on the accountability framework from WDR 2004, this chapter will situate the drive for accountability-enhancingreforms in the context of other interventions to improve education outcomes and will provide the conceptual underpinnings of the subsequent chapters.

-Chapter 2: School-based management (25 pages)

-Chapter 3: Information for accountability (25 pages)

-Chapter 4: Reforming teacher contracts and pay (25 pages)

These chapters will all be structured in a similar three-part way. First, they will review the rationale for each type of intervention and the underlying “theory of change” by which the intervention is expected to affect the performance of education system actors and ultimately translate into increasedstudent learning. Second, they will include a critical review of the evidence from developing and developed countries—evidence which will include the more recent case studies.Third, they will summarize the lessons learned about the conditions under which each type of intervention or reform is likely to improve outcomes, as well as the design features that are important for success.

-Chapter 5: Implications for policies aimed at enhancing service delivery and improving outcomes (15 pages)

This chapter will synthesize the global evidence on the three types of interventions. It will address the issue of when one type might be more appropriate than another, and when the interventions might have synergies and work best in tandem. It will analyze evidence on the comparative cost-effectiveness of different approaches. This chapter will be targeted to the policy-oriented reader who is seeking evidence-based guidance on the implementation ofaccountability-enhancing interventions.

III.Chapter outlines in detail

Overview

This chapter will draw from the accountability framework, the literature reviews and the case studies to provide a succinct summary of the report’s findings. It will be written in the style of a World Development Report Overview which: reflects the main arguments, evidence, and recommendations of the report; is written in an accessible style for a policy oriented audience; and can be circulated as a stand-alone publication.

Chapter 1: Framework for accountability in education service delivery: Why developing country systems fail poor children

This chapter will be structured in three parts. The first part will document failures in service delivery—and low levels of learning—in developing countries and motivate a focus on accountability. This will be set against increases in enrollment and attainment in the poorest countries, with the concomitant strain on quality, and the emphasis on global competitiveness in middle income countries, which is exposing serious deficiencies in education quality in these countries. This part of the chapter will also situate accountability in the context of a broader agenda that includes adequate financial resources; the demand for education (and for quality education); and issues such as curriculum and pedagogy.

The second part of the chapter will focus on the conceptual framework for analyzing accountability. It will draw on the theory underpinning accountability enhancing efforts in developed countries, but will likely stress the value of the WDR 2004 accountability framework. This simple framework, which can be useful for organizing and analyzing interventions to improve service delivery, summarizes the critical relationships among the various actors in the education sector as:

  1. Client Power: The ability of students and their parents—that is the users of services—tohold their local schools accountable;
  2. Voice: The ability of citizens to hold local or central government responsible for their education system;
  3. Compacts: The ability of policymakers to set up an education system which builds on the capacities and motivation of teachers and school staff, by establishing appropriate incentives for high quality service provision.

The third part of the chapter will provide a conceptual model for how changes in accountability affect changes in service delivery, and how these subsequently affect the quantity and quality of teaching and subsequently learning.

Chapter 2: School-based management (SBM)[1]

1) Rationale behind SBM

SBM projects involve the transfer of responsibility and decision-making, usually the responsibility for school operations, to a combination of principals, teachers, parents and other school community members. SBM projects aim to empower principals and teachers, or to strengthen their professional motivation, thereby enhancing their sense of ownership of the school. SBM projects emphasize local decision-making and meaningful community participation.

These school-level actors have to conform to or operate within a set of policies determined by the central government. SBM programs exist in many different forms, both in terms of who has the power to make decisions and in terms of the degree of decision-making devolved to the school level. While some programs transfer authority only to principals or teachers, others encourage or mandate parental and community participation, often as members of school committees (or school councils). In general, SBM programs transfer authority over one or more of the following activities: (i) budget allocation, the hiring and firing teachers and other school staff, (ii) curriculum development, (iii) the procurement of textbooks and other educational material, (iv) infrastructure improvements, and (v) the monitoring and evaluation of teacher performance and student learning outcomes.

In the context of developed countries, the core idea behind SBM is that the efficiency of decisionmaking and resource use will be increased if those who work in the school building and have the best information on local needs, have greater control of the management of what goes on in the building. In developing countries, the stated objective behind SBM is often less ambitious, in that it focuses mainly on involving community and parents in the school decision-making processes rather than putting them entirely in control. However, SBM, in almost all of its manifestations, involves community members in school decision-making.

The theory of action behind SBM is that--becausethese community members are usually parents of children enrolled in the school--theyhave an incentive to improve their children’s education. They can contribute to greater monitoring of school personnel,a closer match between the school’s needs and its policies, and a more efficient use of resources. These improvements in school efficiency are the potential pathway to improved student learning.

SBM can potentially have several other benefits. Under SBM, schools are typically managed more transparently, thus reducing opportunities for corruption. Also, SBM often enables parents and stakeholders with opportunities to increase their skills. In some cases, training in shared decision-making, interpersonal skills and management skills is offered to school council members so that they can become more capable participants in the SBM process and at the same time benefit the community as a whole.

But SBM cannot always be expected to be successful. For example, local democracy and political accountability are often weak in developing countries, and this could mean that elite groups monopolize control of SBM. As a result, central authority may be needed to ensure that the poor or minorities are given fair and full access to education. Also, there might be no culture of accountability in communities, in which case local stakeholders might not question those running the school, and even de jure power transferred to local stakeholders will not result de facto in increased school accountability. Alternatively, the local authorities or agents made responsible for school decision-making under the SBM reform might not have the capacity to do exercise this responsibility effectively,, which means that without large-scale capacity building at the local level, the SBM reform would be likely to fail. Finally, SBM may in fact produce higher efficiency or greater accountability pressures, but school level resources and pedagogical processes may remain so poor that learning gains will not materialize.

2) Critical review of evidence

Starting in the United States, the United Kingdom, Australia, and Canada, SBM programs have been implemented or are being developed in a number of countries, including Hong Kong, Indonesia, El Salvador, Mexico, Nicaragua, Kenya, Nepal and Paraguay. SBM has been typically implemented as a response to some crises in the educational system, for instance, in Chicago, or as a political reform to transfer power to the communities such as in El Salvador, or for teacher empowerment, or even, albeit not very often, to ensure educational quality through management techniques as in Hong Kong (Caldwell 2005). However, despite the theoretical case for SBM, and the number of places where it has been introduced, there are very few rigorous evaluations with respect to its impact.

In the United States in Dade County Florida schools there is evidence of decreased dropout and suspension rates in schools with faculty councils, but no change in test scores over a three year period (Summers and Johnson 1996). For MonroeCounty in Florida, where schools were given the authority to make budget decisions, hire and fire personnel and decide on curriculum there is no evidence of better performance in SAT (Scholastic Assessment Tests) when compared to other schools in Florida as well as rest of the United States (Summers and Johnson 1996). The ChicagoSchool reform, that transfers considerable control over budget allocations to the Local School Councils along with the power of hiring a principal for a contract period of four years, is a frequently quoted success story in SBM, however, no rigorous impact evaluation exists for the program.

Very few of the many known SBM programs in developing countries have been subject to carefully documented, rigorous impact evaluations. Even the most rigorous of the available studies use empirical strategies that are open to question. Nevertheless it is possible to list some findings about the impact of SBM based on the more rigorous analyses. Some studies find that SBM policies changed the dynamics of the school, either because parents got more involved or because teachers’ actions changed, in Nicaragua and El Salvador (King and Ozler 1998; Jimenez and Sawada 1999).

Several studies present evidence that SBM reduced repetition, failure and, to a lesser degree, dropout rates in Honduras, El Salvador, Mexico and Brazil (di Gropello and Marshall 2005; Jimenez and Sawada 2003, 1999; Gertler et al 2006; Paes de Barros and Mendonca, 1998; and Skoufias and Shapiro 2006). The studies that had access to standardized test scores presented mixed evidence (Jimenez and Sawada 2003; King and Ozler, 1998; and Sawada and Ragatz 2005).

The general finding that SBM had had positive results on some variables – mainly, repetition, failure, and attendance rates – in contrast with the mixed results in test scores could be due to the timing and strength of the particular SBM reforms. Research in the United States suggests that SBM reforms may have to be in operation for about five years before fundamental changes are seen at the school level, and only after eight years of implementation can changes be seen in more difficult to modifyindicators such as test scores.