PERFORMANCE MEASUREMENT IN THE PUBLIC SECTOR AT NATIONAL
AND LOCAL LEVELS –SOME APPLICATIONS TO SLOVENIA
Aleksander Aristovnik
Faculty of Administration, University of Ljubljana
Abstract
The main purpose of the paper is to review and to discuss different conceptual and methodological issues related to the performance measurement in public sector at national and local levels. In particular, a composition of Public Sector Performance (PSP) and Public Sector Efficiency (PSE) indicators as well as Free Disposal Hull (FDH) and Data Envelopment Analysis (DEA) techniques are presented and then applied to selected emerging economies in Eastern Europe with a special focus on Slovenia and with some methodological suggestions for measuring (in)efficiency at local levels. The paper employs indicators relating to efficiency/effectiveness at the national as well as at the local levels in a number of major policy areas: education, health care and infrastructure. The results show that efficiency and effectiveness in Slovenia differs significantly at national (in comparisons to other selected countries). However, in general, a relatively average performance in most countries is accomplished with too many inputs so that efficiency is quite low. In other words, the selected governments (including the Slovenian government) could use less resources to attain the same outcomes if they were fully efficient. In addition, the paper points out some deficiencies relating to the employed techniques and definitions. Indeed, the applications of presented techniques are hampered by lack of suitable data to apply those techniques and the precise definition of inputs, outputs and outcomes which may significantly influence the empirical results.
- Introduction
Measuring performance has been increasingly important in public sector recently. In particular, at the time of increased pressures ob public expenditures, stemming from demographic trends and globalization, the improvement of the efficiency and effectiveness of public spending features high on the political agenda. However, for many reasons, both political and technical, performance measurements have become an integral part of relatively few governments' management or decision-making systems yet. The threat of privatization and spending cutbacks, made without due consideration for impact of these changes in the future, has certainly helped increase the interest. In addition, several other factors led to the recent focus on performance measurement such as the pervasive dissatisfaction with government employees’ unresponsiveness to the public, the dynamics of Wagner’s Law, which systematically increases the relative size of government, and hence puts pressure on the public finances, and the implementation of New Public Management paradigm. But the introduction of performance indicators into public management has been also carrying botha potential for greater effectiveness and substantial risk. It is thus necessary to unbundle the concept of performance, and review the country- and sector-specific conditions that make for public sector reform success or failure. The key determinant of success or failure is whether the changes were realistic, introduced gradually, and consistent with both the methodological complexity of the topic and the specific country realities (especially administrative capacity and the governanceregime).
The purpose of the paper is to discuss and review some previous researches on the performance and efficiency of public sector as well as different conceptual and methodological issues related to the performance measurement in public sector. In particular, a definition and measurements of efficiency and effectiveness will be discussed. Additionally, a composition of Public Sector Performance (PSP) and Public Sector Efficiency (PSE) indicators as well as Data Envelopment Analysis (DEA) is presented and then applied to Slovenia and selected EU countries. However, the focus of the paper is not on how to cut public expenditures, but rather more on investigating potential reserves to increase the value for money of public spending, i.e. how to make the most of limited public resources.[1]
The paper is organized as follows. In the next section we present a brief literature review of measuring public spending efficiency. Section 3 shows a theoretical background and empirical results of public sector performance (PSP) and efficiency (PSE) indicators as well asFree Disposal Hull (FDH) and Data Envelopment Analysis (DEA) analyses. This section also presents empirical applications of the presented techniques which are applied to Slovenia and selected countries. Section 4 points out some methodological approaches for measuring public sector performance and efficiency at local levels. Final section provides main concluding remarks.
- A Brief Literature Review of Public Spending Efficiency
The debate of the role of the public sector has shifted in recent years towards empirical assessments of the efficiency and usefulness of public sector activities. A growing academic literature has been investigating the stabilisation, allocation and distribution effects of public expenditure (see e.g.Tanzi and Schuknecht (2000); Afonso et al. (2005, 2006, 2008), Sánchez and Bermejo (2007), Mandl et al. (2008), etc.) as well as the role of rules and institutions, and the scope for privatising public sector activities (see e.g. Rodrik (2000), Strauch and von Hagen (2000), Persson and Tabellini (2001), Drake and Simper (2001), Gwartney (2002)). Most studies conclude that public spending could be much smaller and more efficient than today. However, for this to happen, governments should adopt better institutions and should transfer many non-core activities to the private sector.
Previous studies on the performance and efficiency of the public sector that appliednon-parametric methods find significant divergence of efficiency across countries.Studies include notably Fakin and Crombrugghe (1997) for the public sector, Gupta andVerhoeven (2001) for education and health in Africa, Clements (2002) for education inEurope, St. Aubyn (2003) for education spending in the OECD, Afonso et al. (2005, 2006) for public sector performance expenditure in the OECD and inemerging markets, Afonso and St. Aubyn (2005, 2006a, 2006b) for efficiency inproviding health and education in OECD countries. De Borger and Kerstens (1996), andAfonso and Fernandes (2006) find evidence of spending inefficiencies for the localgovernment sector. Additionally, Afonso et al. (2008) assess the efficiency of public spending in redistributing income. Most studies apply the Data Envelopment Analysis (DEA) methodwhile Afonso and St. Aubyn (2006a) undertook a two-step DEA/Tobit analysis, in the context of a cross-country analysis of secondary education efficiency.
Composite indicators are very often by-products of efficiency measurements since they are constructed to serve as input or output indicator.[2]Afonso et al. (2005) computed composite indicators of public sector performance and public sector efficiency for 23 industrialised (OECD) countries. Indicators suggest notable but not extremely large differences in the public sector performance across countries (with a few exceptions). Countries with the highest values for sub-indicators include Switzerland (administration and infrastructure), Japan (education), Iceland (health), Austria (distribution), Norway (economic stability) and Luxembourg (economic performance). Countries such as Luxembourg, Japan, Norway, Austria and the Netherlands report high total performance indicators. Looking at country groups, small governments (industrialised countries with public spending below 40% of GDP in 2000) on balance report better economic performance than big governments (public spending above 50% of GDP) or medium sized governments (spending between 40 and 50% of GDP). Big governments feature more even income distribution whereas small ones perform better especially in the administrative, stability and economic performance domains. These results are consistent with those found in Tanzi and Schuknecht (2000). When comparing the main economic players of today, it is noteworthy that the US and particularly Japan report above-average performance in most sub-indices and for the total PSP measure. By contrast, the EU (weighted average) performs below average. Additonally, efficiency indicators are computed weighing performance by the amount of relevant public expenditure. They found significant differences in public sector efficiency across countries. Netherlands, Australia, Finland and Luxembourg show the best values for overall efficiency. Looking at country groups, small governments post the highest efficiency amongst industrialised countries. Differences are considerable as small governments on average post a 40% higher scores than big ones.
Other authors (e.g. SCP/CERP (2004)) have tried to improve on the work by Afonso et al. (2005). The country-clusters resulted are very similar. Southern European countries present low general and educational performance, new EU member states show low general performance but high educational one, and the Northern European and Anglo-Saxon countries with high scores in both items (although the differences among countries in the educational performance are high; e.g. Luxembourg with a high macroeconomic score but fairly poor results for the effectiveness of its education system). It should be noted that, though there is considerable correlation between public sector performance in the different areas, it is by no means perfect. Countries that do well in several respects also produce poorer performances in other areas. Finland, for example, records high scores for many policy areas, but has a low score for other, such as crime. In the other side, Poland does badly in many areas, but does well on economic growth, income distribution and its school drop-out rate. Similarly, Afonso et al. (2006) computed these indicators for the new EU member states and emerging markets. They found that expenditure efficiency across new EU member states is rather diverse, especially compared to the group of top performing emerging markets in Asia. From the analysis of composite public sector performance and efficiency scores they found that countries with lean public sectors and public expenditure ratios not far from 30% of GDP tend to be most efficient.
- Assessing Efficiency in Public Sector
The measurement of efficiency generally requires: (a) an estimation of costs; (b) an estimation of output; and (c) the comparison between the two. Applying this concept to the spending activities of governments, we can say that public expenditure is efficient when, given the amount spent, it produces the largest possible benefit for the country’s population.[3]Often efficiency is defined in a comparative sense: the relation between benefits and costs in country X is compared with that of other countries. This can be done for total government expenditure, or for expenditure related to specific functions such as health, education, poverty alleviation, building of infrastructures and so on. If in country X the benefits exceed the costs by a larger margin than in other countries, then public expenditure in country X is considered more efficient. However, the measurement of public efficiency is relatively complicated as comparison and measurement of both costs and benefits may be difficult. Deficient budgetary classifications, lack of reliable data, difficulties in allocating fixed costs to a specific function, and failure to impute some value to the use of public assets used in the activity can also hamper the determination of real costs.[4]
Figure 1:Conceptual Framework of Efficiency and Effectiveness
Source: Mandl et al. (2008, p. 3).
Figure 1 illustrates the link between input, output and outcome, the main components of efficiency and effectiveness indicators.[5] The monetary and non-monetary resources deployed (i.e. the input) produce an output. For example, education spending (input) affects number of students completing a grade (output). The input-output ratio is the most basic measure of efficiency.[6] However, compared to productivity measurement, the efficiency concept incorporates the idea of the production possibility frontier, which indicates feasible output levels given the scale of operations. The greater the output for a given input or the lower the input for a given output, the more efficient the activity is. Productivity, by comparison, is simply the ratio of outputs produced to input used.
On the other hand effectiveness relates the input or the output to the final objectives to be achieved, i.e. the outcome. The outcome is often linked to welfare or growth objectives and therefore may be influenced by multiple factors (including outputs but also exogenous 'environment' factors). The effectiveness is more difficult to assess than efficiency, since the outcome is influenced political choice. The distinction between output and outcome is often blurred and output and outcome are used in an interchangeable manner, even if the importance of the distinction between both concepts is recognized. For example, the outputs of a health system are often measured in terms of the number of operations performed or days spent in a hospital. The final outcome, however, could be how many patients got well enough to return to an active life. Thus, the effectiveness shows the success of the resources used in achieving the objectives set.
3.1.Methods for Measuring Public Sector Performance and Efficiency
3.1.1.Public Sector Performance and Efficiency Indicators
The measurement of public sector performance (PSP) (defined as the outcome of public sector activities) and efficiency (PSE) (defined as the outcome relative to the resources employed) is still very limited. Afonso et al. (2005) provides a proxy for measuring public sector performance and efficiency.[7] Their paper compares the performance of the public sector and relates it to resource use. These authors use indicators relating to effectiveness and in a number of major policy areas: education, health care and infrastructure. In addition, they draw on indicators of the quality of public administration, based on survey data. Finally, the authors operationalize the conventional functions of government: distribution, stabilization and allocation (Musgrave and Musgrave, 1984, p. 184). These indicators are aggregated by means of unweighted totalling of standardised component scores. Performance is then related to resource use on two levels: in each concrete policy area, and for the public sector as a whole.
Figure 2 displays the composition of PSP indicators.[8] As to the “opportunity indicators”, administrative performance of government is measured as a composite of the following indices: corruption, red tape, quality of the judiciary, and the size of the shadow economy. The education indicator contains secondary school enrolment and the OECD educational attainment indicators in order to measure both the quantity and quality of education. The health performance indicator contains infant mortality and life expectancy. The public infrastructure indicator contains a measure of the communication and transport infrastructure quality. All these indicators change slowly so that observations every 10 years provide a good impression of changes over time except in the case of public infrastructure where period averages have been used.
Afonso et al. (2005) distinguished public sector performance (PSP), defined as the outcome of public policies, from public sector efficiency (PSE), defined as the outcome in relation to the resources employed. Assume that public sector performance (PSP) depends on the values of certain economic and social indicators (I). If there are i countries and j areas of government performance which together determine overall performance in country i, PSPi, (where PSPij = f(Ik)) we can then write:
(I)
However, public sector performance must be set in relation to the inputs used to gauge their efficiency. In order to get some values of efficiency of the state, public sector efficiency (PSE) is composed, taking into account the expenditure related to each selected sub-indicator. In this respect, the public sector performance (PSP) indicator is weighted by the relevant category of public expenditure (PEX) as follows[9]:
(II)
Figure 2:Public Sector Performance (PSP) Indicator
Source: Afonso et al. (2005, p. 10).
3.1.2.Other Parametric and Non-Parametric Methods
An alternative approach is based on the concept of efficiency frontier (productivity possibility frontier). There are multiple techniques to calculate or estimate the shape of the efficiency frontier. Most investigations aimed at measuring efficiency are based either on parametric or nonparametric methods. The main difference between the parametric and the non-parametric approach is that parametric frontier functions require the ex-ante definition of the functional form of the efficiency frontier.
A very common parametric approach is the Stochastic Frontier Analysis (SFA). It is a statistical method to fit the frontier and is based on econometric methods. This approach assumes a specific functional form for the relationship between input and output. The advantage of this method is that it is able to cover the effects of exogenous shocks, i.e. nondiscretionary factors. The model can specify the equations based on such assumptions (Mandl et al., 2008, p. 9).
On the other hand, the non-parametric approach constructs an efficiency frontier using input/output data for the whole sample following a mathematical programming method.[10] This frontier provides a benchmark by which the efficiency performance can be judged. This technique is therefore primary data-driven. Among the different non-parametric methods the Free Disposal Hull (FDH) technique imposes the fewest restrictions.[11] It follows a stepwise approach to construct the efficiency frontier. Along this production possibility frontier one can observe the highest possible level of output/outcome for a given level of input. Conversely, it is possible to determine the lowest level of input necessary to attain a given level of output/outcome. This allows identifying inefficient producers both in terms of input efficiency and in terms of output/outcome efficiency (Afonso et al., 2003, p. 18).
An alternative non-parametric technique that has recently started to be applied to public expenditure analysis is Data Envelopment Analysis (DEA).[12] DEA approach is based on a linear combination of input and outputs in order to specify the efficiency frontier. Convexity of the set of input-output combinations is assumed since this method constructs an envelope around the observed combinations. According to DEA methodology, the general relationship can be given by the following function for each country i (Afonso et al., 2006, p. 21):
Yi=f(Xi), i=1,...,n(III)
where we have Yi– a composite indicator reflecting our output measure; Xi– spending or other relevant inputs in country i. If Yi < f (xi), it is said that country i exhibits inefficiency. For the observed input level, the actual output is smaller than the best attainable one and inefficiency can then be measured by computing the distance to the theoretical efficiency frontier.