Workshop on MDG Monitoring, Bangkok, Thailand, 14-16 January 2009 / 2009


Table of contents

Introduction

Opening addresses

Objectives of the Workshop

MDG monitoring at the sub-national level

Discrepancies between national and international data

Net enrolment ratio in primary education and literacy

Employment

Poverty

Water and sanitation

Child mortality

Working groups and plenary discussion

The New MDG Monitoring Framework

National coordination

Recommendations

Annex 1.List of participants

Annex 2.Session on sub-national monitoring: Summary of the Working Groups

Annex 3.Session on discrepancies between national and international data: Summary of the Working Groups

Annex 4.Session on relevance of MDG indicators and differences in definitions used at the national and international levels: Summary of the Working Groups

Annex 5.Summary of gaps and discrepancies between national and international data

Summary of gaps and discrepancies between national and international data

1

Workshop on MDG Monitoring, Bangkok, Thailand, 14-16 January 2009 / 2009

Introduction

  1. The Workshop on the Millennium Development Goals (MDG) Monitoring was held in Bangkokon14-16January 2009. The workshop was organised by the United Nations StatisticsDivision (UNSD) in collaboration with the United Nations Economic and Social Commission for Asia and the Pacific(ESCAP). ESCAPhosted the Workshop.
  2. Participants included representatives from28national statistics offices and governmental agencies -namelyAfghanistan, Bangladesh, Bhutan, Bolivia, Cambodia, China, India, Iran, Kazakhstan, Kiribati, Kyrgyzstan, Lao PDR, Liberia, Malaysia, Micronesia, Mongolia, Morocco, Nepal, Pakistan, Papua New Guinea, Philippines, Sri Lanka, Tanzania, Thailand, Timor-Leste, Tuvalu, Vanuatu and Viet Nam- and from international agencies –International Labour Organization (ILO), United Nations (UN), United Nations Educational Scientific and Cultural Organisation (UNESCO), United Nations Children’s Fund (UNICEF), World Bank and World Health Organization (WHO). The list of participants is available inAnnex 1.

Opening addresses

  1. Two opening addresses were given. The first address was given by Ms. Haishan Fu, ESCAP Statistics Division,who welcomed the participants and explained that the Workshop presented an opportunity (i) to improve data quality of the MDG indicators, (ii) to understand and reduce data inconsistencies between the national and international levels, (iii) to explore ways to improve metadata and (iv) to harmonize indicators at national level. Inconsistencies arise due to lack of coordination both in national statistical systems (NSS) and between national and internationallevels. To tackle data inconsistencies, it is important to improve country statistical capacity, to develop an understanding at the country level of the methodologies used by the international agencies and to assess disparities within the country. Recent national data often do not reach the international agencies in time to be included in international publications.In addition, lack of coordination in NSS also contributes to under-use of administrative records and sample surveys for many MDG indicators. Ms. Haishan Fu concluded by wishing everyone a productive Workshop.
  2. The second address was given by Ms. Francesca Perucci, United Nations Statistics Division, who expressed the concern of the international community regarding lack of data for MDG monitoring. The Inter-agency and Expert Group (IAEG)on MDG Indicators has made a lot of effort to respond to the recommendations of the UN Statistical Commission to improve data availability and quality on MDG indicators.

Objectives of the Workshop

  1. This Workshop is part of theinitiatives to implement recommendations made by international agencies and countries to improve the monitoring of MDG indicators. The objectives of the Workshop are:
  • To review and explore state-of-the-art methodologies tomonitor the MDGs at the sub-national level;
  • To understandexistingdiscrepancies between national and international data, to identify data gaps at the international level and to develop recommendations to address the gaps and discrepancies;
  • To recommend strategies for a better coordination in the national statistical systems (NSS) and betweenthe national and international systems;
  • To present the metadata of thenew MDG indicators recently adopted and review related national definitions, methods and data sources.
  1. The Recommendations of the Workshop will be reported to the UN Statistical Commission.

MDG monitoring at the sub-national level

  1. The session on MDG monitoring at the sub-national level started with two country presentations:

MDG data at sub-national level: the experience of Cambodia(by Mr. Lay Chan, National Statistical Institute ofCambodia).

The presenter listed the key sources for MDG indicators at sub-national level in Cambodia: population censuses, administrative records - like the Health Information System and Communes database - and national and ad-hoc surveys. For example, socio-economic surveys are conducted annually and provide information at the provincial and district level; sub-national data on infant mortality, water and sanitation come from the Communes database. Sub-national data are also available for other indicators. For instance, secondary enrolment are available at the provincial and district level, by urban and rural area, by gender and for disadvantaged groups. The main challenges for sub-national MDG monitoring include lack of statistical capacity andinsufficientdata quality control at local level and lack of donor support for local level projects.

MDG data collection in China (by Ms. Liu Wei, National Bureau of Statistics ofChina).

This presentation informed the Workshop about the data sources for MDG indicators at sub-national level in China. Most data fromadministrative records can be disaggregated forurban/ruralarea and by gender. For instance, administrative recordsprovide data for urban/rural area on the number of students, on population using an improved drinking water source (indicator 7.8) and prevalence of underweight children under-five years of age (indicator 1.8). Surveys and censusescan also provide data by urban/rural area and by gender, includingdata on literacy (for urban/rural area and by gender) and for proportion of population below national poverty line (for urban/rural area). However, both surveys and administrative records fail to provide reliable sub-national data on children and maternal mortality.

  1. After these two country presentations, Ms. Jessamyn Encarnation (National Statistical Coordination Board of Philippines) organized a session to train the participants on basic skills for sub-national MDG monitoring on the basis of the experience of the NSO of Philippines. The session was organized in four parts:

a)The first presentation by Ms. Encarnacion provided an insight into the Philippines experience in sub-national MDG monitoring. It showed examples of the sub-national data produced in the Philippines, the strategies adopted in the Philippines to localize MDGs (like the use of small area estimation techniques), listed sources used in the Philippines for sub-national monitoring, revieweddata availability at sub-national level, and displayed an assessment of progress towards achieving selected MDGs in each region within the Philippines. Despite the remarkable advances in sub-national MDG monitoring, some challenges remain, including budgetary constrains and the low appreciation by local government to support monitoring the MDGs.

b)The participants were then divided in five working groups and discussedthe availability of sub-national data and the use of small area estimation in their countries. The conclusions of each group are available inAnnex 2.

c)The second presentation of Ms. Encarnacion focused on small area estimation (SAE) methodology. She explained how this methodology can be used to estimate indicators for small areaswhen there are no reliable data directly available from surveys, censuses or administrative records. Small area estimation techniques usecensus data on other variables–including dummy variables like location -for the small areas of interest to improvethe survey estimates of the indicators of interest. Since the relation between different variables is likely to change with time, the census data used should not be more than three years old. Ms. Encarnacion explained that her office is currently exploring methods to deal with census data older than three years.The SAE methodology can be used to produce estimates for small geographic areas or small population groups.

The model used to calculate small area estimates includes adjustments to ensure consistency between national and small area estimates, so that aggregation of small area estimates produces a figure very close to the national estimate.

Small area estimates have been produced in the Philippines in recent years for a number of indicators and used as inputs for policy making, like determining priority municipalities and mapping the poorest families.

d)The country participants were then given the opportunity to solve an SAE exercise using real Philippine data and PovMap, a software packagedeveloped by the World Bankfor poverty mapping, available at:

The participants welcomed the training on SAE and noted the lack of capacity of many countries to build SAE models.

  1. On the basis of the discussions, the participants presented the following conclusions:

a)The most needed indicators at sub-national level are the indicators on poverty, education, health and sanitation. There is a need to strengthen the capacity of the key actors in collecting, analyzing and using sub-national data.

b)Some of the countries are producing MDG indicators at sub-national level but not for all indicators. Sub-national data for these indicators are generated from administrative records (usually for education and health indicators), censuses, and household sample surveys.

c)In some countries, the quality of sub-national data is not assured – especially from administrative records due to lack of completeness, close supervision and coordination.

d)For some indicators, the data can not be generated at sub-national level due to insufficient sample sizes. An increase in the sample size will lead to extra costs.Other issues to be considered in increasing sample size include:

  1. The capacity of enumerators and statistical methods;
  2. The frequency of collection;
  3. The coordination between agencies.

Technical assistance and funding to support larger surveys are needed.

e)The main target users of sub-national data on MDG indicators include:

  1. Local government: sub-national data can be useful for decentralizing policy and programmes. Provincial governments can use the data to make their own targets and plans to develop their area.
  2. Central government: sub-national data can be helpful in budget allocation to local governments.
  3. International organizations: as donors and supporters, they will gain from a more detailed monitoring of progress through sub-national data.
  4. Researchers.

f)Many countries are not yet using SAE but expressed their interest in doing so. International organizations need to assist countries in implementing SAE by providing financial and technical assistance and man power.

  1. The Workshop provided directions for future work on sub-national monitoring. Given the limited resources of NSOs, there is a pressing need to develop guidelines:

a)On the essential requirements (sources, data quality) to produce MDG indicators at the sub-national level.

b)To decide when is relevant to produce at MDG indicators sub-national level, according to potential uses and for which groups/areas;

c)On the optimal regularity and frequency of sub-national data;

d)To address inconsistencies between national and sub-national data. Since the data sources are typically different (e.g. administrative records are used for local estimates while the national estimates come from surveys), there may be no connection between the sub-national and national estimates.

e)On the advantages and drawbacks of each data source for sub-national monitoring.

Discrepancies between national and international data

  1. At the session on discrepancies between national and international data, representatives of ILO, UIS-UNESCO, UNICEF, WHO/UNICEF-JMP and the World Bank presented the process used by these agencies to compile data from national sources and to produce international estimates.
  2. Prior to the Workshop, the country participants were requested to provide the data available in their country for selected 18 MDG indicators. The representatives of Bangladesh and Sri Lankawere invited to compare their data with the data provided by the international agencies available on the MDG website. At the Workshop, Mr. Hoque (Bangladesh) presented his findings for the indicators on water and sanitation (MDG indicators 7.8 and 7.9) and Ms. Nigamuni (Sri Lanka) presented her findings for selected education indicators (MDG indicators 2.1 and 2.3).

Net enrolment ratio in primary education and literacy

  1. In the UIS-UNESCO presentation, Mr. Nyi Nyi Thaung (UIS-UNESCO)clarified that adjusted net enrolment ratio in primary educationis used for international monitoring and explained the difference between this indicator and the classical net enrolment ratio in primary education. While the numerator of classical net enrolment ratio only includes the children enrolled in the primary level of education, the numerator of adjusted net enrolment ratio encompasses children enrolled in both the primary and secondary level of education. Otherwise, both indicators use the same denominator: the population of primary school-age. The population data is provided by the UN Population Division and the enrolment data are received from the Ministries of Education. Regarding the literacy data, UIS uses data from censuses and household surveys. Mr. Nyi Nyi Thaung presented the full process and timeline of the production of the estimates. Once the preliminary data are available, UIS shares them with the countries before publishing the data.
  2. Ms. Nigamuni(Department of Census and Statistics of Sri Lanka) listed the different data sources used in her country for the indicators 2.1 and 2.3. There are some data discrepancies in enrolment data and in the population figures used for indicator 2.1 because: (i) Sri Lanka uses the classical net enrolment ratio while theadjusted net enrolment ratio is used at the global level; (ii) The sources of the population data are different.For indicator 2.3 (literacy), the national definition isslightly different from that used by UNESCO, but this usage did not lead to data discrepancies – the national data coincides with the international data.

Employment

  1. In her presentation, Ms. Valentina Stoevska (ILO) focused on indicator 3.2 but pointed out that similar issues are found in the other employment related MDG indicators. She gave an overview of the ILO data gathering mechanism, via annual questionnaires and country websites. ILO uses data from the following sources:

a)Labour force surveys (LFS);

b)Establishment surveys, which provide information on wages and occupationbut usually cover only particular population groups and certain occupational groups. In addition, the data only reflect paid employment;

c)Official estimates;

d)Administrative records and insurance records, which are both very limited in scope;

e)Censuses which provide comprehensive information but are not very frequent;

f)Other surveys.

  1. Metadata are also collected together with the data. ILO conducts consistency checks and countries are contacted if issues arise. However, Ms. Stoevska indicated that not all countries reply.
  2. During the ILO’s presentation, the participants were informed that the International Conference of Labour Statisticians had recently reviewed the definition of employment but decided not to change the definition because, among other reasons, countries already have experience with the current definition. However, the Conferencerecognized the need to use supplementary indicators on under-employment and labour under-utilization to compensate for the employment situation in many developing countries: as most people work at least one hour a week,unemployment rates are very low.
  3. ILO faces many challenges in the international comparisonof data due to the use of different definitions worldwide. In addition, data availability is still low, especially in Sub-Saharan Africa and Oceania.When data are not available, estimations for indicator 3.2 can be produced using auxiliary variables, like total paid employment, total employment in non-agriculture and even economicallyactive population when employment data are absent.
  4. Discrepancies between national and international data arise from theuse of different sources, different series from the same source (for instance, a labour force survey may produce series on total employment and on paid employment; and ILO may end up using one series and the country the other), different definitions and classifications, and use of estimates by ILO when national data are not available for a particular year. Ms. Stoevska provided an illustrative example of estimation of missing values forMalaysia. From the mid 1990s onwards, this country only has data on total employment. Data on this series and on paid employment in the non-agricultural sector from previous years are used to estimate paid employment in the non-agriculturalsector from mid-1990s onwards using the observed relation between the two series in previous years.

Poverty

  1. Mr.