NAME OF MAIN AUTHOR / ARIF HARYANA
(DETAILED AFFILIATION[1]) / NATIONAL DEVELOPMENT PLANNING AGENCY (BAPPENAS)
NAMES OF OTHER AUTHORS
(SHORT AFFILIATION)
PRESENTING AUTHOR NAME / ARIF HARYANA
SUGGESTED NEC SUB-TOPICS[2]
MONITORING AND EVALUATION SYSTEM IN INDONESIA
- Introduction
As part of the public sector reforms, Indonesia has been enacted The Law No.17/2003 Regarding the State Financing and Law No.25/2004Regarding the National Development Planning System. These laws provide regulatory framework on the implementation of performance-based planning and budgeting, which demand performance information from the monitoring and evaluation (M&E) results. M&E become important part of the development management cycle (Figure I), since it provides information that present accountability as well as input for the improvement of next planning and budgeting period. Furthermore, the Government enacted Regulation No.39/2006 that provide legal basis for conducting M&E and mandated the Ministry of National Development Planning/National Development Planning Agency (BAPPENAS) to develop M&E system. In order to implement the mandat, the Deputy of Development Performance Evaluation (DPE) was establihed within BAPPENAS in 2007.
Figure I. PLANNING AND M&E CYCLE
The DPE has three substantive roles: to formulate and coordinate national development M&E; to monitor and assess the annual plan (RKP) and medium-term (five-year) plan (RPJMN); and to mantain the national development M&E partnerships.
The enactment of regulatory framework and institutional establishment for national M&E shows the interest and commitment of the Government of Indonesia in M&E.However, therev still many challenges in the implemention of M&E, at operational level as well as policy level.
The purpose of this paper is to share information and lesson learned from the Indonesian evaluation system and its implementation challenges.
- M&E System in Indonesia
Laws No.25/2005 describes three time horizonsof national development planning: long-term plan (RPJPN, 2005-2025), medium-term plan (RPJMN, 5 years, the current RPJMN covers 2010-2014), and annual plan (RKP). However, GR 39/2006 only mandate to conduct M&E on the RKP andRPJM. Both RKP and RPJM contains programs and activities that is implemented by line ministries. All programs conducted by all ministries in all provincesare subject to be monitored and evaluated.
e-Monev for More Timely Monitoring
The focus of monitoring on the implementation of RKP is to review the progress of projects/activities that is conducted quarterly and to measure achievement of the targeted output at year-end. It also identify the constraints and botlenecks if there exist in the implementation. This information is collected from reports that is submitted quarterly by line ministries and provincial government. Coordination meeting with line ministries as well as provincial government is conducted at least twice a year to discuss the progress and find out the way to solve the problems and constraints. The information from this mechanism is used as input for next year planning process.
There were several problems and constraints in preparing the quarterly monitoring reports. One of the main problems waslow reporting rate lack of reporting promptness due to difficulties in collecting and integrating M&E information from the program implementers around the country.There was only about 30 percentof all the Line Ministriessubmitted the reports in 2010.Reportingrate from provincial government was even less than20 percentIt was surely not a good input for BAPPENAS to analyze the national development performance toward performance based planning and budgeting.Another issue that causes the low reporting rateis the reluctance of the Line Ministries to submit the report as there wasno punishment norrewards for delivering the reports.
In order to improve the reporting rate,BAPPENAS has developed an online application system, calledthe e-Monev.The application is aimed to assist line ministries in collecting and integrating data, and submitting the report. Now, the application has been implemented and the number of line ministries who submit the report substantially increase to 65 percent in first quarter of 2013.
Another purposes of developing the e-Monev is to improve accountability. The e-Monev provides page that contain basic infomation that canbe seen by the public from everywhere. The information includes both the progress of program/activities performance and budget disbursement. The e-Monevalso provides “traffic-light” notification color on the performance progress to every program/activities, which is based on the gap between achievement and target already set. The notification can be used as warning for line ministries to improve the implementation of their program. For each Program/activities, the notification color will be as follows:
-Green: gap < 0%; the program performance meets the target;
-Yellow: gap < 25%; needs more attention and effortsto achieve its target; and
-Red:large gap(gap 25%); potentially cannot achieve its target in the end the year.
The“traffic-light” notifications are eye catching and easy to be understood. Therefore, the notifications might be a good way for BAPPENAS as well as the line ministries to monitor the progress of all the development programs.
BAPPENAS keeps improving the e-Monev to significantly help the Line Ministries and the local government in submiting their M&Ereports. BAPPENASis now developing the e-Monev for local government, and it is by this end of the year and will be tried out in the third quarter of 2013.
Based on information collected from the quarterly monitoring data, BAPPENAS independently conducted evaluation of the government annual plans (RKP). The gap analyses was used in the programs evaluation, that is comparing the output achieved by theprogramto its targets. Moreover, there was also a quadrants analysis to compare each ministry performance to others.
Evaluation of the National Medium-Term Plan (RPJMN)
The evaluation on the implementation of the RPJMN is aimed to measure the outcomes and impactof programs implemented during five years implementation of the plan. The GR No.39/2006 mandated to conduct M&E on RPJM at least oncein the last year of the implementation period of the plan. Basically, the evaluation on each program is conducted by the line ministries as a self-evaluation and the report has to be submitted to BAPPENAS.These reports and data from other sources became the particular input for BAPPENAS to do the analyses and produced the integrated evaluation report
Eventhough there is no obligation to conduct mid-term evaluation (MTE), BAPPENAS has conductedMTEof the current RPJMN in 2012.The purpose was to measure the progress achievement on the outcome or impact of the development programs, especially those programs that are considered as national priority. The MTE utilized gap analysis method, that is by comparing the outcomes or impacts achieved until mid year of 2012 with the planned or targeted outcomes and impacts. In addition, MTE also review thetrends of achievement in the last three years. Based on both gap and trend analysis on key performance indicators we can make a judgement whether the programs already achieved its target, is on track towards achieving its target, or lagged behind.Based on this judgment, weput the color notificationson each performance indicatorsof program:
-Green: target already achieved oron-track/on-trend tomeet the target;
-Yellow: needs more and effortsto achieve its target; and
-Red:difficultto achieve its target in the year-end of the RPJMN.
The eye-catching color notifications is used in order to make it easier for policy makers to grasp and understand the progress of the development programs.
For those programs which has performance indicators falls into the Yellow and Red color notifications, The MTEalso identified the constraintsand problemsthat exist and proposed solutions.
Figure II. Example of Analysis Format in MTE: Health Sector
The evaluation conducted by BAPPENAScan be considered as independent because BAPPENAS is planning institution,not a ministry or institution that execute the development programs. Therefore, the bias in analysis due to conflicting interest can be minimized. BAPPENAS also free to use data from any independent credible sources, not only data from line ministries reports.The use of credible data from independent sources, such as the Statistics Indonesia (Badan Pusat Statistik/BPS) and national as well as international independent institutions assure the credibility and quality input of the evaluation.
In practical, the information provided by the MTEwas very beneficial for policy makers, because the MTE was conducted at the right time. Performance information from MTE was available just in time when the process of planning and budgeting for the fiscal year (FY) 2014 started. The MTE results was presented by the minister and discussedin the cabinet meeting chaired by thePresident. As follow up of the meeting, the President deliver directive policy to all ministers regarding the priority program and activities for FY 2014. The President also instructed the three coordinating ministers to coordinate relevant ministries to take steps needed to deal towards improving the performance of the indicators that fall under Yellow and Red Notifications. This shows how important it is the M&E in the management cycle of development process.
Public Consultation Forum For Planning and M&E
Government should provide a public sphere to give society access to all the public activities, including in the planning and M&E practices.Article 2, Law No. 25/2004 requires and provides space for public involvement in this area.The involvement is intended to capturethe aspiration of people and increase the sense of belonging of stakeholders.In practice, public involvement is accomodate in the planning process. Civil society organizations (CSO)and academia has been playing active role in the Musrenbang,discussion forum in the planning process.
Eventhough there is no obligation to create formal forum for CSO to deliver their views in the M&E,BAPPENAS held public consultation meetings since 2012, involving participants from CSOand universities.Relevant CSO is invited to share and discuss their views regarding specific issues in the meetings. The discussion is held with academia or other independent partiesas facilitators. The forum makes M&E results become more objective. It is also expected to improve the current M&E system and methodology.
- Closing Remarks
Indonesia has realized how important the M&E is. Therefore, Indonesia commits to comprehensively implement it, both regulation and institutionalization part. With the commitment, Indonesia has enacted GR No. 39/2006 and established the DPE within BAPPENAS in 2007. However, there are challenges faced by Indonesia to improve its M&E system and capacities.
The main challenge that is faced by Indonesia is to build the capacity of the M&E practitioners. As a consequence, Indonesia needs to broaden its networking and partnerships to share knowledge and take lesson learned from other countries in implementing a better M&E system.
---oo0oo---
[1]National Development Planning Agency (BAPPENAS), Jl. Taman Suropati No.2, Jakarta 10310, INDONESIA; tel./fax: +62(21)3192-7438; email:
[2] Independence – Credibility - Use of evaluation.