Visit Report to RuLIP, Cambodia
July 25thand 26th, 2013
Undertaken by Digpal Bahadur, APMAS project, AIT
I. Context:
RuLIP has been requesting to organize a training on SPSS and while finalizing the agenda for the same, it became clear thata) the problems they are facing with respect to reporting to IFAD and b) anticipation of getting stuck while doing the impact assessment later in the year; may not be served well with a training on SPSS and they would still be facing the same problems and bottlenecks.
In our understanding, the problem is to do with the non-alignment and inconsistency of various surveys that they have done (e.g. baseline, mid-line, annual outcome survey (AOS) and more recently Participatory Impact Assessment (PIA) tools) with the indicators in their Log Frame.
Such needs became more pressing after the mid-term review report was shared by IFAD, which clearly mentioned the weaknesses of present M&E and data on impact assessment[1].
So, in order to deal with the issues in hand and to support the RuLIP project in dealing with it, a two day meeting was organized with following objectives in mind.
II. Objectives:
- To analyze and gap identification of the baseline data
- To analyze and gap identification of the mid-term review data
- To look at the new PIA questionnaires and seeing their potential to gather data
- To identify the gap areas with respect to available data and expected indicators for impact assessment and final reporting to IFAD
- To identify the challenges in their present Log Frame w.r.t. reporting to IFAD
- Suggesting way forward options and methodology for the impact assessment
- Identifying the SPSS training needs
III. Understanding the Problems:
Meeting started with a detailed discussion on the MTR report and going through each significant observations made by the evaluators. Followed by understanding the AOSprocess, looking at the data and finding out the gaps.
1. Sampling for Data for Impact Assessment:
It became clear that IFAD wants RuLIP to go back to the same household for mid-line and end-line as was done during the base line and this is a real problem for RuLIP due to two reasons a) they have coded everything while entering the data and no reference/key has been kept to un code that data, hence they do not know which data is for which household and b) they claim many households have moved out/migrated since the baseline, so anyways they are not there.
2. Flawed Base Line Data:
Another challenge that they are facing is to do with the base line data timeing – base line is from the time of food prices spike of 2007-08[2]. So, any comparison with a relatively much stable food situation in the country is yielding many folds favorable results of the project.
3. Log frame:
It seems that they have not gone back to their Log frame frequently enough and while designing the survey tools. None of the single survey captures all or most of the indicators that they have agreed to report against to IFAD. Hence, they face problem in reporting as data to report against each indicator is not available to act as support of their achievements.
4. PIA:
Discussion was undertaken on how they are calling it participatory tool. It became clear that while the tools look like any other set of questionnaires but they collect data in a participatory way. I.e. it is not collected individually but in groups in villages and the documents are kept in the village only and they are reviewed and updated periodically by the community. However, since the PIA tools are really exhaustive, they are dealing with a practical problem of how to compile and analyze the data. This was one of their expectations from the SPSS training, that they were proposing.
5. Target Groups:
Another issue is with their target groups and grey/unclear areas of reporting requirements and their amended target group focus.For e.g. they have been advised to not continue to support the Farming Systems Improvement (FSI) groups but their Log Frame[3]still has mention of them as verification indicators.
6. AOS:
RuLIP team claimed that they have been advised by IFAD people not to conduct any more AOS but concentrate on the PIA’s. While discussing and going through each component of their evised Log Frame, AOS is still required for many indicators as Means of Verification (MoV’s).
7. Balance between Quantitative v/s Qualitative data:
One of the observation by a mission report[4] post the MTR mentions that with APMAS support the AOS gives good insights, but lacks in quantitative data to support the improvements and at times gives misleading information. We tried to understand this in detail and how to cater to this requirement.
IV. Suggested Way Forward and Action Steps:
1. Sampling for Data for Impact Assessment:
There are physical/hard copy files with filled in questionnaire of MTR and there is a code for each village; which their district level teams will be able to identify and their village level Commune Workers would know from the names written in the previous questionnaire of their village and hence they can collect the endline and PIA data from the same households.
In cases of households having migrated and not returning to villages often, it has been suggested to ask the Commune Workers to get the data from a household which has similar demographic, socio-economic and political profile. This they need to write to RuLIP in advance as a way out of the bottleneck they are facing to go back to the same households.
2. Flawed Base Line Data:
Nothing can be done to undo the data collected from the food crisis period. To go forward, RuLIP may have to ignore the baseline data and instead, consider MTR data as base line. The Mid Term Review Mission report from IFAD has suggested the same[5] and this, as RuLIP claimed, has been approved by IFAD.
3. Log frame:
Together we went through each indicator in the Log Frame and identified the source of information for each MoV’s. There are some grey areas, that RuLIP has been suggested to get clarity in writing from IFAD as Log Frame requires certain elements to be reported but RuLIP is under empression that they do not have to. This is the revised Log Frame, agreed by both parties post the MTR, so they should have in writing that they do not have to report from ‘AOS’ or about ‘FSI groups’. Revised Log Frame is a recent document and RuLIP could not show any correspondence w.r.t. dropping certain components from their radar.
4. PIA:
Digpal has agreed to prepare a template for them to solve the issue of not able to compile and analyze the data. For this purpose, all the tools are being translated into English (only one is remaining as of now), and template will be prepared once all the tools are translated.
5. Target Groups:
Covered in the point number ‘3’ of this section.
6. AOS:
RuLIP team has been suggested to gain clarity in writing from IFAD and not wait for a situation that they are asked for it and they have no time to do it.
7. Balance between Quantitative v/s Qualitative data:
The new PIA tools have been reviewed in detail and were discussed with the RuLIP team. With this tool, now enough quantitative as well as qualitative data are being generated and recorded periodically.
Annexure: 1
Kingdom of Cambodia
Rural Livelihoods Improvement Project in Kratie, Preah Vihear and Ratanakiri
IFAD Grant No. DSF8005-KH
Mid-Term Review Mission
04-21 July 2011
Aide-mémoire
A.Introduction
1.An IFAD Mission[6] was in Cambodia between 4th and 21st July 2011 to carry out an assessment of the project performance at mid-implementation, in the light of changes in the country’s policy, institutional, economic and social contexts, and to identify the project’s achievements, lessons learned and impact and to make recommendations regarding changes in the project design, management, institutional arrangements and budget allocation, accordingly.
2.An IFAD Grant of SDR 6.4 million (equivalent to approximately USD 9.218 million) under the Debt Sustainability Framework (DSF) was approved by the Executive Board of IFAD on 18th April 2007 for financing the Rural Livelihood Improvement Project in Kratie, Preah Vihear and Ratanakiri (RULIP, IFAD DSF Grant No. DSF-8005-KH). The Project Grant Agreement was signed on 28th May 2007 and became effective on 31st August 2007. The Project Completion Date is 30th September 2014 and the Grant Closing Date is 31st March 2015. In December 2007, the IFAD Executive Board approved the project as one of the projects to be directly supervised by IFAD.
3.The goal of RULIP is to improve the livelihoods of the rural poor in the Project Area and the target communes of the three provinces. The purpose is to assist 22,600 poor households in the Project Area (84 poor communes in 16 districts in the three provinces) achieve positive and sustainable impact on agricultural development.
4.Prior to field visits in the three provinces, the Mission met with the staff of the Project Support Unit (PSU) in the Ministry of Agriculture, Forestry and Fisheries (MAFF) and representatives of the Ministry of Economy and Finance (MEF) and the Ministry of Women’s Affairs (MoWA). Appendix 9 includes a list of those met in the three provinces and at the national level. The Mission would like to thank all those met in Phnom Penh and in Kratie, Preah Vihear and Ratanakiri for their assistance and full cooperation.
5.A draft Aide-memoire (AM) was discussed at a wrap-up meeting on 21st July 2011, chaired by Mr. Ouk Vuthirith, Deputy National Project Coordinator, and attended by representatives from MAFF-PSU, the Provincial Departments of Agriculture (PDAs) and Provincial Departments of Women’s Affairs) (PDoWAs) in the three provinces and those from MEF and MoWA. The current aide-memoire (AM) reflects the agreements and decisions of the meeting, and subsequently signed by H, E. Teng Lao, the National Project Coordinator.
B.Overall Assessment of Project Implementation Progress
6.Assessment at Mid-Term: RULIP has shown moderately satisfactory progress at mid-term, achieving on average about 70% of the planned physical outputs based on Appraisal targets. The cumulative expenditure from 2007 to May 2011 was USD 6,331,761 (IFAD grant of USD 6,044,200 and the Royal Cambodian Government (RGC) counterpart of USD 287,561) which was 62.0% of the total budget of USD 10,220,070 (Appendix 3). The grant disbursed during this period was USD 6,044,200 or 63.5% of the total grant of USD 9,513,000.
7.The Project’s two main impact mechanisms are the creation of farmer groups with capitalisation of Group Revolving Funds (GRFs) and provision of technical training to group members on improved production technologies. These complementary interventions are designed to address the two principal bottlenecks to improved household food security, and then to increase production and household economic growth. Success in both areas is necessary to achieve the Project’s objectives.At MTR, the establishment of farmers groups and GRFs has worked reasonably well, with 70% of Appraisal targets already met but with some concerns of the likely sustainability of around 30% of weak GRFs. The technical training has been delivered in both short and longer duration courses but has not yet achieved satisfactory adoption rates among farmers.
8.The Project has experienced significant cost overruns. According to the current financial plans, the Project is estimated to run out of resources in 2012. Additional resources would be required to complete the AWPBs for the next three years. These plans indicate a projected shortfall of approximately USD 5 million. However, IFAD can mobilise additional resources of the order of USD 2.5 million.
9.Some key achievements of the Project by MTR include: (1) the targeting approach using wealth ranking works well showing clear stratification among different farmer groups (Livelihoods Improvement Groups or LIGs, Most Vulnerable Family Groups or MVFGs and Farming Systems Improvement Groups or FSIGs); (2) introduction of the MVFGs in 2009 was necessary and useful to address the needs the poorest groups in the villages; (3) women’s participation is good in most activities and the gender mainstreaming training is showing good results; (4) setting up the group revolving funds (GRF) was well done and the experience could be built on during the remaining period; (5) training of key project staff (MAFF PSU, PDAs, PDoWAs, the District Agriculture Office or DAO) was completed in time, and performance of the Commune Extension Workers (CEWs) and Farmer Field School (FFS) trainers is generally satisfactory; (6) the technical staff understand social mobilization better; (7) there is good cooperation between national and sub-national levels, and across ministries (MAFF and MoWA); and (8) the project has helped the PDAs operationalize the Decentralization and Deconcentration (D&D) agenda which involves strong cooperation with the Commune Councils (CCs) and DAOs.
10.Follow-Up to Last Supervision Mission Recommendations: The Government has made satisfactory progress in taking action on the 2010 supervision mission’s recommendations (Appendix 2). The Project has completed 28 recommendations and has ongoing activities on 13. Six recommendations have to be discussed with the MTR mission and to be completed in 2011 onwards while 5 recommendations have not been initiated.
C.Outputs and Outcomes, by Component
Component A: Livelihoods Improvement
Group development
11.This has been the primary focus of the Component A with good progress in all three provinces. By MTR the project had set-up a total of 358 LIGs (with a total of 8734 member households of which 16% women-headed) 47 MVFs (with 609 member households, 34% women-headed) and 228 Farming System Improvement Groups (“FSIG” - 5468 households, 11% women headed). Collectively, the total of 634 farmer groups established by mid-term represented 70% of Appraisal target. The introduction of MVFs into the project in 2009/10 in response to the soaring food prices was a positive and effective change in the project.
12.LIGs and MVFs have received similar packages of support, typically consisting of three elements:
(a)A grant element. To LIGs, this is principally in-kind in the form of tools, seeds and group rice banks. For MVFs, in addition to that for LIGs, each household received a grant of USD50 for essential food and health needs (although no rice banks were provided). Grant provision to MVF households has been appropriate and well targeted.
(b)Group Revolving Funds (GRFs) have been established in all LIGs and MVFs and training provided to 3 leaders in each group, including 1 bookkeeper. Groups formed in 2008 and 2009 have now completed at least one full lending and repayment cycle while those from 2010 and 2011 will do so in 2011 and 2012, respectively. The process has worked well for the majority of groups which have achieved good repayment rates and some growth in the total fund. There is however a sizeable minority of groups (for 2008/09 groups: 32% of LIGs and 17% of MVFs with default >25%)where this process has not worked well and there has been a high level of default raising questions as to the sustainability of these groups. The problem is particularly acute in Ratanakiri where 47% of groups from 2008/09 have 25% default or more. There is broad consensus among the PDA teams and Mission that there is a strong correlation between the performance of the CEWs and the groups they are supporting. Action will therefore need to be taken to address under performance of individual CEWs where needed. The priorities for the remaining period must be to provide additional basic training to group leaders and bookkeepers to ensure they have sufficient capacity to be self-managing by the end of the project. This specifically means ensuring sound bookkeeping as well as good transparency and leadership within the groups.
(c)Training packages of 6 days initial training have been provided to all members covering 5-6 topics, followed by two sets of 3 day refresher training in the next two years. From discussions with farmers, the Mission concludes that training on GRFs and gender issues has been most effective. In contrast, technical training has been too short and covering too many topics and consequently adoption rates appear to be low. It is, however, acknowledged that in a few communes training has worked well with high adoption rates. For the remainder of the project, additional training will need to be provided to all existing LIG and MVF groups to increase technology adoption rates using similar training approaches to those already used more effectively with FSIGs and incorporating farm demonstrations into the LIG/MVF training.
13.Although no comprehensive data is available to assess the degree of adoption of new technologies by the beneficiary farmers, the Mission’s field observations and discussions suggest that there is a considerable room for improvement. The limited adoption by farmers of improved technologies is among the greatest risks to the Project achieving its target impacts and outcomes. Deepening and expanding the technical training provided to all LIG and MVF members and ensuring that it genuinely reflects members priorities, is based on participatory group learning and is delivered in time to allow at least one full season of systematic follow-uppost-training must be the highest priority for the remaining period of the Project. Without achieving a significant and widespread improvement in the adoption by farmers on the technologies on which they have been trained the project would be unlikely to achieves its main Outcome and Impact level targets.