Evaluation of the Project

Government of the State of Eritrea

Ministry of Agriculture

Food and Agriculture Organization of the United Nations

European Commission

Joint Evaluation of the Project:

Technical Support to the Ministry of Agriculture

Food/2002/003-041 (GCP/ERI/008/EC)

20 August – 10 September 2005

Preliminary Summary Report

on Major Findings, Conclusions & Recommendations

prepared by

Dr. Manfred Metz

Dr. Hans A. von Zedlitz

Ghebremedhin Haile

Asmara

8 September 2005

Abbreviations and Acronyms

APDDAgricultural Promotion and Development Department of MOA

CTACoordinator Technical Assistance

DANIDADanish International Development Assistance

ECEuropean Commission

ESUExtension Support Unit

FAOFood and Agriculture Organization

FASFarmers Advisory Service

GOEGovernment of Eritrea

GTZGerman Agency for Technical Cooperation

M&EMonitoring and Evaluation

MOAMinistry of Agriculture

NARINational Agricultural Research Inistitute

NFISNational Food Information System

NGONon-governmental organization

NUEWNational Union of Eritrean Women

OVIObjectively Verifiable Indicator

PSDPlanning and Statistics Division of MOA

RSDRegulatory Services Department of MOA

TATechnical Assistance

UNUnited Nations

UNDPUnited Nations Development Programme

WFPWorld Food Programme

The report was prepared by the consultants who accept sole responsibility for its content. The report does not necessarily reflect the opinion of the Government of Eritrea, FAO nor of the European Commission.

Joint Evaluation of the Project:

Technical Support to the Ministry of Agriculture of Eritrea

FOOD/2002/003-041 (GCP/ERI/008/EC)

Contents

page

1. Introduction

2. Background and Context

3. Assessment of Project Objectives and Design

3.1 Justification

3.2 Objectives

3.3 Project Design

3.3.1 Initial Project Design (according to project document)

3.3.2 Modification of project planning during the course of implementation

4. Assessment of Project Implementation, Efficiency and
Management

4.1 Project Budget and Expenditure

4.2 Activities and Outputs

4.3 Government Support

4.4 Project Management

4.5 Technical and Operational Backstopping

5. Assessment of Results and Effectiveness

5.1 Effects and Impact

5.2 Sustainability and Environmental Impact of Results

5.3 Gender Equity in Project Implementation and Results

5.4 Cost-effectiveness

6. Conclusions and Recommendation

6.1 Conclusions

6.2 Recommendations

7. Lessons Learned

1. Introduction

Subject of the evaluation is the project “Technical Support to the Ministry of Agriculture of Eritrea”, FOOD/2002/003-041 (GCP/ERI/008/EC). It is a two-year project funded by the European Commission with a budget of € 1,904,702. It comprises a technical assistance component, implemented by FAO, amounting to € 1,347,502, and provisions for studies and pilots, managed by the EC-Delegation, of € 557,200. The project document (EC – FAO Contract) is dated October 30, 2002, and the related project agreement between the Government of Eritrea (GOE) and FAO was signed in Dec. 03 (by FAO) respectively January 04 (by GOE).

Implementation of the project started with the assignment of the first (of three) resident technical adviser in August 2003. The project was officially launched in December 2003 and will be concluded on October 15, 2005.

The project objective is to enhance the institutional and technical capacity of the Ministry of Agriculture (MOA) to facilitate and manage the sustainable rehabilitation of the agricultural sector, thus contributing to the overall objective of “Reduced national and household food insecurity”.

The evaluation is intended, as the project draws to a close, to appraise the achievements against the objective set in the project documents as well as presented in the Inception Report and to provide recommendations to the Government, FAO and the EC on the further steps necessary to consolidate progress and ensure achievement of objectives. Any further need for external assistance will have to be identified. The Terms of Reference for the evaluation are attached as Annex 1.

The evaluation was conducted during the period August 20 – September 10, 2005, and the evaluation team composed of the three consultants:

Dr. Manfred Metz, Berlin (international consultant and teamleader)

Dr. Hans A. von Zedlitz, Frankfurt (international consultant)

Mr. Ghebremedhin Haile, Asmara (national consultant).[1]

During the evaluation, the team met with the Hon. Minister of Agriculture, Mr. Arefaine Berthe, the representatives of FAO, Dr. Admir Bay, and of EC, Mr. Stéphane Halgand, the national project coordinator, Mr. Solomon Haile, the heads of the MOA departments and representatives of other relevant institutions and organizations (World Bank, WFP, CARE International, NFIS project). Several meetings were held with the project team (M&E/data base Adviser, FAS Adviser, National Consultant for Policy Planning) and with the FAO technical support staff present during the mission (Mr. Materne Maetz and Mr. Mike Connolly). Furthermore, two field trips were made to Zoba Anseba and Zoba Debub. The detailed mission itinerary and the list of persons contacted are attached as Annex 2.

The consultants are grateful for open minded discussions, the ready sharing of information and experience by all persons contacted, and all the support they received throughout the mission. Particular thanks to the FAO office and the project team and secretary for facilitating the mission in many respects.

2. Background and Context

The project was designed in 2002, at a time when Eritrea still heavily suffered from the aftermath of the border conflict with Ethiopia, and recovery, poverty reduction and food security have become primary issues of concern on the political agenda. It was rightly perceived that agriculture, providing the basis of livelihood for the majority of the Eritrean population, has to play a pivotal role in addressing those issues, and that – in this context - the capacity of the Ministry of Agriculture (MOA) needs strengthening in various critical fields, in order to be able to effectively facilitate the process of agricultural rehabilitation and development.

Three critical fields have been identified for capacity building in the MOA: 1stPolicy Analysis and Planning; 2ndDatabase and Monitoring; and 3rdFarmers Advisory Services (FAS). Though there had been preceding initiatives in these fields[2], there remained gaps to be filled. Completion, refinement, deepening, and adjustments were required under the present project, in order to make the proposed policies and measuresimplementableand to initiate the process towards implementation. Training of MOA staff in all of these fields, to ensure effective implementation and sustainability, constituted an important project activity.

With regard to some of its components (policy analysis, data base and monitoring), the project was, to a certain extent, complementary to the already ongoing and FAO supported National Food Information System (NFIS) project.

Under the technical assistance (TA) component, three international project experts were recruited by FAO:

1)A project coordinator (CTA) and strategic planning specialist;

2)A monitoring and evaluation (M&E) specialist;

3)A specialist in agricultural extension for farmers advisory services (FAS).

Project implementation started with the arrival of the first TA expert (M&E specialist) in August 2003. The CTA/strategic planner arrived in September, and the FAS expert in November 2003. The project was officially launched at the inception workshop on December 2, 2003.

As stipulated by the project document, a Tri-Partite Steering Committee was formed, constituted by representatives of MOA, FAO and EC. The first Tri-Partite Steering Committee Meeting was held on December 2, 2003, marking the official launch of the project. A first draft inception report had been prepared and approved by the Steering Committee, subject to further comments to be considered.[3]

Planned major outputs proposed in the project document and also reflected in the Inception Report as well as initial Work Plan were:

• A Project Implementation Plan, Annual Plans, Semi-annual Implementation Reports and minutes of the Project Steering Committee Meetings.

• An Agricultural Policy and Strategy Development Status Report.

• An Agricultural Sectoral Policy and Strategy Framework.

• An Agriculture Projects Database for the MOA.

• A Monitoring and Evaluation System and Procedures for the MOA.

• Training Materials for each of the three project components.

• A Coordination Strategy and Plan for the MOA.

• Investment and Activity Plans for FAS based on the needs assessment.

• Terms of reference for identified specialised studies and pilots, to be financed through the “Special Studies and Pilots” budget managed by the EC Delegation.

During the initial project phase, the project team has organised a series of familiarization workshops for the MOA Departments, conducted a number of Zoba level briefing visits, and established working committees / task forces for each project component.

In early September 2004, the Project Co-ordinator resigned for personal reasons. Project coordination was subsequently entrusted to the Monitoring and Evaluation Specialist. Since November 2003, the work of the policy and planning component of the project has been continued with the recruitment of a Senior International Consultant and a Senior National Consultant.

Late 2004, the EC commissioned a one-week external monitoring of the project. The monitoring report emphasised the limited achievements of the project (compared to the project document) and recommended, among others, that:

• The project focuses on a limited number of activities and results that can realistically be achieved within the few months left.
• A tripartite end-of–project evaluation should be held.

The project partners approved these recommendations. A revised workplan for 2005 was prepared, and the results of the (close-to-the-) end-of-project evaluation[4] are presented in this report.

3. Assessment of Project Objectives and Design

3.1. Justification

Since independence in 1993, the Government of Eritrea has been prioritising achievement of food security and poverty reduction objectives.[5] A demonstration of renewed and reinforced efforts in this regard is the recent adoption of the Interim Poverty Reduction Strategy Paper and of the Food Security Strategy, both released in April 2004. Acknowledging the pivotal role of agriculture in achieving poverty reduction and food security objectives, and the mandate of the MOA to promote agricultural development, the MOA has initiated the process towards defining an Agricultural Sector Policy and Strategy Framework[6] and a Mid-Term Agricultural Sector Development Plan.

The capacity to plan, to monitor and to provide effective services to the farmers have been identified as critical constraints for the MOA to fulfil its role in promoting better sector performance. Therefore, the project with its three components, aiming at capacity building in these fields, is considered to be fully justified. Moreover, efforts in this regard are supportive to GOE’s focus on capacity building, on public sector reform, and to the ongoing process of decentralisation. The objective of improved public sector performance, associated with the measures of public sector reform and decentralisation, can, in fact, only be achieved through massive efforts of capacity building at all levels of the administrative hierarchy. This applies to the Government structures in general, and to the MOA structures in particular.

3.2 Objectives

The overall objective of the project is to contribute to “reduced national and household food security”. By explicitly pointing out national and household food security, the objective incorporates the two sides of the food equation: Availability and access. With regard to both aspects, agriculture plays the most essential role: Increased and ensured food availability at national level is achieved through increased overall food production, and, for most of the Eritrean households, improved access to food is achieved through increased household production and farm income.

The specific project objective “Enhanced institutional and technical capacity of the MOA to facilitate and manage the sustainable rehabilitation of the agricultural sector” will likely contribute to reduced national and household food security, as stated above, hence is s consistent with the overall objective.

The specific project objective is further specified in the planned results of the project, namely, according to the Project Document (par. 2.3) “an enhanced strategic planning, monitoring and evaluation and farmers advisory services capacity in the agricultural sector, with supporting systems developed and institutionalised, with which it[7] can more effectively direct agricultural rehabilitation.”

In summary can be stated that the project objectives, as set out as planned results and as specific and overall objectives in the project document, are consistent and relevant.

3.3 Project Design

3.3.1 Initial Project Design (according to project document)

The selection of the three project components for TA (policy planning, data base and M&E, FAS) well reflect the appropriate fields of intervention to achieve the planned project objectives. The special fund for pilots and studies is supportive to the interventions in these fields.

However, there are some shortcomings in the project design which have adversely affected the later project performance. This refers particularly to:

• The specified project results[8]: They refer partly to activities or conditions rather than results (e.g. result 1), partly they are only vaguely described. The latter issue becomes particularly obvious when the “objectively verifiable indictors” for result achievements in the Logframe Matrix are defined (especially referring to indicators related to results 4-7). The stated results related to FAS (5. in Logframe Matrix), are particularly intangible, and one of the results (inputs delivered) represents rather an important condition/assumption for result achievement than a direct result of the related project activities. Altogether, the Logframe Matrix seems to have been put together in a rush, limiting its role as planning tool and as guideline for monitoring project progress.

• A major condition, highly important for the planned project achievements, has not been explicitly set out in the project document: the counterpart issue. Though the project document bluntly states (par.3.1) that the project has been designed learning from previous interventions in the sector, being reflected in the placement within the MOA, closely counterparted, to maximise capacity development and institutional sustainability, the establishment of counterparts is nowhere put as a (pre-)condition for effective project implementation, not even in the Logframe Matrix. The FAO-MOA project agreement lacks a respective regulation as well.

• Timing: The project was planned to start in December 2002 with a duration of two years (until November 2004), including start-up activities like recruitment of local staff, preparation of plan of implementation, procurement etc. Apart from not considering an inception phase (see following point), a project duration of two years is very short, too short in the case of a capacity building project to achieve consolidated and sustainable results. No provisions have been made for a possible extension or necessary follow-ups, if required.

• Inception phase: There has been no inception phase foreseen in the project document, suggesting that the project could start full swing from the outset. Since the actual project implementation was delayed by some ten months (arrival of CTA in September 2003), the conditions had changed (new policy issues coming up, FAS activities interrupted) which called for reconsideration and adjustments of the planned project activities and results.[9]

• Target group/ beneficiaries: The project document lacks a clear definition of the beneficiaries of its interventions. It can be derived from the project objectives and implicitly understood that the final beneficiaries of the project are the farm households, the immediate target group the MOA and its staff. It remained, however, open, which department(s) of the MOA should be the primary client(s), staff at central and/or zoba- or su-zoba-evel, or whether there should be emphasis on specific areas of the country (with regard to FAS), etc. Such ambiguity has particularly affected the FAS component: During the initial phase of the project, the FAS adviser was placed into the Planning and Statistics Division (PSD) of the MOA and later, more appropriately, transferred to the Agricultural Promotion and Development Department. It also remained open whether FAS activities should concentrate on a those zobas-/sub-zobas, where FAS had been already initially introduced, or on a geographic expansion of FAS.

There is a clear sequence of activities with regard to all three components: First, Elaboration of concepts (policies, data base, M&E system, FAS approach), and second, implementation. While it is understood that the concepts to be elaborated must be realistic and implementable, given existing conditions, capacities and resources, it must have been also clear from the outset that the second step, implementation, requires certain conditions yet to be established (staffing, capacities, financial and material resources). Ideally, this sequence of activities would be reflected by the following sequence of TA inputs: Continued assignments during the elaboration phase, repeated intermittent assignments during the process of implementation, once the conditions are there. Such an approach would not only encourage the partner to establish the necessary conditions (staff, clarification of responsibilities, implementation plans, complementary measures etc.), but would also be more cost-effective (avoiding money and time wasted while waiting for conditions to effectively work to be fulfilled).

3.3.2 Modification of project planning during the course of implementation

Some modification of the project planning have been made during the course of project implementation: 1) during the inception phase, and 2) in response to the findings and recommendations of an EC monitoring mission Nov./Dec. 2004.

1) Modifications during the inception phase:

A first draft of the inception report was presented to and accepted by the first Tri-Partite Steering Committee meeting on December 2, 2005. Thus, the period from August 2003, when the first technical adviser (M&E expert) arrived until end of November can be considered as the inception phase, during which the project got established, adjustments and refinement of activity and output planning were made, and first activities for project implementation have started. The inception report[10] reflects the revised planning. In comparison to the project document, a number of modifications / specifications were made with regard to “proposed”[11] activities under the different project components, and a revised workplan was presented. The planned results or, as stated: “expected outputs” (2.3) had apparently not been substantially reviewed, compared to the project document, remained partly vaguely described, or, where specified, should proof to have been over-ambitious in what could realistically be achieved during project implementation.[12] Furthermore, no attempt has been made to revise the Logframe matrix.