Mountain Partnership Secretariat Mid term assessment

Mid-Term Assessment of the

Mountain Partnership Secretariat

Heather Creech

Ken Caplan

February 13, 2007


Table of contents

A. Introduction

1. Background

2. Acronyms and Terminology

3. Expectations for the assessment

4. The Frame of Reference for the Evaluation Team

5. Methodology

6. Benchmarking against good practice in partnerships: Putting the Mountain Partnership into context

B. Evolution of the Mountain Partnership

Current action on mountains

C. Objectives of the Evaluation

1. Relevance of the project to development priorities and needs

2. Clarity and realism of the project’s development and immediate objectives

3. Quality, clarity and adequacy of project design

Oversight mechanisms and accountability to members

Relationship with the Mountain Forum

Rationale for placing the Secretariat within FAO

4. Efficiency and adequacy of project implementation

5. Project results

1. Initial observations

2. Results according to project objectives

6. Sustaining project results

7. Cost-effectiveness

D. The way forward

1. Revisit and redefine the purpose of the Mountain Partnership

2. Redefine the purpose of the Secretariat

3. Introduce measures of governance, strategy and accountability

4. Review the hosting arrangement for the Secretariat

5. Consider decentralizing Secretariat functions

6. Plan for closure within an agreed time frame

Appendix 1: Terms of Reference

Appendix 2: Informants

Appendix 3: Interview Protocol

Appendix 4: Key events in the history of the Mountain Partnership

Appendix 5: Project proposals/Concepts prepared by the Mountain Partnership Secretariat (including the Mountain Products Programme)

Appendix 6: Key documents

Appendix 7: List of Initiatives, issues and focus areas

Thematic initiatives

Geographic area initiatives

Focus areas

Mountain issues covered in general on the website

Mid-Term Assessment

of the Mountain Partnership Secretariat

A. Introduction

1. Background

The International Institute for Sustainable Development (IISD) has been invited to review and assess the Mountain Partnership Secretariat as a project managed by the Food and Agriculture Organization (FAO) within its Forestry Department, with funds from the Swiss Development and Cooperation Agency (SDC) and Italian Cooperation.

As described on the website, the Mountain Partnership is a voluntary alliance of partners dedicated to improving the lives of mountain people and protecting mountain environments around the world.

The Mountain Partnership Secretariat has been established to provide a variety of services to the Partnershipmembers, including networking and outreach, knowledge management, fundraising intelligence and brokerage services among others. The Secretariat is hosted at FAO headquarters, and consists of three full time, long term professional staff from FAO.

The structure of the project includes two major subcontracted relationships -- funds provided to third parties for selected activities in support of project objectives. These relationships are with UNEP (in particular, the Office of the Carpathian Convention based in Vienna) and the Mountain Forum (based in Kathmandu, with regional nodes in North America, Latin America, and Africa). The UNEP officer is considered to be a part time staff member of the Secretariat.

The “stakeholder group” for this evaluation therefore consists of:

  • The Mountain Partnership Secretariat staff of three: Coordinator, Programme Officer and Communications Officer
  • FAO Forestry Department
  • SDC
  • Italian Cooperation
  • UNEP
  • Mountain Forum

This is a formative evaluation, intended to provide a stock taking of progress to date and guidance on such changes in project design as may be warranted.

As stated in the Terms of Reference (Appendix 1):

The evaluation is intended to assess the relevance of the Project goals and objectives, its achievements and results so far and to provide recommendations to the Governments of Switzerland and Italy, FAO and UNEP on the further steps necessary to consolidate progress and ensure achievement of objectives during the present phase. It will also assess the need for follow-up and identify any necessary changes in the overall design and orientation of the Project.

This evaluation has been conducted by a team of two external experts in network and partnership management:

  • Heather Creech (Team Leader): Director, Knowledge Communications, IISD
  • Ken Caplan: Executive Director, Building Partnerships for Development in Water and Sanitation

2. Acronyms and Terminology

As the Terms of Reference describes those registered with the Mountain Partnership as “members” of the Partnership, we have chosen to use the term “members” rather than “partners”.

The term “Stakeholder group” is used to encompass the core set of relationships: Secretariat, SDC, Italian Cooperation, FAO, UNEP, Mountain Forum.

Acronyms

BPD / Building Partnerships for Development in Water and Sanitation
CSD / Commission on Sustainable Development
FAO / Food and Agriculture Organization
FAO SARD-M / FAO Sustainable Agriculture and Rural Development - Mountains
GAN / Global Action Network
GEF / Global Environment Facility
ICIMOD / International Centre for Integrated Mountain Development
IISD / International Institute for Sustainable Development
IUCN / World Conservation Union
MF / Mountain Forum
MP / Mountain Partnership
MP SARD-M / MP initiative: Sustainable Agriculture and Rural Development - Mountains
MPS / Mountain Partnership Secretariat, or “the Secretariat”
MRI / Mountain Research Initiative
SDC / Swiss Agency for Development and Cooperation
TCP / Technical Cooperation Programme (FAO)
UNCED / United Nations Conference on Environment and Development
UNEP / United Nations Environment Programme
WSSD / World Summit on Sustainable Development

3. Expectations for the assessment

The stakeholder group seeks a neutral assessment that will provide some insight into whether the objectives for this project are in fact the “right” objectives, whether sufficient results have been achieved, and whether there is a relationship between performance and institutional arrangements (oversight and financing) and structures (secretariat roles and responsibilities).

4. The Frame of Reference for the Evaluation Team

While they each have experience in traditional evaluations, it should be noted that both members of the evaluation team bring a different perspective to network and partnership evaluation. Partnerships are different from most traditional projects given their multiple (and more horizontal) accountabilities, and their need for flexible and opportunistic approaches to structure and incentives for engagement and interventions.

A number of tensions become evident as a partnership approach evolves. Towards this end, and given the stage in the MP’s life-cycle, the evaluators have been primarily interested in better understanding what would make the greatest contribution to improving the effectiveness of the Mountain Partnership and the ability of and scope for the MP Secretariat to best serve this interest. Our interests were in forging the assessment process as part of a facilitated conversation between the primary stakeholders on how best to bolster areas where the work of the MPS could become even more effective and to address some of the weaknesses in the structures of and support mechanisms for the MPS. Though perhaps not employing a traditional project evaluation approach, the evaluation team understood that they were brought on board precisely for this particular frame of reference and expertise.

5. Methodology

Four evaluation tools were used for this assessment:

  • Documentation review
  • Structured interviews (in person and by telephone) with a sample of informants from the membership of the Mountain Partnership
  • Workshop with stakeholder group
  • Benchmarking against partnership and network practice

These tools were deployed as follows:

  • Preliminary briefings by the Secretariat and SDC
  • Documentation review
  • Structured in person interviews with representatives of the stakeholder group, in Rome (Secretariat, FAO, UNEP, Italian Cooperation) and Berne (SDC)
  • Structured phone interviews with representatives of the stakeholder group (Mountain Forum and Government of Italy’s UN office)
  • Twogroup meetings with Secretariat staff
  • First group of structured phone interviews with selected members of the Partnership
  • Informants selected by the evaluation team with a view to securing greater regional input from members in Latin America, Francophone and Anglophone Africa, Asia and the Pacific.
  • Informants nominated by SDC
  • Benchmarking: review of data gathered against partnership and network practice
  • Draft report
  • Workshop with the stakeholder group to present an overview of good partnership practice,and to review as a group the original objectives of the project, for greater clarity among the group
  • Additional group meeting with Secretariat staff
  • Second group of phone interviews with selected members of the Partnership
  • Informants nominated by the Secretariat
  • Further benchmarking against current practice
  • Final report

The original evaluation design proposed by the evaluation team included a survey of all members of the Mountain Partnership. However, the team learned that at least three surveys had been conducted by the Secretariat in the recent past (for varying purposes), the most recent of which had a very low response rate [below 5%].The team was concerned that findings from another survey with a low response rate would not be generalizable to the membership as a whole and would therefore be of little value; and further, that a low response would give a negative signal about the support of members for the Partnership. Also, as the in-person interviews progressed, the evaluators recognized the need for informants to be able to speak freely about their hopes and concerns for both the Partnership and the Secretariat, which they might not have been as prepared to do in written responses.

We therefore focused on collecting input through the structured interview process. Including the stakeholder group, we targeted approximately 30% of the Partnership. In the first group of interviews, the evaluators selected members of the Partnership according to their sector and geographic location in order to secure a range of sectoral and regional perspectives. The evaluators also interviewed a selection of members recommended by the SDC. A second round of interviews was conducted of members recommended by the Secretariat.

A total of 49 informants participated in the process out of 57 contacted (an over 85% response rate). The totalnumber of unique institutions within that group is 38, or 27% of the Partnership family (excluding the Secretariat but including 4 regional nodes of the Mountain Forum).

In addition to respondents representing global, North America and European based institutions, we note the following regional diversity of informants:

  • Africa: 2 francophone; 3 anglophone
  • Near east and North Africa: 2
  • Asia: Central Asia: 2; South Asia: 5; Southeast Asia: 1
  • Latin America: 1 Brazil; 7 others

Even with personal interviews, it must be noted that a small percentage (10%) of the informant group (including informants in the second interview group,excluding the Secretariat and donors) specifically indicated that their comments were to be “off the record” or felt that they were not in a position to comment on issues of accountability, leadership or roles in decision making. Anonymity has been provided to all informants, with exceptions only where attribution could not be avoided because of the specificity of the issue.

A list of informants is provided in Appendix 2 together with an analysis by region and sector.

The interview protocol is provided in Appendix 3.

6. Benchmarking against good practice in partnerships: Putting the Mountain Partnership into context

Partnerships bring organizations together to capitalize on synergistic goals and opportunities to address particular issues or deliver specified tasks that single organisations cannot accomplish on their own as effectively. They are also based on relationships in which individual partner organisations cannot purchase the appropriate resources or competencies purely through a market transaction.[1] Thus partnerships are less about the contract and more about the spirit of horizontal accountability and decision-making. Partnerships are also not meant to be permanent, but a transitional mechanism until practices becomes institutionalised or transactions-based.

Beyond this definition above, The Mountain Partnership broadly fits into a relatively new organizational form referred to as Global Action Networks (GANs), Global Public Policy Networks or Multi-Stakeholder Platforms. A Global Action Network is an action oriented network[2], involving multiple partners, led by strong advocates for change, and is focused on accelerating societal change within its domain of influence.

The number of such initiatives is growing rapidly with examples across the health, education, water and sanitation and other sectors. The overarching essence of these institutions or initiatives could be categorized by their attempt to:

address issues of global importance, even while recognizing that solutions may be at the local or national level; and

bring together all relevant stakeholder groups towards a common set of objectives, even while recognizing that the different stakeholder groups will have different understandings and preferences for how to meet those objectives.

Within this broad category of Global Action Network, at least three different types of initiative have emerged. These include initiatives that fulfill the function of:

  1. Resource transfer - whereby the primary function is to make funding or materials available to local or national level projects and programs. (Examples include the Global Fund to Fight Aids, Malaria and Tuberculosis and the Global Alliance for Vaccines and Immunization (GAVI).)
  2. A verification agent- whereby the primary function is to determine and promote greater accountability by jointly creating new standards. (Examples include the Global Reporting Initiative, the Forest Stewardship Council, and the Extractive Industries Transparency Initiative.)
  3. A learning network/platform- while all GANs have a learning function, some initiatives are set up exclusively around understanding and promoting innovation and learning in order to move debates and practice forward. (Examples include Building Partnerships for Development in Water and Sanitation (BPD), and the International Centre for Trade and Sustainable Development (ICTSD).)

The distinctions between these three types are important particularly around:

Who makes decisions within the partnership or how inclusive the initiative is (for example, for the standard setting initiatives, legitimacy is derived from ensuring that all stakeholders are represented in some way – thus inclusion is critical);

What benefits are provided to participating individuals and organizations (i.e., funding; labeling and certification; knowledge and learning) and thereby what are their incentives for participating, and

How free is the initiative to explore and anticipate new trends (this relates to how mainstream or cutting edge the initiative is expected to be).

All three types fulfill legitimate functions and can advance global aims but it is unlikely that any one initiative would be able to mix and match the different functions to any significant degree. Blending the different "personalities" or spirits required of each category would prove to be very difficult.

GANs display certain “personality traits” based on their unique institutional form. Such personality traits could be summarized as follows:

Multiple accountabilities – Given their need to be neutral and independent amongst a cast of different stakeholders, GAN staff need to carefully balance perceived influences on their actions. Thus, though they may be “surrounded by stakeholders”, GAN constituencies may be encouraged not to overstep their influence. This may ultimately result in a lack of clear guidance for GAN staff.

Process driven rather than outputs driven – GAN staff must constantly view their activities through a process lens to ensure that inclusion is maximized and their optimum convening power is maintained. Given limited capacity, hard decisions may also have to be negotiated that keep people on-board even if these decisions result in not meeting the needs of some groups immediately. This process focus or facilitation behind the scenes tries to ensure that partners rather than GAN secretariats take credit for the work that is done. Thus attribution is often a problem.

Technical expertise – GAN staff are expected to have a clear technical expertise in the area in which they are working. Their credibility comes not only from their ability to facilitate the sharing of knowledge and mediate between conflicting interests, but also from their ability to speak knowledgeably on the subjects in which they are engaged.

Champions, Dynamism, Leadership – As GANs are meant to push and promote certain agendas, GAN staff are expected to galvanize and motivate various constituencies in order to build and sustain the “action” momentum of the network. This requires certain personality traits that can inspire and lead people and institutions.

Often under-funded – In later stages of their development, the mandate of GANs is often larger than the funding available. Thus again, difficult choices need to be made as to where the GAN will focus its energy and efforts.

Learning mechanisms / Cutting edge – As a learning organization, GAN staff are expected to frame learning around the area in which they are working. Bringing together a wide range of stakeholders provides enormous opportunities to galvanize exchange. Staff should be able to enthusiastically extract and share information. This requires objectivity and creativity.

B. Evolution of the Mountain Partnership

The Mountain Partnership has emerged out ofnearly four decades of research and international cooperation on mountain ecosystem protection and sustainable development for mountain peoples. As the scientific community began to focus on the importance of this ecosystem (noting that one quarter of the world’s land mass is in mountains and 75% of the world’s fresh water resources come from mountains) so too the international policy community began to both draw public attention to mountain regions and to negotiate approaches for the sustainable development of those regions. Decision makers agreed to include mountains within the international work plan of Agenda 21, made further commitments during the International Year of the Mountains, and reinforced the momentum with the establishment of the Mountain Partnership at the World Summit on Sustainable Development. Regional conventions established during this period include the Andean Convention and the Carpathian Convention. Attention to mountains has also been included in other multilateral environmental agreements, in particular the Convention on Biological Diversity.