End of Project Evaluation, of WASH for Life, Pakistan

AGA KHAN FOUNDATION

(Pakistan)

The Aga Khan Foundation (AKF), an agency of the Aga Khan Development Network (AKDN), is a private, non-denominational, international development agency established in 1967. It seeks sustainable solutions to long-term problems of poverty, hunger, illiteracy and ill-health with special emphasis on the needs of rural communities in mountainous and other resource poor regions, primarily in Asia, Africa and the Middle East. Its main areas of focus are education, health, rural development, environment and strengthening civil society.

The Aga Khan Foundation (Pakistan) [AKF (P)] is seeking high calibre Consultant/Consultancy firm for

“End of Project Evaluation, of WASH for Life, Pakistan”

The project is being implemented in 41 villages in District Chitral of Khyber Pakhtunkhwa and the Consultant/Consultancy firm will be required to travel to Chitral to carry out the task.

1.  Background and Context

The floods resulting from the GLOF events of June 2015 in Chitral district and then a strong earthquake in October hit the district causing widespread damage to critical infrastructure including jeep-able roads, bridges and tracks, suspension bridges over waterways, protection walls, water supply and irrigation channels and mirco-hydel electricity generation units. These calamities severely affected several water supply schemes across the district, which got damaged due to rock falls, debris falling, and damages to water tanks, feeder lines and distribution system. In total about 117 water supply schemes (WSS) were adversely affected. As a result residents were forced to use contaminated water or had no or limited water available for their needs.

2.  Introduction to Project

WASH for Life is a project of humanitarian assistance for natural disasters and conflict victims which is funded by European Commission Humanitarian Aid and Civil Protection (ECHO) and co-funded by You Foundation through HOPE'87 and AKPBSP. The project is jointly implemented by HOPE’87 and AKPBSP with Aga Khan Foundation Pakistan providing grant management, compliance and monitoring, planning and reporting support to the Implementing Partners. The principal objective of the project is to ensure timely and dignified access to sufficient and safe WASH services for populations threatened by on-going humanitarian crises and to increase their resilience to withstand water stress and shocks. More specifically the project focuses on saving and preserving life and alleviates the suffering of populations facing severe environmental health risks and/or water insecurity caused by humanitarian crises in Pakistan. WAFL is being implemented in 41villages of district Chitral of Khyber Pakhtunkhuwa province in Pakistan. It was started on September 01, 2016 and will continue until December 31, 2017, with a total duration of 16 months. This period includes a No Cost extension of 04 months from

1st September 2017 till end of December 2017.

3.  Purpose of the Assignment

The overall purpose of the study is to evaluate “WASH for Life in District Chitral of Pakistan”, with a particular emphasis on the effectiveness, sustainability, efficiency and relevance of the project interventions and extract lessons learnt to chalk out recommendations for on-going and future programming by partner agencies. It should also capture achievements of the project having short and medium term impact on target populations. The evaluation will focus on all interventions planned under “WASH for Life (WAFL) in 41 settlements in District Chitral for sufficient and safe WASH services. Also to look into the involvement of populations threatened by on-going humanitarian crises in Chitral and to gauge the increase in their resilience to withstand water stress and shocks in selected target communities.

More specifically, the key objectives of the evaluation are:

a.  To assess the relevance, effectiveness, efficiency, sustainability and impact of the ECHO supported WASH Action.

b.  To assess the extent to which the project has contributed to agreed results to help replicate and scale up interventions.

c.  To draw operational recommendations and lessons learned for further improvement and enhancement of relevant sector policies, plans, strategies through analyses of the factors contributing to the success or failure of the project.

d.  To assess appropriateness of consortium based approach to achieve the expected results and provide recommendations for consortium arrangements in future for such projects.

4.  Evaluation Criteria and Questions

The evaluation criteria are mainly for five areas. Below are some examples of the questions per criteria. Consultants for this evaluation are recommended to elaborate the list of questions during the submission of their technical proposals, which will be used for the selection process by AKF/ECHO. The list of evaluation questions will be discussed and finalized with AKF(P) and its partners and the technical committee during the inception phase, before submission of the inception report. The final list of questions will have to be realistic given the budget and timeframe proposed by the evaluation team. The impact will be assessed mainly by using both qualitative and quantitative methodologies by taking into account the perception of beneficiaries and stakeholders.

Relevance / -  Did the intervention respond to the needs of the various target groups?
-  Was the logical framework coherent enough to achieve the results?
-  Did the project followed the integrated water and sanitation extension program (WASEP) approach that provides not only the physical infrastructure and promotion of proper sanitation practices but ensures long term sustainability of the interventions by creating and empowering local governance structures in the form of Water and Sanitation Committees (WSCs) and cascading the behavior change education programs
-  Did the project contribute in adopting a coherent approach for a broader, transversal and cross-sectoral response to humanitarian needs by building synergies with other partners/actors and local authorities?
Efficiency / -  Were resources utilized and managed in efficient manner?
-  How well did the operation co-ordinate with other, similar interventions (if any) for synergy and in order to avoid overlaps?
-  Would it have been possible to achieve the same results at a lower cost?
-  Have the activities been conducted as planned w.r.t timelines, budget and other agreed modalities (if any).
Effectiveness / -  Were the outputs delivered as planned and in a coherent manner? How well the agreed indicators are met?
-  What was the level of quality and compliance of activities to the norms and standards previously established by the program, both for the construction and social intermediation components?
-  Did the project impact the lives of its targeted population, and the rural settings bordering the target communities?
-  Were the behavioural change communication sessions (CHIP and SHIP) effective in promoting improved health and hygiene practices in the target communities?
-  Were the disaster risk reduction and resilience measures incorporated in to the project design and implementation?
-  How have project activities impacted local environment and if project design and implementation approach were cognizant of mitigating adverse impacts?
Sustainability / -  What is the likelihood that the operation and maintenance and repair of the installed facilities being financed at the local and sub-national level for continuity of the services (access to clean water etc.) after the end of the project?
-  To what degree did the project consider the existing structure or resources to enhance the sustainability after the end of the intervention?
-  What could the project have done different to improve sustainability?
Impact / -  What were the main impacts (positive/negative, expected/unexpected) as perceived by the different actors and beneficiaries of the project?
Other areas of project specific concern
Equity / -  Was the project aligned with ECHO’s specific equity agenda in addressing the needs of the target groups (i.e.to what extent the initiative reached different groups including the most vulnerable and marginalized)?
-  Did the project contribute to equitable participation and benefits to various groups (men, women, children and differently abled people)?

5.  Methodology

A mixed-method approach is required for this evaluation including the systematic use of qualitative (e.g, structured interviews and focus groups) and quantitative (e.g. recent survey results, existing routine data) methods. The overall methodology will be participatory and involve various social groups including the most marginalized to capture their opinion. A detailed methodology is to be submitted by the applicant (consultant) at the time of submission of the technical proposal which will be used as a basis for proposal assessment by ECHO and its partners. Afterwards, the consultant will be requested to develop a more holistic evaluation plan having study work plan, approach and design of the study, the required sampling methods, list of questionnaires and tools, information collection and analysis methods etc.

The below is general proposition to develop technical proposal for the applicant.

i)  Desk Review and Refinement of Methodology

The list of references and documentations will be shared with the selected consultant. In relation to this project, the baseline and end-line results of a Knowledge, Attitude and Practice (KAP) study are available. This data together with other existing documents e.g. Workplan, Monitoring Report, Project progress reports, etc. as appropriate shall be reviewed and utilized for the evaluation by considering the disparities based on geography, gender and other key variables based on the availability of disaggregated data.

ii)  Data Collection, Field Work ( site visit/interviews)

The systematic use of qualitative (e.g, structured interviews and focus groups) and quantitative (e.g. recent survey results, existing routine data) methods are required to collect the additional evidence during the field visit. The meeting with key stakeholders will be accompanied by HOPE’87 and AKPBSP staff during the field work in Chitral. Due to the nature of this “end of the project evaluation” with time and budget limitation, it is highly recommended to appropriately and reasonably sample no less than 14 among 41 target villages (effectively covering evaluation requirements) for the project site visit and data collection, focus group discussion in the field based on the desk review analysis. The respondents and methods/tools (interview, questionnaires, and focus group) need to be strategically selected based on the evaluation questions.

6.  Deliverables

·  Inception report outlining analysis of project documents, existing data, identified villages and proposed study design with sample size, qualitative and quantitative instruments and field implementation plan

·  First draft of End of Project (EoP) evaluation report

·  Presentation of the key findings to AKF(P)

·  Final EoP evaluation report

All material prepared and collected during the EoP evaluation will be the sole property of AKF (P). The consultant/Consultancy firm will be required to provide the client with all primary data collected in electronic form.

7.  Implementation Time

A total of 20 days in December 2017

Activity / Number of Days
Preparation of inception report with tools / 03
Meetings, data collection and field work / 8
Data entry and data analysis / 5
Report / 04
Total / 20

8.  Required Qualifications

·  Post-graduate degree in Humanitarian Studies, Disaster Management, Social Sciences and/or related field.

·  At least 10 years’ experience of conducting evaluations of emergency/ humanitarian programmes.

·  Previous experience of conducting evaluations of NGO consortia/alliances in emergency/disaster management contexts;

·  Familiarity with WASH, emergency food security and livelihoods issues, as well as knowledge of financial and economic analysis;

·  Familiarity with International quality and accountability standards applied in disaster management;

·  Experience in the use of participatory methodologies and developing gender sensitive evaluation methodologies;

·  Experience with ECHO funded projects an advantage.

·  Excellent written and spoken communications skills in English. Knowledge of the Pakistani regional context and local languages will be an advantage

·  Experience in assessing organizational capacity and gaps and ability to recommend the corrective measures.

A team of two-three people will also be considered, utilising one consultant with good local language and gender specific skills.

9.  Preparation and Submission of Bids

Qualified and interested parties are asked to submit bids with the following:

1)  Cover letter: A one-page cover letter signed by a responsible official of the bidding agency/individual consultant.

2)  Detailed technical proposal: A proposed plan of 4-5 pages (excluding Annexes) including:

a)  Introduction: A brief overview of the assignment as understood by the bidding agency and a brief statement of the team’s overall technical approach

b)  Sampling strategy and rationale, proposed methodology for data collection and analysis, proposed approach for an evaluability assessment, quality assurance procedures and ethical considerations

c)  Study team composition including their roles and respective level of effort

d)  Timetable for completion of major activities

3)  Annex A - Cost proposal: A detailed budget for completing the proposed work.

4)  Annex B - Resumes: CVs of all staff members assigned to the consultancy.

5)  Annex C - References: For at least the lead consultant (if not also the others), a list of three former or current clients for whom similar work has been done, together with their addresses and telephone numbers.

AKF reserves the right to add or delete information, or otherwise amend the contents of this RFP during the period of preparation of proposals. AKF reserves the right to extend the period specified for presentation of proposals.

All proposals, once submitted, shall constitute firm offers and may not be retracted for a period of ninety (90) days following their submission. It is understood that bidding agencies assume the cost of preparation and submission of proposals.

AKF(P) offers a competitive remuneration package and conducive working environment. We are an equal opportunity employer - females are encouraged to apply.

Only online applications will be entertained. The deadline for applications is November 29, 2017.

Only short-listed candidates will be contacted.

For more information about AKDN, please visit www.akdn.org

Page 3 of 6