Read the Instructions Before Filling out This Template

ANNOTATED INDEX

FOR DRAFTING

TERMS OF REFERENCE

FOROPERATIONAL EVALUATIONS

TEMPLATE

Read the instructions before filling out this template

February 2015

INDEX

1.JUSTIFICATION AND OBJECTIVESOF THE EVALUATION

2.BACKGROUND AND CONTEXT

3.SCOPE OF THE EVALUATION

4.EVALUATION QUESTIONS

5.METHODOLOGY

6.EVALUATION MANAGEMENT

7.SCHEDULE AND DELIVERABLES

8.EVALUATIONPRINCIPLES, AUTHORSHIP AND PUBLICATION

9.PROFILE OF THE EVALUATION TEAM

10.SUBMISSION OF BIDS, BUDGET AND BIDS’ APPRAISAL CRITERIA

APPENDIX I. PRELIMINARY LIST OF STAKEHOLDERS

APPENDIX II. PRELIMINARY LIST OF REFERENCE DOCUMENTS

APPENDIX III. STYLE GUIDE FOR EVALUATION REPORTS

APPENDIX IV. DECLARATION OF ABSENCE OF CONFLICT OF INTEREST

APPENDIX V. EVALUATION MATRIX MODEL

TITLE OF THE EVALUATION

JUSTIFICATION AND EVALUATIONOBJECTIVES

Grounds for evaluation:

Evaluation objectives:

Intended use of the evaluation:

BACKGROUND AND CONTEXT

SCOPE OF THE EVALUATION

Definition of the evaluation object:

Geographic scope:

Time frame:

Considerations regarding cross-cutting priorities:

Other relevant information to be included in Appendix

Appendix I should present a preliminary list of themain stakeholders involved in the evaluation process[1].

Annex II should detail main reference documentsrelated to the evaluation object as well as a set of links to websites containing updated information and publications relevant to the object of this evaluation[2].

EVALUATION QUESTIONS

…

In consultation with the different stakeholders and after a preliminary documentary review, the evaluation team will revise the evaluation questions and analyse the explicit interest in and feasibility of including them, so that they can be adjusted in a well-reasoned manner based on the information required, the evaluation deadlines and the resources earmarked for the evaluation. This adjustment will be included in the inception report, and should be validated by the management team.

METHODOLOGY

To articulate in a systematic and logical matter the comprehensive judgement of the evaluation object, the technical proposal shall revolve around an evaluation matrix[3]. The matrix is a tool for the operationalization of the evaluation questions, but it cannot be a substitute for the theoretical and methodological approach guiding the evaluation, which should be reflected clearly in its own section of the proposal. Furthermore, it is worth bearing in mind that methodology is not merely a list of techniques; rather, it involves a theoretical and epistemological stance, orienting the manner in which the evaluation is to be focused in order to meet the evaluation objectives, and setting out the form (specific techniques) in which the data is to be collected, classified, analysed and presented, with the aim of producing solid findings and evaluation deliverables meeting quality standards.

A preliminary evaluabilityanalysis should be included, indicating which are the main a priori/enabling factors and limitations for satisfying the evaluation objectives meeting the planned aims for this evaluation, and answering the questions set forth.

It is the evaluation team’s responsibility to present in its proposal an appropriate theoretical and methodological framework for the purposes, objectives and utility of the evaluation(adjusted to the time and resources available for carrying this out), as well as a coherent approach that makes possible to operationalize in the evaluation matrix the evaluation’s different objectives, levels of analysis, questions and sub-questions. In this regard, it is important to clarify that this operationalization does not have to involve, in principle, any modification to the evaluation questions presented, but rather their development in subquestions, in line with the rest of the formulations of each of the columns in the evaluation matrix.

In its proposal, the evaluation team will ensure: the complementarity and diversityof methods and information sources, making clear the existing limitations and specifying how and to what point the analysis is going to include the cross-cutting approaches included in Spanish Cooperation’s Evaluation Policy (human rights, gender, cultural diversity and environmental sustainability).

These techniques should be consistent with the methodological approach chosen and appropriate for the nature of the information that is expected to be available in order to answer the evaluation’s different questions and subquestions. The proposal will justify in a specific and concrete manner the contribution of and need for each technique used in this evaluation, avoiding generic paragraphs on the technique and its characteristics. Likewise, it should avoid the mere juxtaposition or listing of basic techniques (e.g., interviews, surveys, focus groups) without specifying their details, relevance, and complementarity. As long as the information available so permits, quantitative and qualitative methods will be combined, which means the use of data collection techniques and specific analysis procedures for providing information that is sufficiently precise and robust and adequately explainedand put into context..

Among the selection of techniques that the evaluation team considers opportune to propose, at least XXXX should be included (indicate the techniques appropriate for each case, e.g.review of the literature, interviews, discussion groups, case studies, surveys).

(Only when appropriate): Taking into account the scope, levels of analysis and evaluation questions andin order to adequately address the evaluation object, it is recommended that the evaluation team follow the methodological proposal of XXX.

EVALUATIONMANAGEMENT

The evaluation will be carried out ensuring the participation of the main stakeholders involved in the evaluation object. The management structure of the evaluation will comprise:

(Only for joint evaluations) A Management Committeecomposed of XXX and ZZZ, with the following functions

(Only for individual evaluations) The management unit of the evaluation, in this case XXX, with the following functions:

-Provide broad guidance and direction in drafting ToR in consultation with the Reference Group, and give approval to the ToR.

-Appraise bidding proposals and select the evaluation team.

-Maintain a close, efficient and regular liaison and dialogue with the evaluation team and the Reference Group.

-Provide technical advice and methodological oversight to the evaluation process.

-Carry out the evaluation quality control and supervise the accomplishment of deadlines.

-Validate the inception report and approve all the evaluation products in consultation with the Reference Group.

-Release the evaluation products and facilitate the dissemination of the evaluation results.

-Coordinate filling out the management response document and promote use of the evaluation recommendations.

To enhance the appraisal process and introduce appropriate participation channels for the different stakeholders, a Reference Groupwill be created, including the participation of XXX (specify the principal stakeholders involved in the intervention evaluated)

The ReferenceGroup’s main functions will be to:

Bring comments, suggestions and informative requirements to the Management Committee(whichever is appropriate) to draft the ToR.
Be closely involved in the evaluation process by providing comments and suggestions on the evaluation products (inception and final reports and other products) to the Management Committee/unit (whichever is appropriate).
Provide the evaluation team with all needed contacts and access to all the relevant information to carry out the evaluation.
Contribute to the dissemination of the evaluation results.
Support the implementation of the evaluation recommendations.

The secretariat and coordination services for the Management Committee shall be the responsibility of XXX. This secretariat will be involved throughout the evaluation, serving as a liaison between the Reference Groupand the management /unit (whichever is appropriate)and will provide support and advisory services to the latter during the entire process.

SCHEDULE AND DELIVERABLES

The time scheduled for this evaluation is XXX weeks since the formalization of the contract, and will follow the distribution of phases, tasks, deliverables and times indicated on the table below. The scheduled deadlines may be extended with the consent of the evaluation’sManagement Committee/unit (whichever is appropriate), until the evaluation team’s deliverables are considered of sufficient quality.

During Phase I, the evaluation team will be accompanied by the evaluation’s Management Committee/unit (whichever is appropriate) in order to facilitate adjustment of the work and drafting of the preparatory report.

Unless expressly indicated otherwise, all deliverables shall be in (indicate language). All deliverables shall be presented in Word, to guarantee their subsequent publication under adequate conditions, in whatever formats are considered appropriate. The final report shall include photographs, maps, tables, infographics and other visual resources to make the report more user-friendly and easily understood.

Follow the style guide included as Appendix III. We recommend using the OECD DAC Glossary of Key Terms in Evaluation and Results Based Managementas a means toclarifyterms (e.g. instrument).

Phase / Activities / Duration
Phase I. Preparatory Activities and Desk Study / -Initial meeting between the evaluation team and theManagement Committee/unit (whichever is appropriate), gathering of information and workplan adjustment
-Stakeholder analysis and identification of the evaluation key informants.
-Preliminary literature review and collectionof new documentation at the office and on the ground.
-Description of the evaluation object
-Preliminary mapping of interventions (only when faced with a complex evaluation involving many interventions).
-Preliminary reconstruction of the intervention’s rationale
-Adjustment of the evaluation questions, methodology and work plan.
-Review of the documentation submitted by the evaluation team as part of the preparatory reportby Management Committee/unit (whichever is appropriate) and by the Reference Group
-Adjustment of the deliverables by the evaluation team and their formal approval, when applicable. / X weeks
(a minimum of 2 additional weeks are to be included for review, adjustment and approval of the inceptionreport)
Phase I deliverable: InceptionReport
The report is to contain the finalevaluation design, which must include:i) a description of the evaluation object; ii) the intervention’s rationale (reconstructed if it were not expressly formulated); iii) the conceptual analytical framework; iv) a methodological proposal, including a brief evaluability analysis (describing the limitations found or foreseen and justifying, when applicable, any changes with regard to the initial plan), specification of the techniques and data collection tools, and a detailed analysis plan; v) the updated work plan; and vi) a definitive evaluation matrix.
Phase II. Fieldwork / Application of data collection tools according to what was set forth in the InceptionReport. / X weeks
(this requires prior approval of the Inception Report)
Phase II deliverables: Presentations at the beginning and end of the fieldwork
The fieldwork will begin by briefing the Reference Groupand the Management Committee/unit (whichever is appropriate). Other stakeholders may also be invited to this briefing. Once the fieldwork phase has finished, the evaluation team will make a presentation that will include the activities carried out, and it will present preliminary findings for discussion with the Management Committee/unit and with the Reference Group
Phase III. Analysis and interpretation of the information.Drafting and submission of the final report / -In-depth analysis and interpretation of the information collected.
-Drawing up of the first draft of the final report, which will include the corresponding appendices.
-Integration of observations and comments forwarded by the Management Committee/unit of the evaluation. The evaluation team will explain how these observations have been included in the document and will present, when applicable, the arguments for their non-consideration, preserving at all times the independence of the evaluation, in line with the Quality Standards for Development Evaluation of the OECD’s Development Assistance Committee (DAC).
-Drafting of the final report.
-Submission of the final report. / XXweeks
(a minimum of 3 additional weeks are to be included for review, adjustment and approval of the final report)
Phase III deliverable: Final Report
This shall comprise the following documents:
-The final report itself(maximum 70 pages without appendices), which must respond to the evaluation objectives and answer the questions set forth. The conclusions and recommendations in the report must stem from the findings.
-A list of appendices, which will include, among other documents, the detailed methodology, the list of techniques and instruments applied, and a list of the principal sources of information.
-An executive summary, written in Spanish, English and in the language of the partner country if different from these two (maximum 5 pages in each language), the format and wording of which must be tailored for its dissemination and facilitating decision-making. It must include, as a minimum: an introduction to the object of the evaluation and to the methodology used and a summary of the principal findings, conclusions, lessons learned and recommendations. This executive summary is to be submitted once the final report has been formally approved.

Below is a guideline for the content structure of the final evaluation report. However, the index and definitive outline of the report shall be agreed upon between the Management Committee/unit and the evaluation team.

In any case, the findings and conclusions must be structured so that they respond clearly to the evaluation questions.

Lessons learned (either positive or negative) should include generalizations that go beyond the immediate intervention being evaluated, highlighting the wider-ranging relevance that the knowledge generated during the process might have. It is not a matter of repeating the conclusions with a different wording, and it must be understood that not all evaluations generate a large number of lessons learned. Lessons must only be drawn if they represent actual contributions to general knowledge. They must be solidly backed by the evaluation findings and conclusions. They may strengthen or complement commonly-accepted lessons, but they must not be a mere repetition of common knowledge. Lessons learned must be worded simply; they must specify the context from which they were drawn; they must explain the knowledge generated, identifying factors that explain why things happened in a certain way; and, with the necessary adaptations to each new context, they must be able to serve as guidelines for future interventions.

Recommendations should be limited in number, avoid generic wording, and state to whom they are addressed.

1. Introduction.

1.1. Structure of the Report.

1.2. Background, Scope and Evaluation objectives

1.3. Evaluation Questions and Levels of Analysis .

1.4. Theoretical and Methodological Framework.

1.5. Data Collection Tools and Analysis s.

1.6. Determining Factors and Limitations of the Evaluation.

2. Analysis and Interpretation of the Data (Findings).

3. Conclusions

4. Lessons Learned

5. Recommendations (Strategic and Operational).

Appendices

EVALUATION PRINCIPLES, AUTHORSHIP AND PUBLICATION

The evaluation shall follow the OECD DAC QualityStandards for DevelopmentEvaluation and Spanish Cooperation’s Evaluation Policy. Throughout the evaluation process, the evaluation team must respect theprinciplespublished in the Spanish Cooperation website.

Regarding authorship, without prejudice to recognition of the evaluation team’s moral rights, the (corresponding contracting entity) is to be responsible for designing the layout, printing and publishing the documents.

The contract shall comprise at least a presentation of the results of the evaluation in (the corresponding partner country) by the evaluation team. The evaluation team may also be required to participate in other activities involving presentation and delivery of results; in such cases, the costs that may be incurred by said activities are to be paid independently.

PROFILE OF THE EVALUATION TEAM

The evaluation team is to comprise at least XXX people. It is recommended that there be an appropriate balance between men and women, and that it include local professionals.

The team coordinator must have a university degree and specialized training in evaluation or social research, and at least 3 years’ experience in carrying out evaluations.

XXX members of the team shall be required to have a university degree, preferably in development cooperation, evaluation, or public policies.

As a whole, the evaluation team must prove to have:

Wide-ranging knowledge of the socio-economic context and public policies of the country where the evaluation is being carried out.
Knowledge of Spanish Cooperation.
Specific knowledge of XXX (sectors, languages, etc. depending on the object of the evaluation)
Knowledge of cross-cutting approaches. This requirement will be adapted depending on the specific weight of the different approaches in each case.

The technical proposal is to include the tasks to be carried out and the period of time that each professional will dedicate to the evaluation, as well as their formal commitment to being a member of the evaluation team during the validity of the contract. Any change to the makeup of the evaluation team must be previously agreed upon with the Management Committee.

SUBMISSION OF BIDS, BUDGET AND BIDS’ APPRAISAL CRITERIA

Bids are to be submitted in XXX before XXX(time)XXX (day).

The following documentation must be included:

Presentation of the company (if applicable) andthe résumés of the evaluation team. It shall be stated that the presentation of documents accrediting the accuracy of this information may be requiredat any time.
Technical proposal, which is to include a description of the object to be evaluated, a methodological proposal including the operationalization of the evaluation matrix, and a work plan.
Financial proposal. This must be broken down into the different types of expenses (travel, accommodation, meals, fees, materials, etc.). In the case of fees, the tasks and fees of each member of the evaluation team are to be specified, indicating the amount per person and per day.

The maximum budget for this evaluation is XXX euros (excluding VAT).The price shall be paid in three instalments, following the validation of the corresponding deliverables (adjust on a case-by-case basis):

-After approval of the inception report (30%)