TERMS OF REFERENCE

External final evaluation of the project

International consultant

Project title: Promoting media and information literacy and building media capacities in quality non-news programming in Uzbekistan

Duration of appointment:20 days

Expected start date:1 March 2018

Background

The project “Promoting media and information literacy and building media capacities in quality non-news programming in Uzbekistan” was developed and submitted by UNESCO Office in Tashkent for approval to the Conflict Stability and Security Fund (CSSF) in Central Asia in late 2016. The approval was granted in December 2016 and in January 2017 the implementation of the project started. According to the UNESCO rules, all extra-budgetary projects need to get national government endorsement before entering into the actual implementation phase. The endorsement is a signature of authorized government official on the Project Agreement document. The endorsement was obtained only in late March 2017, because of internal administrative changes inside the government apparatus; it was granted by the Deputy Prime-minister Ms. Narbaeva Tanzilya. This situation caused postponements in the project implementation; for this reason, the activities scheduled for February and March 2017 had to be moved to the following quarter. Since then, all the project activities were implemented more or less in time, without significant delays. The completion of the project is scheduled for 31 March 2018.

The project consists of two components: Education and Media, and has overall budget of 400,700 USD (or 320, 560 GBP). The project is in line with UNESCO’s mandate and promotes quality education and communication (media) in order to further respect for justice, human rights and fundamental freedoms which affirmed for people of the world by the UN Charter. The project considers youth as one of the priority areas development and thus, focuses on developing skills on Media and Information Literacy mainly among young teachers and journalists.

Key partners of the project are Center for Secondary Specialized and Professional Education (CSSPE) under the Ministry of Higher and Secondary Specialized Education (MHSSE), National TV and Radio Company (NTRC), National Association of Electronic Mass Media (NAEMM), and In-service Training Center for Journalists (ITCJ). UNESCO Office in Tashkent worked with all these organizations within the previous projects and regular activities. The need for the project occurred due to the rapid change in the world, especially changes concerned with information flows. Thus, critical thinking and quality media products became vital in our days and youth, who often become victims of mis- and disinformation, need to be educated in this regard as users of the information and creators of the information.

In accordance with UNESCO administrative guidelines and requirements of the donor – CSSF – the final project evaluation is mandatory. The evaluation should identify project successes and gaps, so that future initiatives are planned more effectively and avoid challenges faced by the evaluated project.

Purpose/ objectives of the Evaluation

  • To review, evaluate and document the Project achievements, effectiveness, relevance and efficiency;
  • To assess and document sustainability of outcomes of the Project;
  • To identify weaknesses and strengths of the Project design and implementation strategy;
  • To draw lessons, conclusions and make recommendations that can improve design, implementation and sustainability of project results.

It is expected that the results of the Final Evaluation will be used by UNESCO for accountability purposes to generate evidence of achievements and challenges to inform donors and UNESCO management, and for learning purposes to identify effective mechanisms of project implementation and improve project management activities, in general.

Scope

The scope of the Final Evaluation will cover all activities undertaken within the framework of the project. The evaluators shall compare planned outputs to actual outputs and assess actual results andachievements to determine their contribution to attainment of the project outcomes.

The geographical scope includes 14 regions of Uzbekistan (Education component) and Tashkent and Nukus (Media component).

The Evaluator shall assess the Project using the criteria of relevance, effectiveness, efficiency, sustainability, and impact as defined by OECD-DAC.

The Final Evaluation will be guided by the following overarching questions, which will be validated and further refined during the inception phase. It is important to note that certain aspects are more important than others, they will be clarified to the Evaluator after completion of the selection process.A set of sub-questions will be identified for each of these key dimensions. Indicative evaluation questions for each dimension are as follows:

Relevance: The Final Evaluation will examine the extent to which Project’s support on MIL is aligned with the UNESCO’s mandate including institutional priorities, global development goals, and the development needs and priorities of key stakeholders, in particular the British Embassy in Tashkent. It will also look at the relative importance of the initiatives and the comparative advantage of UNESCO in spearheading them.

-To what extent is the Project’s work on MIL aligned with national needs and priorities, as well as national, regional, international priorities, strategies and frameworks?

-What is the added value of the Project in spearheading and supporting these programs?

-Has the Project contributed to the creation of favorable conditions for gender equality?

Effectiveness: The Final Evaluation will examine the extent to which the Project activities on MIL are carried out effectively.

-To what extent was the applied approach effective considering the scope and size of the Project?

Efficiency: The Final Evaluation will examine managerial and operational aspects of the Projectimplemented by UNESCO to support MIL in Uzbekistan.

-To what extent was the Project implemented in accordance with plans, target groups, timeline and what were the key enabling factors and obstacles?

-Has the implementation of the Project been efficient with respect to gender equality?

Results: The Final Evaluation will examine the extent to which the Project’s support to MIL has contributed to enhancing the performance of Uzbekistan to address population security, information literacy, media and education quality, freedom of expression, and access to information issues. Where possible, the evaluation will also seek to identify the extent to which the Project have contributed to positive changes in the determined fields.

-To what extent has the Project strengthened national capacities to address lack of critical thinking among teachers and journalists, impartiality of mass media, quality of media content and how are national project partners and other stakeholders using enhanced knowledge/skills/systems to transform the way they work on issues related to MIL?

-To what extent did the Project contribute to improving and expanding MIL in Uzbekistan?

Sustainability: The Final Evaluation will examine the extent to which Project can be scaled up or taken forward by government and project partners (CSSPE, NTRC, NAEMM, ITCJ) independently.

-To what extent are the benefits of the Project likely to be sustained or taken forward independently by government and project partners after the completion of the Project?

-To what extent have the Project contributed to influencing national, local or institutional priorities on MIL through increased awareness, strengthened capacity and exposure to good practice?

The Final Evaluation should provide evidence-based information which is reliable, credible, and useful.

Methodology

The Evaluator shall collect quantitative and qualitative data on Project outputs and expected and unexpected results/ achievements. S/he shall visit at least four regions of Uzbekistan where activities on Education component of the project were conducted: Tashkent, Nukus (Karakalpakistan), Namangan (Namangan region), and Gulistan (Syrdarya region). This means that the Evaluator will have at least four field visits. The totalnumber of consultancy days, including review period, design, field visits, and final report writing, should not exceed 20 days.

The Evaluatorshallundertake the following:

  1. Review all relevant reports, proposals and relevant literature on the project and KAP reports on Education and Media components;
  2. Design a study to evaluate outcomes of the project as stated above;
  3. Identify and conduct interviews with a sample of stakeholders in project area to gather data on project results and outcomes;
  4. Discuss with UNESCO project team findings of the evaluation prior to finalizing the report.

As part of UNESCO strategies and priorities and CSSF requirements, the Final Evaluation should be gender sensitive, participatory and promote a learning approach. The Evaluator should ensure that the assessment covers these elements in the inception and final reports.In addition, the Evaluator should test the Theory of Change (ToC) and reflect this in the reports.

Roles and Responsibilities

UNESCO will facilitate the Final Evaluation process by providing contact information, such asemail addresses and/ or phone numbers of relevant stakeholders and networks to be consulted during the Final Evaluation. Also, UNESCO will provide round-trip air tickets/ train tickets to Tashkent and required regions, as well as hotel booking.

The Evaluator will be responsible for managing the Final Evaluation, its quality, data collection, analysis, and report writing.

Qualifications Evaluation Team

It is anticipated that the mission team will comprise of the following:

1 international consultant (team leader)

The team leader is responsible for the quality of the final evaluation. (S)he will author the final evaluation report and lead the final evaluation team through the implementation of the evaluation methodology.

The development of the evaluation tools and strategies will be of primary importance. The team leader will develop the evaluation methodology, tool and evaluation questions in consultation with UNESCO Office in Tashkent. Other team members,under his/her leadership,will refine the tools and evaluation questions for which they will gather information. This will require attention to the area of overall project organization and processes.

The team leader should have the following qualifications:

  • Advanced academic degree (MSc or PhD) and substantive experience in reviewing and evaluating similar projects, preferable those involved with UN agencies and focused on education and media development;
  • Excellent English writing and communication skills. The Consultant should bring his/her own computer equipment;
  • Demonstrate ability to assess complex situations, effectively/accurately resolve critical issues, and draw forward-looking conclusions and recommendations;
  • Highly knowledgeable of monitoring and evaluation processes;
  • Familiarity with the challenges developing countries face in providing access to information and freedom of expression;
  • Excellent in management, human relations, coordination, planning, and team work;
  • Excellent feedback-giving skills and culture sensitiveness.

1 national consultant

The national consultant will give specific insights of the situation in Uzbekistan. S/he will be key in collecting data to ensure that findings are clear and sound. S/he will take the lead in facilitating face-to-face meetings and interviews within the community in order to understand the progress that has been made since project’s start.

Qualifications of the national consultant should be as follows:

  • Academic degree (BSc or MSc) and experience in reviewing and evaluating similar projects, preferable those involved with UN agencies and focused on education and media development;
  • Excellent Uzbek, Russian, and English writing and communication skills. The Consultant should bring his/her own computer equipment;
  • Ability to interpret to the international counterpart and also translate necessary documents to English;
  • Understand the context and background of education and media in Uzbekistan;
  • Demonstrated skills and knowledge in monitoring and evaluation processes;
  • Excellent in human relations, coordination, planning, and team work.

Deliverables and Schedule

Key deliverables are the following:

  1. An Inception report detailing out: methodology for evaluation, delivery schedule, budget, and sample tools;
  2. Final report in both soft and hard copies;
  3. Follow-on recommendations on potential future activities that could be built on the work already done and thus increase impact and sustainability of the current project;
  4. Develop a power-point presentation on the evaluation process/approach, findings, conclusions and key recommendation.

The final report should be structured as follows and not exceed 20 pages, excluding Annexes:

  • Executive Summary (no more than 2 pages)
  • Programme description
  • Evaluation purpose
  • Evaluation methodology
  • Findings
  • Lessons learned
  • Recommendations
  • Annexes (including the list of stakeholders consulted during the evaluation, key documents reviewed, ToR, survey forms and aggregate findings, budgetary analysis, etc.)

The schedule should be presented in the following table:

# / Activity / Estimated time (days) / Key outputs
1
2
Time allocated to the Assignment / 20 days

Process for submitting the proposal

Interested Individuals or Firms are invited to submit their proposals before 22 February 2018 at 23:59 pm CET by email to

The technical proposal and the financial proposal shall be submitted in separate files.

The technical proposal should include:

-a brief description of the proposed evaluation methodology

-a workplan, including the proposed timeline

-full CV

The financial proposal should contain:

-the total cost in US dollars, detailing the cost of professional fees and travel.

Should you require additional information, email ver, any delay in providing potential information will not be considered a reason for extending the date for submission of your proposal. Additionally, kindly note that until the selection of the contractor is made official by a contract formally signed by UNESCO, the Organization has not undertaken any commitment, whether oral or written, to any prospective contractor.

Page 1 of 6