MULTI-COUNTRY EVALUATION OFREGIONAL KNOWLEDGE AND LEADERSHIP AREAS

INCLUDING ALL CHILDREN IN QUALITY LEARNING IN CEE/CIS

2002 - 2012

Terms of Reference

BACKGROUND

For about 10 years now, UNICEF has adoptedin the CEE/CIS region a ‘system’ approach to programming with the view to accompany reforms and contribute to concrete changes at institutional, societal and individual level.The 2009 and 2010‘Mid-Term Review’ submissionsto the Executive Board reported a number of outcome and impact resultsachieved at country level in the CEE/CIS Region. This representeda clear evidencethat UNICEF’supstream work in a significant number of countries and sub-regions yieldstangible results not only in terms of system changes (outcome results) but also in terms of changes in the life of children (impact results).

In April 2012, the Regional Office initiated a participatory process involvingallCountry Offices in CEE/CIS with the view to identify few strategic result areas where UNICEF has the capacity to deliver high-quality and relevant results that contribute to address child rights violations and close equity gaps.

A Regional Knowledge and Leadership Agenda(RKLA) emergedfrom this process, with a double purpose:

(1)Inform and guide UNICEF’sfuture work in CEE/CIS, with a strategic focus on few areas where results for children can be achieved over the next two-three years across a number of sectors and countries (prospective approach); and

(2)Generate evidence and document - in an aggregated picture -how UNICEF contributed over the past decade to outcome and impact results for childrenin a significant number of countries, (retrospective approach). Thisprocess is expectedto contribute to a regional knowledge exchange agenda and generate mutual learning among countries on practices and strategies in order to inform the prospective approach.

Philippe TESTOT-FERRY, 22 November 2018

The result areas that were selected for the retrospective approachshare the following features:

a)They address key violations of child rights in terms of magnitude and/or severity;

b)They explicitly contribute to closing equity gaps (except for results in health and HIV/AIDS, which aim at universal coverage or overall elimination);

c)They are being achieved in a significant number of countries;

d)And they can be documented and evaluated.

The strategic intent is to explore not only the link between system reforms and reduction in child rights violations but also to track the reduction or equity gap and assess the extent to which the indicators related to the most disadvantaged children are catching up with national averages.

For each of the following areas, multi-country evaluations will be undertaken to demonstrate how reduction of equity gaps and impact results (in terms of changes in the life of children) were made possible through changes in the national (regional/local) systems and document UNICEF’s contribution to these changes.

(1)Children’s right to be raised in a family environment (2012)

(2)Juvenile Justice: Children’s right to support to re-integration into society (2013)

(3)Children’s right to early learning / school readiness (2013)

(4)Inclusion ofall Out of School Children in Quality Learning (2012)

(5)Children’s right to health:infant and under 5 and mortality (2013)

For each of these areas, Reference Groups(composed of representatives of the Regional Office and concerned Country Offices) were set up in order to identify common elements and differences in the country approaches and policy options that led to the achievement of various impact and outcomeresults.

The present Terms of Reference concerns the multi-country evaluation planned for the fourth key result area: Inclusion of all Out of School Children in Quality Learning.

CONTEXT

National average primary school enrolment rates in the CEECIS region are all over 90-95%; thus, nearly every child enrolls in school at some point in their life. However, the national picture belies sub-national disparities that leave certain groups of children completely excluded from basic education. There are 3.7 million children of primary and lower-secondary school age and 1.6 million children of pre-primary school age out of school in CEE/CIS (UNESCO, 2010 GMR); an additional 12 million adolescents are estimated to be out of school. While on the whole, there are no big gender disparities in access and participation in basic education, with a few exceptions, notably in Tajikistan and few years ago in Turkey, gender disparities in learning outcomes (at the disadvantage of boys) particularly in reading but also in sciences and mathematics, become increasingly reported. Children and adolescents out of school are those from the most socially, culturally and economically marginalized communities and thus are the hardest to reach.Major equity gaps in both access to and outcomes of education exist between these children and their peers from majority populations.

The profiles of children that are most likely to be out of school and thus represent the most urgent challenges for governments and partners in the region are: (a) Children from ethnic minorities, especially the Roma; (b) Children with disabilities; (c) girls in few countries, and boys in others; (d) Children from the poorest households; (e) Working children; (f) Children performing below academic standards; (g) Children of pre-primary age - from all the above groups; (h)Adolescents - from all the above groups; and (i) Children with multiple disadvantages (for instance, being a girl, from an ethnic minority and living in a poor household and therefore not being enrolled in pre-primary education).

APPROACH TO INCLUSIVE EDUCATION IN CEE/CIS AND KEY RESULTS ACHIEVED

For the past 10 years, the overarching result pursued byUNICEF in CEE/CIS is that “Innovative and effective approaches for addressing exclusion in education are scaled-up by governments”.

Based on UNICEF’s Education Strategy and MTSP, the vision that has guided UNICEF’s work in education in CEE/CIS for the past 10 years is that ‘Every child in the region will access and complete basic education of good quality’. Toward this vision, UNICEF has focused its education interventions in the region on four long-term (2015) goals, adapted to the specific context of CEE/CIS, in particular the two following ones: (1) reaching the last 10-15% of children who are out of school; and (2) improving the quality and relevance of basic education in order to reduce school drop-outs and increase completion/achievement rates. These two objectives are clearly mentioned in the 2007 Regional Education Strategy Note.

In order to support the long-term vision and goals for education in CEE/CIS, the Regional Officeset for itself the followingobjective: ‘Establish a programmatic environment that will enable Country Offices to better position their education programmes, achieve large scale results and remain relevant’. More specifically, four operational strategies to be supported at regional, sub-regional and country levels were formulated, as follows:

  1. Critical knowledge will be generated, analyzed, packaged and disseminated in order to:
  • Create a body of evidences on key issues in basic education (exclusion, disparities, quality, relevance…) that will allow evidence-based advocacy among policy and decision-makers, and influencing sector reforms;
  • Create a critical mass of best practices or successful projects within countries that have the potential to influence and steer changes within education systems.
  1. Strategic partnerships with key actors in education at the regional and international levels will be built or strengthened in order to: (1) mobilize funding and leverage influence towards sector reforms; and (2) develop programmatic synergies for greater impact of partners’ interventions.
  1. Quality assurance and oversight will be provided by the Regional Office in order to improve the quality, effectiveness, and sustainability of UNICEF country programming approaches in education.
  1. Technical assistance will be made available to Country Offices through maintaining and nurturing high calibre technical expertise at regional and international levels in relevant areas of education.

Significant resultsin terms of changes in children’s life (impact results)have been achieved during this period in 5 countries: Armenia, Kosovo[1], Serbia, the Former Yugoslav Republic of Macedonia and Turkey.

In Armenia: (1) The number of children in special schools decreased by 60% in 2011-12 compared to 2007-2008 (National Centre for Education Technologies); and (2) The percentage of children with certified disabilities attending regular schools is 70% (Source: UNICEF Armenia Survey on the access of children with disabilities to education, health and social protection services); in regions with higher concentration of inclusive schools (Yerevan and Tavush) where UNICEF programs have been implemented, the proportion is higher - 75 and 80 % respectively.

In Kosovo, the proportion of drop-outs at primary and lower secondary education levels was divided by three in 8 years, from 1.67% in 2002/03 to 0.5%2010/11.In addition, the gender parity index in primary school increased from 0.79 in 2004 to 0.93 in 2011, hence implying that more girls are now participating in primary education.

In Serbia, the primary school enrolment of Roma children increased from 66% in 2005 (Source: MICS 3) to 91% in 2010 (Source: MICS 4).

In The Former Yugoslav Republic of Macedonia (TFYRoM), the primary school enrolment of Roma children increased from 61.1% in 2006 (Source: MICS 3) to 85.6% in 2011 (Source: MICS 4).In addition to that:(1) Fourth grade student knowledge in numeracy increased by at least 10 per cent against the baseline; (2) Early grade teachers (1-3 grades) knowledge in numeracy increased by at least 10 per cent against the baseline; (3) Fourth grade student knowledge in literacy (reading and writing) increased by at least 10 per cent against the baseline; and (4) Early grade teachers (1-3 grades) knowledge in literacy (reading and writing) increased by at least 10 per cent against the baseline (Source:assessments in progress of student outcomes and early grade teacher knowledge).

In Turkey: (1) It is estimated that about 350,000 children (250.000 girls and 100.000 boys) were enrolled in schoolas a result of the 2003-2007 Girls Education Campaign(Source: Ministry of National Education); and (2)Gender disparity in primary education has been reduced from 7.15% in 2003 to 0.37% in 2010-2011 (Source: Ministry of National Education).

The results listed above– which translate into concrete changes in the life of thousands of children -were made possible because of the reforms in education systems achieved over the past 10 years by governments with the support of their partners,including UNICEF, both at country and regional level.

These concrete changes in the life of children are measured by the following impact indicators:

  • Primary, secondary and pre-primary school enrolment rates, disaggregated by gender, location, personal characteristics (disability) and ethnicity (Roma, Ashkali and Egyptians) ;
  • Number/percentage of children out of primary, secondary and pre-primary school, disaggregated by gender, location, personal characteristics (disability) and ethnicity(Roma, Ashkali and Egyptians);
  • Gender party index;
  • Percentage of children with special needs attending regular schools versus special schools;
  • Percentage of increase in grade 4 student outcomes in literacy reading and writing) and numeracy;
  • Participation in preparatory pre-school programme.

RATIONALE

The following section is meant toexplain the regional theory of changethat led to these impact results, and to which UNICEF made a contribution. A Theory of change is defined as a “Blueprint of building blocks needed to achieve the long-term goals of a social change initiative[2]”. In this case, it refers to the reduction of equity gaps and impact results achieved through changes in the national (regional/local) systems, partly due to the contribution of UNICEF.

Theory of Change

The reality is that 10 years ago, there was no explicit theory of change in the CEE/CIS region, neither in the Regional Office, nor in Country Offices. However, what has guided the work of UNICEF in this region over the past decade (regional story line or pathway) is a consensus that the progressive realization of child rights and reduction of equity gaps is best achieved through changes in systems at national/sub-national levels, and that sustained UNICEF engagement through its core roles contributes to these system changes (see also the Generic Theory of Change underlying UNICEF’s approach in the CEE/CIS region).

So, this TOR attempts to re-construct a regional theory of change which has been implicit (regional pathway), is based on the mainstreaming the UNICEF Child-Friendly School framework in education systems at national/regional/local, and which contributed to significant changes in children’s life, as measured by the impact results listed above. UNICEF being a very decentralized organization (the Country Programme is central to its work), this theory of change was led, facilitated and supported by the Regional Office (as described below) and implemented in various ways (but with similar goals: the progressive realization of child rights and reduction of equity gaps in basic education) by Country Offices.

Since 1995, UNICEF’s work in education globally has been guided by the Child-Friendly School (CFS) approach. Initially, the CFS concept was presented as an ‘umbrella’ under which the diverse activities and goals of UNICEF’s work on schools might be consolidated and rationalized. By early 2000, UNICEF expanded the definition of quality for key elements of child-friendly schools. By the end of 2001, UNICEF had conceptualized a comprehensive and complex CFS approach which is a framework for rights-based and child-friendly educational systems and schools, which are characterized as inclusive of all children, effective for learning, healthy and protective of children, gender-sensitive, and promoting the participation of children, families and communities.

The CFS concept was designed as a tool for fulfilling the right of children to have access to an education of good quality. At the national level, for ministries, development agencies, and civil society organizations, the CFS approach and principles were meant to be used as a human rights-based normative framework for education policies and programmes, leading to child-friendly education systems and environments. At the community level, for school staff, parents, and other community members, the framework was meant to serve as a tool for quality improvement through localized self-assessment, planning and management, and as a means for mobilizing the community around education.

While the number of countries in which UNICEF is promoting and using the CFS approach increased from 33 countries in 2004 to 56 countries in 2007, theCFS package was adaptedbyUNICEF Country Offices to specific country realities and this has resulted in important variations in its application within UNICEF programmes.The main issue has been a tendency to overprescribe on child-friendly schools (project approach) and to under-emphasize normative and upstream work, policy development and capacity building of duty-bearers (system approach).

The Eastern Asia and Pacific region is the part of the world where UNICEF made the fastest and deepest progress in promoting and implementing the CFS approach. In 2005, there were more than 800 schools certified as child-friendly in Thailand, a similar number in the Philippines, and the Regional Office in Bangkok had developed a set of standards and indicators[3] for each of the five components of the CFS framework.

In the CEE/CIS region, the CFS approach was introduced in the context of the breakup of the Soviet Union and Yugoslavia, and the beginning of the transition from socialism to pluralist societies, with education becoming a key vehicle for managing changes in values, respects for child rights and democratic participation. Since the mid-nineties, the focusin the region has been mainly on the ‘effectiveness’ component of the CFS framework, with the implementation of child-centered approaches in teaching and learning such as the Active Learning(AL) projects in Azerbaijan, Bosnia and Herzegovina, Macedonia and Serbia or the Global Education Project in four Central Asian countries.

In some other countries, the focus has been more on the ‘inclusiveness’ component of the CFS framework, with the Girls Education Campaign (project) in Turkey, as well as the inclusion of children with disabilities in Armenia orRoma children in Kosovo and Serbia, among other countries.

The transition period also corresponded to and was followed by a phase of wars and conflicts – between Armenia and Azerbaijan from 1988 to 1994, in countries of the former Yugoslavia (Bosnia & Herzegovina, Croatia and Slovenia) from 1991 to 1995, the civil war in Tajikistan in the 1990s, the Kosovo war from 1998 to 1999…etc. The impact of these conflicts and the need to restore education services led a number of UNICEF Country Officesto focus on service delivery (infrastructure, basic commodities and supplies) and to adopt a fragmented approach.

Indeed, the common feature of education interventions in most of these CEE/CIS countries has been the project approach (pilot projects) in very small numbers of schools (often 10-15), with low potential for replication and expansion at national level, and limited impact on education systems. This fact was confirmed by a number of evaluations commissioned or supported by the Regional Office between 2005 and 2008, such as the multi-country evaluation of Global Education in Central Asia as well as CFS or AL evaluations and assessments in Azerbaijan, Bosnia and Herzegovina, Kosovo, Macedonia and Moldova.

In the mid-2000s, the acquisition of the EU Candidate status by some countries in the region (mainly in the Balkans but also previously Turkey) triggered significant education sector reforms and system changes. In 2005, the Regional Office initiated a Regional Analysis of Education in CEE/CIS (‘Education for Some More than Others?’), which laid the foundations for a regionalvision for Basic Education which, in turn, were later formalized in the Regional Education Strategy and Regional Education Strategy Note.

In 2006, the Regional Office organized a CFS Study Tourin Thailand for12 countries involved in implementing the CFS approach in the region - Armenia, Albania, Bosnia & Herzegovina, Kosovo, Macedonia, Montenegro, Serbia, Tajikistan, Turkmenistan, Turkey, Uzbekistan. This study tour represents a key milestone (or starting point) for the shift made by most countries in CEE/CIS from small CFS projects to a more systemic approach using the CFS concepts and principles as an overarching conceptual framework for policy development and sector reform rather than for applying them at school level only. Returning from Thailand, a number of countries undertook evaluations or assessments of their CFS projects (as mentioned above), some decided to scale up their initiatives (Azerbaijan, Uzbekistan), and others initiated a system approach, working at various levels (central, regional, local) and with different components of education sectors (policy and planning, curriculum, pedagogic centres, teachers training, assessment departments …); this has been the case in Macedonia Serbia and Turkey.