ii
Submitted to:
Sri Lanka Program
Asia Branch
Foreign Affairs, Trade and Development Canada (DFATD)
Submitted by:
Susanne Duska
DADA International Inc.
May 27, 2014
Table of Contents
Acknowledgements iv
List of Abbreviations v
Executive Summary vi
CHAPTER ONE: BACKGROUND 1
1.0 INTRODUCTION 1
1.1 EVALUATION PROFILE 2
1.1.1 Rationale, Purpose and Objectives of the Evaluation 2
1.1.2 Evaluation Criteria, Questions and Strategic Considerations 3
1.1.3 Evaluation Approach, Sampling and Data Gathering Methods 5
1.1.4 Evaluation Challenges and Limitations 7
1.1.5 Structure of the Report 7
1.2 DEVELOPMENT CONTEXT AND NLP PROFILE 8
1.2.1 Development Context 8
1.2.2 DFATD Engagement in Sri Lanka 13
1.2.3 Overview of Official Language Policy in Sri Lanka 14
1.2.4 NLP Project Description 15
CHAPTER TWO: MAJOR FINDINGS 18
2.1 DEVELOPMENT RESULTS AT IMMEDIATE OUTCOME LEVEL 18
Component One – Immediate Outcome 1100 19
Component Two – Immediate Outcome 1200 29
Component Three – Immediate Outcome 2100 35
Component Four – Immediate Outcome 2200 41
2.2 DEVELOPMENT RESULTS AT INTERMEDIATE OUTCOME LEVEL 49
Component One – Intermediate Outcome 1000 49
Component Two – Intermediate Outcome 2000 54
2.3 GENDER EQUALITY AS A CROSS-CUTTING THEME 57
2.4 EFFICIENCY 62
2.5 RELEVANCE 66
2.6 SUSTAINABILITY 68
CHAPTER THREE: CONCLUSIONS, RECOMMENDATIONS, LESSONS LEARNED 70
3.1 CONCLUSIONS 70
3.2 RECOMMENDATIONS 75
3.3 LESSONS LEARNED 77
Annexes 79
1. Sampling Framework 80
2. List of Persons Consulted in Canada 82
3. List of Persons/Groups Met in Sri Lanka 83
4. List of Documents Consulted 86
5. Government Structures and Processes 89
6. Status of Gender Equality in Sri Lanka 92
7. CIDA’s Governance Programming in Sri Lanka – Promoting Language Rights 96
List of Tables
Table 1: Distribution of Budget by Year and Component 17
Table 2: Progress Toward Targets at Immediate Outcome Level 19
Table 3: NLP Media Scans Jan 2012 to Dec 2013 35
Table 4: Breakdown of Model Sites by Type, Region, and Language Context 45
Table 5: Budget Allocations for MNLSI and its Institutions (2011 to 2014) 54
Table 6: Progress Toward Immediate Outcomes, Indicator By Indicator 72
Table 7: Progress Toward Intermediate Outcomes, Indicator By Indicator 74
Acknowledgements
The Evaluation team extends its thanks to those who contributed their insights and thoughtful commentary on matters relating to this evaluation, including DFATD-HQ and the Canadian High Commission.
Special thanks are due to the Ministry of National Languages and Social Integration for accommodating the many interviews and meetings requested at a busy time.
We are similarly grateful to Agriteam and the NLP office staff for providing substantive input and logistical support at every turn. They too were caught in a busy stretch but never failed to be a supportive presence. Canadian technical assistance providers were most generous with their observations.
Also appreciated were the staff of the Asia Foundation who accompanied the Evaluation Team on the visits to model sites. Ample discussions with all partners in the NLP initiative helped to ground our perceptions of how this project has evolved.
Finally, not to be forgotten, are the many beneficiaries who took time out of their day to share their experiences with us.
Susanne Duska, Team Leader
Ramanaish Katheravelu, Team Member
Inoka Priyadarshani, Team Member
List of Abbreviations
CIDA Canadian International Development Agency
CBO Community Based Organization
CSO Civil Society Organization
DAC/OECD Development Assistance Committee of the Organization for Economic Cooperation and Development
DFATD Department of Foreign Affairs, Trade and Development
DOL Department of Official Languages
DSD Divisional Secretariat Division
GDP Gross Domestic Product
GE Gender Equality
GHRD Green Tech Human Resource Development
GISP Governance and Institutional Strengthening Project
GIZ German Aid Agency – Deutsche Gesellschaft fur Internationale Zusammenarbeit
GN Grama Niladari
GNI Gross National Income
GoC Government of Canada
GoSL Government of Sri Lanka
LA Local Authority
MC Municipal Council
MNLSI Ministry of National Languages and Social Integration
MOU Memorandum of Understanding
MoHE Ministry of Higher Education
MPAHA Ministry of Public Administration and Home Affairs
NGO Non Governmental Organization
NILET National Institute for Language Education and Training
NIPA National Integration Promotion Assistant
NIPC National Integration Promotion Coordinator
OLBI Official Languages and Bilingualism Institute
OLC Official Languages Commission
OLP Official Languages Policy
PEF Public Engagement Fund
PIP Project Implementation Plan
PMF Performance Measurement Framework
PS Pradeshiya Sabha
PSC Project Steering Committee
RBM Result Based Management
SWOT Strengths, Weaknesses, Opportunities, Threats
TAF The Asia Foundation
TEM Technical Exchange Mission
TORs Terms of Reference
ToT Training of Trainers
UC Urban Council
UGC University Grants Commission
Executive Summary
The National Languages Project (NLP) has been referred to as a flagship project for Canada because it is an opportunity to engage the Government of Sri Lanka (GoSL) in constructive ways, using Canadian comparative advantage in official bilingualism. The $5.6 million initiative is the result of a design-and- deliver modality, implemented by Agriteam Canada Consulting Ltd. The Project was expected to run for four years and nine months, to March 31, 2016, but activities are expected to wrap up by June 2015.
Evaluation Profile
The Department of Foreign Affairs, Trade and Development (DFATD) commissioned DADA International to undertake a mid-term evaluation of NLP, roughly three and a half years into the project. The rationale was that the project has matured enough to determine whether overall goals are likely to be met, yet early enough to accommodate any lessons learned.
The purpose of the evaluation is to ensure that the targeted public institutions at central and local level are benefitting from the project in accordance with stated goals of institutional strengthening for OLP implementation. The objectives are to assess criteria of effectiveness, efficiency, relevance and sustainability, as well as Gender Equality (GE) as a cross-cutting theme. DFATD provided an extensive list of questions, which were folded into an evaluation matrix and guided interview protocols. Broadly speaking: What worked well? What worked less well? What critical tasks remain?
Four features define the evaluation methodology: 1) Consultative and participatory approach in line with the project’s own orientation; 2) Process analysis to capture change; 3) Contribution analysis to pin-point NLP’s role in the stream of institutional reforms; and 4) Constant comparison to detect patterns and variations. Methods for data gathering included individual and group meetings with almost all key stakeholders in Colombo, and a purposive sample of beneficiaries at local sites. Two local professionals joined the Evaluator on mission to provide interpretation in Tamil and Sinhala as needed, in addition to research and interview tasks as assigned.
Chapter One provides information on context. Chapter Two reports on project achievements, indicator by indicator, at the immediate outcome level, and then at intermediate level. Separate sections deal with gender integration, efficiency, relevance and sustainability. Chapter Three provides conclusions with their analytical backing, recommendations that flow from the conclusions and lessons learned.
Development context and NLP
The language policies of Sri Lanka have been widely cited as a root cause of the 30-year conflict that left a divided citizenry. There is a strong constitutional and legislative framework establishing Tamil and Sinhala as official languages, with English as a link language. Ample regulations and circulars have been issued to govern the implementation of Official Languages Policy (OLP) at all levels of the administrative and political structures of the country. There was no strategy for a coordinated approach, however. There also were serious impediments to bilingual/trilingual public service delivery, now seen as critical for promoting reconciliation between citizens, and between citizens and state.
CIDA began carving out a niche in the area of language rights as early as 2003. There were at least four antecedents to NLP, but results and corporate memory on them largely have dissipated. Most likely they were too brief and not systematic enough to effect change on a sustainable scale.
The time was right for NLP. The Ministry of National Languages and Social Integration (MNLSI) had just been created with a mandate to lead and coordinate the implementation of OLP nationwide. Three institutes were placed under its purview:
§ Department of Official Languages (DOL), responsible for government translation services;
§ National Institute for Language Education and Training (NILET), responsible for training public officials and translators, trainers for the task, and citizens in registered language societies; and
§ Official Languages Commission (OLC), responsible for promoting language rights, and auditing public institutions for compliance with OLP.
NLP has four components: 1) The heart of the project is institutional strengthening of the new Ministry and its portfolio institutions; 2) A small responsive fund is included to support civil society engagement in promoting language rights; 3) Translation service improvement is targeted through university curricula and capacity building of DOL; and 4) Improvement in the bilingual/trilingual service capacity of public institutions is piloted at model sites. Under sub-contract with Agriteam are two core partners: Official Languages and Bilingualism Institute (OLBI) in Canada; and The Asian Foundation (TAF) in Sri Lanka.
Major Findings – Immediate Outcome Level
The Evaluation Team found that NLP offered a systematic and sustained approach to institutional strengthening for the Ministry and each of its institutions. It consisted of: 1) Technical Exchange Missions (TEMs); 2) Visits, reports and follow-up mentoring by Canadian technical assistance providers; 3) Capacity building workshops; 4) Development of resources and tools; and 5) Financial or material contributions. With many results achieved by NLP and its MNLSI partners, it often was a matter of triggering dormant or emergent intention with an infusion of focus and financial leverage.
From TEM reports to technical advisory reports the Evaluation saw a strong continuity of ideas and action plans. Recommendations were readily taken up for the most part, confirming that these inputs were relevant to needs and tailored to absorption capacity. Senior officials met by the Evaluation Team cited several advances in their area of work, but it was not always clear whether this was a direct or indirect result of NLP support. There was general concurrence, however, on the following major achievements, directly attributable to NLP contributions:
ü Development of five-year corporate plans, something that had never been done before with as much rigor and strategic thinking. The process sometimes involved functional reviews, organizational re-structuring and even legal amendment. Such changes can be far-reaching but also lengthy. A wild card for NILET, and even more for OLC, will be the approval of legal amendments for expanded mandates, still in progress.
ü Financing of two surveys to underscore needs in the area of translations (for DOL) and language training (for NILET). Five year corporate plans and the conceptualization of what is needed for expanded mandates require serious justification before the Treasury will consider new allocations.
ü Establishment of a Translation Center, a plan for 15 years and a reality today. Equipped with new computers, translation software and 40 specialized dictionaries, the Center has increased DOL’s productivity substantially. The Canadian Translation Bureau is the model, but it will take more software, more training, streamlining of operations and repositioning within the public service to bring the Center to that level.
ü Model curricula for a degree in translation studies at three universities, again a vague plan that received new impetus. Technical assistance from the School of Translation and Interpretation at Ottawa University made this a reality for the first time in Sri Lanka. The challenge now will be to find teachers capable of teaching in the new practical methods modeled in Canada.
ü Model sites of innovative local service delivery, ready for replication. As noted by the Secretary of MNLSI, officers often felt it was not possible to transform attitudes but the model sites have demonstrated otherwise. Using self-assessment checklists, TAF helped public institutions take stock of their degree of OLP compliance, as a basis for developing language plans. Small funds were made available to take some first steps with trilingual/bilingual signage. The process was systematic and well documented. It provides a clear pathway and the needed tools for replication among different kinds of public institution. MNLSI is currently working with TAF/NLP to develop a replication plan.
ü Production of the “Guidebook for the Preparation of Language Plans by Public Institutions, a key tool for managers and government entities to plan and monitor the implementation of OLP. The guidebook is foundational for bringing all ministries into a comprehensive national plan for OLP implementation, with consistency of approach across all central and local government institutions.
Major Findings – Intermediate Outcome Level
In her meeting with the Evaluation Team, the MNLSI Secretary commented: “Looking at the logic model, we have come to the level of intermediate outcomes, although there is more to be done.”
At this level of outcome it is the comprehensive national plan or roadmap for OLP implementation that is the main results indicator. The vision of the roadmap goes beyond corporate plans. It requires strong baseline evidence to justify an enlarged budget with which to address shortcomings across the nation. The Ministry received intensive technical support from NLP/OLBI to embark on a process of gathering data on the bilingual capacity of 5000 public institutions. The resulting database will inform the language action plans of these institutions and the Ministry’s overarching one. The Evaluation Team is confident that the end result will be a high quality comprehensive national plan, based on the mechanisms in place, and the resources and energy committed by MNLSI leadership and staff.
The national vision for OLP implementation requires a long-term financial commitment from the GoSL. Funding for the Ministry and its portfolio institutes has been trending upward. Action plans for the Ministry and its institutions will be submitted to Cabinet imminently. They will showcase the coordinated, phased approach being taken by MNLSI. Ministry expectations are high that its financial allocation for 2015 will see substantial increase, based on the evidence it will be able to present.
Gender as a cross-cutting theme
NLP has developed a very strong agenda for mainstreaming GE considerations through all its activities. It has promoted these effectively to its partners, including MNLSI, DOL OLC, NILET, PEF awardees, TAF and model site institutions. It has seeded and nurtured core ideas, developed mechanisms to monitor the outcomes, and reported quite systematically. What is needed is more time for results to materialize, some research on the intersection between language rights and gender equality, and possibly some adjusted expectations of what can be achieved