Table of Contents
Terms of Reference (TOR) 2
Background 2
Progress in the Implementation of IOWP 3
Objectives of the Process Evaluation and Expected Deliverables 4
Scope of the Work 4
Final output 4
Provision of concrete recommendations for scalability and sustainability of the program. 5
Qualifications (including team composition) 5
Minimum Staffing Requirements 5
Reporting 7
Time Frame 7
Evaluation of proposals 7
Payments Terms: 8
DRAFT
Terms of Reference (TOR)
for
A Process Evaluation of the Integrated Outreach Worker Program (IOWP) under the Project “National Unified Registry and Integrated Outreach Worker Program for Targeting Social Assistance”
Background
The Government of Jordan, with support from the World Bank, is implementing the project "Support to Implementation of a National Unified Registry and Outreach Program for Targeting Social Assistance”. The project development objective (PDO) is to improve the targeting of social safety net programs and developing an efficient outreach mechanism. It has two main components: Component 1: Building and Using the National Unified Registry (NUR) for Targeting; and Component 2: Piloting Integrated Outreach Worker Program (IOWP).
Under the IOWP component, a Case Management System (CMS) has been developed and is being piloted in three Governorates: Zarqa, Irbid, and Ma’an. Through the CMS, social workers are reaching out to the poorest households in these Governorates to (i) identify their needs; and (ii) refer them to locally-available services that meet those needs based on select eligibility criteria. IOWP is managed by the Ministry of Planning and International Cooperation (MOPIC) and is being implemented by the International Youth Foundation (IYF) in partnership with participating service providers. IYF was contracted to develop the CMS methodology and implement in Zarqa, Irbid, and Ma’an to reach out about 23,470poor Jordanian households. The ultimate objective is to transfer the knowledge from the pilot to the Government of Jordan so it can be refined and scaled up. The implementation of the program has been organized in three batches of households (outreach cycles), each of them supported through the following sequence of activities:
Progress in the Implementation of IOWP
IYF has so far reached out to 25,868 households. About 23,471 households have been surveyed, of which 13,200 households were found eligible for referral supporting services and 1,120 have been accepted by the referral services. The cycle of activities for the third and final batch of households is expected to be completed by September 2017.
The IOWP pilot has already developed some basic elements of a modern Case Management System (CMS), including the methodologies for home visits and service referral, Case Management Information System (CMIS), and training of social workers. The CMIS storages and processes information collected from each beneficiary household starting with the needs assessment until the closing of the intervention. One of the main functions of the CMIS is to automatically identify referral services available for providing correct information to the beneficiaries and tracking of the referral process based on select eligibility criteria with each supporting entity .
IYF has deployed and maintained a workforce of about 500 outreach social workers and supervisors. An effective system of supervision and support to social workers and supervisors has been put in place. Data collected through home visits is subject to thorough quality controls, and the CMIS is constantly being enhanced to reflect feedback on the ground.
In order to facilitate the transfer of the IOWP pilot to the government, IYF will be documenting the experience with, lessons learnt, data collected and tools developed under the IOWP. The process evaluation will also play a key role in documenting the IOWP experience including success and challenges for future scale up of the system.
Objectives of the Process Evaluation and Expected Deliverables
This TOR pertains to the selection of firm to conduct a process evaluation of IOWP in three governorates covered by the program.
Scope of the Work
I. Design of the process evaluation methodology based on the Case Management intervention implemented by IYF in the three governorates covered by the program. The methodology will include the set of instruments to collect information from different actors of the program.
II. Analysis of the administrative data collected through the CMIS
1. Admin data on households
a. Profiling of beneficiary households (using available variables and data in CMS)
b. Disaggregation by governorate and locality
c. Disaggregation by cohort
d. Results (at the level of the objectives of the program) – based on the results matrix agreed with IYF
e. Results (at process level – use of performance indicators agreed with IYF)
f. Linkages between profiling and results
2. Admin data on social workers and supervisors
a. Profiling of social workers and supervisors
b. Disaggregation by governorate
c. Performance (based on performance indicators)
d. Linkages between profiling, performance and results at the household level
III. Beneficiary satisfaction survey (representative sample of beneficiaries at national and governorate level)
IV. In-depth/ semi-structured interviews with
1. Beneficiaries (stratified by profile and level of results achieved)
2. Social workers and supervisors in each governorate
3. Service providers in each governorate
V. Online survey to 100% of social workers and supervisors
Final output
Final report of the process evaluation of the program should include, at least:
1. A detailed description of the implementation of the program (main features, work plan, organization of the service delivery, management tools, organization at governorate and locality level, training and qualifying outreach social workers and supervisors)
2. Description and analysis of the effectiveness and efficiency of the program through results achieved including the program’s impact on beneficiaries
i. Based on review of the results matrix of the program (indicators at PDO level and intermediate results, the consultant firm is expected to interpret results regarding the effectiveness and efficiency of the program by government)
3. Assessment of the satisfaction level of social workers and beneficiaries with the program set-up and delivery system (project trainings, meetings, implementation approach and follow-up).
Provision of concrete recommendations for scalability and sustainability of the program.
Qualifications (including team composition)
- Track record in carrying out process evaluations of social and development programs in Jordan using mixed methods.
- Track-record and demonstrated experience in designing, conducting and analyzing data from Focus Group Discussions (FGD).
- Track-record and demonstrated experience in analyzing quantitative data as well as designing, implementing and analyzing data from beneficiary satisfaction surveys.
Minimum Staffing Requirements
The table below presents a tentative team composition. The firm is free to propose an alternative staff allocation plan with proper justification of roles, experiences, and qualification.
Position / Role description / Qualifications / Number of staff1. Team Leader (International or national)
Key / - Overall responsibility for the study
- Elaborate study design based on the TOR
- Selection of respondents
- Finalize topic guide
- Training of interviewers
- Quality control and supervision
- Development of coding methodology
- Oversight of coding
- Analysis
- Report writing / - At least a Masters degree in social science (sociology, anthropology, political science, economics, etc.)
- At least 7 years of experience in analysis of social programs
- Experience leading at least 5 qualitative studies of comparable scale and scope
-
- Excellent proven report writing skills
- Knowledge of the Social Protection sector in Jordan desirable
- Proficiency in Arabic desirable / 1
2. Researchers (one quantitative and the other qualitative (national)
Key / - Supporting of TL in developing methodology, topic guides,
- Training interviewers
- Implementation of coding methodology
- Supporting the TL in analysis and report writing / - At least a Masters degree in social science (sociology, anthropology, political science, economics, etc.)
- At least 4 years of experience in qualitative analysis
- Experience leading analytical work on at least 3 qualitative studies
- Experience in using qualitative analysis software
- Excellent proven report writing skills
- Proficiency in Arabic required / 1
3. Field work supervisor
Key / - Training of field staff (interviewers)
- Supervising and as needed conducting in-depth interviews
- Quality control
- Coordination of logistics related to fieldwork / - At least a university degree in social science (sociology, anthropology, political science, economics, etc.)
- At least 3 years of experience in qualitative analysis of social programs
- Experience supervising at least 3 qualitative studies of comparable scale and scope / 1
4. Interviewers/ facilitators / Conducting in-depth interviews / - At least a university degree in social science (sociology, anthropology, political science, economics, etc.)
- At least 3 years of experience in qualitative analysis of social programs
- Prior experience as an interviewer in at least 2 qualitative studies / 5
5. Support staff / Logistical and administrative support as required / Adequate to the tasks assigned / As required
Reporting
The Consultant will report to the PMU Project at MOPIC, Jordan. All reports should be in English language.
Time Frame
The intended assignment is expected to commence in September 2017 through January 2018. Below is an indicative time frame:
Deliverable No. / Deliverable / Estimated Timeline1 / Draft Inception report / September, 2017
2 / Final Inception Report / October, 2017
3 / Draft Topic Guides and Survey Instrument / October, 2017
4 / Survey and Focus Group Piloting / October, 2017
5 / Survey and Focus Group Implementation / November, 2017
6 / Draft Program Assessment & CMS Data Analysis / November, 2017
7 / Draft Report and Presentation of Survey and Focus Group Results / December, 2017
8 / Focus Group Transcripts and Full Survey Results Transmitted / October – January, 2018
9 / Final Report and presentation (finalization of all deliverables) / January, 2018
Evaluation of proposals
All proposals will be evaluated in two different stages, the first being the technical evaluation which will compose 80% of the grade and the second being the financial evaluation which will compose 20%.
The responses to this RFP will be scored according to the following evaluation criteria:
Category / Description / WeightTechnical proposal = 80%
Specific experience of the Consultant (as a firm) Relevant to the assignment / The Bidder possesses extensive and recent (in the past 3 years) experience in the conduct of at least three (3) projects similar in scope and nature to the work described in this TOR. / 15%
Team Profile / The proposed personnel have qualification, skills and experience similar in scope and nature to the proposed project
Name of each key assignment and their score should be indicated here
Team leader
Researchers
Filed team
Interviewers / Facilitators / 35%
Detailed Project Plan / The proposed plan and methodology for providing the proposed services is adequate, comprehensive to meet the scope and work described in this RFP. / 50%
Financial proposal = 20%
Financial proposal / 20%
Payments Terms:
Payments to the consultant will be made upon the completion of milestone activities (A, B, C, D) according to the following schedule:
Milestone(Deliverables’ Groups) / Deliverables
(see above delivery table) / Percentage of the total value of the contract paid / Estimated Timeline
Group A / (1) Inception report and (4) Piloting and finalized instruments (6) completed analysis of administrative information / 25% / October, 2017
Group B / (5) Survey and Focus Group Implementation status (progress) report
(8) Focus Group Transcripts and Full Survey Results Transmitted / 15% / November, 2017
Group C / (7) Draft report and presentation (and full transcripts transmitted to MOPIC) / 30% / December, 2017
Group D / (9) Final report and presentation / 30% / December, 2017
TOTAL / 100%
4 | Page