Report on the Qualitative Evaluation of the Global Fund Round 7 LLIN Distribution Activities in the Urban Areas of Kampala and Wakiso, Uganda, 2010
This report was written by the Stop Malaria Project
June 2012
Suggested quotation:
Strachan C, Gudoi S, Balayo C, Leitao J, Protopopoff N, Anguko A, Akora C, Meier J, Magumba G, Meek S. Report on the Qualitative Evaluation of the Global Fund Round 7 LLIN Distribution Activities in the Urban Areas of Kampala and Wakiso, Uganda, 2010. Malaria Consortium/Stop Malaria Project, June 2012.
Table of contents
List of acronyms
Executive summary
1. Introduction
2. Methodology
2.1. Data collection
2.2. Analysis of data
2.3. Limitations
3. Key findings
3.1. Macro-planning
3.2. Micro-planning
3.3. Sensitisation
3.4. Training
3.5. Mobilisation of communities and behaviour change communications
3.6. Registration of beneficiaries
3.7. Analysis of households’ registration data
3.8. Distribution of LLINs to beneficiaries
3.9. Hang up
3.10. Reporting
3.11. Support supervision
3.12. Monitoring
3.13. Follow up
4. Discussion
5. Recommendations
5.1. General recommendations
5.2. Recommendations for urban distributions
List of acronyms
ADRAAdventist Development and Relief Agency
BCCBehaviour Change Communication
CSOCivil Society Organisation
DHTDistrict Health Team
DPDistribution Point
GFATMGlobal Fund to Fight AIDS, TB and Malaria
GRNGoods Received Note
IECInformation Education Communication
LCLocal Council
LLIN Long Lasting Insecticidal Nets
MCMalaria Consortium
MFPMalaria Focal Person
MoHMinistry of Health
NMCPNational Malaria Control Programme
NDANational Drug Authority
PACEProgram for Accessible Health, Communication and Education
PMIPresident’s Malaria Initiative
SMPStop Malaria Project
ToRTerms of Reference
UBOSUganda Bureau of Statistics
Executive summary
In August and September of 2010, Uganda’s first distribution of Long Lasting Insecticide Treated Net (LLINs) aiming at universal coverage in urban areas took place in Kampala and Wakiso districts. The distribution was funded by Round 7 of the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM), through which a target of 17,666,984 LLINs were to be distributed in Uganda through rolling mass distributions in two phases. Phase 1 involved a distribution of 7,295,850 LLINs to cover the vulnerable population groups, pregnant women and children under five in 78 districts of Uganda. In the districts of Kampala and Wakiso, three Civil Society Organisations (CSOs), namely the Program for Accessible Health, Communication and Education (PACE), the Adventist Development and Relief Agency (ADRA) and Crane Health Services, led the delivery of the LLINs with support from the Ministry of Health and the USAID/PMI funded Stop Malaria Project.
It was decided by the Ministry of Health (MoH) that the distribution of LLINs in Kampala and Wakiso, given their size and urban nature, would require a slightly different approach from the conventional model used in Uganda, which has been developed based on the rural context. The MoH approved LLIN campaign distribution guidelines were not formally adapted to suit the urban context but a number of changes were made in practice.
An evaluation of Phase 1 of the distribution in Kampala and Wakiso was conducted between October 2010 and January 2011, led by the MoH in coordination with Malaria Consortium, which undertook this activity on behalf of SMP. The objective of this study was to evaluate this LLIN distribution activity in order to review the value and appropriateness of any operational adaptations made to suit the urban context and to make practical recommendations for a more formal adaptation of the distribution guidelines to suit the urban context to be used in subsequent distribution phases, particularly Phase 2 in Uganda.The review was conducted as a retrospective evaluation using qualitative methodologies, primarily utilising key informant interviews (KIIs) and focus group discussions (FGDs). All data was collected between October 2010 and January 2011.
In practice, specific operational changes were made to key aspects of each distribution phase during both the macro- and micro-planning processes as well as during implementation itself, in response to the differing demands raised by the urban context, including relating to behaviours, attitudes, lifestyles and knowledge of malaria and LLINs, infrastructure, security and community structures.Whilst many of these field level adaptations were effective in meeting the aims of the specific distribution phases, others were less effective, and the lack of uniformity in planning and delivery across some activities created challenges for central level coordination, support supervision and resource allocation.
Valuable experience has been gathered from this distribution which, if utilised effectively in the planning of subsequent distribution rounds in urban areas, may result in better overall outcomes, including higher targeting and coverage, and ultimately higher levels of sustained use of nets.It is recommended that a formal adaptation of components of the distribution guidelines be made for LLIN campaign distributions taking place in urban settings in Uganda. This will require more time, commitment and resources to be invested during the earlier planning phases. This will ultimately enable a more efficient allocation of resources to the distribution and help ensure more effective planning and the provision of appropriate support at all stages. The activity phases which require most operational adaptation to suit the urban context include macro- and micro-planning, mobilisation and behaviour change communications (BCC), registration, distribution and follow up.This evaluation highlights specific recommendations for the planning and delivery of an effective distribution of LLINs in urban areas of Uganda.
1. Introduction
Under Round 7 of the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM), a target of 17,666,984 LLINs were to be distributed in Uganda through rolling mass distributions in two phases. Phase 1 involved a distribution of 7,295,850 LLINs to cover the vulnerable population groups, pregnant women and children under five, in 78 districts of Uganda in 2010. Under Phase 2,which is expected tobe initiated during 2012, the remaining gaps will be ‘filled’ in order to reach universal coverage. In Uganda, universal coverage is defined and quantified as ‘one net for every two people’.
At the beginning of 2010, the President’s Malaria Initiative (PMI) undertook support for the distribution of the GFATM Round 7 LLINs within the 15 districts in the Central region of Uganda, channeled through the Stop Malaria Project (SMP)[1]. Of the 15 districts, it was decided by the Ministry of Health (MoH) that Kampala and Wakiso, given their size and urban nature, would require a slightly different approach from the conventional model used in Uganda, which has been developed based on the rural context. The distribution in these two districts thereforetook place at a later date, after further consideration of an appropriate urban distribution strategy. The MoH approved LLIN campaign distribution guidelines were not formally adapted to suit the urban context but a number of changes were made in practice. Phase 1 of the distribution in Kampala and Wakiso was conducted during August and September of 2010.
In the districts of Kampala and Wakiso, SMP worked closely with three Civil Society Organisations (CSOs) to deliver the nets, namely the Program for Accessible Health, Communication and Education (PACE), the Adventist Development and Relief Agency (ADRA) and Crane Health Services.Both ADRA and CraneHealth Services were selected as sub-recipients under the Global Fund grant, with management support from PACE. In coordination with the National Malaria Control Programme (NMCP), SMP was responsible for the provision of the overall technical and operational support supervision tothe CSOs to ensure an effective distribution ofLLINs in line with the implementation guidelines, as agreed by the MoH and partners.
An evaluation ofPhase 1 of the distribution in Kampala and Wakiso was conducted between October 2010 and January 2011, led by the MoH in coordination with Malaria Consortium, which undertook this activity on behalf of SMP. The objective of this study was to evaluate this LLIN distribution activity through qualitative means in order to review the value and appropriateness of any operational adaptations made to suit the urban context and to make practical recommendations for a more formal adaptation of the distribution guidelines to suit the urban context to be used in subsequent distribution phases, particularly Phase 2 in Uganda, as well as beyond. This was the first urban distribution of LLINs at district wide scale ever to conducted in Uganda and thus it was critical to document and review the experience for future benefit. The literature on urban LLIN distribution strategies is also scarce and it was felt important to add to this evidence base.
Specific objectivesof the evaluation included:
- To capture and document the lessons learnt from the Kampala and Wakiso distribution, given its specific urban nature, in relation to all aspects of the distribution model outlined in the MoH distribution guidelines
- To review the support supervision needs of the CSOs to ensure the most effective support supervision in subsequent distribution rounds
- Provide valuable information for comprehensive reporting to the Global Fund
- To add to the evidence base on LLIN distribution models in urban contexts
Page | 1
2. Methodology
The review was conducted as a retrospective evaluation using qualitative methodologies, primarily utilising key informant interviews (KIIs) and focus group discussions (FGDs). This approach enabled opinion to be gathered from all key stakeholders involved in the distribution on the conduct of different activity phases of the distribution, including aspects which were successful as well as challenges faced. All data was collected between October 2010 and January 2011.
2.1. Data collection
Participants were selected purposively from specific strata, as outlined in Table 1. A total of 11 FGDs and 35 KIIs were conducted, with a total of 140 stakeholders in total interviewed. All KIIs were semi-structured in nature and topic guides were developed for each strata. The guides were adapted from those used in the Central region evaluation of the LLIN distribution[2] conducted in May 2010 and therefore most did not require further pre-testing, although the VHTs and beneficiaries guides were pre-tested as these groups were not previously targeted.
All FGDs took place at a central location in Kampala, except for discussions with VHTs for which a more local, easily accessible location was identified in each district. The KIIs were conducted in places of convenience for the informants, usually their offices. FGDs typically took two hours and the KIIs lasted on average one hour and forty five minutes. Each FGD had a maximum of 12 participants.
All data collectors were trained in qualitative data collection techniques and good practice, and the specific data collection tools, process and requirements. The process of data collection was supervised by SMP/Malaria Consortium staff. All tools and data collection were written and conducted in English, apart from the FGDs with VHTs and beneficiaries which were conducted in the local language, Luganda, with transcripts later translated in English for the analysis. All KIIs and FGDs were recorded to facilitate verbatim transcription. All quotes included in the Key Findings section of the report are coded with the strata code (A, B, C etc) and the data collected method noted.
Table 1:Data collection stratum
Strata / Expected role in distribution / Data collection method[3] / No. of participants / Location / Notes on sampleNMCP / A / Oversight of distribution / KII / 2 / Central / Comprising LLIN and Behaviour Change Communication (BCC) focal points
CSOs / B / Implementers of distribution / KII / 2 / Kampala, Wakiso / One representative from each of the implementing local CSOs, ADRA and Crane Health Services
DHTs / C / Key role in planning and coordination at the district level / KII / 4 / Kampala, Wakiso / Including Malaria Focal People from both districts
Central supervisors / D / Supervisors recruited by the NMCP to support the CSOs and sub-county supervisors / FGD / 5 (1 FGD) / Kampala, Wakiso / -
Sub-county supervisors / E / Support supervision of VHTs across designated sub-counties in coordination with the CSO staff / FGD / 24 (2 FGDs) / Kampala, Wakiso / Purposively selected from sub-counties where the distribution was considered to have gone well and where there were challenges
VHTs / F / Community focal point for distribution, lead on village level mobilisation, registration, actual distribution and follow up / FGD, KII / 44 (4 FGDs), 1 / Kampala, Wakiso / 2 FGDs in each district. Villages purposively selected based on where the distribution was considered to have gone well and where there were challenges
Political leadership, district level (LC5s) / G / Involvement in macro-planning at the district level and communication and advocacy support / KII / 2 / Wakiso / Kampala district leadership were not available for interview. They didn’t consider themselves to be involved in the distribution
Political leadership, sub-county level (LC3s) / H / Involvement in micro-planning at the sub-county level and communication and advocacy support / KII / 4 / Kampala, Wakiso / Secretaries of Health, District Health Inspectors, and District Councillors were targeted
Political leadership, community level (LC1s) / I / Involvement in micro-planning at the community level and communication and advocacy support / KII / 20 / Kampala, Wakiso
Beneficiaries / J / Community members receiving the nets / FGD / 32 (4 FGDs) / Kampala, Wakiso / 2 FGDs from each district. Intentional as well as actual beneficiaries were targeted thus selection was from the community level as opposed to the registration lists
2.2. Analysis of data
Qualitative research methods involve the systematic collection, organisation, and interpretation of textual material derived from talk or observation[4] and a systematic approach to the cleaning and analysis of data is critical to ensure conclusions represent well the opinions of informants or activity observed.
Thematic analysis of the qualitative data broadly followed the ‘framework approach’[5]. The framework approach has been developed specifically for applied or policy relevant qualitative research in which the objectives of the investigation are typically set in advance and shaped by the information requirements[6]. For these reasons, although the framework approach reflects the original accounts and observations of the people studied (that is, “grounded” and inductive), it starts deductively from pre-set aims and objectives. The analysis followed five distinct stages:
- Familiarisation—reading of all the data, with key emergent themes identified
- Identifying a thematic framework or coding frame—identifying all the key issues, concepts, and themes by which the data could be examined and referenced were identified. This was carried out by drawing on a priori issues and questions derived from the aims and objectives of the study as well as issues raised by the respondents themselves and views or experiences that recurred in the data. The end product of this stage was a detailed index of the data, which labeled the data into manageable chunks for subsequent retrieval and exploration
- Indexing—the thematic framework or index was applied systematically to all the data in textual form so all data was coded by (sometimes multiple) themes
- Charting—thematic areas were distilled into summaries
- Mapping and interpretation—each thematic area wasanalysed and contextualized
2.3. Limitations
All data collection and analysis was quality controlled through the selection of appropriate data collection methods, the selection of experienced data collectors based on specific criteria, the delivery of high quality training by qualitative research experts, effective support supervision throughout the data collection phase, an emphasis on verbatim transcription, double coding of randomly selected transcriptions and discussion as to the emerging themes amongst key authors throughout the analysis process. Observation would have enhanced the study outcomes although the wide range of stakeholders targeted in the study enables capture of second-hand observation data. The outcomes of this study would benefit from a review in the context of quantitative findings, yet no quantitative follow up survey was budgeted as part of this activity and at the time of writing the monitoring and output data were not fully cleaned and available.
Page | 1
3. Key findings
Key findings have been represented according to each key activity phase, as outlined in Figure 1. Attempts have been made to effectively represent the full range of discussions, while giving appropriate emphasis to more frequently made feedback. The different strata/stratum or stakeholder group(s) have been mentioned as far as possible where it is felt the source is relevant for appreciating the context of the feedback.
Figure 1: Key distribution phase or activity
3.1. Macro-planning
3.1.1. Overview of activity phase
According to the guidelines, the macro-planning was to be conducted by three different sub-committees; the Logistics Sub-Committee (LSC), the Operations Sub-Committee (OSC), and the Advocacy and Social Mobilisation Sub-Committee (ASMSC). All committees were led by the MoH/NMCP and in particular the Central Coordination Committee (CCC).The macro-planning phase included the identification and selection of the target area for the campaign, estimation of population by geographic sub-units, LLIN quantification, and initial planning in relation to logistics, human resources, budgeting, timeframes, LLIN supply chain and distribution route, storage and security. The development of training materials and review of stakeholder overall roles,as well as agreement on support and tools required to conduct the various activities, were also part of the macro-planning process.
3.1.2. Adapting implementation guidelines to the urban setting
In general, all stakeholders reported that there was no MacSome key aspects of the adaption are included here as they related to the macro-planning process but other aspects of the adaptation are included in relation to each of the distribution phases throughout the Key Findings section.
“We didn’t have specific methods identified at the beginning for the urban context. Plans were general really. Then we came to operationalise it, we split by district - each district had its different plan. Kampala and Wakiso districts led the plans...” (A-K)