Page | 1
Assessing innovations in malaria control service delivery: Impact evaluation under India’s proposed National Vector Borne Disease Control Program
A Revised Project Concept Note
October, 2008
Table of Contents
List of Acronyms……….………………………………………………..3
Overview and background……………………………………………….4
Why impact evaluation? – A brief overview…………………………….5
What is to be evaluated? ………………………………………………...6
Case Management………………………………………………………..6
Prevention Activities ……………………………………………....…….7
Identification Strategy……………………………………………………8
Size of Study……………………………………………………....……..9
Data Collection and Analysis……………………………………….……9
Survey Instruments………………………………………………….…...9
Main Outcomes Measured………………………………………….……10
Timeline…………………………………………………………….……11
Budget……………………………………………………………………11
References………………………………………………………...……..13
List of Acronyms
ACT / Artemisinin Combination TherapyAPI / Annual Parasite Incidence
ASHA / Accredited Social Health Activist
BCC / Behavior Change Communication
CBO / Community Based Organization
FTD / Fever Treatment Depot
GoI / Government of India
HRP / Histidine-rich protein
IE / Impact Evaluation
IRS / Indoor Residual Spray
ITNs / Insecticide Treated Nets
LLINs / Long-lasting insecticidal nets
MTS / Malaria Technical Officer
NABARD / National Bank for Agriculture and Rural Development
NGO / Non-Governmental Organization
NSS / National Sample Survey
NVBDCP / National Vector-Borne Disease Control Program
PF / Plasmodium Falciparum
PHC / PrimaryHealthCenter
RCH / Reproductive and Child Health
RDT / Rapid Diagnostic Test
SGSY / Swarnajayanti Gram Swarozgar Yojana
SHG / Self-Help Group
SFR / Slide FalciparumRate (# Falciparum Positives / Blood Slides Examined)
Assessing innovations in malaria control service delivery: Impact evaluation under India’s proposed National Vector Borne Disease Control Program
A Project Concept Note
Overview and background
Malaria continues to represent a serious health threat to the Indian population. The World Bank, through preparation of the National Vector-Borne Disease Control Program (NVBDCP), is assisting the government of India to develop the new national response strategy.Among the Project Development Objectivesof the NVBDCP are two critical goals in disease control efforts: (1) to increase the number of people benefiting from effective prevention, including the promotion of long-lasting insecticidal nets (LLINs), and (2) to provide timely diagnosis and treatment services for malaria control, most notably through the introduction of Artemisinin Combination Therapy (ACT) in endemic areas. The proposed impact evaluation studies described in this note will generate valuable evidence to assist in the realization of both goals.In addition, the impact evaluation studies will contribute to NVBDCP intentions to strengthen central and state capacities for evidence-based policy development and program management for effective malaria control.
A strong international evidence base supports the NVBDCPs efforts to promote LLINs. The insecticide treated bednet (ITN) is one of the most efficacious prophylactics against malaria and a leading public health intervention world-wide. Multiple studies have demonstrated the effectiveness of ITNs in reducing malarial incidence, particularly among pregnant women and children.[1]At the same time there is a substantial body of evidence documenting the extremely low rates of ITN use in many malarial areas. For instance, using data for a sample of African countries from 1998-2002, Monaschetal. (2004) noted that the median number of under-fives who slept under any net or ITNs were 15% and 2% respectively.
These results, along with the existence of externalities from vector and carrier reduction from the insecticide[2] has led some scholars to advocate free large-scale provision of ITNs and retreatment as the only feasible provision method (see Curtis et al. (2003) and Sachs (2005), for example).Others have argued that such universal public interventions are not sustainable in the long run and some measure of cost recovery will be necessary to achieve the desired levels of coverage.[3]However, programs which shift the ITN cost to the consumer often report low take up rates.More recent evidence has demonstrated,in an experimental setting, severe reductions in ITN demand when recipients face even nominal charges for use (Cohen and Dupas, 2007). Recent reviews of national programs have also found the most widespread and equitable coverage attained through free mass distribution as opposed to commercial retail and heavily subsidized clinic distribution (Noor et al., 2007). Under the NVBDCP program, two LLINs per household will be distributed at no cost to highly endemic areas through government channels.
Besides cost, other commonly cited reasons for low ITN penetration concern the lack of sufficient informationas well as lack of experience of net adoption and proper usageand hence lowered community acceptance of ITNs. Delivery mechanismspiloted inthisIE work will considerdistribution through alternativenon-state channelsand, consequently, will systematically vary the information and motivation through which households receive bed nets. This approach should assist policy makers better understand effective delivery approaches in the context of endemic regions in India.
In addition to prevention activities, the current quality of malaria case management in endemic areas leaves much scope for improvement – populations in many districts face inadequate access to curative services and delays occur at various stages in the process of malaria diagnosis and treatment. Thesedelays may be due to shortages of medical personnel or medicines, difficult terrain, or financial reasons. Patients are often treated by less than fully qualified practitioners who do not have access to effective malaria drugs.In this challenging environment GOI now hopes to ensure rapid diagnosis and treatment through the enlistment of volunteer Village Health Workers (ASHA) into the case management guidelines and to provide ACTs at the village level in chloroquine-resistant areas. The goal is for all febrile cases presenting to a health worker to be tested and, if positive, treated within 24 hours of onset of symptoms.
The emerging evidence base for the efficacy of ACT is already substantial.[4]The current challenge for the GoIis to achieve its goals for case management within the constraints of the existing rural health system. Case management alternativesregarding the degree of ASHA supervision and management will be evaluated as part of this IE work and will help in this endeavour.
Why impact evaluation? – A brief overview
In an environment of limited resources, policy makers hope to attain priority health targets at the lowest possible cost. Operational impact evaluation is a tool to determine the causal impact of policy innovations, and hence can help guide policy decisions by assessing the cost-effectiveness of competing policy options. Evaluation also provides information that can be integral to sustaining a program insofar as information assists in the negotiation of budgets and informs stake-holders and the press.
Given budgetary and on-the-ground needs and the realities of implementation, only innovations that are able to be adopted by the state or national program should be evaluated. In order to create germane policy lessons, the innovations should be chosen so that they are (a) scalable, and (b) conducted under the normal circumstances and capabilities found in NVPDCP project districts. Evaluating innovations that cannot be replicable within the context of a nationally led health program is in many ways a meaningless exercise for project purposes.
Rigorous evaluation demands thatany observed change in outcomes in areas that receive treatment must be compared with a valid counterfactual area that represents the course of events that would have occurred in the absence of the intervention. This counterfactual proxies for what would have happened without the program.There are various methodological approaches to the construction of a valid counterfactual, but by far the most rigorous evaluation design is an experimental design where treatment/control status is assigned to a locale on a randomized basis. Randomization assures that all units have an equal chance of control or treatment status and satisfies the conditions of a valid counterfactual comparison, that is (1) all pre-intervention factors/characteristics will be on average equal between groups and (2) the only difference in observed outcomes is due to the intervention and not to other observed or unobserved factors. Impact evaluation under the NVBDCP will utilize this randomized treatment/control design.
External validity is a critical concern. A key criterion for intervention choice will be the ability to replicate any successful intervention on a wide scale under the auspices of the overall program. The NVBDCP in general, and the specific activities suggested under the IE component, have received strong expressions of interest and support from every level of government. While the number of study districts will be necessarily small, great effort will be made to screen districts and blocks for representativeness along the dimensions of disease transmission patterns, socio-economic conditions, and government capacity.
Because the main reason to undertake such work is to inform the conduct of the program going forward, we will only consider relatively low cost interventions for evaluation, due to this focus on scalability. One caveat, sincethe study will chiefly be concerned with highly endemic areas and will most likely operate in the highly endemic blocks within highly endemic districts, certain lessons may not be applicable to other areas in India with lower malaria burdens.Nevertheless over 100 districts in India, compromising a population well over 100 million people, are considered to be endemic.
What is to be evaluated?
The rigorous evaluation in service delivery trials that will occur under the NVBDCP will help GoI ensure more effective delivery of case management and more effective distribution of prevention services thus improving priority health outcomes. The impact evaluation component of the NVBDCP will separately focus on two complementary dimensions of health service delivery: case management including provision of treatment, and preventive services with a focus on the distribution of LLINs. Critical lessons learned from previous malaria control efforts in India directly inform the policy alternatives to be evaluated. These lessons include:
-Improving the quality of case management requires more than just policy change: Successful efforts also need include sufficient training and supportive supervision for implementing new policies. Intensified training and supportive supervision by local NGOs will be provided to help ASHAs better diagnose and provide timely treatment to their patients.
-Community acceptance of ITNs is not automatic and requires innovative approaches: Under the proposed IE work, local NGOswill be contracted for distribution and community outreachto increase take-up as well as usage of ITNs
The end result of both components of the proposed IE design is to reduce the burden of malaria in the targeted populations. Each component is discussed in turn.
Case management
Under the new program, the NVBDCP will introduce the revised policy for malaria diagnosis and treatment in a phased manner, prioritizing states and districts with high PF burden. Access to early diagnosis of PF based on rapid diagnostic kits (RDTs) will be implemented in 50districts – concentrated in the states of Orissa, Jharkand, Chattisgarh, and Madhya Pradesh – in the first two years, and an additional 50 districts in the third year. Each district will be stratified according to the risk of PF malaria.In PHC blocks with SFR >= 2%, all fever patients will have an RDT for PF, except if a microscopy result can be available within 24 hours.In other PHC blocks an RDT will only be taken for patients at high risk. This selective use of RDTs should optimize cost-effectiveness.
The new front line health worker for the treatment of fever/malaria in rural areas will be the ASHA, a multipurpose village-based volunteer health worker. ASHA responsibilities will replace the existing Fever Treatment Depots and DrugDistributionCenters, although where these Depots and Centers continue to operate their work will supplement the ASHA’s activities. Her planned responsibilities in regards to Malaria are numerous: 1. conduct routine fever surveillance and intensive surveillance during transmission season,2. conduct rapid diagnostic tests for diagnosis of malaria, 3. act as Fever Treatment Depot with regards to dispensation of medication and collection of slides for diagnosis, 4. refer severe malaria cases to hospitals, 5. assist the health system during malaria outbreak situations, 6. promote the adoption and proper use of protective nets, 7. assist in village coordination and promotion of IRS, and 8. raise general awareness on malaria-control activities.
The ASHA health worker, however, will be the local representative of the new Rural Health Mission and will be responsible for numerous health related activities including family planning, nutrition, and vaccination. Hence the degree to which she will have the resources, support, and time to do her job effectively is yet to be determined. ASHA training for these proposed malaria control activities constitutes a mere 15 hours over a three day period. The relatively short training exposure combined with multiple and competing demands for the ASHA’s time raises the distinct possibility of under-performance with regards to malaria treatment and control.
The impact evaluation component of NVBDCP will pilot an intervention intensifying the support and training ASHA receives in regards to malaria control. Specifically, the intervention will evaluate the effectiveness and cost-effectiveness ofadditional training and supportive supervision in fever and malaria case management provided by competent and contracted local NGOs.Under this designthe level of supervision will ensure that each ASHA worker will be directly supervised at least once monthly. NGP activities will also include enhanced training to the ASHA, BCC for villagers in her catchement area, and a strengthened supply chain of key malaria control commodities such as RDTs and ACTswith the aim of minimizing stock outages and commodity leakages from health center stores.
To the extent possible, the studywill not only evaluate the malaria-relevant impact of additional monitoring and supportive supervision of ASHA workers by local NGOs but also inform ASHA performance in general (including possible substitution away from other activities as a result of enhanced malaria supervision).
Prevention
National policy for use of LLIN in India is currently crystallizing on a position to scale up as rapidly as possible for universal coverage of LLIN target populations.Presently, the target population is village or sub-center areas with intense transmission (indicated by API above 5/1000), and poor accessibility for IRS operations (for example, roads unpassable in rainy season). Two LLINsper household would be distributed by the public sector free of charge.
The early years of the project provide the opportunity to evaluate an alternativemodelof distribution and social mobilization activities to increase the usage at the household level.In this evaluation, the control will be constituted by the public sector distribution of two LLINs to eachhousehold at no cost. Against this control activity, LLINS will be distributed free of charge by local NGOs and with follow-up demand generation activities/community mobilization conducted by the same NGO.
In this design, local NGOs will work with the government distribution mechanisms and take responsibilityfor social mobilization efforts as well as the monitoring of distribution and initial use.NGOS will most likely enlist the help of local Self-Help Groups. SHGshave traditionally been a small group of persons (primarily women) who come together with the intention of accessing micro-credit programs. National and state government initiatives, as well as NGO efforts, haveprimarily used SHGs to implement poverty alleviation programs through microcredit initiatives by NABARD and SGSY.These initiatives require thata SHGconsist of 10-20 members mostly belonging to families below the Poverty Line. SHGs are CBOs that may afford easy and credible access to the community, and thus have the potential of playing a significant role in strengthening prevention activities at the village level.
In addition to the NGOs serving as a distribution channel at the village level, this treatment arm will also include social mobilization processes through the NGOs and SHGs.It has been observed in the HIV/AIDS behavior change that information concerning the risks of infection is not sufficient for a drastic change in behavior literature.[5]More sizable behavior change has been observed in prevention activities that contain skill related information, for example, condom usage under different scenarios.One possible explanation is that behavioral skills increase one’s effectiveness at sorting through obstacles in utilizing a product.Applying these lessons to malaria prevention, SHGs will be instructed in direct demonstration of bednet usage and will in turn present to households with the goal of increasing acceptability and usage of bednets.
Intervention summary description
Case management
Control: Under the Rural Health Mission, ASHA workers will serve as the local frontline representative for fever/malaria control.
Treatment: Case managementwill be supplemented with intensified training, supervision, and support of ASHA by local NGOs based at the block level and each responsible for a maximum of 75 ASHA per block.