Assessing Provincial and District Health System Capacity to Sustain HIV/AIDS Care and Treatment Services—A Literature Review
Ilona Varallyay, Jennifer Yourkavitch, and Eric Sarriot. ICF Macro- 2010
Introduction
As PEPFAR transitions from an emergency response to promoting sustainable country programs,[1] there is a need to create a benchmarking process through which one can determine the status of the transition of service delivery from implementing organizations to local entities. ICF is creating a rapid assessment process structured around standard health system domains to monitor provincial and district MOH capacity, and thereby to assess the status of the transition efforts. Issues of local ownership and institutionalization of programs within national structures are now key priorities and critical to these aims is the transition of a wide range of essential functions to national government structures. As this transfer of responsibilities gains ground, the role of peripheral/sub-national management structures, such as those at district or provincial level, will become increasingly important in making this local ownership viable. The question then arises, how can we assess the readiness and/or capacity of these lower level government structures (at the district or province levels) to undertake this transition? To address this question, this literature review aims to inform the development of a new tool and process to assess sub-national level capacity of the health system.
This literature review supports the effort to strengthen analytical approaches to assess provincial and district health system capacities to support HIV services by providing a rationale for the inclusion of selected core domains of assessment, and by identifying certain methodologies and contextual factors to consider when designing the assessment tool. The authors reviewed more than 40 sources on health systems strengthening, specifically for HIV services and other relevant areas, and discussed related issues with health systems experts (Annexes 1 and 2). The findings and recommendations gleaned from this review are summarized in this document. This literature review lays a solid foundation for developing a rapid assessment tool on provincial and district health system capacity to sustain HIV care and treatment services.
Methodology of the review
We performed a systematic review of published and unpublished (grey) literature on the assessment of health systems capacity at various levels and also specifically relating to HIV service delivery, as described below.
Search Strategy
We identified sources from a systematic search of computerized databases (Medline, Popline, Dialog –includes Global Health, Federal Research in Progress (FEDRIP), EMCare, The Lancet), a search of the electronic archives of relevant international organizations using the Google search engine (Abt, Health Systems 20/20, MEASURE Evaluation, WHO, the World Bank, FHI, etc.), and by studying the bibliographies and reference lists of identified sources.
Searches used combinations of the following keywords:
· Peripheral health system capacity \ District health team capacity \ Provincial level capacity
· HIV/AIDS care and treatment \ HIV integration\ HIV \ HIV infections \ HIV transmission
· Health Services Performance \ health planning
· Management capacity \ Essential functions \ Supervision of
· Assessment of… \ Review of… \ Assessment tool \ Methodology \ capacity assessment
Temporal limits were set to include sources from the past 20 years; language restrictions were set to include references in Spanish, French, and English.
Article Selection
Documents were eligible for inclusion in the literature review if they addressed health systems strengthening at the sub-national level; if they focused on health systems assessment at a broader level but had lower level applications; if they addressed ART programs specifically; if they addressed capacity assessment along one of the core domain areas[2]; or if they addressed issues of sustainability of health services.
For HIV-related resources, a temporal limit was set to all documents dated post-2001, as this was the approximate time that international organizations started scaling up the introduction of ARVs in developing countries.
Overall, 43 documents were identified, including 17 assessment tools, 7 of which were HIV-focused.
Key Informant Interviews
In addition to the literature reviewed, we conducted semi-structured key informant interviews with several international health experts, whom we felt could contribute to the discussions around the selection of domain areas for the development of this tool. The selection of these contacts was based on initial recommendations from colleagues involved in health systems strengthening and facility assessment work, and then spanned out as each contact made references to others in a relevant field. Individuals from Abt Associates Inc, AIDSTAR-Two, MCHIP, MSH, USAID, WHO provided information included in this literature review. All contacts with informants took place between April 26th and May 11th, 2010 (See Annex 2 for list of informants).
District Capacity within a Health Systems Perspective
The dominant framework for assessing the capacity and performance of the health system as a whole is the WHO Six Building Blocks Model (WHO, 2007). This model breaks the health system functions into 6 broad categories (Figure below):
1. Service delivery: packages; delivery models; infrastructure; management; safety & quality; demand for care
2. Health workforce: national workforce policies and investment plans; advocacy; norms, standards and data
3. Information: facility and population based information & surveillance systems; global standards, tools
4. Medical products, vaccines & technologies: norms, standards, policies; reliable procurement; equitable access; quality
5. Financing: national health financing policies; tools and data on health expenditures; costing
6. Leadership and governance: health sector policies; harmonization and alignment; oversight and regulation
Source: WHO. Everybody’s business: strengthening health systems to improve health outcomes : WHO’s framework for action. World Health Organization, Geneva (2007).
Most of the more recent health systems literature and tools adhere to this comprehensive conceptualization. We focused our review on the operational definition and measure of capacity at provincial and district levels and did not find reason to challenge the ‘building blocks’ as an overarching model for looking at capacity and performance in the health section. However, given the importance of community-level interventions in HIV prevention and treatment work, particularly around linkages between the community and health facilities, we added a seventh building block, ‘community linkages,’ which looks at various aspects of client participation or engagement in the production and consumption of health-related services (for example, outreach services, referral mechanisms, the role of community health workers, etc.) This inclusion is consistent with WHO’s conceptualization of a health system.[3]
It is useful at this point to state important premises of our review.
Capacity is not Performance
Organizational Capacity is the ability of an organization to meet its mandate and achieve its objectives.[4] Performance is how effectively the same organization implements its activities and delivers its services. Those two concepts are obviously related, but they are distinct. Performance can be assessed by the delivery of a good or service to clients. This can be translated into a relatively small number of simple metrics. A given organizational performance (i.e. providing a skilled care provider, trained and equipped in a timely fashion to a client of counseling services) requires the expression of different capabilities from a district: sound human resources management, from recruiting to training to supervising; appropriate resourcing of finances, goods and commodities; sound planning and work organization, etc. Capacity, on the other hand, is a far more multidimensional concept, with strong interdependency between its different dimensions. Its development does not necessarily fit linear patterns. It is important to bear in mind that both capacity and performance are influenced by context. The translation of capacity into performance is affected by the context, and possibly a number of unknown variables. Consequently, the same capacity is not always expressed in the same level of performance because it is affected by external circumstances and intervening events
These two characteristics have provided numerous challenges to the measurement of capacity over the years.[5] Such difficulties also affect the assessment of capacity of health districts, as a linchpin organization within national health systems. The literature is rich in facility assessment tools, but – as shown in this review – far leaner in assessments conducted at district level (and even more so with an HIV/AIDS care and treatment focus).
The District has a unique and pivotal role in the health system
Brown and LaFond again reviewed the state of the art in assessing and measuring capacity in the health sector. In the figure below[6], their model helps visualize how assessing each level has to be analyzed within an overall system.
Under this model, the District fits at the level of the Organization.
Depending on size and political administrative structures of each country, districts report directly to a central level or through a regional administrative level. Provinces sometimes play the role of districts, other times act as regional or quasi-central structures. Where health districts as operational units are essential elements of a health system,[7] the role of the province should be clearly defined and may include monitoring the performance of district health systems, training district-level staff, and conveying central health policies, among others.[8] Our focus here is on structures – district or province – directly overseeing and organizing the work of service delivery units and health workers.
Assessment challenges
The first challenge lies with the understanding and measurement of capacity. The reader is referred again to Brown and LaFond for a thorough treatment of the concepts and measurement challenges. In summary: capacity supports performance, but the relationship between the two is multidimensional, non-linear, and complex.
Then, the unique position of districts within the broader health system creates some challenges in the evaluation of its performance but also its capacity. For example, assessing how a district organizes and implements supervision, or secures drugs and commodities to service providers, is certainly better assessed at health facility level than in the district health office itself. While this tension mostly affects measures of performance, it may on occasion be relevant to the assessment of the district’s capacity as well. Similarly, some elements of district capacity are determined by central level policy or resource allocation decisions. For example, a district capacity assessment may identify human resources weaknesses, but these may be the expression of inadequate policies or interventions at a higher level.
The take-home message is that a ‘district capacity assessment’, by which an assessment is conducted at district level exclusively, must be understood with a proper appreciation for the boundaries and limitations it carries-- that is, some capabilities of the districts are better analyzed upstream or downstream. For example, the capacity of district management to fill vacant positions is limited by policy directives “upstream,” at the national level. Similarly, if we want to better understand the district’s capacity to supervise health workers, we need to ask questions of the health workers themselves (“downstream”). A comprehensive and rapid assessment tool has to be flexible enough to capture elements of the district/provincial capacity that are influenced by, and manifested in, other levels of the health system.
Review of existing assessment models
Nature of assessment methodologies and type of indicators
The tools reviewed included qualitative and quantitative assessment methodologies; a few incorporated both methods. Approaches included self-assessments, facilitated self-assessments, and traditional survey methodologies, albeit mainly at facility level. Assessments varied in the level of applicability from national to provincial to district to facility levels; for purposes of this search, we focused on relevant district and provincial level resources. National level tools that could serve as a guide for the development of relevant sub-national assessment criteria were also included. The most common data collection methodologies among the tools reviewed involved using secondary data, document review, and stakeholder interviews. Some are rapid assessments; others are intended for longer-term implementation.
Both qualitative and quantitative indicators are included, in order to provide a measure of capacity and/or performance and to describe factors that may affect capacity or performance . The different tools capture similar domain areas with distinct indicators; these will be assessed for relevance. Many will have to be modified to fit the context of district/province, according to the roles of the health system at this level.[9]
Domains of assessment at provincial / district levels
Sambo presents a type of framework, which looks specifically at the District level and assesses the functionality of the system according to broader categories, which in sum capture all of the WHO building blocks: 1) the existence of functional district level management structures (village/town health committee, health facility management committee, DHMT, etc) 2) the managerial process (planning, collaboration, guidelines, supervision/ monitoring, drug management, referral mechanisms, HMIS) 3) the health activities/interventions delivered 4) the community health initiatives 5) the availability of locally managed health-related resources (facility budget, cost recovery, human resources, infrastructure, equipment, supplies).[10] This framework does not look at health system performance.
One health system assessment tool which adheres to the WHO framework was adapted for use at provincial level, adjusting the framework for applicability at sub-national level.[11] A close analysis of the specific indicators used at provincial level should inform the development of a district-level assessment. The governance core domain is captured through indicators such as: 1) Responsiveness of government to public needs, 2) Voice of the people 3) Exercising local technical oversight of health service quality 4) Production of services needed by the public 5) Information and reporting 6) Direction, oversight and resource allocation tasks carried by government. The health financing core domain is captured through indicators such as: 1) revenue collection 2) pooling and allocation of financial resources 3) purchasing and provider payments. Service delivery is assessed with indicators such as: 1) availability 2) general access, coverage and utilization 3) service outcomes 4) service delivery access and utilization 5) organization of service delivery 6) quality assurance of care 7) community participation in service delivery. Human resources information such as HR planning, HR policies, performance management, and training and education are assessed. The pharmaceutical management system domain captures elements including relevant budget, policies/laws/regulations, procurement, storage & distribution, appropriate use, access to quality products, and financing. The HIS component includes information on resources/policies/regulation, data collection and quality, data analysis, and use of information for management.