GNCBP Data Report Democratic Republic Congo
This report presents an overview of the data collected by GNCBP in the Democratic Republic of Congo with reference to: 1 .Measurement of program objectives, 2. Alignment with PEPFAR 3.0 strategic direction 3 .Gaps in data submitted to HRSA, 4. Utilization of data for programmatic adjustment, 5. Recommendations for improvement. Data reviewed was obtained from GNCPB annual and quarterly reports submitted to HRSA and external midterm evaluation completed July 2015. GNCBP consists of two subprojects, The Nursing Education Partnership Initiative (NEPI) and General Nursing (GN).
1. The NEPI program was launched in DRC in late 2011 with the aim of increasing the quantity, quality, and relevance of nursing and midwifery education in four institutions: The schools were selected for their locations in regions with high burden of HIV/AIDS.
2. The DRC General Nursing program (GN) was launched in October 2014. It seeks to maintain an appropriately skilled nursing and midwifery workforce; scale-up capacity and retention of nurses, strengthen regulatory bodies; develop and strengthen national nursing and midwifery related strategies and support policy-level nurse leadership
Results: Summary of Data Findings: GNCBP has collected an enormous amount of data on processes and outputs but has been deficient in collecting data on outcomes and impact. The reason for this was the lack of clear evaluation questions and M&E design at the start of the project.
NEPI
A total of 2,746 students graduated from schools receiving NEPI support. In total, 3109 students were enrolled during PY 7, reaching 99.6% of annual target. During quarter 4 a total of 486 students were newly enrolled. In total, 3109 students were enrolled during the project year, reaching 99.6% of annual target. During PY 7 Q 4, 953 students graduated from pre service training programs.
This is 59% of the annual target including 626 students from ISTM Kinshasa, and 327 students from ISTM Lubumbashi. The cumulative total for the project year is 1,407students graduated, or 59% of the annual target.
An exception to decreasing enrollment was made for training midwives or nurses with midwifery skills, since they are in short supply in the country. With NEPI support, first-year enrollment in the midwifery course at ISTM Kinshasa increased from 33 in 2011 to 194 in 2012, then leveled out to 141 in 2013. These cohorts have not yet graduated. At ISTM Lubumbashi, enrollment increased from 57 in 2011 to 82 in 2012, to 130 in 2013.
There were faculties, tutors, mentors or administrators trained to plan and manage nurse workforce development activities. The quarterly target for this indicator (35) was exceeded with a total of 50 faculty trained in research (25faculty) and in curriculum implementation (25 faculty).
Baseline data was not collected on student knowledge and skills. Therefore it was not possible to measure changes to demonstrate effectiveness of innovative teaching methods. There was no instrument identified or implemented to measure competencies of students at graduation, therefore there are no outcomes demonstrating the quality of the graduates. There is an opportunity to measure impact by comparison of pre and post NEPI student enrollment by using retrospective data from 2008 onwards provided by schools. The quality of this data is unknown and there was no standardized data base implemented across NEPI schools to collect student data. DRC struggles to provide employment opportunities to absorb graduate nurses. At this time, there is no system to track deployment of graduates to HIV high volume HIV areas. There is survey data that describes the impact of the NEPI program based on faculty and student perceptions of change before and after NEPI was implemented. This type of data is informative and valuable to supplement quantitative data but it cannot be statistically analyzed to demonstrate impact. Although there were mostly positive changes reported, there was no action taken in areas that were deficient such as HIV competencies (Table 5).
General Nursing There is output data to support general nursing capacity building activities including strengthening of regulatory bodies, national nursing and midwifery related strategies and policy-level nurse leadership.
Alignment with PEPFAR
The NEPI schools were selected for their locations in regions with high burden of HIV/AIDS. Faculty and students have been trained in PMTCT, clinical management of HIV/AIDS and universal precautions to prevent HIV exposure in clinical settings. 586 nursing and midwifery students trained in HIV/AIDS essential competencies. However, there has not been an objective measurement of these competencies. There was strengthening of competency based in-service training for practicing nurses and midwives to provide high-quality HIV care and treatment including PMTCT and Option B+ but these competencies were not measured.
Progress towards sustainability of the NEPI schools should be more closely monitored. This was addressed in the midterm evaluation. While significant progress has been made, the evaluators were not convinced the DRC government is adequately committed and has the necessary resources to sustain GN and NEPI activities. Such commitment is essential to sustainability.
Table 1. Summary of GNCBP Data Submitted
Outcomes / Reported to / Needed / Type of dataQUANTITY
PEPFAR indicator
H2.1 Number of nurses graduated from PEPFAR approved pre service training program
PEPFAR H.2.3.D Number of nurses our midwives completing in service training courses including Option B+ /
OGAC
APR
SAPR / Standardized data base for NEPI schools / quantitative
Quantity of students
Admitted, retained and graduated
Organizational capacity / HRSA annual report
OGAC (number graduated)
NEPI Midterm evaluation
Not reported / student demographics Standardized data base in NEPI schools
organizational assessment / Quantitative data
Targets
Quantitative data
Innovative teaching methods / Not collected / Pre and posttests knowledge and skills / Quantitative data
Quality and relevance of teaching methods / Incomplete
Quarterly reports / Student competencies after trainings and at graduation / Quantitative data
GN strengthen capacity
Nursing councils
Professional organizations
TWG meetings / Quarterly reports
Annual report
Midterm evaluation / Outputs and Qualitative data
Success stories
Varied / 2013, 2014, 2015
Program information
Site visits,
Midterm evaluation, CLASS / Coordination of data / Qualitative data
and quantitative data
Quality / DATIM HRSA / SIMS assessment
Types of data submitted to HRSA (Continued)
See Appendix 1 for list of indicators submitted to HRSA
Progress to targets reported from HRSA quarterly and annual reports. When progress against targets is lower than expected (>25%) explanations for overcoming hurdles are included in quarterly reports. Following submission of reports, program and M&E teams review bottlenecks and discuss how to support country teams to reach expected targets. Trend analysis: NY SI staff reviews data on a semi-annual basis to assess progress made in meeting targets. SI Liaisons share analyses with country teams for further discussion and program planning.
Workplan review: Submitted to HRSA quarterly and following NOAs. Barriers to conducting planned activities are described in quarterly reports and a plan is developed to address program gaps.
Periodic Stakeholder Meeting and Reports: review progress, lessons learned
Midterm evaluation: examines project progress against goals and objectives, analysis of routinely collected data and successes and challenges; results are used to guide activities for the remainder of the project. Completed June 2015.
Final project evaluation: to be implemented at end of project. Retrospective data on number graduated before and after NEPI is available by year and not by school may demonstrate impact. Data on graduates may be included if standardized data base is implemented in NEPI schools.
Organizational Assessment: to be administered at all NEPI supported schools to further assess and document the developments and outcomes associated with NEPI inputs, as well as provide supplementary information on program alignment with PEPFAR 3.0 priorities.
SIMS data on quality assurance reported periodically
DATIM integrated data on epidemiology, quality, EA and SIMS
Outcomes and impact of GNCBP
Outcomes
1. Improve quantity, quality and relevance of nurses and midwives to address essential population-based health care needs, including HIV/AIDS. NEPI assist training institutions and primarily targets pre-service nurse and midwifery education. Quantity is measured by the number students graduated each year and as a proportion of the annual target or percent of target reached.
Target total enrollment for all the schools was lowered from 3,560 students in first year of operations to 3,096 in the second year, and 2,500 in the third year. While enrollment rates were lowered, the schools enrolled more students than was intended in the second and third year—or 3,105 and 2,691 respectively. The reason for over-enrollment might be that publicly funded education institutions do not receive adequate funds from the government and therefore rely on tuition fees to pay for maintenance and other operating expenses
A total of 2,746 students graduated from schools receiving NEPI support. In total, 3109 students were enrolled during PY 7, reaching 99.6% of annual target. During quarter 4 a total of 486 students were newly enrolled. In total, 3109 students were enrolled during the project year, reaching 99.6% of annual target. During PY 7 Q 4, 953 students graduated from pre service training programs. The cumulative total for the project year is 11407. This is 59% of the annual target.
With NEPI support, first-year enrollment in the midwifery course at ISTM Kinshasa increased from 33 in 2011 to 194 in 2012, then leveled out to 141 in 2013. These cohorts have not yet graduated. NEPI supported the training of a total of 1,594 faculty, clinical preceptors and administrators on a broad range of topics from improving administration to clinical mentoring.
2. Identify, evaluate and disseminate innovative models and practices of nursing education hat are generalizable for national scale-up of nursing and midwifery education.
Comment: There is outcome data was collected on quantity of students enrolled and graduated. There is little data on quality. There was no measurement of student competencies at graduation or at clinical facilities. Innovative models of nursing education could not be evaluated since baseline data was not collected.
3. Strengthen capacity to plan and manage nurse and midwifery workforce development through advocacy initiatives, national regulatory authority, regional partnerships for technical and capacity building, support for nursing policy and regulatory development, retention and leadership. Comment: There is a great deal of output data on strengthening capacity. (Appendix 1)
Output data reports activities for strengthening capacity to plan and manage nurse and midwifery workforce development through advocacy initiatives, local and regional partnerships for technical and capacity building, support for nursing policy and regulatory development, retention and leadership, development of nursing council. There is a Quality Improvement Initiative: 2014/2015 to build capacity of nursing leaders from DRC to lead and support high-impact quality improvement initiatives
Impact
To measure impact, a variety of qualitative and quantitative methods may be used to build the evidence. There is an opportunity to measure change in the quantity of graduates to demonstrate impact. NEPI-supported schools provide preliminary retrospective data on enrolment and graduation totals from 2008-present (Table 2). In this way, pre and post NEPI data on enrolled students and graduates could be compared to demonstrate impact. This initiative captures data that had not previously been collected. By the end of Q4, all country teams submitted an adapted M&E plan that includes specific school level data collection forms and data quality assurance (DQA) plans. The disaggregation of data at the school level allows for more robust tracking of outcomes of students. See table below on data available on enrolled students pre and post NEPI.
Table 2. Annual Enrollment in DRC Nursing Schools Before and After NEPI
Survey data from midterm evaluation reports the impact of NEPI by changes in students and institutions as reported by faculty. This type of data is valuable and indicative of an impact but sample sizes were small and data is based on perceptions of change and cannot be statistically analyzed to demonstrate impact. (Tables 3, 4, 5)
The tables below report survey data on the impact of NEPI on the institution and student body based on perceptions of faculty. Survey data from faculty reports competencies in the management of HIV and PMTCT is low (see highlights). This data could be used for program improvement but there was no mention of data utilization from this survey to address highlighted areas. Another area identified from the survey as needing improvement was confidence of students in treatment of HIV patients.
Table 3.Impact of NEPI in the institution and student body since program implementation. Perceptions by Faculty (n=40)
Overall impact of NEPI program / Average rating (from -5 to +5)Averaged rating from -5 (very negative impact) to +5 (very positive impact), and 0 = ‘no impact’
Overall level of impact on your training institution from all NEPI/GN funded resources and training programs? (rating -5 to +5) / + 3.8
Changes in student competencies to address HIV/AIDS Faculty have observed since implementation of the NEPI program / No change / Small changes / Big changes
Competencies relevant to national health priorities / 3% / 50% / 47%
Competencies in the management of HIV/AIDS & co-morbidities / 17% / 60% / 23%
Competencies in HIV/AIDS prevention & treatment (PMTCT, ART) / 10% / 58% / 32%
Changes in student characteristics Faculty have observed since implementation of the NEPI program / No change / Small changes / Big changes
Student skills & knowledge / 6% / 56% / 39%
Clinical reasoning/problem-solving/critical thinking/reflection / 3% / 61% / 36%
Assessment/exam scores (e.g., OSCE, license exam) / 8% / 69% / 23%
Study practices / 6% / 61% / 33%
Overall performance/abilities / 0% / 59% / 41%
A more consistent level of knowledge & skills across the cohort / 18% / 53% / 29%
Integrating theory into practice / 6% / 32% / 62%
Future leadership potential in nursing / 3% / 55% / 42%
Professional behavior in the wards and skill labs / 3% / 39% / 58%
Reduced drop-out / 7% / 50% / 43%
Progress towards independent practice / 0% / 41% / 59%
Use of learning technologies / 21% / 36% / 43%
Changes in attitudes Faculty have observed in students since implementation of the NEPI program / No change / Small changes / Big changes
Confidence in their knowledge & skills / 3% / 46% / 51%
Confidence in caring for HIV/AIDS patients / 12% / 49% / 39%
Motivation to learn / 3% / 39% / 58%
Morale / 6% / 42% / 53%
Commitment to patient care / 0% / 38% / 62%
Intention to stay in nursing / 0% / 26% / 74%
T