EQA Overview
OVERVIEW OF EXTERNAL QUALITY ASSESSMENT (National, TOT and Higher Intermediate Level)
Purpose: / To provide participants with the basics of external quality assessmentModule Time: / 1 hour 30 minutes
Learning objectives: / At the end of this module, participants will be able to:
· Give the definition of EQA
· State that EQA is an indispensable part of a laboratory quality management system
· Name three methods of EQA that are used for AFB sputum smear microscopy:
o Panel testing
o Blinded rechecking
o On-site supervision
· Describe indications for use, advantages and disadvantages of each method
Module Overview
Step / Time / Activity/Method / Content / Resources
Needed
1 / 15 min / Presentation / What is EQA? / Slides 3 - 6
2 / 20 min / Presentation / EQA for AFB Smear Microscopy
- Panel testing
- Blinded rechecking
- On-site evaluation / Slides 7 - 13
3 / 20 min / Presentation / EQA methods: advantages and disadvantages, indications for use / Slides 14 - 20
4 / 30 min / Presentation and Discussion (Situational Analysis) / Identification of the most feasible EQA method for a country / Slides 21 -23
5 / 5 min / Summary / Key messages / Slide 24 - 25
Material/Equipment Checklists
§ PowerPoint slides
§ Computer w/LCD projector
§ Flipchart
Teaching Guide
1 / Module: Overview of External Quality Assessment
DISPLAY this slide before you begin the module. Make sure participants are aware of the transition into a new module.
2 Flipchart
/ Content Overview
(Suggested technique for presentation)
WRITE the content outline before beginning this session.
REFER to flipchart frequently to orient participants to where they are in the module.
EXPLAIN that these are the topics that will be covered in this module.
3 / External Quality Assessment (EQA)
STATE the message on the slide:
MAKE SURE to DEFINE EQA as a system for objectively checking the laboratory’s performance using an external agency or facility.
STRESS EQA is an indispensable part of a laboratory quality management system as it ensures customers (physicians, patients and health authorities) that the laboratory can produce reliable results.
4 / The Quality System
EXPLAIN that the laboratory operation consists of a series of complex events that begin with the ordering of a test, to the reporting of results back to the provider. It is very important that all aspects of the system work correctly. Any error in any part of the system can produce a poor laboratory result. Therefore, all aspects of the laboratory operation need to be addressed to assure quality; this constitutes a quality system.
OUTLINE that the guidelines of the Clinical and Laboratory Standards Institute (CLSI) describe a quality management system model that consists of 12 essentials. DEFINE quality system essentials as the set of coordinated activities that function as building blocks for quality management.
STRESS that according to CLSI quality management system model, EQA forms a part of the Assessment quality system essential and therefore EQA is an important part of a laboratory quality management system.
5 / ISO 15189:2007. Requirements Regarding EQA
EXPLAIN that international standards stress the importance of laboratories’ participation in EQA schemes.
ADD that the International Organization for Standardization has developed the standard named ISO 15189:2007 – Medical laboratories: particular requirements for quality and competence. for medical (clinical) laboratories. An independent external quality assessment of laboratory performance by a recognized scheme is the requirement of this standard.
STRESS that NTP should establish a functional mechanism to monitor the EQA results, review and investigate discordant findings and provide corrective and preventive actions. All findings and recommendations for actions should be appropriately documented.
6 / EQA Benefits
EXPLAIN that participation in EQA schemes is of benefit both for public health programs and individual laboratories:
NOTE that laboratory oriented objectives of EQA include:
• Identification of possible insufficiencies in laboratory practices, and guidance of participants in corrective actions
• Identification of reliability characteristics of methods, materials and equipment in routine-use and suggestion of appropriate corrections;
• Evaluation and monitoring of training impact; assistance in preparation of future training; and,
• Opportunities for continuous self-education.
ADD that public health oriented objectives of EQA include:
• Establishment of bases for results comparability during epidemiological surveillance and disease control;
• Collection of information on laboratory measurements (intra- and inter-laboratory) in order to alert professionals and/or governments on results, traceability and harmonisation problems, and establishment of appropriate acceptable limits for results in a given prospective;
• Collection of information on operating authorisations or laboratory accreditation;
• Further education; and,
• Creation of a channel for communication.
7 / EQA for AFB Smear Microscopy
EXPLAIN that in 2002 a group of international experts set the guidelines on External Quality Assessment for AFB Smear Microscopy.
STATE this consensus document was a collaborative effort of: APHL, CDC, IUATLD, KNCV, RIT and WHO.
8 / EQA Methods for AFB Smear Microscopy
EXPLAIN that EQA for AFB smear microscopy consists of three methods that can and should be combined to evaluate laboratory performance. These methods include panel testing, blinded rechecking and on-site supervision.
ADD that the EQA for AFB Smear Microscopy consensus document defines EQA as a process to assess laboratory performance; EQA allows assessing labs’ capabilities and performances by comparing their results with those obtained in other labs in the network.
9 / Panel Testing Process
LIST panel testing steps as stated on the slide.
10 / A Possible Scheme of A Panel Testing Round
DESCRIBE the possible scheme of a panel testing round following the previous slide message.
STRESS that organization of the panel testing rounds will usually be NRL responsibility. But panel testing rounds are best organized in close cooperation with intermediate laboratories.
NOTE that white color arrows show panel sets flow from NRL to peripheral laboratory technicians via intermediate laboratories. The broken blue colored arrows demonstrate the panel testing results’ reporting flow.
11 / Blinded Rechecking
DEFINE blinded rechecking as stated on the slide. This is considered to be the best method for evaluating performance, because it can show what is happening in reality.
EXPLAIN that the old sampling system “10% of negative and 100% of positive” is not efficient, with over-sampling in some laboratories and under-sampling in others. Lot Quality Assurance System (LQAS) sampling allows the smallest possible but appropriate sample sizes; detailed LQAS instructions on Lot Quality Assurance System are available in the Blinded Rechecking Module of this WIB.
12 / A Proposed Scheme of Organizing Blinded Rechecking for AFB Smear Microscopy
DESCRIBE a possible scheme of organizing blinded rechecking for AFB smear microscopy.
EXPLAIN that smears are randomly sampled from peripheral laboratories and then sent to the first controller for blinded rechecking. The first controller can be a microscopist located at an intermediate laboratory or a quality assurance center. After the first controller re-reads the slides and his/her results are compared with the results of peripheral laboratories, the discordant slides are sent to the second controller for final judgment. According to the proposed scheme the second controller are located at NRL.
13 / On-Site Supervision
EXPLAIN another EQA method – on-site supervision: this method looks at specific testing areas of the laboratory, especially those that have been identified by a bad result to a panel testing or blinded rechecking scheme.
STATE on-site supervision is periodic visits to the laboratory by laboratory or TB supervisors to obtain a realistic picture of laboratory practices, provide assistance with problem areas, including training.
NOTE on-site visits require thorough planning.
14 / EQA methods: Which one is better?
STATE an effective use of each of these methods will highly depend on the availability of resources, goals set up by the NTP, TB prevalence, centralization/decentralization of health services and the stage of laboratory quality assurance activities development in a country.
15 / Panel Testing
LIST advantages and disadvantages of panel testing as stated on the slide.
16 / Panel Testing: Indications for Use
STATE the message on the slide.
17 / Blinded Rechecking
LIST advantages / disadvantages and indication for use of blinded rechecking as stated on the slide.
STRESS that blinded rechecking cannot be used when almost no positives are detected. In the areas where the smear positivity rates in the laboratory registers are low, around 5%, stratified sampling focusing on follow-up smears may be applicable.
18 / On-Site Supervision
LIST advantages and disadvantages of on-site supervision as stated on the slide.
STRESS that on-site supervision is ineffective unless done by a well experienced / educated personnel.
19 / On-Site Supervision: Indications for Use
STATE the message on the slide.
20 / EQA: Important Issue
STATE the message on the slide.
21 / Situational Analysis: How to Identify the Most Feasible EQA method(s) For a Country? (I)
DESCRIBE the situation for Country A and DISCUSS IN A GROUP (if possible) an appropriate strategy that could be suggested for development of EQA activities in Country A. REFER to the summary below for possible conclusions / recommendations.
1. Large numbers of decentralized peripheral laboratories and, in particular, poor developed infrastructure of intermediate laboratories make implementation of all EQA methods difficult:
• Panel testing will result in high workload for NRL (preparation and distribution of uniform panels, review of results and preparation of feedback reports);.
• Blinded rechecking is impossible without properly established intermediate laboratory network: it will be difficult to assign the required number of controllers to ensure even distribution of rechecking workload and, consequently, appropriate quality of controllers’ work.
• On-site supervision: NRL staff will not be able to visit all peripheral laboratories alone.
2. Extra thorough consideration and critical assessment of all strengths and weaknesses of the situation should be recommended; data on smear positivity rates should be collected as well.
3. As it is too difficult to implement blinded rechecking, a panel testing program should be used first while developing intermediate level laboratories for gradual implementation of the blinded rechecking program.
4. Information on AFB smear positivity rates will help determine the sampling approach for the blinded rechecking program. In areas with medium positivity rates (around 5%), stratified sampling would be applicable.
22 / Situational Analysis: How to Identify the Most Feasible EQA method(s) For a Country? (II)
DESCRIBE the situation for Country B and DISCUSS WITH A GROUP (if possible) an appropriate strategy that could be suggested for development of EQA activities in Country B. REFER to the summary below for possible conclusions / recommendations.
The Country B situation is characterized by a relatively well developed TB laboratory network and would include:
1. Panel testing – if there is only sketchy information on laboratory proficiency with a need for rapid assessment.
2. Gradual implementation of blinded rechecking program. For medium prevalence stratified sampling could be a solution.
3. With progressing intermediate laboratories strengthening, panel testing may be limited to training pre- and post-tests, while the blinded rechecking system will be further developed and expanded countrywide.
23 / Situational Analysis: How to Identify the Most Feasible EQA method(s) For a Country? (III)
DESCRIBE the situation for Country C and DISCUSS IN A GROUP (if possible) an appropriate strategy that could be suggested for development of EQA activities in Country C. REFER to the summary below for possible conclusions/recommendations.
1. Low volume TB sputum smear microscopy examinations and very low smear positivity rates exclude blinded rechecking.
2. Panel testing would be more cost-effective. Moreover, a panel test may additionally have a “refresher” advantage (i.e. regarding the appearance of a positive smear – for laboratories where positives are rare).
STATE that NTP managers and NRL of all three countries should also plan training of laboratory and TB supervisors for targeted problem-oriented on-site supervision.
24 / Key Messages
STATE the message on the slide.
25 / Key Messages
STATE the message on the slide. ANSWER any questions the participants may have.
AFB Smear Microscopy EQA WIB TRAINER’S GUIDE
Page 8 of 9