Laboratory

Quality Management System

Assessment Tool

User Manual

a - Introduction

The Laboratory Quality Management System Assessment Tool (LQMSAT) is available on the Excel file “Laboratory QMS Assessment Tool.xls”. This document is the user manual of the Excel file.

The Laboratory Quality Management System Assessment Toolis an evaluation tool designed to:

  • assess any individual laboratory in a standardized way with regards to the implementation of a Quality Management System
  • automatically generate numerical indicators related to the laboratory capacities and quality in different parts called «modules»
  • follow the improvement of the same laboratory over time
  • perform an evaluation based on the technical and management requirements expected according to the level of the laboratory (reference, intermediate, periphery)

The tool is an all-purpose assessment tool that can be used to assess laboratories with different specialties and from different levels of the health system (from basic peripheral level to specialized central level). Recommended tests and instruments may vary from one country to another. It is the assessor’s responsibility to compare the practice in the assessed laboratory with the national recommended practices or regulations. In case such regulations do not exist, the assessor needs to compare the observed practices with the state of art in similar settings or with the international recommendations (WHO, scientific societies, and literature). The tool can also be adapted to include some specific questions according to the purpose or scope of the assessment (refer to the section “editing the tool”).

The current edition of the assessment tool is a Microsoft Excel® file, and is run only using calculations (no macros). Absence of macros enables the LQMSAT to be used on any computer, independent of the operating system language.

b - Presentation of the tool

The file includes fourteen worksheets, titledin English. Altering or renaming of these titles may result in calculation error and possibly compromise the interpretation of the data.

A laboratory offers adequate services only if its results are delivered in a safe, reliable, and timely manner. Therefore, the main objectives of the tool are to describe and assess objective elements that contribute to the capacity, performance, and quality of the laboratory. Quality requirements have been described widely these last years, by various organizations. Some of the most popular and used documents are the ISO standard 15189:2007[1] and the CLSI GP26-A3 guidelines[2]. These documents describe the essential technical and management requirements that ensure that the laboratory is functioning well and delivers quality results. These documents are very comprehensive and used widely by inspection teams or accreditation bodies to design assessment checklists[3][4]. Some countries have adapted these comprehensive documents, extracting the most relevant items for their local purpose[5].

The proposed assessment tool is based on the 12 quality essentials described by the CLSI. These essentials are very similar to the elements described in the ISO 15189 standard, even if organized in a different manner. These essentials apply to simple and complex laboratory operations. Therefore, the assessment tool can be used to assess a wide spectrum of laboratories, such as multi-disciplinary or specialized in one discipline or even one disease.

Figure 1: The various worksheets

Titles and descriptions of each worksheet:

 Assessment results modules

  • Summary:A full summary of evaluation and assessment of the laboratory with LQMSAT. Two pages when printed.
  • All indicators:A summary of the evaluation of each indicator. One page when printed.

 Specific Modules

  • 1-Lab identification:this module gathersinformation concerning the laboratory being assessed and the respondent. The tests (type and number) must also be indicated.
  • 2-Personnel:this module includes questions related to staff management.
  • 3-Documents and records:this module deals with the management of all the documents handled by the laboratory: procedures, forms, reports etc.
  • 4-Purchasing and inventory (of services and supplies):this module gathers information onthe reagents and supplies purchased or prepared by the laboratory (e.g., stock management). The module also includes questions concerning other servicescontracted by the laboratory, such as the referral of samples to other laboratories.
  • 5-Equipment:this module assesses and lists the laboratory equipment and its maintenance.
  • 6-Process control (quality of pre-, post- and examination procedures):this module reviewsmost of the operations and quality requirements during the preexamination phase (sample collection), the examination phase (internal quality control, written procedures) and the postexamination phase (release of results, storage of samples).
  • 7-Information management:this module examinesthe laboratory results and data management and reporting.
  • 8-Assessment: this module deals with the internal on-site assessments (audit), external on-site assessment visits, and other interlaboratory comparisons through the participation to external quality assessment schemes (e.g. proficiency-testing schemes) or rechecking systems.
  • 9-Occurrence management, process improvement, and customer service: this module reviews how the errors are detected and monitored, and how corrective actions are implemented for a continuous improvement of the laboratory performance.
  • 10-Facility and safety: this module assesses the infrastructure conditions and the biosafety measures.

 Language

The LQMSAT can be used either in English or in other languages. Its initial language is English. Prior to switching languages, other fonts may need to be installed on your computer.

To switch from English to another language, do the following:

1)Select worksheet names “language”

2)Translate the English words in the other language in column D (just as the French example)

3)Toggle the cell number B3 from “1” (English) to “2” (other language).

Any modification to the wording in the modules worksheets should be done ONLY in this sheet as the cells of the other modules reports to the “Language” sheet.

c- The assessment process

IMPORTANT NOTE: It is necessary that the assessment is carried out by a laboratory specialist or a person who has in-depth knowledge in laboratory assessment and understands the general laboratory operations.

1) Recommended length of the assessment

A complete assessment of a laboratory should take at least half a day for a reference laboratory and should take about 2~3 hours for a peripheral laboratory. This assessment must be carried out during opening hours, in order to be able to observe staff at work.

In addition to using the electronic and paper version of LQMSAT, it is pertinent to have a notebook and a camera (ideally digital) for documentation.

2) Approach to the assessment and securing cooperation from the laboratory

Before you begin the assessment of the laboratory, it is recommended that the assessor explains the main purpose of the assessment. He/shemust point out that this is not a control process that may lead to punitive measures.

3) Assessment procedures and guidelines

The assessment should begin with a short meeting with the laboratory manager or whoever is in charge in order to describe why the assessment will be conducted, what the process will be and what the expected outcome might be. This meeting could be useful to get some information about the laboratory organization and many management items (staff, procurement of equipment, financing, etc…).

After the preliminary meeting, the assessor should visit the laboratory, following the “sample path”:

  • sample or patient reception
  • sampling rooms
  • recording
  • technical rooms (microscopy, chemistry, hematology, culture, bacteriology, serology, molecular biology)
  • support rooms (glassware washing room, sterilization, restroom, guard room, stock room, repair room, secretary room, offices)

The assessor should look at the global cleanliness, the global organization, biosafety level while manipulating samples.

In each room (if applicable), the assessor may be required to perform the following tasks:

  • Inspect the condition and use of refrigerators

1) Are the refrigerators clean?

2) Randomly take 3 or 4 old-looking products and inspect their expiration dates.

3) Is there an internal thermometer available? If not, are there any temperature charts?

  • Inspect the condition and organization of incubators

1)Look at the thermostat adjustments and daily temperature chart records of each incubator (ideally, it is recommended to have a thermometer inside a water can in the incubator)

2)Inspect the overall cleanliness of the incubator

3)Inspect the maintenance of special atmospheres (e.g., 5% CO2, anaerobic, micro-aerophilic conditions)

4)Inspect blood culture bottles Are there any old bottles? Do all the bottles have identifications?

5)Inspect the organization of Petri dishes  Are there any old Petri dishes? Do all of Petri dishes have identifications?

  • Inspect the condition and organization of freezers How is the sera collection organized?
  • Check the conditions of all the other equipment (use, maintenance…)
  • Visually evaluate the condition and cleanliness of the laboratory benches and facilities
  • Examine the condition and organization of stock shelves, cards, and expiration dates of the reagents on the shelves
  • Inspectthe laboratory logbooks, patient records or information (inspectif they correspond to the right patient), and management of the results
  • Inspectthe archives
  • Inspectthe condition of the microscopes: always assess one or two slides in order to check: 1) the quality of the films and stains, 2) the condition of the microscope, and 3) the Kohler adjustment (centering of optical axes).
  • Observe and evaluate how laboratory technologists work

1)Do they use personal protective equipment, e.g., lab coats, gloves, glasses, masks? (Is Personal Protective Equipment available?)

2)Is the bench clean? Do technologists disinfect the bench at the end of their work? Is regular disinfection of the laboratory bench likely?

  • Inspectand evaluate how the laboratory waste is managed inside technical rooms. This is important information as it will tell about the laboratory’s activities within the past 24 hours of the assessment.

1)Is the waste regularly emptied?

2)Are there separate waste (lid-covered?) containers for noncontaminated and contaminated wastes?

3)Is there a special solvent container (any for acids?)

Recommendations should be made following the observation and inspection. It is important to note that assessments are performed to improve the condition of the laboratory and understand that keeping good relations between the assessor and the laboratory personnel is vital. Hence, all recommendations and advice should be made in friendly manner. Comments that may be quite embarrassing or upsetting for the laboratory should only be mentioned to the manager of the laboratory (or the person in charge).

During the visit, document the assessment by taking pictures of the rooms, working staff, equipment, packaging delivery procedures, etc. These pictures will help to illustrate and explain the conditions of the laboratory for the final report. Always ask for permission to take photographs.

Once the initial visual evaluation of the laboratory is completed, immediate assessment of the laboratory using LQMSAT is recommended. The assessor can initially use the paper version of the tool for his convenience, but it is recommended to directly use the electronic version. Using the electronic versionenables immediate discussions with the manager about the indicators of LQMSAT questions. Recommendations for improvement of the laboratory can be made following the discussions.The assessor does not have to ask all the questions and can enter directly some answers according to the observations made during the visit. The assessor should ask only the questions to be highlighted (for instance for an educational purpose) or the questions for which the visit cannot give clear answers.

The following aspects are recommended to be noted at the conclusion of the assessment:

  • Strengths of the laboratory, in order to highlight positive aspects of the laboratory
  • Weaknesses of the laboratory in order to highlight future actions for improvement of the laboratory

IMPORTANT:the observations are kept confidential. Establishing a confidential climate will make the assessment flow easier.

d- General recommendations for completing each module

General rule: all gray cells of the sheets“Lab ID” to sheet #10 must be filled in.

For each question, and for the entire tool, the assessor has a limited number of possible answers:

  • «Y» for yes
  • «N» for no
  • «na» for «non applicable», should be entered if the question doesn’t apply to the laboratory.
  • «1, 2, 3, 4 or 5» for questions with different possible answers
  • Different values other than those above can entered when asking:
  • Number of equipment
  • Number of tests performed weekly
  • etc.

Figure 2: Limited possibility for answers (here Y, N and na)

Clicking the small arrow at the right side of the cell opens a box with authorized values.Error message will appear as below, when trying to enter values other than the ones listed in the drop down list:

Figure 3: Error message in case of unauthorized value

Hence, THE ASSESSOR can only enter an authorized value in the listbox.

IMPORTANTNOTE:

If the assessor thinksthat the answer to the question applies to both Y and N, he/she must SYSTEMATICALLY choose the most accurate answer of the two.

Examples:

1) If the answer is «yes» in few cases but «no», for most cases, the assessor has to choose «no». This systematic «rigorousness» will allow the observations made by different assessors more comparable.

e- Calculations of the indicators

Calculations of modules indicators are automatically performed when answering to each question.

The calculations are based on the following principles:

-answering Yes (“Y”) gives 1 point (or “100%”)to the question

-answering No (“No”) gives 0 (or “0%”) point to the question

-answering Not applicable “na” excludes the question from the calculation

-for some questions, the respondent can choose between other options such as “regularly”, “sometimes” or “never”, “bad”, “medium” or “good”. In that case, answering each answer can give the following scores:

Regularly / 0 point = 0%
Sometimes / 0.5 point = 50%
Never / 1 point = 100%
Bad / 0 point = 0%
Medium / 0.5 point = 50%
Good / 1 point = 100%

The indicator is calculated from the average of all the questions and appears usually at the top in the cell D2 (or E2). Calculations can be seen on the right hand of the questions columns. They are hidden by a white panel that should moved away after the sheet is unprotected (see below section g-editing the tool).

Figure 4 shows the calculations for module 1.

Figure 4: Calculations

f- Analysis of the results and summary

  • General overview in analyzing indicator summary

The analysis of the results of the indicatorsshould be done with the manager of the laboratory who has been collaborating in this assessment. This should be done at the end of the assessment and after entering all the data in the LQMSAT.

NOTE:the assessor should discuss not only the weaknesses, but also the strengths of the laboratory on which an improvement plan can be built.

After analysis, the assessor should highlight only the three most important indicators that need to be implemented for the laboratory’s improvement in the future.

Figure 5 shows an example of the graphic representation of indicators in the LQMSAT with background color ranging from red to green. This allows the assessor to easily assess conditions of each indicator:

  • Red: Below 50%, requires significant improvement
  • Yellow: Between 50 and 80%, some improvement is necessary
  • Green: Above 80%, the laboratory is in good standing

Figure 5 : “All indicators” worksheet.

  • General indicator summary

This calculation is an average of all modules. It is reported both in the “all indicators sheet” and the “summary” sheet.

g- editing the tool

Each worksheet in the LQMSAT is protected against any modification. This is to avoid performing incorrect manipulations that may compromise calculation. Some white «panels» have been placed in certain locations to hide cells that are used for calculations, particularly on the right hand of the grey cells.

NOTE: To unprotect these worksheets in LQMSAT, click on “tools” in the menu bar in Excel to find “protection.” Then click “unprotect sheet” as shown in the diagram below (Figure 6).

Figure 6: active sheet unprotected.

This will unprotect each worksheet in the LQMSAT and the assessor may be able to move the white panels to reveal calculation section, add or delete questions, change indicator calculations, or add/delete indicators.

CAUTION: In doing this task, the assessor must first save his/her work in a different name so that he/she may go back to the original file.

1

[1] ISO 15189:2007. Medical laboratories- Particular requirements for quality and competence

[2]GP26-A3 Vol. 24 No.36. Application of a Quality Management System Model for Laboratory Services; Approved Guideline – Third Edition. Clinical and Laboratory Standards Institute.

[3]Check-list standard pour l’accréditation de laboratoires médicaux selon ISO/CEI 17025 (1999) respectivement ISO 15189 (2003). Swiss Accreditation Service.

[4]A Guidance Document for the Application of ISO 15189:2003 to Medical Laboratories. Caribbean Laboratory Accreditation Service. 2004.

[5]Quality Standards in Health Laboratories. Implementation in Thailand: A Novel Approach. SEA-HLM-386. World Health Organization. 2005.