Qualitative Validation Overview

SMILE

JohnsHopkinsUniversity

Baltimore, MDUSA

Qualitative Validation Overview

Qualitative Validation Overview

Validation of a qualitative laboratory test consists of an established set of required experiments. This overview is intended to be applicable to both CLIA Waived and non-waived methods. A Waived test is one that has been granted Waived status by the US Clinical Laboratory Improvement Act of 1988 (CLIA ’88). As indicated in the document, additional validation testing may be required for non-FDA Approved test methods. After completing all of the validation experiments, raw data and results should be compiled and filed in an organized manner. All validation records should be retained indefinitely. Avalidation summary should be prepared that containsthe signature of the Laboratory Director, indicating the validation has been reviewed and approved.

The following are the required components of validation:

1. Precision is reproducibility –the agreement of the measurement of replicate runs of the samesample. It is the process of determining the range of random error. The precision is measured in terms of coefficient ofvariation (CV). Precision testing is required for qualitative tests that meet the following criteria:
2. Qualitative results are derived from a quantitative value such as an OD.
3. Manufacturer’s package insert describes precision specifications for the assay.

For qualitative tests that do require precision testing follow the guidelines listed below:

1. Within run and between run reproducibility will be determined by running the negative control and positive control as follows:
2. For within run, at least 20 replicates of negative control and at least 20 replicates positive control will be tested in one run.
3. For between run reproducibility, both negative and positive control will be tested at least once per day but not more than 5 times per day to obtain a total of 20 replicates each.
4. Acceptability criteria: The results obtained from the negative and positive controls will be used to calculate the CV and compared to the manufacturer’s claims for reproducibility. The laboratory CV should be less than or equal to the manufacturer’s stated CV. In the event that an assay does not perform as expected, the Laboratory Director will determine acceptability.

1. Accuracy is the true value of a substance being measured. Verification of accuracy is the process of determining that the test system is producing true, valid results.
2. Determine your comparison or reference method
3. The comparison method must be previously validated.
4. The comparison method must be currently performing successfully on EQA.
5. Comparison to a method currently in use in the laboratory is preferred if this method meets the above criteria.
6. Samples with known values, such as proficiency testing samples or commercial standards, may be used as the reference method.
7. Examples of possible reference methods for qualitative tests include in-house or external quantitative methods, use of a reference lab, correlation with another lab using a previously validated method, use of EQA samples or other commercially prepared reference material with known values.
8. Sample Criteria
9. A minimum of10 samples for each expected result. For example, if a test method gives results of “Positive/Negative”, the accuracy study must include 10 known positives and 10 known negatives.
10. Testing: Two levels of quality control must be run each day that testing is performed, not including controls internal to the kit cartridge/testing device.
11. Acceptability criteria:

The performance of qualitative tests is most commonly described in terms of sensitivity and specificity. The table below is a contingency table that compares the results of a qualitative test with the outcome of the diagnostic accuracy criteria. The entry in each cell of the table represents the number of specimens corresponding to the labels in the margins.

1. Calculate the estimated Diagnostic Sensitivity(True positive rate) = 100 x [TP/(TP+FN)]
2. Calculate the estimated Diagnostic Specificity(True negative rate) = 100 x [TN/(FP+TN)]
3. Calculate the percent Positive Agreement (Positive Predictive Value)

=100 x TP/(TP+FP)

1. Calculate the percent Negative Agreement (Negative Predictive Value)

=100 x TN/(TN+FN)

1. Compare the results calculated above with the manufacturer’s stated claims for Sensitivity, Specificity and Agreement found in the test kit package insert.
2. Results must be equal to, or greater than, the manufacturer’s claims for the method to be considered accurate.

1. Linearitywith AMR and CRR–Not applicable for qualitative tests.
2. Analytical Sensitivity is the lowest concentration of an analyte that can be measured. For an FDA approved, unmodified method, the manufacturer’s stated sensitivity (cut-off value) will be used.
3. AnalyticalSpecificity is the determination of the effect of interfering substances. For an FDAapproved, unmodified method, the manufacturer’s stated specificity will be used.
4. Reference Ranges–Can be determined by the laboratory with laboratory director approval. Verification of manufacturer’s stated reference range is not required.
5. Summary and Approval. See SMILE Qualitative Testing Validation Summarytemplate.
6. References
7. GCLP Workshop and Workbook18-20 May 2008, Verification of Performance Specifications, pages 1-33.
8. Clinical and Laboratory Standards Institutes (CLSI), User Protocol for Evaluation of Qualitative Test Performance; Approved Guideline--Second Edition. CLSI document EP12-A2 (ISBN 1-56238-654-9). Clinical and Laboratory Standards Institute, 940 West Valley Road, Suite 1400, Wayne, PA 19087-1898 USA, 2008.
9. EP Evaluator Release 8, David G. Rhoads Associates Inc.,
10. James O. Westgard, Online Validation Training, Westgard QC, Inc. Sections 11-Determining Bias,12- Estimating Trueness, and 13- Judging Method Acceptability.

1000_SMILE Qualitative Validation Overview v.1.2.doc Page 1 of 4