Every laboratory bringing a non-waived test into service faces the same requirement. In the United States the requirement comes from the Clinical Laboratory Improvement Amendments of 1988 (CLIA), and is set out in 42 CFR 493.1253. That regulation is worth reading directly, because it is more specific than its reputation suggests, and considerably shorter.
The regulation does two useful things. It names the performance characteristics you are answerable for, and it splits laboratories into two groups with very different workloads. Everything else it leaves to you: how many samples to run, and what counts as acceptable.
The requirement covers any test system introduced after 24 April 2003, so every method a laboratory brings up today falls inside it. Test systems already in use before that date are exempt. The requirement covers non-waived testing, both moderate and high complexity. Verification or establishment must be complete before the first patient result is reported, not at the first inspection and not retrospectively.
The regulation separates laboratories by what they are putting into service.
If you are introducing an unmodified, FDA-cleared or approved test system, used exactly as the manufacturer intended, you are verifying. You must demonstrate performance comparable to the manufacturer’s claims for three characteristics: accuracy, precision and reportable range. Separately, you must verify that the manufacturer’s reference intervals are appropriate for your own patient population. Four items.
You are establishing if you modify a cleared system or use one for which the manufacturer supplies no performance claims. The same applies to a test not subject to FDA clearance, such as a laboratory-developed test or a standardised textbook procedure. Seven items.
| Performance characteristic | Verification | Establishment |
|---|---|---|
| Accuracy | Required | Required |
| Precision | Required | Required |
| Reportable range | Required | Required |
| Reference intervals | Verify they suit your population | Establish your own |
| Analytical sensitivity | — | Required |
| Analytical specificity, including interfering substances | — | Required |
| Any other characteristic required for test performance | — | Required, as applicable |
Whether you face four short studies or seven full ones largely decides how long a method takes to bring into service. Which path you are on turns entirely on whether the assay is used exactly as cleared. A cleared assay run on a different specimen type, at a different dilution or on an instrument it was not cleared for has been modified. Validation vs verification works through that decision in detail. If there is any doubt, treat the change as a modification.
The regulation names the characteristics but not the methods. The EP series from the Clinical and Laboratory Standards Institute (CLSI) is the standard reference for how each one is done. CLIA does not require it, but the protocols are what accrediting organisations and reviewers expect to see behind the numbers. Which CLSI EP protocol do you need maps the whole series.
Accuracy. In practice you assess accuracy with a method comparison, which gives the bias at each medical decision point. What CLIA calls accuracy is therefore closer to what metrology calls trueness, agreement with a comparative or reference method, than to the combination of trueness and precision. See trueness, precision and accuracy for the distinction. The choice of regression matters here, and correlation is not the statistic for it.
Precision. For verification you run a short study confirming the manufacturer’s claim, as in verifying manufacturer precision claims. For establishment you run a fuller design separating repeatability and within-laboratory precision. The two paths are different studies rather than the same study at different lengths. The verification guide also sets out which of the two you need.
Reportable range. A linearity study fixes the interval over which the response stays linear and within the allowable deviation. The reportable range and the analytical measuring interval are not the same thing once dilution is involved. Reporting a diluted result outside a validated dilution is a common inspection finding.
Reference intervals. For verification you are confirming the manufacturer’s interval suits your population, which is a much smaller study than building one. See transferring and verifying a reference interval. For establishment you are producing your own, which raises questions of method, partitioning and outliers.
Analytical sensitivity (establishment only). Analytical sensitivity covers the low end of the method: the limit of blank, detection and quantitation. The limit of quantitation is set by a precision goal. Where the outcome is detected or not detected, the limit of detection comes from probit analysis.
Analytical specificity (establishment only). The regulation names interfering substances explicitly. Interference and recovery testing covers the design and the decision against an allowable limit.
Nothing in 493.1253 tells you how many samples to run or what result counts as acceptable. The regulation leaves both to the laboratory, and setting the acceptance criteria is the step most easily skipped. A bias of 4% is a pass or a failure depending entirely on a limit you set before the study.
You also have to be able to say where that limit came from. The source should be one of the Milan models (clinical outcome, biological variation or state of the art), or a regulatory or proficiency-testing limit. Choosing the criterion after seeing the data invalidates an otherwise sound verification.
Sample sizes come from the protocols rather than the regulation. The individual guides work through how many samples a method comparison needs and the precision study designs. The numbers differ substantially between the verification and establishment paths.
The regulation carries two requirements beyond the characteristics themselves.
The first is that your calibration and quality control procedures follow from the specifications you verified or established. Neither is chosen independently. The precision you measured is what determines sensible control limits, which is why the control chart you run afterwards depends on the study you did first.
The second is documentation. The regulation requires the laboratory to document all of these activities. A separate section, 42 CFR 493.1105, requires the records to be kept for as long as the method is in use, and for at least two years. A study you ran but cannot produce evidence for does not count as done.
Download the CLSI EP15-A3 example workbook (.xlsx) — a ferritin precision verification of the kind the verification path calls for, ready to open in the Analyse-it trial. Three samples are measured over five runs of five replicates and tested against the manufacturer’s claim.
The example workbook is downloading.
It opens in Excel on its own — the data and the finished results are both in it. Analyse-it is what lets you change the analysis and re-run it, try the same study on your own data, or work through it to see how the software handles it.
Every feature from all five editions for 15 days.
Treating a modified assay as a verification. A specimen type, dilution, or instrument the assay was not cleared for moves you from four characteristics to seven. This is the most serious error here, because it tends to be discovered at inspection rather than when the method is introduced.
Setting the acceptance criterion after the study. The limit has to exist, and be justifiable, before the data arrive. Judging a bias against a limit chosen once you have seen it is not a verification.
Confusing accuracy with precision. CLIA lists them separately because they are separate studies answering separate questions. A method can be highly repeatable and consistently wrong.
Adopting the insert reference interval without checking it. Taking the manufacturer’s values because the assay is cleared is not what the regulation asks for. It asks you to demonstrate the interval is appropriate for your own patient population.
Stopping at the numbers. Verification is not finished when the statistics pass. The calibration and control procedures have to be set from what the studies found, and the records kept for as long as the method runs.
Analyse-it reports the statistics each verification study produces, following the CLSI protocol it is based on, inside Excel:
Every feature from all five editions for 15 days. These analyses are in the Method Validation and Ultimate editions, from US$ 475 a year. Validated against NIST and CLSI reference datasets. If you are accredited by the College of American Pathologists (CAP), the checklists ask for more than CLIA, and some of it must be repeated on a schedule. See CAP accreditation and AMR verification and comparing instruments within one laboratory. For the whole set in one place, what a clinical laboratory has to verify maps each requirement to the study that answers it.