Establishing a method’s precision means measuring how much its results vary when nothing about the sample has changed. The measurement must also be thorough enough to separate the different sources of that variation. Establishing precision is the manufacturer’s job during development, and the laboratory’s job for a test it has developed or modified itself. Establishment is the fuller, more demanding counterpart to precision verification. Verification confirms a stated claim with a short study, while establishment derives the numbers from scratch, following the protocol in CLSI EP05-A3.
Precision has structure, and the study is built to expose it. Measure a sample twice in one run and the two results differ a little, which is short-term, within-run noise. Measure it again in a later run the same day, and on other days, and further variation appears. Conditions differ slightly from run to run, and calibration, reagents and operators can change from day to day.
EP05-A3 captures all of this with a nested design: several replicates within each run, several runs within each day, across many days. The classic single-site design runs 20 days, two runs a day, two replicates a run (80 results per sample) at two or more concentrations spanning the measuring interval. Twenty operating days is about a month of work, which is why a short evaluation before you commit is worth running first.
Analysed with a nested random-effects model, the design partitions the total variation into its components rather than lumping it into one number. Separating those components is the purpose of the design, and the precision components guide names them in full.
A single-site study yields two principal figures. Repeatability is the within-run precision: the irreducible short-term scatter of replicates measured close together. Within-laboratory precision (intermediate precision) combines the within-run, between-run and between-day variation into the total spread the laboratory will actually see over time. Within-laboratory precision is always at least as large as repeatability, and it is the figure to report for routine use. A result reported next week can be expected to vary by the within-laboratory figure, not by the more favourable figure from replicates run one after another.
Report each as a standard deviation and a coefficient of variation, at each concentration studied. The CV matters because precision is rarely constant across the range, which is the subject of the next section.
A single-site study characterises precision within one laboratory. When the question is whether a method performs consistently across laboratories, instruments or sites, the design extends to several sites. That question is the manufacturer’s concern for a product going to market. A further component then appears, the between-laboratory variation. Combined with within-laboratory precision, the between-laboratory component gives reproducibility, the broadest precision figure. Reproducibility describes how much results vary wherever the method is used, rather than within one laboratory.
Precision usually depends on concentration. Often the standard deviation grows with the analyte level while the coefficient of variation stays roughly constant. Measuring precision at a single concentration and quoting it for the whole range is therefore misleading.
A precision profile measures precision at several concentrations across the interval and fits a variance function describing how imprecision changes with the result. The function may be constant, proportional, a mix of the two, or a power model with more parameters. The profile lets you state the precision at any point in the measuring range, and it feeds directly into a sound limit of quantitation.
The workbook below fits exactly such a profile for a multi-site CA19-9 study, with a three-parameter variance function across the range.
Establishing precision is a substantial study. Twenty days of measurement is not undertaken lightly, and much of the time it is not what a laboratory needs. If you are bringing an unmodified, cleared assay into service, the manufacturer has already established its precision. Your job is the far shorter one of confirming the claim holds in your own laboratory. Establish only when you have to: for a laboratory-developed test, a modified method or a use outside the cleared indications. The guide to verifying a manufacturer’s precision claims works through which one applies.
Two worked examples, ready to open in the Analyse-it trial:
The example workbook is downloading.
It opens in Excel on its own — the data and the finished results are both in it. Analyse-it is what lets you change the analysis and re-run it, try the same study on your own data, or work through it to see how the software handles it.
Every feature from all five editions for 15 days.
Reporting repeatability as though it were the method’s precision. Replicates run one after another understate the variation a laboratory sees day to day. Within-laboratory precision is the figure that matters for reported results.
Measuring precision at one concentration. Precision changes across the range. Use a precision profile rather than quoting a single-level figure for the whole interval.
Running the study in too few batches. Measuring everything in a few large batches destroys the between-run and between-day structure the study exists to measure. Spread the runs across days as the design specifies.
Establishing when you could verify. A full EP05 study for an unmodified cleared assay is effort spent re-deriving documented claims. Verify unless your use is non-standard.
Analyse-it runs the EP05 establishment on your own nested design, inside Excel:
Every feature from all five editions for 15 days. Precision is in the Method Validation and Ultimate editions, from US$ 475 a year. Validated against NIST and CLSI reference datasets. See also the precision components guide.