Establishing a method’s precision means measuring how much its results vary when nothing about the sample has changed, and doing it thoroughly enough to separate the different sources of that variation. Establishing precision is the manufacturer’s job during development, and the laboratory’s job for a test it has developed or modified itself. The work is the fuller, more demanding counterpart to precision verification: verification confirms a stated claim with a short study, while establishment derives the numbers from scratch. CLSI EP05-A3 is the protocol.
Precision has structure, and the study is built to expose it. Measure a sample twice in one run and the two results differ a little. That is short-term, within-run noise. Measure it again in a later run the same day, and on other days, and further variation appears: runs drift, calibrations settle, operators and conditions change between days.
EP05-A3 captures all of this with a nested design: several replicates within each run, several runs within each day, across many days. The classic single-site design runs 20 days, two runs a day, two replicates a run (80 results per sample) at two or more concentrations spanning the measuring interval. That is a month of work, which is why a short evaluation before you commit is worth running first.
Analysed with a nested random-effects model, that design partitions the total variation into its components rather than lumping it into one number. Those components are what the design is for. The precision components guide names them in full.
A single-site study yields two principal figures. Repeatability is the within-run precision: the irreducible short-term scatter of replicates measured close together. Within-laboratory precision (intermediate precision) combines the within-run, between-run, and between-day variation into the total spread the laboratory will actually see over time. Within-laboratory precision is always the larger of the two, and the figure to report for routine use. It is what a result reported next week can be expected to vary by, not the more favourable figure from replicates run one after another.
Report each as a standard deviation and a coefficient of variation, at each concentration studied. The CV matters because precision is rarely constant across the range, which is the subject of the next section.
A single-site study characterises precision within one laboratory. When the question is whether a method performs consistently across laboratories, instruments, or sites (the manufacturer’s concern for a product going to market), the design extends to several sites. A further component appears: the between-laboratory variation. Combined with within-laboratory precision, it gives reproducibility, the broadest precision figure. It describes how much results vary across the full intended footprint of the method rather than within one lab.
Precision usually depends on concentration. Often the standard deviation grows with the analyte level while the coefficient of variation stays roughly constant. Measuring precision at a single concentration and quoting it for the whole range is therefore misleading.
A precision profile measures precision at several concentrations across the interval and fits a variance function (constant, proportional, or a combination of the two) describing how imprecision changes with the result. That profile lets you state the precision at any point in the measuring range. It feeds directly into a sound limit of quantitation.
The workbook below fits exactly such a profile for a multi-site CA19-9 study, with a three-parameter variance function across the range.
Establishing precision is a substantial study. Twenty days of measurement is not undertaken lightly, and much of the time it is not what a laboratory needs. If you are bringing an unmodified, cleared assay into service, the manufacturer has already established its precision. Your job is the far shorter one of confirming the claim holds in your own laboratory. Establish only when you have to: for a laboratory-developed test, a modified method, or a use outside the cleared indications. The guide to verifying a manufacturer’s precision claims works through which one applies.
Two worked examples, ready to open in the Analyse-it trial:
The example workbook is downloading.
It opens in Excel on its own — the data and the finished results are both in it. Analyse-it is what lets you change the analysis and re-run it, try the same study on your own data, or work through it to see how the software handles it.
Every feature from all five editions for 15 days.
Reporting repeatability as though it were the method’s precision. Replicates run one after another understate the variation a laboratory sees day to day. Within-laboratory precision is the figure that matters for reported results.
Measuring precision at one concentration. Precision changes across the range. Use a precision profile rather than quoting a single-level figure for the whole interval.
Running the study in too few batches. Running everything in a few big batches destroys the between-run and between-day structure the study exists to measure. Spread the runs across days as the design specifies.
Establishing when you could verify. A full EP05 study for an unmodified cleared assay is effort spent re-deriving documented claims. Verify unless your use is non-standard.
Analyse-it runs the EP05 establishment on your own nested design, inside Excel:
Every feature from all five editions for 15 days. Precision is in the Method Validation and Ultimate editions, from US$ 475 a year. Validated against NIST and CLSI reference datasets. See also the precision components guide.