Assessing linearity of a measurement procedure Linearity fixes the range over which you can trust a result. How to establish it under CLSI EP06-Ed2 — and why a correlation coefficient is the wrong tool for the job.

A quantitative method is linear over the range where its measured result follows a straight line against the true concentration. Equal steps in concentration should produce equal steps in the reading, whatever the starting point. That range is the interval over which you are entitled to report numbers. Outside it, results curve away from the truth and mislead.

Establishing it is the job of CLSI EP06-Ed2. A high correlation coefficient is a common shortcut, but it does not do the job, for much the same reason it fails in method comparison.

The dilution series

Linearity is assessed with a series of samples of known relative concentration spanning the claimed measuring interval. Typically you mix a high and a low pool in known proportions, or use dilution. Measure each level in replicate. The levels should reach the ends of the interval and space out across it. Nonlinearity most often appears at the extremes: saturation at the top, or a disproportionate response near the bottom. Too few levels, or levels clustered in the middle, leave the regions where curvature hides unexamined.

Fitting and testing for curvature

Fit the measured means with polynomials of increasing order: a straight line, then a second-order (quadratic) curve, then a third-order. Compare the best-fitting of those against the straight line. The quantity that matters is the deviation from linearity at each level. That is the gap between the best-fitting curve and the straight line, reported with a confidence interval. A straight-line fit alone cannot expose it. It will draw a line through curved data and leave the curvature in the residuals.

Judge against an allowable nonlinearity

Statistical significance is not the same as clinical importance. EP06-Ed2 rests on this distinction. A large enough study will flag a trivial, clinically irrelevant curvature as significant.

So the deviation from linearity at each level is judged against an allowable nonlinearity. That is a criterion set from clinical or analytical requirements, often expressed as a percentage. Use an equivalence test rather than a test against zero. A difference plot showing each level’s deviation against an allowable band makes this immediate. Where deviations sit inside the band, the method is linear enough. Where they break out, it is not.

A linearity plot showing measured values against equally spaced dilution levels: a dashed straight-line fit, a shaded allowable-nonlinearity band around it, and a red curve of the true response that bends away and leaves the band at the top two levels.
Linearity is judged as deviation from a straight line, not proportionality. The measured response (red) curves away from the fitted straight line at the top; where a level’s deviation falls outside the allowable-nonlinearity band, the departure is real curvature rather than scatter.

Report the linear interval

The output is not a yes/no pass or fail on the whole range. It is the interval over which the method is linear within the allowable nonlinearity. If the extremes deviate but the middle holds, the reportable range is narrower than claimed. Results outside it should be diluted into range or flagged. The workbook below shows exactly this pattern for an IgM assay. Some dilutions meet the requirement and others do not. The reported measuring interval is the span that does.

Linearity difference plot showing each dilution's deviation from the best-fit line against an allowable nonlinearity band.
A linearity evaluation with deviations judged against an allowable nonlinearity band (IgM, dataset from CLSI EP06-A Appendix C). The method is linear over the interval where the deviations stay within the band.

Downloads

Two worked examples, ready to open in the Analyse-it trial:

  • IgM linearity — five dilutions with linear and polynomial fits and a ±5% allowable nonlinearity band.
  • Calcium linearity — the full and a reduced measuring range compared against allowable nonlinearity.

Common mistakes

Using a correlation coefficient or R² as the test. Both can look excellent while the data curve. Fit polynomials and judge the deviation from linearity against an allowable limit.

Too few levels, or none at the extremes. Curvature hides where you did not sample. Span the interval and reach its ends.

Reading statistical significance as clinical significance. A large study flags trivial curvature. Judge the deviation against an allowable nonlinearity, not against zero.

Treating linearity as a one-off. A linearity study establishes the interval at bring-up. The same analysis is what recurring AMR verification rests on under the CAP checklists.

Confusing linearity with calibration. A linearity study asks whether response stays proportional across the interval. A calibration curve is the fitted relationship you invert to report results. See fitting a calibration curve.

Reporting outside the linear interval. If the extremes fail, the reportable range is narrower than claimed. Dilute into range or flag. Do not report numbers the method cannot support.

Assess linearity with Analyse-it

Analyse-it runs the EP06-Ed2 assessment on your own dilution series, inside Excel:

  • Polynomial fitting through to 5th order, so the departure from a straight line is modelled rather than eyeballed
  • Equality and equivalence testing of the deviation from linearity, using Hsieh-Liu confidence intervals
  • A difference plot against an allowable nonlinearity band, level by level

Every feature from all five editions for 15 days. Linearity is in the Method Validation and Ultimate editions, from US$ 475 a year. Validated against NIST and CLSI reference datasets. The linear interval established here is the basis of the analytical measuring interval and reportable range. Full detail in the trueness reference guide.