A reference interval is defined by its 2.5th and 97.5th percentiles, so the estimate comes from the tails of the distribution. The tails are also where two common features of reference data do the most harm.
A single erroneous extreme value can drag a limit noticeably. An untreated skew can put a limit in the wrong place altogether. Screening for outliers and handling the distribution’s shape are therefore not preliminary steps to complete before the real analysis. Those two steps largely are the analysis.
Reference samples come from apparently healthy volunteers, but errors occur: a transcription slip, a mislabelled tube, or an individual who was not as healthy as assumed. Outlier screening finds candidates for these.
But a value being extreme is not proof it is wrong. Some genuinely healthy people sit at the edges of a distribution. Deleting them because they are inconvenient narrows the interval artificially. The right approach is to flag extreme values, investigate for a cause, and remove only those with a justified reason. Do not trim the tails simply to make the interval look neat.
EP28-A3C points to outlier rules designed for this setting rather than a generic one. Tukey’s fences (flagging values beyond a multiple of the interquartile range) are robust because the quartiles are not themselves distorted by the extremes being tested. Rules of the Dixon and Reed type judge a candidate by its distance from its neighbours relative to the overall range.
What these share is resistance to masking. A single method that uses the mean and SD to find outliers is undermined by the very outliers it is looking for. Rank- and range-based rules are not. Apply the screen on the appropriate scale too, after any transformation rather than before it. A value that looks extreme on a skewed raw scale may be unremarkable once the skew is removed.
Reference distributions are frequently right-skewed. Many analytes have a long upper tail. Skew matters because the parametric method, which computes the limits from the mean and SD, assumes a Gaussian shape.
Apply it to skewed data untransformed and the limits come out wrong: the upper limit too low, the lower limit implausible or negative. Confirm the shape first (a normal Q-Q plot and a formal test such as Shapiro–Wilk or Anderson–Darling) rather than assuming normality.
Two sound routes handle skew. The first is to transform the data to normality, then apply the parametric method on the transformed scale and back-transform the limits. A logarithm often suffices, or a Box–Cox or one of the other transformations chosen to fit. The second is to sidestep the assumption entirely with the non-parametric or robust methods, which make no claim about the distribution’s shape.
The transformation route keeps the efficiency of the parametric method when it works. The non-parametric route is simpler and safer when the skew is awkward or the sample small enough that fitting a transformation is itself uncertain. The worked example below screens an ALT dataset and handles its skew by transformation.
Download the CLSI EP28-A3C example workbook (.xlsx): an ALT reference interval with outlier screening and a transformation for skew, ready to open in the Analyse-it trial.
The example workbook is downloading.
It opens in Excel on its own — the data and the finished results are both in it. Analyse-it is what lets you change the analysis and re-run it, try the same study on your own data, or work through it to see how the software handles it.
Every feature from all five editions for 15 days.
Deleting extreme values because they are inconvenient. Some healthy people sit in the tails. Remove outliers only with a justified cause, not to tidy the interval.
Using a mean-and-SD outlier rule. The outliers distort the very statistics used to find them. Prefer rank- and range-based criteria that resist masking.
Applying the parametric method to skewed data. It assumes normality. Untransformed skew puts the limits in the wrong place. Transform first, or go non-parametric.
Screening on the wrong scale. A value extreme on a skewed raw scale may be ordinary once transformed. Screen after transformation, not before.
Analyse-it does the screening before the estimate, on your own reference data, inside Excel:
Every feature from all five editions for 15 days. Reference intervals are in the Medical, Method Validation and Ultimate editions, from US$ 340 a year. Validated against NIST and CLSI reference datasets. See choosing a reference interval method.