Estimating nonparametric limits of agreement in non-normally distributed data

An assumption of the Bland-Altman limits of agreement is that the differences (or residuals, when fitting a regression) are normally distributed. In many cases, there will not be a big impact on the limits of agreement when the distribution of the differences is not normal. However, there may be cases where it is preferable to estimate the limits using a nonparametric method.

A histogram of the differences is useful for assessing the assumption of normality. If the distribution is skewed or has very long tails, the assumption of normality may not be valid.

To illustrate these concepts, we will use another example from Bland & Altman’s 1999 paper. This example shows the differences in systolic blood pressure measurements by a device and those by sphygmomanometer.

Open the file tutorials\SBP.xlsx.
The analysis report on the SBP worksheet shows that the differences are fairly constant but contain a few large discrepancies.
On the Analyse-it ribbon tab, in the Report group, click Edit.
The analysis task pane opens.
On the Fit Differences task pane, in the LoA estimator drop-down list, select Percentiles.
Click Recalculate.
The results are recalculated and the analysis report updates.

The difference plot shows the limits of agreement estimated using the 2.5th and 97.5th percentiles and the average bias estimated as the median of the differences.

The limits of agreement estimated by the nonparametric method are wider than the limits estimated using the parametric method. Roughly 2.5% of the observations are above, with a similar percentage below the limits of agreement. In contrast, the narrower parametric-based limits of agreement show all observations outside the lower limits of agreement and none above the upper limit.