Estimating the average bias and the limits of agreement

If the differences are not related to magnitude, the mean of the differences provides an estimate of the average bias between the methods. The limits of agreement estimate the interval that a given proportion of differences between measurements is likely to lie within. The limits can be used to determine if the methods can be used interchangeably, or if a new method can replace an old method without changing the interpretation of the results.

  1. On the Analyse-it ribbon tab, in the Method Comparison group, click Estimate Agreement Limits.
    The Fit Differences task is added to the analysis task pane. Provided the differences are normally distributed, the default settings are suitable for analyzing methods with constant bias and variability.
  2. Click Recalculate.
    The results are recalculated and the analysis report updates.

The mean difference estimate is 2.1, which indicates that the Mini Wright meter reads on average 2.1 l/min higher than the Wright meter. A hypothesis test could be performed to determine if this is significantly different from zero. If it is, adjusting readings from the Mini Wright meter by subtracting 2.1 will make them agree more closely with the Wright meter.

(click to enlarge)

The limits of agreement estimate an interval of -73.9 to 78.1, which indicates that the Mini Wright meter may measure as much as 73.9 l/min below and 78.1 l/min above the large meter. This would be unacceptable for clinical purposes.

The confidence intervals for the mean difference and limits of agreement indicate the uncertainty in the estimates. The wide intervals are due to the small sample size and large variation of the differences. Even the most optimistic interpretation would conclude that the agreement is unacceptable.