Difference in area under curve (AUC)

The difference in areas under the ROC curves compares two or more diagnostic tests.

It is imperative when comparing tests that you choose the correct type of analysis dependent on how you collect the data. If the tests are performed on the same subjects (paired design) the test results are usually correlated. Less commonly, you may perform the different tests on different groups of subjects or the same test on different groups of subjects, and the test results are independent. A paired design is more efficient and is preferred whenever possible.

A point estimate of the difference between the area under two curves is a single value that is the best estimate of the true unknown parameter; a confidence interval indicates the uncertainty of the estimate. If the tests are independent, the confidence interval is computed using the combined variance of the curves and a large sample Wald approximation. If the tests are paired, the standard error incorporating the covariance (DeLong et al., 1998) and a large sample Wald approximation is used.

A hypothesis test for the difference in AUC can test equality, equivalence, or non-inferiority of the diagnostic tests. Inferences about the difference between AUC are made using a Z test. The three hypotheses of interest are:
  • Equality

    The null hypothesis states that the difference is equal to a hypothesized value (usually 0), against the alternative hypothesis that it is not equal to the hypothesized value (usually 0). When the test p-value is small, you can reject the null hypothesis and conclude the difference is not equal to the hypothesized value (usually 0, the tests are different).

  • Equivalence

    The null hypothesis states that the difference is less than a lower bound of practical equivalence or greater than an upper bound of practical equivalence, against the alternative hypothesis that the difference is within an interval considered practically equivalent. When the test p-value is small, you can reject the null hypothesis and conclude that the tests are equivalent.

  • Non-inferiority

    The null hypothesis states that the difference from a standard test is greater than the smallest practical difference against the alternative hypothesis that the difference from the standard test is less than the smallest practical difference. When the test p-value is small, you can reject the null hypothesis and conclude that the test is not inferior to the standard test.

When the ROC curves cross, the difference between the AUC does not provide much useful information.