Asymmetric agreement measures are dependent on the assignment of X and Y variables. They are often useful due to the natural interpretation as the proportion of the comparative method (X) results in which the new method (Y) results are the same.
For a binary test, with positive/negative results, the results can be expressed as a 2x2 contingency table:
The positive agreement (PPA) and negative agreement (NPA) have a natural interpretation:
Symmetric agreement measures are not affected by interchanging the X and Y variable. These are useful in many other cases, such as comparing observers, laboratories, or other factors where neither is a natural comparator. There are various measures based on the mean of the proportions to which X agrees with Y, and Y agrees with X. The Kulczynski, Dice-Sørensen, and Ochiai are three such measures that use arithmetic, harmonic, and geometric mean of the proportions, respectively.
The harmonic mean weights the smaller proportion more heavily and produces the smallest value amongst the three measures. The geometric mean is the square root of Bangdiwala’s B statistic, which is the ratio of the observed agreement to maximum possible agreement in the agreement plot. The arithmetic mean has the greatest value. In most cases, of moderate to high agreement, there is very little to choose between the measures.
The average positive agreement and average negative agreement using the Dice-Sørensen measure are:
The overall proportion of agreement is the sum of the diagonal entries divided by the total.
From the Statistical Reference Guide for Analyse-it 6.24.0: https://analyse-it.com/docs/user-guide/method-comparison/agreementmeasures
We use essential cookies to run the site. With your permission we'd also like to use analytics and
advertising cookies to see how visitors find and use Analyse-it, so we can keep improving it and reach
more people like you.
You can change your mind at any time, see our
Privacy policy.