Agreement measures for binary and semi-quantitative data
Agreement measures summarize the similarity of the results of two binary or semi-quantitative methods.
Asymmetric agreement measures are dependent on the assignment of X and Y variables. They are often useful due to the natural interpretation as the proportion of the comparative method (X) results in which the new method (Y) results are the same.
For a binary test, with positive/negative results, the results can be expressed as a 2x2 contingency table:

- Positive agreement is the proportion of comparative/reference method positive results in
which the test method result is positive.

- Negative agreement is the proportion of comparative/reference method negative results in
which the test method result is negative.

Symmetric agreement measures are not affected by interchanging the X and Y variable. These are useful in many other cases, such as comparing observers, laboratories, or other factors where neither is a natural comparator. There are various measures based on the mean of the proportions to which X agrees with Y, and Y agrees with X. The Kulczynski, Dice-Sørensen, and Ochiai are three such measures that use arithmetic, harmonic, and geometric mean of the proportions, respectively.
The harmonic mean weights the smaller proportion more heavily and produces the smallest value amongst the three measures. The geometric mean is the square root of Bangdiwala’s B statistic, which is the ratio of the observed agreement to maximum possible agreement in the agreement plot. The arithmetic mean has the greatest value. In most cases, of moderate to high agreement, there is very little to choose between the measures.
- Average positive agreement is the number of positive matches as a proportion of the
average of the number of positive results by X, Y.

- Average negative agreement is the number of negative matches as a proportion of the
average of the number of negative results by X, Y.

The overall proportion of agreement is the sum of the diagonal entries divided by the total.