# 7-Apr-2020 Why the diagnostic test 'accuracy' statistic is useless

In our last post, we mentioned that the *'accuracy'* statistic, also known as the probability of a correct result, was a useless measure for diagnostic test performance. Today we'll explain why.

Let's take a hypothetical test with a sensitivity of 86% and specificity of 98%.

As a first scenario we simulated test results on 200 subjects with, and 200 without, the condition. The *accuracy *statistic (TP+TN)/N is equal to (172+196)/400 = 92%. See below:

In a second scenario we again simulated test results on 400 subjects, but only 50 with, and 350 without, the condition. The *accuracy *statistic is (43+343)/400 = 96.5%. See below:

The *accuracy* statistic is effectively a weighted average of sensitivity and specificity, with weights equal to the sample prevalence P(D=1) and the complement of the prevalence (that is, P(D=0) = 1-P(D=1)).

Accuracy = P(TP or TN) = (TP+TN)/N = Sensitivity * P(D=1) + Specificity * P(D=0)

Therefore as the prevalence in the sample changes so does the statistic. The prevalence of the condition in the sample may vary due to the availability of subjects or it may be fixed during the design of the study. It's easy to see how to manipulate the accuracy statistic to weigh in favor of the measure that performs best.

Finally, the 'accuracy' statistic is not a good way of comparing diagnostic tests. As illustrated above, it is affected by the prevalence in the samples of the tests. And, even given two tests, with sample prevalence 50%, suppose test A has a sensitivity of 100%, but specificity 0%, and test B has a sensitivity of 0% and specificity 100%. Both tests give the same probability of a correct result, yet the two tests are radically different.

The measures of the accuracy of a diagnostic test are sensitivity and specificity. The *accuracy *statistic, the probability of a correct test result, is **not** a measure of the intrinsic accuracy of a test.

For more information, see our online documentation:

Measures of diagnostic accuracy

##### Tags

##### Latest posts

- A New Year. A New Edition.
- Analyse-it v6.10: Survival Analysis and other improvements
- Analyse-it v5.90: Support for the updated CLSI EP6-Ed2 protocol and inverse predictions
- Analyse-it 5.50 to 5.65: Recent improvements
- COVID-19: Calculating the detection limit for a SARS-CoV-2 RT-PCR assay using Analyse-it
- COVID-19: Calculating PPA/NPA agreement measures using Analyse-it
- Diagnostic accuracy (sensitivity/specificity) versus agreement (PPA/NPA) statistics
- COVID-19: Establishing the diagnostic accuracy (sensitivity/specificity) of a test using Analyse-it
- Why the diagnostic test 'accuracy' statistic is useless
- Sensitivity/Specificity and The Importance of Predictive Values for a COVID-19 test

##### Most popular posts

- Announcing the Analyse-it Quality Control & Improvement Edition
- Analyse-it 4.0 released: Support for CLSI guidelines, and Measurement Systems Analysis
- Analyse-it 3.80 released: Principal Component Analysis (PCA)
- Recent improvements in Analyse-it 3.76 and our first video tutorial!
- Our software development and validation process
- The numerical accuracy of Analyse-it against the NIST StRD
- Quantiles, Percentiles: Why so many ways to calculate them?
- Handbook of Parametric & Non-parametric Statistical procedures
- A sombre note: Professor Rick Jones

## Comments

Comments are now closed.