You are viewing documentation for the old version 2.30 of Analyse-it. If you are using version 3.00 or later we recommend you go to the latest documentation
Qualitative (Sensitivity/Specificity) examines the performance of a diagnostic test and it's ability to correctly identify non-normal (abnormal/diseased) cases. Specific decision levels (medical decision points) can be evaluated if the diagnostic test is measured on a continuous scale.
The requirements of the test are:
- A dichotomous nominal sample containing the true state of the case: normal or non-normal.
- A continuous sample containing the observations of the diagnostic test, or a dichotomous nominal sample containing the diagnosis of each case: normal or non-normal.
Examining diagnostic ability
Diagnostic test performance can be evaluated when only the true state and diagnosis, determined by the diagnostic test, is known for each case.
Data in existing Excel worksheets can be used and should be arranged in the List dataset layout, or alternatively the number of cases correctly & incorrectly identified can be summarised as a 2x2 table dataset containing counts. Any names can be used for the positive and negative states, e.g. Abnormal/Normal, as long as the true state and test diagnosis use the same state names.
When entering new data we recommend using New Dataset to create a new test performance or 2x2 contingency table dataset.
To start the test:
- Excel 2007:
Select any cell in the range containing the dataset to analyse, then click Test performance on the Analyse-it tab, then click Qualitative (Sensitivity / Specificity).
Excel 97, 2000, 2002 & 2003:
Select any cell in the range containing the dataset to analyse, then click Analyse on the Analyse-it toolbar, click Test performance then click Qualitative (Sensitivity / Specificity).
- Click True classification then select the dichotomous variable containing the true state (positive or negative) of the case.
- Click Positive indicated by and select the group indicating a positive case. The list shows the two groups of the classification variable.
- Click Test and select the variable containing the observations of the diagnostic test.
- Tick Predictive value to show positive and negative predictive values. Predictive values will be based on the sample prevalence unless the prevalence in the population is entered (see Prevalence in the population below).
- Click OK to run the test.
The report shows the number of observations analysed, and, if applicable, how many missing values were excluded. The table shows number of actual positive and negative cases against the number identified by the diagnostic test.
Sample prevalence shows the proportion of positive cases (as indicated by true classification) in the sample and is used to calculate predictive values and correct- and mis-classification rates. Predictive values use the population prevalence, if entered.
Sensitivity TP (true positive), Specificity TN (true negative), FP (false positive) and FN (false negative) proportions and confidence intervals, with positive and negative likelihood ratios, describe the performance of the test.
Examining diagnostic performance of a decision level
When the diagnostic test is observed on a continuous scale individual decision levels (medical decision points) can be evaluated. Potential decision levels can be determined from a ROC analysis.
Data in existing Excel worksheets can be used and should be arranged in a List dataset layout containing a nominal variable indicating the true state, and a continuous variable containing the observations of the diagnostic test for each case.
When entering new data we recommend using New Dataset to create a new test performance dataset.
- Select any cell in the range containing the dataset to analyse, then click Test Performance on the Analyse toolbar of the Analyse-it tab, then click Qualitative (Sensitivity/Specificity).
- Click Test classification and select the dichotomous variable containing the test diagnosis of each case (normal or non-normal).
- Enter Cut-off, the decision level where cases are dichotomised as normal or non-normal, then click Abnormal if and select Less than if observations below the cut-off value are to be deemed non-normal, or click Greater than if observations above it are non-normal.
- Click Classification and select the dichotomous variable containing the true state of the case.Click Abnormal indicated by then select the group that indicates a subject is non-normal.
- Enter the Prevalence of the condition in the population. See below for more information about prevalence.
- Click OK to run the test.
The report shows the number of observations analysed, and, if applicable, how many missing values were listwise excluded.
A table shows number of true normal and non-normal cases against those identified by the diagnostic test at the chosen decision level. Sensitivity, specificity, positive & negative predictive values and efficiency show the performance of the diagnostic test.
Prevalence in the population
When evaluating a test the subjects studied are usually hand selected to ensure the sample contains enough normal and abnormal cases to properly evaluate the performance of the test. However in the population at large the incidence or prevalence of the non-normal state will probably be very different to that in the sample. In most populations the condition will be rarer than the hand selected sample, causing a small false-positive rate in the sample to be magnified when applied to the population (more so than a small false-negative rate).
Positive & negative predictive values, the probability of the test correctly diagnosing the state of a case, use the prevalence (when specified) to better express the performance of the test in the population.
To enter the prevalence of the condition in the population:
- If the Qualitative dialog box is not visible click Edit on the Report toolbar on the Analyse-it tab.
- Enter Prevalence of the condition in the population, as a proportion between 0 and 1. For example if the prevalence is 5% of the population enter 0.05.
- Click OK.
The positive and negative predictive values will now show the ability of the diagnostic test to correctly identify positive (non-normal) and negative (normal) cases drawn at random from the population.
References to further reading
- Statistical Methods in Diagnostic Medicine.
Xiao-Hua Zhou, Nancy A. Obuchowski, Donna K. McClish 2002; ISBN 0-471-34772-8
- Practical Statistics for Medical Research.
ISBN 0-412-27630-5 1991; 409-19.
- Statistical methods in Laboratory Medicine.
ISBN 0-7506-1345-9 1991; 336-46.
(click to enlarge)