We are receiving a lot of questions about relevant analyses in the Analyse-it Method Validation edition to help in evaluating new diagnostic tests in the fight against COVID-19. Below are some quick links that will help, but contact us if you have questions - we are working as normal.
Also see our latest blog post: Sensitivity/Specificity and The Importance of Predictive Values for a COVID-19 test
Three different ways are used to make large sample inferences all using a X2 statistic, so it is important to understand the differences to avoid confusion.
The score, likelihood ratio, and Wald evidence functions are useful when analyzing categorical data. All have approximately chi-squared (X2) distribution when the sample size is sufficiently large.
Because of their underlying use of the X2 distribution, mixing of the techniques often occurs when testing hypotheses and constructing interval estimates. Introductory textbooks often use a score evidence function for the hypothesis test then use a Wald evidence function for the confidence interval. When you use different evidence functions the results can be inconsistent. The hypothesis test may be statistically significant, but the confidence interval may include the
hypothesized value suggesting the result is not significant. Where possible you should use the same underlying evidence function to form the confidence interval and test the hypotheses.
Note that when the degrees of freedom is 1 the Pearson X2 statistic is equivalent to a squared score Z statistic because they are both based on the score evidence function. A 2-tailed score Z test produces the same p-value as a Pearson X2 test.
Many Wald-based interval estimates have poor coverage and defy conventional wisdom. Modern computing power allows the use of computationally intensive score evidence functions for interval estimates, such that we recommended them for general use.