We are receiving a lot of questions about relevant analyses in the Analyse-it Method Validation edition to help in evaluating new diagnostic tests in the fight against COVID-19. Below are some quick links that will help, but contact us if you have questions - we are working as normal.
Also see our latest blog post: Sensitivity/Specificity and The Importance of Predictive Values for a COVID-19 test
Test if the precision meets the manufacturer's performance claim.
You should use this procedure when you already have a performance claim from a manufacturer's package insert and you want to test whether your precision is significantly greater than the claim. It is possible for the imprecision from a study to be greater than the manufacturer's claim due to the chance alone. This procedure ensures that the manufacturer's claims are only falsely rejected 5% of the time when they are in fact true.
CLSI EP15 uses a familywise significance level so the overall significance level is a maximum of 5% regardless of the number of levels (for example, for 1 level the significance level is 5%, for 2 levels the significance level is 5%/2 = 2.5% for each level, for 3 levels the significance level is 5%/3 = 1.6% for each level).
The analysis report updates.
The detailed variance components table shows the observed and expected SD/CV along with the hypothesis test p-value for each level.
All the hypothesis tests are not significant and highlighted green in this example. If any are significant they are highlighted in red and you should contact the manufacturer for further assistance in diagnosing the problem. If the hypothesis test is not statistically significant but the imprecision estimate is much larger than the claim, you may want to repeat the study with more data to be able to detect smaller departures from the claim.