Estimating the precision of a measurement procedure (CLSI EP05-A3)

Learn how to estimate the precision of a measurement procedure for product performance characteristics, for FDA 510k submissions and product marketing.

In this tutorial you will use the CLSI EP05-A3 procedure to establish the precision.

1. Estimating precision
   Estimate the precision of the measurement procedure at a single site.
2. Estimating precision for a multi-site study
   Estimate the precision of the measurement procedure at multiple sites and samples.
3. Fitting a precision profile
   Fit a precision profile function to describe the imprecision across the measuring interval.

Estimate the precision of the measurement procedure at a single site.

1. Open the file tutorials\EP05-A3 Appendix A.xlsx.

   The worksheet opens showing 4 columns. The Sample column identifies the levels of the analyte in the sample (1 level). Day identifies the day number (20 days), Run identifies the run within each day (2 runs per day). Glucose (mg/dL) identifies the measured value for 2 replicates of sample in each run.

2. Click a cell in the dataset.
3. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Precision, and then click 2 Factor Nested.
   The analysis task pane opens.
4. In the Y drop-down list, select Glucose.
5. In the Factor A drop-down list, select Day.
6. In the Factor B nested within Factor A drop-down list, select Run.
7. In the By drop-down list, select Sample.
8. In the Estimator drop-down list, select SD.
9. In the Confidence Interval edit box, type 95%, and then in the Method drop-down list, select Exact / Satterthwaite.
10. In the Conditions drop-down list, select Laboratory / Repeatability to label the innermost variance component repeatability and the outermost within laboratory.
11. On the Analyse-it ribbon tab, in the MSA group, click Variability plot.
12. On the Analyse-it ribbon tab, in the MSA group, click Identify Outliers.
13. Click Calculate.

    The analysis report opens.

The variability plots show a simple visual assessment of the closeness of agreement between the measured quantity values. The purple lines show the mean of each run, the light blue lines show the mean of each day, and the dark blue line the overall grand mean.

You should observe the scatter of the points to ensure there are no obvious problems. No points stand out as atypical relative to the bulk of the data, and there are no discernible trends (such as drift).

Note: To see an example of a dataset that contains outliers and how to deal with them see the EP15-A3 tutorial.

The abbreviated variance components table shows the required precision statistics expressed numerically as the standard deviation (SD) and coefficient of variation (CV).

The detailed variance components table show the precision expressed numerically as the chosen measure of imprecision along with a confidence interval for each component.

Note: You can change the imprecision to be expressed as SD, Variance, or CV and you can also change the method of computing the confidence intervals. We prefer the Exact / MLS method as the coverage is closer the nominal level in most cases.

Estimate the precision of the measurement procedure at multiple sites and samples.

1. Open the file tutorials\EP05-A3 Appendix B.xlsx.

   The worksheet opens showing 4 columns. The Sample column identifies the levels of the analyte in the sample (6 levels). Laboratory identifies the site (3 laboratories), Run identifies the run (5 runs per site). CA19-9 (kU/L) identifies the measured value for 5 replicates of sample in each run.

2. Click a cell in the dataset.
3. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Precision, and then click 2 Factor Nested.
   The analysis task pane opens.
4. In the Y drop-down list, select CA-19.
5. In the Factor A drop-down list, select Laboratory.
6. In the Factor B nested within Factor A drop-down list, select Run.
7. In the By drop-down list, select Sample.
8. In the Estimator drop-down list, select SD.
9. In the Confidence Interval edit box, type 95%, and then in the Method drop-down list, select Exact / Satterthwaite.
10. In the Conditions drop-down list, select Reproducibility / Repeatability to label the innermost variance component repeatability and the outermost reproducibility.
11. On the Analyse-it ribbon tab, in the MSA group, click Variability plot.
12. On the Analyse-it ribbon tab, in the MSA group, click Identify Outliers, and then on the analysis task panel in the Granularity drop-down list, select Level, Factor A so that outliers are detected for each sample and laboratory.
13. Click Calculate.

    The analysis report opens.

Fit a precision profile function to describe the imprecision across the measuring interval.

1. On the Analyse-it ribbon tab, in the MSA group, click Profile.
   The Precision Profile panel opens in the analysis task pane.
2. In the Precision profile drop-down list, select CV%.
3. Select the Reproducibility, Within Factor A, and Repeatability check boxes.
4. Select the Logarithmic X-axis scale check box so that the scale of the precision profile will show the lower levels more clearly.
5. Select the Fit variance function check-box and then in the Model drop-down, select 3-parameter.
6. Click Recalculate.

   The analysis report updates.