Prepare standards at known concentrations, measure the response of each, fit a curve through the points, and you have a calibration. From then on the curve runs backwards: an unknown sample produces a response, and you invert the fit to get its concentration. That inversion is what makes calibration different from ordinary regression. The quantity you care about, the estimated concentration, is not the quantity the fit was optimised for.
Start with the simplest model the data support, and let the residuals decide whether it is enough. A straight line is right for many detectors over a limited range. A quadratic fits gentle curvature. Immunoassays and other binding assays are usually sigmoidal and need a four- or five-parameter logistic. Forcing a straight line through the linear-looking middle throws away both ends of the working range.
Resist the temptation to add terms until the fit improves. Every extra parameter buys a closer pass through the calibrators you happen to have measured. A curve that follows its own calibration noise will estimate unknowns worse, not better. The model should reflect the response mechanism, not the scatter of one run.
A four- or five-parameter logistic is a different class of problem from everything else here. It is non-linear in its parameters, so fitting one takes iterative non-linear least squares rather than the algebra that fits a least-squares line. That is what dedicated immunoassay curve-fitting software is for; Analyse-it has no non-linear least-squares facility and does not fit 4PL or 5PL curves. The rest of this guide covers the models least squares can fit: the straight line, polynomials to 6th order, and the linearisable forms — logarithmic, exponential and power — which between them cover most chromatographic, spectrophotometric and electrochemical calibrations.
One naming collision is worth getting out of the way, because it costs people time. A 4PL is not the four-parameter variance function used in a precision profile. A variance function models imprecision — SD or CV — against concentration, and is what sets the limit of quantitation. A 4PL models response against concentration. The two share a number and nothing else.
Weighting is the decision that matters most, and the one most often left at its default. Ordinary least squares minimises the sum of squared residuals in the response units, treating an error of 100 units at the top of the curve as exactly as important as an error of 100 units at the bottom. Detector response is almost always heteroscedastic: absolute variability grows with concentration, so the top standards have far larger residuals in absolute terms. Under unweighted fitting those top standards dominate the fit entirely.
The consequence is a curve pulled to fit the high standards while the low end drifts. The low end is where the limit of quantitation lives and where most clinically or environmentally interesting samples sit.
Weighting each point by 1/x or 1/x² restores the balance, so that a 5% error means the same thing everywhere on the curve. Where replicate data are available to estimate it, the reciprocal of the variance is better still. The same logic drives the choice between ordinary and weighted regression in method comparison: constant variance means unweighted, proportional variance means weighted.
The way to decide is to look. Run replicates at several levels, plot the standard deviation of the response against concentration, and see whether it is flat or rising. A rising profile means weighting.
In Analyse-it, weighted fitting sits in two analyses. The dilution-series linearity analysis fits weighted polynomial models for non-constant precision across the measuring interval. Method comparison offers weighted ordinary and weighted Deming regression.
A calibration with an r² of 0.999 can still be systematically wrong at the concentrations you care about. The coefficient measures how much of the variation in response the model accounts for. Over a range spanning two or three orders of magnitude, r² will be close to one for almost any monotonic fit, including a straight line through an obviously curved set of points. This is the same failure that makes correlation the wrong statistic for method comparison: a measure of association is not a measure of agreement.
What does the job is the residual plot. Plot residuals against concentration and look for structure. Random scatter around zero means the model is adequate. A curve, a fan, or a run of points on one side means it is not, whatever r² says.
Where you have replicates at each level, a lack-of-fit test separates the two questions properly. The test compares the variation of points around the fitted curve with the variation between replicates at the same level, and tells you whether the misfit is larger than measurement noise can explain.
Since the curve is used in inverse, test it in inverse. Take each calibrator, read its concentration back off the fitted curve, and compare that with its assigned value. This back-calculation gives a recovery at every level, and exposes exactly what the residual plot only hints at. A curve that is fine at the top and reads 20% low at the bottom standard shows that here, in units anyone can act on.
Set an acceptance criterion for that recovery before you look, level by level, and expect a wider tolerance at the lowest standard than in the middle. A calibrator that fails its own recovery check should not be dropped just to improve the fit. The reason it failed is useful data.
In Analyse-it, do this in the worksheet. Fit Model gives you the model equation and predicts response from concentration. Invert the equation yourself to recover concentration from response, then put the recovery column beside it. Inverse prediction proper — solving for X with a confidence interval — is reported for logistic and probit models, where the question is the concentration at a given probability. See probit analysis for the limit of detection.
The curve is only evidence over the interval the standards covered. Extrapolating above the top calibrator or below the bottom one estimates concentrations from a model shape that was never tested there, and the error is unbounded.
The working range is bounded at the bottom by the limit of quantitation (the concentration where imprecision first meets your goal) and at the top by the highest calibrator. Results outside it are reported as below or above the range rather than as numbers. The measuring interval and the reportable range covers where dilution extends that and where it does not.
Download the dilution-series linearity example workbook (.xlsx) — five levels with replicates, fitted with linear and polynomial models, with the deviation from linearity assessed at each level. It is framed as a CLSI EP06 linearity study rather than a calibration, but the structure is the same: known levels, replicates, a fitted model, and the fit judged level by level.
The example workbook is downloading.
It opens in Excel on its own — the data and the finished results are both in it. Analyse-it is what lets you change the analysis and re-run it, try the same study on your own data, or work through it to see how the software handles it.
Every feature from all five editions for 15 days, with no sign-up and no licence key.
Leaving the weighting at unweighted. Where response variability grows with concentration, an unweighted fit is decided by the top standards and reads badly at the bottom, which is usually where the samples that matter are.
Judging the fit by r². It will be high for a curved set of points fitted with a line. Read the residual plot, and test lack of fit against replicate variation where you can.
Adding terms to improve the fit. A higher-order polynomial passes closer to the calibrators and estimates unknowns worse. Match the model to the response mechanism.
Extrapolating beyond the standards. The curve is evidence only between the lowest and highest calibrator. Outside that, report the result as beyond the range.
Dropping an inconvenient calibrator. Removing a level because it spoils the fit hides the problem rather than solving it. Investigate why it failed.
Analyse-it fits the model and reports what shows whether it holds, inside Excel:
Every feature from all five editions for 15 days, with no sign-up and no licence key. Model fitting is in every edition, from US$ 155 a year; the dilution-series linearity analysis, which adds weighted models and deviation testing level by level, is in the Method Validation and Ultimate editions, from US$ 475 a year. Validated against NIST and CLSI reference datasets. See assessing linearity of a measurement procedure for judging deviation against an allowable limit, or the trueness reference guide for the technical detail.