Standard Edition

ANOVA and ANCOVA with the right multiple comparison One-way, two-way and multi-factor ANOVA and ANCOVA — with eight multiple comparison procedures, interaction plots, effect means, leverage plots and full residual diagnostics.

Download 15-day trial Buy from US$ 155

Every feature from all five editions for 15 days, no sign-up.
Standard edition from US$ 155 a year · 30-day money-back guarantee.

Microsoft Excel with the Analyse-it tab selected, showing the Multiple Comparisons section of a one-way ANOVA report: the Tukey-Kramer all-pairs table with simultaneous confidence intervals, p-values and coloured significance bars, the Mean-Mean scatter plot, the Compare menu open on the ribbon listing the eight procedures, and the Compare Groups task pane. Handwritten notes: Runs inside Excel: every analysis is on the Analyse-it tab; Eight multiple comparison procedures under one Compare menu; Every pair: difference, simultaneous 95% CI, p-value, colour-flagged; Mean-Mean scatter plot: every pairwise difference at a glance. Red: different, green: not different; Type, procedure and the confidence level; The report is an ordinary Excel worksheet: share it, archive it, open it on any PC with Excel.

Beyond the one-way case

It allows me to have a large amount of tests and statistical analysis of low and medium complexity within a database such as excel which saves time associated with eventual data imports from other sources, that the data to be analyzed can be calculated and organized using standard Excel tools, such as links, formulas and dynamic tables.
Rafael B.
Head of Oceanography department

Excel’s built-in ANOVA produces a one-way table and stops. No multiple comparisons, no covariates, no multi-factor models, no diagnostics. Most real experiments have more than one factor. Knowing that “something differs” is rarely the question you need answered.

Which groups differ? By how much? Does adding a covariate change the conclusion? Does the effect of one factor depend on the level of another?

One-way ANOVA, Welch ANOVA and Kruskal-Wallis

Start with a straightforward question: do these groups differ? One-way ANOVA tests that, with Welch’s ANOVA when the variance assumption does not hold and Kruskal-Wallis when normality is in doubt. Side-by-side dot plots with confidence diamonds show the data before you commit to a test.

  • One-way between-subjects ANOVA
  • Welch’s ANOVA for unequal variances
  • Kruskal-Wallis non-parametric test
  • Bartlett, Levene, Brown-Forsythe homogeneity of variance tests
  • One-way within-subjects ANOVA (repeated measures)
  • Friedman non-parametric test
Microsoft Excel with the Analyse-it tab selected, a one-way ANOVA report with dot plots, confidence diamonds and the descriptives table, and the Compare Groups task pane open. Handwritten notes: Compare Groups: normality and equal variances, then means (ANOVA, Welch ANOVA) or medians (Kruskal-Wallis); Dot plots with 95% confidence diamonds: see the data before you test; Each group summarised: N, mean, 95% CI, SE and SD; Mean plot: confidence diamonds, at 95%.
One-way ANOVA: dot plots with 95% confidence diamonds, the descriptives table and the Compare Groups task pane.

Two-way and multi-factor ANOVA, and ANCOVA

Add a second factor to test for an interaction, include a continuous covariate to adjust for baseline differences or build a full factorial to see every combination. Two-way, multi-factor, ANCOVA — the model matches the design, and Type I and Type III tests show which terms matter.

  • Two-way and multi-factor designs
  • Crossed factors, polynomial terms, interactions
  • Continuous covariates (ANCOVA)
  • Automatic dummy variable coding
  • Full factorial and custom model structures
  • Type I (sequential) and Type III (partial) SS
  • ANOVA table (SS, DF, MS, F, p-value)
  • R², adjusted R²
  • Parameter estimates with confidence intervals
  • Model equation
Microsoft Excel showing a two-way ANOVA report: the Effect of Terms table for primer type, application method and their interaction, then the main-effect plots, with the Fit Model task pane open on Effect Means. Handwritten notes: Fit Model: two-way, multi-factor and ANCOVA designs; test the model, test each term, then check the residuals; Effect of Terms: both factors matter, their interaction does not (p = 0.27); Effect means for each term, plotted as main effects; Terms: two factors and their interaction.
Two-way ANOVA: the Effect of Terms table and the main-effect plots for primer type and application method.

Multiple comparisons: Tukey-Kramer, Dunnett, Hsu, Scheffé and more

A significant ANOVA is the start, not the answer. After a one-way ANOVA, eight multiple comparison procedures answer the specific follow-up question: all pairs with Tukey-Kramer, against a control with Dunnett, with the best using Hsu. Each controls the family-wise error rate for its comparison structure, with a non-parametric counterpart where there is one. After a Fit Model ANOVA or ANCOVA the five parametric procedures apply. The Mean-Mean scatter plot shows every pairwise difference in a single plot.

  • After one-way ANOVA (Compare Groups), eight:
  • Tukey-Kramer, Dwass-Steel-Critchlow-Fligner, Dunnett, Steel, Hsu, Scheffé, Student’s t, Wilcoxon-Mann-Whitney
  • After Fit Model ANOVA or ANCOVA, five:
  • Tukey-Kramer, Dunnett, Hsu, Scheffé, Student’s t
  • Both:
  • Mean-Mean scatter plot; simultaneous confidence intervals, p-values, colour-flagged significance
Microsoft Excel showing the Multiple Comparisons section of a one-way ANOVA report: the Tukey-Kramer all-pairs table with simultaneous 95% confidence intervals, p-values and red and green significance bars, the Mean-Mean scatter plot, and the Compare Groups task pane open on Multiple Comparisons with the type, method and confidence interval options. Handwritten notes: Simultaneous 95% CIs for every pair; bars red where the pair differs, green where it does not; Mean-Mean scatter plot: every pairwise difference at a glance; Comparison type, procedure and CI level.
Tukey-Kramer all-pairs comparisons with the Mean-Mean scatter plot, and the comparison type, procedure and confidence level in the task pane.

Effect means, interaction plots and leverage plots

Effect means and interaction plots show whether factors work together or against each other. A main effect that looks strong on its own can disappear — or reverse — in the presence of an interaction. Leverage plots isolate each term’s contribution after accounting for everything else in the model.

  • Adjusted effect means with confidence intervals
  • Main effect plots
  • Interaction plots
  • Leverage (partial residual) plots
Microsoft Excel showing the 2-way Interaction Effects matrix of a three-factor ANOVA report: an interaction plot for every pair of factors A, B and C, with the Fit Model task pane open on Effect Means. Handwritten notes: Effect Means, on the Terms group, adds the main-effect and interaction plots to the report; An interaction plot for every pair of factors. Parallel lines: no interaction. Lines that diverge or cross: the factors combine; Interaction effect plots: one tick.
A three-factor experiment, each factor at two levels: an interaction plot for every pair of factors.

Residual diagnostics: outliers, leverage and influence

Residual plots reveal the non-normality and non-constant variance that the ANOVA table will not show. The Outliers, Leverage, Influence plot marks every observation with a large studentised residual or Cook’s D. You can then see whether the conclusion rests on a few unusual cases. Save residuals, leverage and Cook’s D back to the dataset for further investigation.

  • Raw and standardised residual plots
  • Sequence and lag plots
  • Residual distribution plot
  • Outliers, Leverage, Influence plot (studentised residuals against leverage, bubble size Cook’s D)
  • Save fitted values, residuals, standardised and studentised residuals, leverage, Cook’s D
Microsoft Excel showing the Residuals section of a two-way ANOVA report: standardised residuals against predicted values, a histogram and a normal probability plot of the residuals, with the Fit Model task pane open on Residuals. Handwritten notes: Residuals, and Outliers and Influence, on the Diagnostics group: the plots that show whether the model holds; Standardised residuals against predicted values: no pattern, no trend in the spread; Histogram and normal plot: are the residuals normal?; Histogram, normal, sequence and lag plots.
Residual diagnostics for the two-way ANOVA: standardised residuals against predicted values, their histogram and normal plot.

Example analyses

See ANOVA results in detail — ANOVA tables, multiple comparisons, interaction plots and diagnostics — using real datasets you can download and follow along with.

ANOVA 1 5 pages One-way ANOVA, two ways
Tensile strength of paper by hardwood concentration. Montgomery 2001. Four groups of six observations, analysed through both Compare Groups and Fit Model so the two routes can be compared. Dot plots with confidence diamonds, ANOVA, Tukey-Kramer over six contrasts and a Mean-Mean scatter plot.
ANOVA 2 p2 3 pages Two-way ANOVA
Aircraft primer paint adhesion force. Montgomery 2001, page 572. Three primer types × two application methods, 18 observations, R² = 0.908. F tests for each term — both main effects significant, the interaction not — LS means, main effect plots and Tukey-Kramer on both factors.
ANOVA 3 p3 3 pages Three-factor ANOVA
Surface finish, 2³ factorial design. Montgomery 2001, page 587. 16 observations with every two-way and three-way interaction fitted. Only factor A is significant at 5%. Main effect plots and interaction effect plots.
ANOVA 4 p2 2 pages ANCOVA
Three dose levels with a covariate, 30 participants. Field 2003. A three-level factor — placebo, low dose, high dose — with a participant-level covariate, 30 observations. F tests for dose and covariate, LS means, main effect plot and Dunnett comparisons against the placebo control.
Compare Groups 1 3 pages One-way ANOVA
Y by brand, 7 groups, 19 observations. Descriptive statistics by group, Levene test for homogeneity of variance, one-way ANOVA, Tukey-Kramer all-pairs comparisons over 21 contrasts with simultaneous 95% CIs and a Mean-Mean scatter plot.
Compare Groups 2 2 pages Independent-samples t-test
Calcium supplementation and blood pressure, 21 observations. Five-number summary by group with side-by-side plots, Fisher F test for the variance ratio, and Student’s t test of the difference in means against a hypothesised difference of 3.
Compare Pairs 2 pages Compare pairs
Body fat before and after an exercise programme. 28 paired observations. Descriptive statistics for both conditions and for the differences, Hodges-Lehmann shift estimate with a 95.49% CI and the Wilcoxon signed-ranks test.

Part of the Standard edition

ANOVA and ANCOVA are one part of a complete statistical analysis toolkit. The Standard edition also includes simple and multiple regression, logistic regression, PCA and factor analysis, descriptive statistics, hypothesis testing, correlation and categorical data analysis. See everything in the Standard edition →

Related guides in the Learn section: robustness testing with factorial designs and the precision components explained.

Software you can trust

Validated calculations Every statistic tested against the NIST Statistical Reference Datasets, published datasets and thousands of internal test cases. No reliance on Excel’s built-in functions. How Analyse-it is developed and validated →
Data stays on your PC No cloud processing, no uploads, no third-party access. Your data never leaves your computer — essential when working with sensitive, confidential or patient-identifiable data.
Standard Excel workbooks Analyses are ordinary Excel workbooks that you can share with colleagues, archive for audit and open on any machine with Excel — no Analyse-it licence required.
No formulas to break Results contain no formulas, so there is nothing to overwrite and no cell reference to break. The results you reported will be exactly what you find when you reopen the workbook.

Free trial and pricing

Try it on your own data first. The 15-day trial is every feature from all five editions, with no sign-up and no licence key — install it and start straight away.

Download 15-day trial See pricing or get a quote →

Standard edition: US$ 155 per year or US$ 395 for a perpetual licence. Every purchase carries a 30-day money-back guarantee. Need a quote for purchasing? Add the licence to the cart and save it as a PDF quote.