A hypothesis test formally tests if two or more population means/medians are equal.
The hypotheses to test depends on the number of samples:
- For two samples, the null hypothesis states that the difference between the mean/medians of the populations is equal to a hypothesized value (0 indicating no difference), against the alternative hypothesis that it is not equal to (or less than, or greater than) the hypothesized value.
- For more than two samples, the null hypothesis states that the means/medians of the populations are equal, against the alternative hypothesis that at least one population mean/median is different.
When the test p-value is small, you can reject the null hypothesis and conclude that the populations differ in means/medians.
Tests for more than two samples are omnibus tests and do not tell you which groups differ from each other. You should use multiple comparisons to make these inferences.