Math 115: Introduction to Statistics – 15. Inference for Multiple Means

Where We’ve Been — Comparing Means

Topic	Comparison	Statistic	Example
11	One mean	\(\bar{x}\)	Cherry Blossom
12	Two independent means	\(\bar{x}_1 - \bar{x}_2\)	Birth weights
13	Two paired means	\(\bar{x}_d\)	Textbook prices
15 (today)	Three or more means	\(F\)	Vocabulary by class

Topic 15 extends topic 12. Topic 12 compared two means with a single difference \(\bar{x}_1 - \bar{x}_2\). With three or more means, one difference can’t summarize the picture — so we need a new statistic: \(F\).

Vocabulary Scores by Social Class

Do vocabulary test scores differ by self-identified social class? General Social Survey — random sample of 795 U.S. adults.

Response: \(0\)–\(10\) score on a 10-question vocabulary test (numerical)
Explanatory: self-identified social class — Lower, Working, Middle, Upper (\(k = 4\) groups)

Hypotheses.

\(H_0: \mu_{\text{Lower}} = \mu_{\text{Working}} = \mu_{\text{Middle}} = \mu_{\text{Upper}}\)
\(H_A:\) at least one class has a different mean vocabulary score

Today’s running example comes from the General Social Survey — a random sample of seven hundred ninety-five U.S. adults. Each respondent took a ten-question vocabulary test, scored zero through ten, and identified their own social class as Lower Class, Working Class, Middle Class, or Upper Class. So the response is numerical — the wordsum score — and the explanatory variable is categorical with four levels. The research question is whether mean vocabulary score differs by self-identified social class. The null hypothesis says the population mean wordsum score is the same in all four classes, while the alternative says at least one class has a different mean. Notice the alternative isn’t directional — we’re not predicting which class would be higher. Any pattern of inequality counts against the null.

EDA: Vocabulary Scores

Class	n	\(\bar{x}_i\)	\(s_i\)
Lower	41	5.07	2.24
Working	407	5.75	1.87
Middle	331	6.76	1.89
Upper	16	6.19	2.34

Boxes show each class’s median and spread; ◆ = group mean (what ANOVA compares); dashed line = overall mean \(\approx 6.14\). Middle sits highest, Lower lowest; Upper is noisy with only 16 people. Real differences in the means, or sampling noise?

Why Not Just Run Many T-Tests?

The obvious idea: compare the classes two at a time with the t-tests we already know.

Comparing all four classes pairwise takes 6 separate t-tests — Lower–Working, Lower–Middle, and so on, every class against every other.

Each t-test carries its own 5% chance of a Type I error (a false positive) when its null is true.

That’s fine for one test. But we’re about to run six — what happens to the overall risk?

When Errors Add Up

If all six nulls were true, the chance of at least one false rejection is:

\[1 - (0.95)^6 \approx 0.265\]

That’s about 26.5% — over five times the 5% we intended.

The overall error balloons. Running many separate tests trades a controlled per-test error for a much larger overall chance of a false alarm.

We need a single overall test that asks one yes/no question with one \(\alpha\).

One Overall Test: ANOVA

ANOVA (Analysis of Variance) tests whether any of \(k\) group means differ:

\(H_0: \mu_1 = \mu_2 = \cdots = \mu_k\) (all population means equal)
\(H_A:\) at least one \(\mu_i\) differs from the others

The idea: weigh the variability between the group means against the variability within the groups — developed next.

Yes/no, not which. ANOVA tells us whether at least one group differs — it doesn’t say which groups. Pinpointing the specific pair(s) is the job of topic 16 (multiple comparisons).

Between vs. Within

Two sources of variability in the data:

Between groups — how spread out the group means are around the overall mean.
Within groups — how spread out individual observations are around their own group’s mean.

The logic of ANOVA. If \(H_0\) is true, the group means should sit as close to the overall mean as random sampling within each group would predict — the between-group spread carries no signal beyond that sampling noise.

If between is much larger than within → evidence that real group differences are pulling the means apart.

The whole logic of ANOVA rests on two complementary spreads. Between-group spread asks how far the four group means sit from each other and from the overall mean. Within-group spread asks how far the individual scores sit from their own group’s mean — the noise inside each class. If the null is true, all four groups are drawing from the same population, and any apparent spread between their sample means is just sampling noise — the same noise that produces the scatter within each group. So under the null, the group means are only as spread out as that within-group sampling noise would produce, with no signal on top. If the between-group spread is much larger than the within-group noise can account for, that’s evidence that real, systematic differences among the population means are pulling the sample means apart further than chance alone would explain. Before we turn that comparison into a formula, let’s check the intuition with a quick picture.

Same Means, Different Evidence

Two made-up datasets, three groups each, differing only in within-group scatter. Judge from the raw data alone.

In which panel is there stronger evidence that the three population means truly differ — judging from the data alone? Same means in both panels — 4, 5, 6 (diamonds)! Between-group spread is identical; only within-group scatter differs. Left: tight scatter → means stand clear → strong evidence. Right: means buried in scatter → weak. This contrast is exactly what ANOVA measures.

Here’s a quick gut check before we build the formula. There are two made-up datasets, each with three groups. Look at the scatter of points: in which of the two panels does it look like the three groups genuinely have different centers? Take a side, and advance when you’re ready to see the answer. Here’s the twist. The group means are exactly the same in both panels — four, five, and six — now shown as the black diamonds. So the spread between the group means is identical on the two sides, and the only thing that differs is the scatter within each group. On the left, the points hug their group means tightly, so the three centers stand clearly apart from the noise, and that’s strong evidence the means differ. On the right, those same three means are swimming in scatter, and you can barely tell the groups apart, so the evidence is weak. This is the whole idea of ANOVA in one picture. It doesn’t just ask how far apart the group means are; it asks how far apart they are relative to the scatter within each group.

Two Sums of Squares

Back to the vocabulary data — each spread as a number (from software):

Between groups (means vs overall):

\[SSG = \sum n_i (\bar{x}_i - \bar{x})^2 \approx 237\]

Within groups (values vs group mean):

\[SSE = \sum \sum (x_{ij} - \bar{x}_i)^2 \approx 2870\]

A raw sum of squares isn’t a fair yardstick on its own — it grows as more data piles in, so \(SSE\) and \(SSG\) can’t be compared head-to-head. Next: turn each into an average (a mean square).

Mean Squares: A Fair Comparison

Divide each sum of squares by its own degrees of freedom to get an average — a mean square:

Between (MSG) — \(df_1 = k - 1 = 3\)

\[MSG = \frac{SSG}{df_1} = \frac{237}{3} = \; ?\]

Within (MSE) — \(df_2 = n - k = 791\)

\[MSE = \frac{SSE}{df_2} = \frac{2870}{791} = \; ?\]

Try it: compute both mean squares, then compare them. \(MSG = 79\) and \(MSE \approx 3.6\). Both estimate the same underlying noise — how much individual scores vary within a group. \(MSE\) measures it directly; under \(H_0\), \(MSG\) estimates that same noise, so the two land close. Here \(MSG \gg MSE\) — the class means vary more than noise alone, a hint they really differ.

To compare the two spreads fairly, we divide each sum of squares by its own degrees of freedom, giving an average called a mean square. The two degrees of freedom are different, and that difference is the whole point. The between-groups degrees of freedom, df one, is k minus one — four classes minus one, or three. The within-groups degrees of freedom, df two, is n minus k — seven hundred ninety-five minus four, or seven hundred ninety-one. So we divide S-S-G, about two hundred thirty-seven, by three, and we divide S-S-E, about two thousand eight hundred seventy, by seven hundred ninety-one. Work out both mean squares, then compare them. Advance when you’re ready. The between-group mean square is about seventy-nine, and the within-group mean square is only about three point six. Here’s the key idea. Both mean squares are trying to estimate the same thing — the underlying noise, meaning how much individual scores naturally vary within a group. The within-group mean square measures that noise directly. And when the null hypothesis is true, when all the class means are equal, the between-group mean square estimates that same noise, so the two land close together and their ratio is near one. But here the between-group mean square, seventy-nine, is far bigger than three point six. The class means are varying more than noise alone would produce, which is a strong hint that they really differ.

The F-Statistic

The test statistic is the ratio of the two mean squares:

\[F = \frac{MSG}{MSE}\]

\(F \approx 1\): between-group variability looks like within-group — consistent with \(H_0\).
\(F \gg 1\): the group means are spread out more than noise alone → evidence they differ.

Try it: compute \(F\) for vocabulary (\(MSG \approx 79\), \(MSE \approx 3.6\)). \(F = \dfrac{79}{3.6} \approx 22\) — between-group variability is about 22× the within-group variability. Far above 1: suggestive evidence against \(H_0\).

The ANOVA test statistic, F, is simply the ratio of the two mean squares — the between-group mean square divided by the within-group mean square. The logic follows from how they’re built. If the null is true and the class means are really equal, the two mean squares estimate the same variance, so F sits near one. If the class means differ, the between-group mean square grows and F climbs well above one. Try computing F for vocabulary from the two mean squares we found. Advance when you’re ready. Seventy-nine divided by three point six is about twenty-two. So the between-group variability is roughly twenty-two times the within-group variability — far above the value of one we’d expect if all the class means were equal. That’s already suggestive evidence against the null, even before we compute a p-value.

Where Does the F Come From?

\(SSG = \sum_i n_i (\bar{x}_i - \bar{x})^2\) has one piece per class (overall mean \(\bar{x} \approx 6.14\)):

Class	\(n_i\)	\(\bar{x}_i\)	\((\bar{x}_i - \bar{x})\)
Lower	41	5.07	-1.07
Working	407	5.75	-0.4
Middle	331	6.76	0.62
Upper	16	6.19	0.04

Compute the contribution to \(SSG\) from the Lower class: \(n_{\text{Lower}} (\bar{x}_{\text{Lower}} - \bar{x})^2\). \(41 \times (-1.07)^2 \approx 41 \times 1.14 \approx 47\) — about \(20\)% of \(SSG = 237\).
Lower has the biggest gap from the overall mean (\(-1.07\)). Which class contributes the most to \(SSG\) — and why? Middle dominates: \(331 \times (0.62)^2 \approx 127\) — about half of \(SSG\). Its gap is smaller than Lower’s, but its weight \(n_i\) is far larger. \(SSG\) rewards groups that are both off-center and large.

S-S-G has one piece per group: each piece is the group’s sample size times the squared distance from that group’s mean to the overall mean of six point one four. Lower is the furthest below the overall mean, at minus one point oh seven. Middle is the furthest above, at plus point six two. Working has a modest gap at minus point four, and Upper is nearly right at the overall mean, with a gap of just plus point oh four. Two questions to work through. First, compute Lower’s contribution to S-S-G. Second, ask which class contributes the most overall — and think about why. Advance when you are ready to see the answers. Lower’s contribution is forty-one times negative one point oh seven squared, which comes out to about forty-seven — about twenty percent of the total S-S-G of two hundred thirty-seven. A meaningful contribution, but not the biggest. Middle dominates: three hundred thirty-one times point six two squared is about one hundred twenty-seven, which is roughly half of S-S-G. Notice the structure. Lower has the biggest gap from the overall mean at minus one point oh seven, larger than Middle’s plus point six two. But Lower only has forty-one people, while Middle has three hundred thirty-one. The weight n sub i matters — a lot. A modest gap with many people behind it outweighs a bigger gap with few people behind it. S-S-G rewards groups that are both off-center and large.

What \(F\) Does — and Doesn’t — Tell Us

\(F \approx 22\) for vocabulary — much bigger than 1. But is that “big” relative to what \(H_0\) would produce by chance?
\(F\) scores the whole group structure at once; it doesn’t say which classes differ.
To turn \(F\) into a decision, we need a reference distribution — what \(F\) values would look like under \(H_0\).

Next time (15b): the F-distribution, conditions, the p-value, and the decision.

Let’s step back for a moment. We’ve built a single number, F, that compares between-group spread to within-group spread on a per-degree-of-freedom scale. For vocabulary, F is about twenty-two — way larger than one. That feels like strong evidence against the null, but we haven’t formalized what “big” means yet. We need a reference distribution — the typical values of F we’d see across many samples if the null were true, when all four population means really were equal. That’s the F-distribution, indexed by two degrees-of-freedom parameters, and we’ll set it up in fifteen B along with the conditions and the p-value. One useful thing F doesn’t tell us, even after we get the p-value — F doesn’t say which class differs from the others. That’s a separate question that topic sixteen tackles.

References

Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
Section 22.3
General Social Survey — vocabulary score (wordsum) by self-identified social class.