15. Inference for Multiple Means
b. ANOVA
Math 115
Yurk
Recap: The Vocabulary ANOVA
Do mean vocabulary scores differ across self-identified social class? In 15a we looked at GSS wordsum scores — a 0–10 vocabulary test — for 795 U.S. adults, split into four classes. In the sample, Middle scored highest and Lower lowest:
| Class | n_i | \bar{x}_i | s_i |
|---|---|---|---|
| Lower | 41 | 5.07 | 2.24 |
| Working | 407 | 5.75 | 1.87 |
| Middle | 331 | 6.76 | 1.89 |
| Upper | 16 | 6.19 | 2.34 |
The four sample means clearly differ — but is that real, or just sampling noise? 15a’s ANOVA compares the spread between the class means to the spread within the classes. Next we lay that computation out in a standard ANOVA table.
The ANOVA Table
15a’s arithmetic — sums of squares, mean squares, and F — is conventionally organized into an ANOVA table, the standard summary every stats package (Jamovi included) prints. Read each row left to right: a mean square is that row’s SS over its df, and F is the ratio of the two.
| Source | df | SS | MS | F |
|---|---|---|---|---|
| Between groups (class) | 3 | 236.6 | 78.9 | 21.73 |
| Within groups (residual) | 791 | 2869.8 | 3.63 | |
| Total | 794 | 3106.4 |
- MSG = SSG / df_1 = 236.6 / 3 \approx 78.9
- MSE = SSE / df_2 = 2869.8 / 791 \approx 3.63
- F = MSG / MSE = 78.9 / 3.63 \approx 21.73
- SST = SSG + SSE = 236.6 + 2869.8 = 3106.4 (df_T = n-1 = 794)
Is F \approx 21.73 big enough to reject H_0? We need a reference distribution — the F values we’d expect if H_0 were true.
The F-Distribution
Under H_0, F follows an F-distribution with two df — df_1 (numerator) and df_2 (denominator); conditions next slide.
- Always non-negative — F is a ratio of two non-negative mean squares.
- Right-skewed — bell-shaped at larger df, but still skewed.
- Around 1 under H_0 — MSG and MSE estimate the same variance, so F sits near 1 (peak below 1 from the skew).
- Bigger F → farther into the right tail → stronger evidence against H_0.
Degrees of Freedom for ANOVA
For k groups and total sample size n:
df_1 = k - 1 \qquad df_2 = n - k
- df_1 counts the number of between-group comparisons that are free once we fix the overall mean.
- df_2 counts the within-group observations free once we fix each group’s mean.
Try it: what are df_1 and df_2 for the vocabulary ANOVA? Recall k = 4, n = 795.
df_1 = 4 - 1 = 3 and df_2 = 795 - 4 = 791. So we compare F = 21.73 to an F-distribution with df_1 = 3 and df_2 = 791.
Conditions for ANOVA
Three conditions for the F-distribution to be a valid reference:
| Condition | Rule |
|---|---|
| Independence | Independent within and between groups — random samples or random assignment (else justify from context). |
| Normality | Each group roughly normal, or n_i \geq 30 (CLT covers the means). |
| Equal variance | Group SDs similar — s_{\max} / s_{\min} < 2. |
Try it: do the conditions hold for the vocabulary ANOVA? Recall the smallest group is Upper (n = 16); group SDs range from 1.87 to 2.34.
Independence: GSS random sample ✓. Normality: n_i \geq 30 for Lower, Working, Middle ✓; Upper’s n = 16 is small, but scores are bounded 0–10 with no big outliers — F approximation still OK (mild caution). Equal variance: 2.34 / 1.87 \approx 1.25 < 2 ✓.
The P-Value for Vocabulary
- P-value = area to the right of F = 21.73 under the F-curve with df_1 = 3, df_2 = 791.
- Any difference among the \mu_i only pushes F up, so — as with chi-square — use the right tail only (no doubling).
- From Jamovi (or
pf): p-value < 0.001 — essentially zero, far below 0.05.
Conclusion for Vocabulary
Results: F = 21.73, df_1 = 3, df_2 = 791, p-value < 0.001 → reject H_0.
In context: mean vocabulary scores differ across self-identified social classes in the U.S. adult population — at least one class mean is different from the others.
Scope of inference.
- Generalization: GSS uses a random sample of U.S. adults, so the conclusion generalizes to the U.S. adult population.
- Causation: observational — social class was just measured, not assigned — so no causal claim. We’re describing a real population-level association between class identification and vocabulary score.
Heads up: ANOVA tells us at least one class differs from the others — it doesn’t say which classes differ from each other or in which direction. Topic 16a picks up that thread with pairwise follow-ups (and a correction for doing many comparisons).
Plant Growth — Setup
Does a treatment change plant yield? A controlled experiment grew 30 plants under three conditions — control (ctrl), treatment 1 (trt1), treatment 2 (trt2) — and measured dried weight (Dobson, 1983).
| Group | n_i | \bar{x}_i (g) | s_i (g) |
|---|---|---|---|
| ctrl | 10 | 5.03 | 0.58 |
| trt1 | 10 | 4.66 | 0.79 |
| trt2 | 10 | 5.53 | 0.44 |
- State H_0 / H_A, then check the three ANOVA conditions. H_0: \mu_{\text{ctrl}} = \mu_{\text{trt1}} = \mu_{\text{trt2}}; H_A: at least one mean differs. Independence: controlled experiment with random assignment ✓. Normality: n_i = 10 in each group — small, so each group should look roughly normal; with only 10 per group, the F approximation is more sensitive to skew or outliers. Proceed with caution. Equal variance: 0.79/0.44 \approx 1.8 < 2 ✓.
Plant Growth — Complete the ANOVA Table
Software reports the two sums of squares: SSG = 3.77, SSE = 10.49. With k = 3 groups and n = 30 plants, fill in the rest of the table (each MS = SS/df; F = MSG/MSE):
| Source | df | SS | MS | F |
|---|---|---|---|---|
| Between groups (group) | ?2 | 3.77 | ?1.88 | ?4.85 |
| Within groups (residual) | ?27 | 10.49 | ?0.39 | |
| Total | ?29 | ?14.26 |
- df_1 = k-1 = 2, df_2 = n-k = 27, df_T = n-1 = 29
- SST = SSG + SSE = 3.77 + 10.49 = 14.26
- MSG = SSG/df_1 = 3.77/2 \approx 1.88; MSE = SSE/df_2 = 10.49/27 \approx 0.39
- F = MSG/MSE = 1.88/0.39 \approx 4.85
Plant Growth — P-Value
Recall: from your table, F \approx 4.85 on df_1 = 2, df_2 = 27. The shaded area to the right of F is the p-value, out of a total area of 1.
Eyeball the shaded tail: is the p-value below \alpha = 0.05? Conclude, with scope.
The shaded tail is a tiny slice of the area → p-value < 0.05 (\approx 0.02), so reject H_0 (moderate). Conclusion: the group means aren’t all equal — at least one condition’s mean yield differs from the others. Scope: random assignment → the difference is causal, for these plants; no random sample → limited generalization.
What ANOVA Does — and Doesn’t — Tell Us
ANOVA tells us: whether at least one group mean differs from the others — one yes/no question about the whole collection of group means.
ANOVA doesn’t tell us:
- Which specific groups differ from each other.
- The direction or magnitude of differences.
- Which pairs are “significantly different” individually.
Topic 16a (next class — same Friday) picks up where ANOVA leaves off: pairwise comparisons of means, with a correction for the inflated error from running many comparisons — the problem from the start of 15a.
Connection to Two-Means T-Test
ANOVA is the natural generalization of the two-sample (independent-means) t-test from topic 12:
| Two Means (Topic 12) | Multiple Means (Topic 15) | |
|---|---|---|
| Number of groups | exactly 2 | 3 or more |
| Statistic | \bar{x}_1 - \bar{x}_2 | F = MSG / MSE |
| Reference distribution | t with df = n - 2 | F with df_1 = k-1, df_2 = n-k |
| What we test | \mu_1 = \mu_2 | \mu_1 = \mu_2 = \cdots = \mu_k |
| Tail(s) used for p-value | One- or two-sided, depending on H_A | Always right tail |
For k = 2 groups the two methods are equivalent — the two-sided t-test p-value matches the ANOVA p-value exactly, and F = T^2.
References
- Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
- Section 22.3
- General Social Survey — vocabulary score by self-identified social class.
- Dobson, A. J. (1983). An Introduction to Statistical Modelling. London: Chapman and Hall — plant growth example.