15. Inference for Multiple Means

a. The F-Statistic
Math 115

Yurk

Where We’ve Been — Comparing Means

Topic Comparison Statistic Example
11 One mean \bar{x} Cherry Blossom
12 Two independent means \bar{x}_1 - \bar{x}_2 Birth weights
13 Two paired means \bar{x}_d Textbook prices
15 (today) Three or more means F Vocabulary by class

Topic 15 extends topic 12. Topic 12 compared two means with a single difference \bar{x}_1 - \bar{x}_2. With three or more means, one difference can’t summarize the picture — so we need a new statistic: F.

Vocabulary Scores by Social Class

Do vocabulary test scores differ by self-identified social class? General Social Survey — random sample of 795 U.S. adults.

  • Response: 0–10 score on a 10-question vocabulary test (numerical)
  • Explanatory: self-identified social class — Lower, Working, Middle, Upper (k = 4 groups)

Hypotheses.

  • H_0: \mu_{\text{Lower}} = \mu_{\text{Working}} = \mu_{\text{Middle}} = \mu_{\text{Upper}}
  • H_A: at least one class has a different mean vocabulary score

EDA: Vocabulary Scores

Class n \bar{x}_i s_i
Lower 41 5.07 2.24
Working 407 5.75 1.87
Middle 331 6.76 1.89
Upper 16 6.19 2.34

Boxes show each class’s median and spread; ◆ = group mean (what ANOVA compares); dashed line = overall mean \approx 6.14. Middle sits highest, Lower lowest; Upper is noisy with only 16 people. Real differences in the means, or sampling noise?

Why Not Just Run Many T-Tests?

The obvious idea: compare the classes two at a time with the t-tests we already know.

Comparing all four classes pairwise takes 6 separate t-tests — Lower–Working, Lower–Middle, and so on, every class against every other.

Each t-test carries its own 5% chance of a Type I error (a false positive) when its null is true.

That’s fine for one test. But we’re about to run six — what happens to the overall risk?

When Errors Add Up

If all six nulls were true, the chance of at least one false rejection is:

1 - (0.95)^6 \approx 0.265

That’s about 26.5% — over five times the 5% we intended.

The overall error balloons. Running many separate tests trades a controlled per-test error for a much larger overall chance of a false alarm.

We need a single overall test that asks one yes/no question with one \alpha.

One Overall Test: ANOVA

ANOVA (Analysis of Variance) tests whether any of k group means differ:

  • H_0: \mu_1 = \mu_2 = \cdots = \mu_k (all population means equal)
  • H_A: at least one \mu_i differs from the others

The idea: weigh the variability between the group means against the variability within the groups — developed next.

Yes/no, not which. ANOVA tells us whether at least one group differs — it doesn’t say which groups. Pinpointing the specific pair(s) is the job of topic 16 (multiple comparisons).

Between vs. Within

Two sources of variability in the data:

  • Between groups — how spread out the group means are around the overall mean.
  • Within groups — how spread out individual observations are around their own group’s mean.

The logic of ANOVA. If H_0 is true, the group means should sit as close to the overall mean as random sampling within each group would predict — the between-group spread carries no signal beyond that sampling noise.

If between is much larger than within → evidence that real group differences are pulling the means apart.

Same Means, Different Evidence

Two made-up datasets, three groups each, differing only in within-group scatter. Judge from the raw data alone.

  1. In which panel is there stronger evidence that the three population means truly differ — judging from the data alone? Same means in both panels — 4, 5, 6 (diamonds)! Between-group spread is identical; only within-group scatter differs. Left: tight scatter → means stand clear → strong evidence. Right: means buried in scatter → weak. This contrast is exactly what ANOVA measures.

Two Sums of Squares

Back to the vocabulary data — each spread as a number (from software):

Between groups (means vs overall):

SSG = \sum n_i (\bar{x}_i - \bar{x})^2 \approx 237

Within groups (values vs group mean):

SSE = \sum \sum (x_{ij} - \bar{x}_i)^2 \approx 2870

A raw sum of squares isn’t a fair yardstick on its own — it grows as more data piles in, so SSE and SSG can’t be compared head-to-head. Next: turn each into an average (a mean square).

Mean Squares: A Fair Comparison

Divide each sum of squares by its own degrees of freedom to get an average — a mean square:

Between (MSG) — df_1 = k - 1 = 3

MSG = \frac{SSG}{df_1} = \frac{237}{3} = \; ?

Within (MSE) — df_2 = n - k = 791

MSE = \frac{SSE}{df_2} = \frac{2870}{791} = \; ?

Try it: compute both mean squares, then compare them. MSG = 79 and MSE \approx 3.6. Both estimate the same underlying noise — how much individual scores vary within a group. MSE measures it directly; under H_0, MSG estimates that same noise, so the two land close. Here MSG \gg MSE — the class means vary more than noise alone, a hint they really differ.

The F-Statistic

The test statistic is the ratio of the two mean squares:

F = \frac{MSG}{MSE}

  • F \approx 1: between-group variability looks like within-group — consistent with H_0.
  • F \gg 1: the group means are spread out more than noise alone → evidence they differ.

Try it: compute F for vocabulary (MSG \approx 79, MSE \approx 3.6). F = \dfrac{79}{3.6} \approx 22 — between-group variability is about 22× the within-group variability. Far above 1: suggestive evidence against H_0.

Where Does the F Come From?

SSG = \sum_i n_i (\bar{x}_i - \bar{x})^2 has one piece per class (overall mean \bar{x} \approx 6.14):

Class n_i \bar{x}_i (\bar{x}_i - \bar{x})
Lower 41 5.07 -1.07
Working 407 5.75 -0.4
Middle 331 6.76 0.62
Upper 16 6.19 0.04

  1. Compute the contribution to SSG from the Lower class: n_{\text{Lower}} (\bar{x}_{\text{Lower}} - \bar{x})^2. 41 \times (-1.07)^2 \approx 41 \times 1.14 \approx 47 — about 20% of SSG = 237.

  2. Lower has the biggest gap from the overall mean (-1.07). Which class contributes the most to SSG — and why? Middle dominates: 331 \times (0.62)^2 \approx 127 — about half of SSG. Its gap is smaller than Lower’s, but its weight n_i is far larger. SSG rewards groups that are both off-center and large.

What F Does — and Doesn’t — Tell Us

  • F \approx 22 for vocabulary — much bigger than 1. But is that “big” relative to what H_0 would produce by chance?
  • F scores the whole group structure at once; it doesn’t say which classes differ.
  • To turn F into a decision, we need a reference distribution — what F values would look like under H_0.

Next time (15b): the F-distribution, conditions, the p-value, and the decision.

References

  • Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
  • Section 22.3
  • General Social Survey — vocabulary score (wordsum) by self-identified social class.
Course Home