15. Inference for Multiple Means

b. ANOVA
Math 115

Yurk

Recap: The Vocabulary ANOVA

Do mean vocabulary scores differ across self-identified social class? In 15a we looked at GSS wordsum scores — a 0–10 vocabulary test — for 795 U.S. adults, split into four classes. In the sample, Middle scored highest and Lower lowest:

Class	\(n_i\)	\(\bar{x}_i\)	\(s_i\)
Lower	41	5.07	2.24
Working	407	5.75	1.87
Middle	331	6.76	1.89
Upper	16	6.19	2.34

The four sample means clearly differ — but is that real, or just sampling noise? 15a’s ANOVA compares the spread between the class means to the spread within the classes. Next we lay that computation out in a standard ANOVA table.

Let’s pick the vocabulary example back up from fifteen A. The question is whether mean vocabulary scores differ across self-identified social class. We have wordsum scores — a zero to ten vocabulary test — for seven hundred ninety-five U.S. adults from the General Social Survey, split into four classes: lower, working, middle, and upper. In this sample the four class means clearly differ — middle scores highest, lower lowest, with working and upper in between. The real question is whether that gap reflects a true difference across the whole population, or just the sampling noise we’d expect even if every class had the same mean. To answer it, ANOVA compares the spread between the class means to the spread within the classes. Next we lay that computation out in the standard ANOVA table.

The ANOVA Table

15a’s arithmetic — sums of squares, mean squares, and \(F\) — is conventionally organized into an ANOVA table, the standard summary every stats package (Jamovi included) prints. Read each row left to right: a mean square is that row’s SS over its df, and \(F\) is the ratio of the two.

Source	df	SS	MS	F
Between groups (class)	3	236.6	78.9	21.73
Within groups (residual)	791	2869.8	3.63
Total	794	3106.4

\(MSG = SSG / df_1 = 236.6 / 3 \approx 78.9\)

\(MSE = SSE / df_2 = 2869.8 / 791 \approx 3.63\)

\(F = MSG / MSE = 78.9 / 3.63 \approx 21.73\)

\(SST = SSG + SSE = 236.6 + 2869.8 = 3106.4\) (\(df_T = n-1 = 794\))

Is \(F \approx 21.73\) big enough to reject \(H_0\)? We need a reference distribution — the \(F\) values we’d expect if \(H_0\) were true.

Everything we computed in fifteen A — the sums of squares, the mean squares, and the F-statistic — is conventionally laid out in a single summary called the ANOVA table. Every statistics package, including Jamovi, prints this same table. Each row is a source of variation — one slice of the overall spread in the scores. The between-group source, labeled by our grouping variable, social class, measures how much the class means differ from one another; the within-group source, often called residual, measures the spread of individuals inside each class; and the total row is the two combined. The columns are degrees of freedom, sum of squares, mean square, and F. Read each row from left to right: a mean square is simply that row’s sum of squares divided by its degrees of freedom. For the between-group row, two hundred thirty-six point six divided by three gives a mean square of about seventy-eight point nine. For the within-group row, two thousand eight hundred seventy divided by seven hundred ninety-one gives a mean square of about three point six three. The F-statistic is the ratio of those two mean squares — seventy-eight point nine over three point six three — which lands at about twenty-one point seven, exactly the F we built in fifteen A. The total row captures the whole idea of ANOVA: the total variability, with n minus one degrees of freedom, splits cleanly into a between-group part and a within-group part. So the computation is settled. What remains is whether an F of about twenty-one point seven is big enough to reject the equal-means null. For that we need a reference distribution — the values of F we’d expect if the null were true, when all the population means are equal. That benchmark is the F-distribution, which we set up next.

The F-Distribution

Under \(H_0\), \(F\) follows an F-distribution with two df — \(df_1\) (numerator) and \(df_2\) (denominator); conditions next slide.

Always non-negative — \(F\) is a ratio of two non-negative mean squares.
Right-skewed — bell-shaped at larger \(df\), but still skewed.
Around 1 under \(H_0\) — \(MSG\) and \(MSE\) estimate the same variance, so \(F\) sits near 1 (peak below 1 from the skew).
Bigger \(F\) → farther into the right tail → stronger evidence against \(H_0\).

The F-distribution is the reference shape that F follows when the null is true, given the conditions we’ll see next. Unlike the t or chi-square distributions, the F-distribution has two shape parameters — degrees of freedom for the numerator and degrees of freedom for the denominator — because F itself is a ratio of two mean squares. The plot shows three curves with different df pairs: a red curve with df equal to two and ten, a blue curve with df equal to three and thirty, and a green curve with df equal to ten and one hundred. There are four properties to notice. First, every curve stays on the positive side of zero — F is a ratio of two non-negative numbers. Second, every curve is right-skewed, with a long tail to the right. Third, under the null, F tends to sit around one — because MSG and MSE both estimate the same variance, so their ratio is typically close to one. The visual peak of each curve actually sits below one, since the distribution is right-skewed. Fourth, the same right-tail logic from chi-square carries over: bigger F means farther into the right tail, which means stronger evidence against the null.

Degrees of Freedom for ANOVA

For \(k\) groups and total sample size \(n\):

\[df_1 = k - 1 \qquad df_2 = n - k\]

\(df_1\) counts the number of between-group comparisons that are free once we fix the overall mean.
\(df_2\) counts the within-group observations free once we fix each group’s mean.

Try it: what are \(df_1\) and \(df_2\) for the vocabulary ANOVA? Recall \(k = 4\), \(n = 795\).

\(df_1 = 4 - 1 = 3\) and \(df_2 = 795 - 4 = 791\). So we compare \(F = 21.73\) to an F-distribution with \(df_1 = 3\) and \(df_2 = 791\).

The F-distribution depends on two degrees of freedom because the test statistic is a ratio. The numerator degrees of freedom — df one — is k minus one, where k is the number of groups. The denominator degrees of freedom — df two — is n minus k, where n is the total sample size. Try both for the vocabulary ANOVA before advancing. K equals four social classes, n equals seven hundred ninety-five respondents. Df one equals four minus one, which is three. Df two equals seven hundred ninety-five minus four, which is seven hundred ninety-one. So we compare our observed F of twenty-one point seven to an F-distribution with df one equal to three and df two equal to seven hundred ninety-one. To spoil the ending, twenty-one point seven is way out in the right tail of that curve, and we’ll see the p-value picture shortly.

Conditions for ANOVA

Three conditions for the F-distribution to be a valid reference:

Condition	Rule
Independence	Independent within and between groups — random samples or random assignment (else justify from context).
Normality	Each group roughly normal, or \(n_i \geq 30\) (CLT covers the means).
Equal variance	Group SDs similar — \(s_{\max} / s_{\min} < 2\).

Try it: do the conditions hold for the vocabulary ANOVA? Recall the smallest group is Upper (\(n = 16\)); group SDs range from 1.87 to 2.34.

Independence: GSS random sample ✓. Normality: \(n_i \geq 30\) for Lower, Working, Middle ✓; Upper’s \(n = 16\) is small, but scores are bounded \(0\)–\(10\) with no big outliers — F approximation still OK (mild caution). Equal variance: \(2.34 / 1.87 \approx 1.25 < 2\) ✓.

ANOVA has three conditions for the F-distribution to be a valid reference. First, independence: observations need to be independent both within and between groups. The standard paths are random samples or random assignment; otherwise we justify independence from study context. Second, normality: the response should be roughly normal within each group, or each group should have at least about thirty observations so the Central Limit Theorem can cover the group means. Third, equal variance: the group standard deviations should be similar — the practical rule of thumb is that the ratio of the largest to the smallest should be less than two. Try the three conditions for vocabulary before advancing. For vocabulary, independence holds because the General Social Survey draws a random sample of U.S. adults. Normality is mostly fine — Lower, Working, and Middle all have well over thirty respondents, so the Central Limit Theorem covers them. The Upper class has only sixteen people — small — but the wordsum score is bounded zero through ten with no extreme outliers, so the F approximation should still be reasonable. Proceed with mild caution. For equal variance, the group standard deviations range from one point eight seven to two point three four, a ratio of about one point two five, well under two. So all three conditions are met, with that small caution about the Upper group’s sample size.

The P-Value for Vocabulary

P-value = area to the right of \(F = 21.73\) under the F-curve with \(df_1 = 3\), \(df_2 = 791\).
Any difference among the \(\mu_i\) only pushes \(F\) up, so — as with chi-square — use the right tail only (no doubling).
From Jamovi (or pf): p-value \(< 0.001\) — essentially zero, far below 0.05.

Conclusion for Vocabulary

Results: \(F = 21.73\), \(df_1 = 3\), \(df_2 = 791\), p-value \(< 0.001\) → reject \(H_0\).

In context: mean vocabulary scores differ across self-identified social classes in the U.S. adult population — at least one class mean is different from the others.

Scope of inference.

Generalization: GSS uses a random sample of U.S. adults, so the conclusion generalizes to the U.S. adult population.
Causation: observational — social class was just measured, not assigned — so no causal claim. We’re describing a real population-level association between class identification and vocabulary score.

Heads up: ANOVA tells us at least one class differs from the others — it doesn’t say which classes differ from each other or in which direction. Topic 16a picks up that thread with pairwise follow-ups (and a correction for doing many comparisons).

Let’s put the result together. F equals about twenty-one point seven on three and seven hundred ninety-one degrees of freedom. The p-value is essentially zero — well below zero point zero five — so we reject the null. In plain terms: mean vocabulary scores really do differ across self-identified social classes in the U.S. adult population. At least one of those four class means is different from the others. Now consider the scope of inference. The General Social Survey draws a random sample of U.S. adults, so the conclusion generalizes to the U.S. adult population. But social class was just measured, not assigned — this is observational data — so there’s no causal claim. We’re describing a real population-level association between class identification and vocabulary score, not saying class “causes” vocabulary or the reverse. There’s one important limitation. ANOVA only answers whether at least one class differs from the others — it doesn’t tell us which classes, or in which direction. Looking at the means, Middle is the highest and Lower is the lowest, but to formally compare individual pairs we need a second tool. Topic sixteen A — coming up right after this one — picks up that thread.

Plant Growth — Setup

Does a treatment change plant yield? A controlled experiment grew 30 plants under three conditions — control (ctrl), treatment 1 (trt1), treatment 2 (trt2) — and measured dried weight (Dobson, 1983).

Group	\(n_i\)	\(\bar{x}_i\) (g)	\(s_i\) (g)
ctrl	10	5.03	0.58
trt1	10	4.66	0.79
trt2	10	5.53	0.44

State \(H_0\) / \(H_A\), then check the three ANOVA conditions. \(H_0: \mu_{\text{ctrl}} = \mu_{\text{trt1}} = \mu_{\text{trt2}}\); \(H_A:\) at least one mean differs. Independence: controlled experiment with random assignment ✓. Normality: \(n_i = 10\) in each group — small, so each group should look roughly normal; with only 10 per group, the F approximation is more sensitive to skew or outliers. Proceed with caution. Equal variance: \(0.79/0.44 \approx 1.8 < 2\) ✓.

Now it’s your turn, on a second example with a different feel. A controlled experiment grew thirty plants in total — ten under standard control conditions, ten under treatment one, and ten under treatment two — and measured the dried weight of each plant. The original study is by Dobson, an introduction to statistical modelling textbook from nineteen eighty-three. Group means are about five point oh three grams for control, four point six six for treatment one, and five point five three for treatment two — so trt one looks below the control mean and trt two looks above. State the hypotheses and check the three ANOVA conditions before advancing. Hypotheses follow the ANOVA pattern. The null says the three population means — control, treatment one, and treatment two — are all equal, and the alternative says at least one of them differs. Now check the conditions. Independence holds because this is a controlled experiment with random assignment to the three conditions. Normality is more uncertain — each group has only ten plants, well below thirty, so each group should look roughly normal on its own. With small group sizes the F approximation is more sensitive to skew or outliers than it would be at large n. Proceed with caution. Equal variance: standard deviations range from about point four four to point seven nine — a ratio of about one point eight — borderline but under two. So the conditions hold with a caution about normality.

Plant Growth — Complete the ANOVA Table

Software reports the two sums of squares: \(SSG = 3.77\), \(SSE = 10.49\). With \(k = 3\) groups and \(n = 30\) plants, fill in the rest of the table (each \(MS = SS/df\); \(F = MSG/MSE\)):

Source	df	SS	MS	F
Between groups (group)	?2	3.77	?1.88	?4.85
Within groups (residual)	?27	10.49	?0.39
Total	?29	?14.26

\(df_1 = k-1 = 2\), \(df_2 = n-k = 27\), \(df_T = n-1 = 29\)

\(SST = SSG + SSE = 3.77 + 10.49 = 14.26\)

\(MSG = SSG/df_1 = 3.77/2 \approx 1.88\); \(MSE = SSE/df_2 = 10.49/27 \approx 0.39\)

\(F = MSG/MSE = 1.88/0.39 \approx 4.85\)

Here’s a your-turn that rebuilds the whole ANOVA table from just two numbers. Software reports the between-group sum of squares as three point seven seven and the within-group sum of squares as ten point four nine. You also know there are three groups and thirty plants in total. From those, fill in every other cell of the table — the three degrees of freedom, the total sum of squares, both mean squares, and the F-statistic. Pause here and work through it before advancing. Start with degrees of freedom. The numerator degrees of freedom is k minus one — three minus one is two. The denominator is n minus k — thirty minus three is twenty-seven. And the total is n minus one, which is twenty-nine. Next, the total sum of squares. The between and within parts add to the whole, so three point seven seven plus ten point four nine is fourteen point two six. Now the mean squares — each one is its own sum of squares divided by its own degrees of freedom. The between-group mean square is three point seven seven over two, about one point eight eight. The within-group mean square is ten point four nine over twenty-seven, about zero point three nine. Finally, the F-statistic is the ratio of the two mean squares, which comes out to about four point eight five. That’s the F we carry into the next slide to find the p-value.

Plant Growth — P-Value

Recall: from your table, \(F \approx 4.85\) on \(df_1 = 2\), \(df_2 = 27\). The shaded area to the right of \(F\) is the p-value, out of a total area of 1.

Eyeball the shaded tail: is the p-value below \(\alpha = 0.05\)? Conclude, with scope.

The shaded tail is a tiny slice of the area → p-value < 0.05 (\(\approx 0.02\)), so reject \(H_0\) (moderate). Conclusion: the group means aren’t all equal — at least one condition’s mean yield differs from the others. Scope: random assignment → the difference is causal, for these plants; no random sample → limited generalization.

Time to read the result off the F-curve, using the F you just computed, about four point eight five. The shaded region to the right of that value is the p-value, and the whole area under the curve is one. So just eyeball it: about what fraction of the total area is shaded? Decide whether the p-value is below zero point zero five, then reach a conclusion with scope before advancing. The shaded tail is only a tiny sliver of the total area of one — clearly smaller than zero point zero five — so the p-value is below zero point zero five, about zero point zero two. We reject the null at alpha equals zero point zero five. The evidence is moderate — not as overwhelming as the vocabulary case, where the p-value was below zero point zero zero one. Conclusion: not all three group means are equal — at least one condition differs in mean yield from the others, but the F-test alone doesn’t tell us which one. Turn now to the scope of inference. This was a controlled experiment with random assignment, so a causal claim is justified — at least one condition really does differ in mean yield from the others, causally, for plants like these. But the plants themselves were not a random sample of any wider population, so generalization beyond this experimental setting is limited. Notice the contrast with the vocabulary example: there, random sampling let us generalize but observational data ruled out causation. Here, random assignment lets us claim causation but the lack of random sampling limits generalization. Same toolkit, different scope conclusions, because of the study design.

What ANOVA Does — and Doesn’t — Tell Us

ANOVA tells us: whether at least one group mean differs from the others — one yes/no question about the whole collection of group means.

ANOVA doesn’t tell us:

Which specific groups differ from each other.
The direction or magnitude of differences.
Which pairs are “significantly different” individually.

Topic 16a (next class — same Friday) picks up where ANOVA leaves off: pairwise comparisons of means, with a correction for the inflated error from running many comparisons — the problem from the start of 15a.

Worth pausing on what ANOVA does and does not tell us — because the gap is what motivates topic sixteen. What ANOVA tells us is whether at least one group mean differs from the others — a single yes-or-no question about the whole collection of group means. What ANOVA doesn’t tell us is which specific groups differ, the direction or size of the differences, or which individual pairs are “significantly different” from each other. For vocabulary, the F-test told us some class is different — but a careful analyst still wants to know whether it’s Lower versus Middle, Middle versus Working, all pairs, or just one pair. Topic sixteen A — coming up right after this lecture — picks up exactly that question. It does pairwise comparisons of group means, but with a correction for the way the overall error balloons when you run many comparisons — the very problem we raised at the start of fifteen A.

Connection to Two-Means T-Test

ANOVA is the natural generalization of the two-sample (independent-means) t-test from topic 12:

	Two Means (Topic 12)	Multiple Means (Topic 15)
Number of groups	exactly 2	3 or more
Statistic	\(\bar{x}_1 - \bar{x}_2\)	\(F = MSG / MSE\)
Reference distribution	\(t\) with \(df = n - 2\)	F with \(df_1 = k-1\), \(df_2 = n-k\)
What we test	\(\mu_1 = \mu_2\)	\(\mu_1 = \mu_2 = \cdots = \mu_k\)
Tail(s) used for p-value	One- or two-sided, depending on \(H_A\)	Always right tail

For \(k = 2\) groups the two methods are equivalent — the two-sided t-test p-value matches the ANOVA p-value exactly, and \(F = T^2\).

ANOVA is the natural generalization of the independent-samples t-test from topic twelve. Topic twelve compared exactly two group means using the difference of two sample means, with a t reference distribution. Today the number of groups can be three or more, the statistic is F equal to M-S-G over M-S-E, and the reference is the F-distribution with two df parameters. The hypotheses generalize the same way: in topic twelve, null was mu one equals mu two; today the null is all k mu’s equal. The biggest change is the tail use — topic twelve let us choose one-sided or two-sided depending on the question, while ANOVA always uses the right tail of F, for the reason we walked through earlier. Nice confirmation of the connection: when there are exactly two groups, the F-test gives exactly the same p-value as a two-sided independent-samples t-test, and the F statistic equals T squared. Nothing is lost by using ANOVA — we just gain the ability to compare three or more groups.

References

Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
Section 22.3
General Social Survey — vocabulary score by self-identified social class.
Dobson, A. J. (1983). An Introduction to Statistical Modelling. London: Chapman and Hall — plant growth example.