14. Inference for Contingency Tables

a. The Chi-Square Statistic
Math 115

Yurk

Where We’ve Been — and Where We’re Going

We’ve been building inference for categorical data in stages:

Topic	What we tested	Statistic	Example
6	One categorical variable	\(\hat{p}\)	Payday loans
10	Two groups × a binary outcome (2×2)	\(\hat{p}_1 - \hat{p}_2\)	CPR survival
14 (today)	Two categorical variables (any r×c)	\(X^2\)	Party × spending

Today continues the thread from topics 6 and 10. Once a variable has more than two categories, the table grows past 2×2 and a single difference in proportions can’t summarize it — so we score the whole table with one number, \(X^2\).

Before we start today’s topic, let’s place it in a thread we’ve already been pulling on. Back in topic six, we did inference on a single categorical variable — one proportion, p hat, like the fraction of borrowers who take out a payday loan. In topic ten, we compared two groups on a binary outcome — a two by two table, summarized by the difference of two proportions, as in the CPR survival example. Topics eleven through thirteen took a detour into numerical data and means; today we pick the categorical thread back up. Topic fourteen is the next step in that same story. Now both variables can have any number of categories — three parties, three opinion levels, or more. Once the table grows past two by two, a single difference in proportions can’t summarize it anymore. So we build one number that scores the entire table at once. That number is the chi-square statistic, written capital X squared, and today we’ll learn to compute it.

Party × Military Spending

Is opinion on military spending associated with political party? General Social Survey, 2016 — random sample of 149 U.S. adults.

Explanatory: party (Dem / Ind / Rep) — 3 categories
Response: “Do we spend too little / about right / too much on the military?” — 3 categories

“No relationship” between two categorical variables is called independence:

\(H_0\): party and military-spending opinion are independent (no association)
\(H_A\): party and military-spending opinion are associated (not independent)

Today’s running example comes from the General Social Survey, a long-running, broadly representative national survey of U.S. adults. Our sample uses one hundred forty-nine respondents from the twenty sixteen wave, each paired with their answer to a question about military spending. The explanatory variable is party, with three categories: Democrat, Independent, Republican. The response is opinion on military spending, with three categories: too little, about right, too much. For two categorical variables, “no relationship” has a name: independence. The null hypothesis is that party and military-spending opinion are independent — knowing someone’s party tells you nothing about which opinion they’re more likely to give. The alternative is that there’s an association — the opinion mix differs across the three parties.

The Observed Table

Party (3 rows) × opinion (3 columns) — a 3×3 table, bigger than topic 10’s 2×2 in both directions.

Party	Too Little	About Right	Too Much	Row total
Dem	17	14	12	43
Ind	20	28	24	72
Rep	24	8	2	34
Col total	61	50	38	149

Looking row by row, Republicans look noticeably more likely to say “Too Little” and noticeably less likely to say “Too Much” than Democrats or Independents — in this sample. Real association, or sampling noise?

To answer that, we need to compare these counts to what we’d expect if party and opinion were truly independent.

Here’s the full observed table — three parties as rows, three opinion levels as columns, so it’s a three by three, bigger than the two by two tables of topic ten in both directions. The row totals on the right give the sample sizes: forty-three Democrats, seventy-two Independents, thirty-four Republicans. The column totals across the bottom give the overall opinion split: sixty-one said too little, fifty said about right, thirty-eight said too much. And the grand total, in the bottom corner, is one hundred forty-nine. Now compare the rows. Republicans really stand out — twenty-four of their thirty-four, about seventy percent, said too little, and only two said too much. Democrats and Independents spread their opinions more evenly across the three levels, but in different directions: Democrats slightly favor too little, while Independents tilt the other way toward too much. In this sample, Republicans are noticeably more pro-military-spending than the other two groups. But sample sizes are modest, so we have to ask the usual question: is this a real association in the U.S. adult population, or could it just be sampling noise? To answer that, we need to compare what we observed to what we’d expect if party and opinion were truly independent.

Expected Count for One Cell

Each cell has an expected count — the value we’d see if \(H_0\) (independence) held:

Party	Too Little	About Right	Too Much	Row total
Dem	17	14	12	43
Ind	20	28	24	72
Rep	24	8	2	34
Col total	61	50	38	149

Focus on one cell — Democrats who said “Too Little” (observed 17). If \(H_0\) held, what count would we expect here?

Overall, \(\dfrac{61}{149} \approx 41\%\) said “Too Little.” Under \(H_0\), that same share appears in every row.

So expect that share of the 43 Democrats: \(E_{\text{Dem, TL}} = \underbrace{43}_{\text{Democrats}} \times \underbrace{\tfrac{61}{149}}_{H_0\text{ proportion}} \approx 17.6\).

Expected 17.6 vs observed 17 — almost exactly what independence predicts.

Every cell in the table has an expected count — the value we would see if the null, independence, were true. Let’s work out one of them. Focus on a single cell: Democrats who said “too little.” In our data, the observed count there is seventeen. The question we’re answering is this — if party and opinion were truly independent, what count would we expect in that cell? Let’s build it. Start with the whole sample. The fraction saying “too little” is sixty-one over one hundred forty-nine — about forty-one percent. Under the null, that same fraction should appear in every row. So take the forty-three Democrats and expect that same fraction of them. The expected count is the number of Democrats times the proportion — forty-three, times sixty-one over one hundred forty-nine — which comes to about seventeen and six tenths. So we expected about seventeen point six, and the observed count was seventeen — almost exactly what independence predicts. The bigger departures will come from other cells.

The Expected-Count Formula

We just built the Democrat cell as row total \(\times\) \(H_0\) proportion:

\[E_{\text{Dem, TL}} = \underbrace{43}_{\text{row total}} \times \underbrace{\tfrac{61}{149}}_{H_0\text{ proportion}} \approx 17.6\]

Every cell works the same way — just swap in that cell’s own row total and column:

\[\text{Expected} = (\text{row total}) \times \underbrace{\frac{\text{column total}}{\text{grand total}}}_{H_0\text{ proportion}} = \frac{(\text{row total}) \times (\text{column total})}{\text{grand total}}\]

Compute One Expected Count

Republicans saying “Too Little” — if \(H_0\) (independence) held, what count would we expect? Read the totals you need off the table, then use \(\text{Expected} = \dfrac{\text{row} \times \text{column}}{\text{grand total}}\).

Party	Too Little	About Right	Too Much	Row total
Dem	17	14	12	43
Ind	20	28	24	72
Rep	24	8	2	34
Col total	61	50	38	149

\(E_{\text{Rep, TL}} = \dfrac{34 \times 61}{149} = \dfrac{2074}{149} \approx 13.92\). Observed was 24 — far more than \(H_0\) predicts. This cell is where the association starts to show.

Observed vs Expected

Observed (Expected) for every cell:

Party	Too Little	About Right	Too Much
Dem	17 (17.6)	14 (14.43)	12 (10.97)
Ind	20 (29.48)	28 (24.16)	24 (18.36)
Rep	24 (13.92)	8 (11.41)	2 (8.67)

Democrats: observed ≈ expected across all three — 17 vs 17.6, 14 vs 14.4, 12 vs 11.

Independents: fewer “Too Little” than expected, more “Too Much” — they lean toward “Too Much.”

Republicans: far more “Too Little” than expected, far fewer “Too Much” — they lean toward “Too Little.”

Independents and Republicans pull in opposite directions. We need one number that combines all nine gaps.

Now compare observed to expected, cell by cell — the observed count first, the expected count in parentheses. Three patterns stand out. Democrats sit almost exactly on their expected counts across all three opinions — seventeen versus seventeen point six, fourteen versus fourteen point four, and twelve versus eleven. So in this sample, Democrats look like the overall population. Independents go the opposite way at the two ends. Fewer said “too little” than expected — twenty versus about twenty-nine and a half — and more said “too much” — twenty-four versus about eighteen. So Independents lean toward “too much.” Republicans lean the other way, and more sharply. Far more said “too little” than expected — twenty-four against about fourteen — and far fewer said “too much” — only two against about nine. So Republicans lean toward “too little.” Independents and Republicans pull in opposite directions from the overall mix. To turn these nine separate gaps into a single test, we need one number that combines them all.

The Chi-Square Statistic

Score each cell, then sum across the whole table:

\[X^2 = \sum_{\text{all cells}} \frac{(\text{Observed} - \text{Expected})^2}{\text{Expected}}\]

Two design choices in the formula:

Squaring \((\text{O} - \text{E})\) — gaps become positive, so they can’t cancel across the table.
Dividing by Expected — a gap of 5 matters more when the cell expects 10 than 100.

Bigger \(X^2\) = more total deviation from independence = stronger evidence against \(H_0\).

The chi-square statistic sums one piece per cell. For each cell, take the observed count minus the expected count, square that difference, and divide by the expected count. Then add the result across every cell in the table. Two design choices are worth flagging. First, the squaring. Without it, positive and negative gaps would cancel — in fact, within any row of a contingency table, the raw observed-minus-expected gaps always sum to exactly zero, so simply summing the raw gaps across cells would give zero no matter how much association is in the data. Squaring forces every gap to count toward the total, regardless of sign. This is also why every kind of departure from independence ends up pushing chi-square up rather than down — and why the test ends up one-tailed in the right direction. Second, dividing by expected. This standardizes each gap to its scale. A gap of five matters a lot more when the expected count is ten — that’s a fifty-percent miss — than when the expected count is one hundred — a five-percent miss. So bigger chi-square means a bigger total deviation from independence — and stronger evidence against the null.

Computing \(X^2\) for Military

For each cell, compute \(\dfrac{(O - E)^2}{E}\):

Party	Too Little	About Right	Too Much
Dem	\(\frac{(17 - 17.6)^2}{17.6} \approx 0.02\)	\(\frac{(14 - 14.43)^2}{14.43} \approx 0.01\)	\(\frac{(12 - 10.97)^2}{10.97} \approx 0.1\)
Ind	\(\frac{(20 - 29.48)^2}{29.48} \approx 3.05\)	\(\frac{(28 - 24.16)^2}{24.16} \approx 0.61\)	\(\frac{(24 - 18.36)^2}{18.36} \approx 1.73\)
Rep	\(\frac{(24 - 13.92)^2}{13.92} \approx 7.3\)	\(\frac{(8 - 11.41)^2}{11.41} \approx 1.02\)	\(\frac{(2 - 8.67)^2}{8.67} \approx 5.13\)

\[X^2 \approx 0.02 + 0.01 + 0.1 + 3.05 + 0.61 + 1.73 + 7.3 + 1.02 + 5.13 \approx 18.97\]

Plugging in for the military table — one ratio for each of the nine cells. The Democrat row contributes almost nothing: zero point zero two, zero point zero one, and zero point one. The Independent row contributes more — about three for “too little,” about six tenths for “about right,” and about one point seven for “too much.” The Republican row is the biggest contributor — seven point three for “too little,” about one for “about right,” and five point one for “too much.” Add up all nine pieces. Chi-square comes out to about eighteen point nine seven. That number summarizes how far the whole table sits from no association. The next question is how big eighteen point nine seven is relative to what we’d see if the null were true — and that’s what fourteen B is for.

Where Does \(X^2\) Come From?

The full sum is \(X^2 \approx 18.97\). Each cell’s contribution sits in the table above. Which two cells push \(X^2\) up the most — and what’s the story those cells tell?

Identify the two largest cell contributions. Republicans, “Too Little” (\(\approx 7.3\)) and Republicans, “Too Much” (\(\approx 5.13\)). Together they account for about \(12.4\) out of \(18.97\) — roughly two-thirds of the total.
What do those two cells say about where the association is? Republicans were more likely than expected to say “Too Little” (24 observed vs 13.9 expected) and less likely than expected to say “Too Much” (2 observed vs 8.7 expected). \(X^2\) summarizes how different the table is from “no association”; the cell-by-cell contributions tell us where the difference is.

The chi-square total was about eighteen point nine seven, but that single number hides a lot of structure. Each cell makes its own contribution; some cells push it up much more than others. Look at the nine values and find the two largest contributions, then ask what those cells are telling you about the data. Advance when you are ready to see the answers. The two largest contributions both come from the Republican row. Republicans saying “too little” contributes about seven point three; Republicans saying “too much” contributes about five point one. Together those two cells add to about twelve point four out of the total eighteen point nine seven — roughly two-thirds of the total, coming from just those two cells. Those two cells tell a directional story. Republicans were more likely than expected to say “too little” — twenty-four observed against about thirteen point nine expected — and less likely than expected to say “too much” — only two observed against about eight point seven expected. So in this sample, Republicans were more pro-military-spending than we would predict if there were no association. Chi-square just summarizes how far the whole table sits from no association; the cell-by-cell contributions tell you where the difference is concentrated.

What \(X^2\) Does — and Doesn’t — Tell Us

\(X^2 = 0\) means observed = expected in every cell — perfect independence.
\(X^2 = 18.97\) for the military table — but is that “big”?
\(X^2\) scores total deviation from independence; it doesn’t say which groups differ or which direction.
To turn \(X^2\) into a yes/no decision, we need a reference distribution — what \(X^2\) values would look like under \(H_0\)

Next time (14b): the chi-square distribution, conditions, the p-value, and the decision.

Stepping back. We built a single number — the chi-square statistic — that scores an entire contingency table against the independence null. For the military example it’s about eighteen point nine seven. We’ve also seen that chi-square has two important limitations: it doesn’t tell us which groups differ, and it doesn’t tell us the direction of any differences — that takes a careful look at the cell-by-cell contributions. The real question still hanging is whether eighteen point nine seven is “big enough” to reject the null. Big compared to what? We need a reference distribution — the typical range of chi-square values we’d see across many samples if the null were true. That’s the chi-square distribution itself, and we’ll set it up in fourteen B, along with the conditions, the p-value, and the decision.

References

Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
Section 18.2
General Social Survey, 2016 — National Opinion Research Center.