Extending Our Framework

Previously, we compared two groups with a binary outcome (2×2 table).

• Used difference in proportions: \(\hat{p}_1 - \hat{p}_2\)
• Used Z-statistic for hypothesis testing

But what if we have:

• More than 2 groups? (e.g., Dem, Ind, Rep)
• Non-binary outcomes? (e.g., Spending: Too Little, About Right, Too Much)

We need a new approach!

Two Ways to Generalize

Generalization 1: More groups

• Explanatory variable has 3+ categories
• Example: Democrat, Independent, Republican

Generalization 2: Non-binary response

• Response variable has 3+ categories
• Example (Government Spending): “Too Little”, “About Right”, “Too Much”

With either generalization, we can’t use a single difference in proportions!

The Hypothesis Framework

Same concept as before, different statistic:

• \(H_0\): No association between the variables (independence)
• \(H_A\): There is an association (variables are not independent)

New statistic: Chi-square, \(\chi^2\), measures deviation from independence.

Military Spending

What Would We Expect Under \(H_0\)?

If \(H_0\) is true (no association), each party should have the same distribution of opinions.

Expected count for any cell:

\[\text{Expected} = \frac{\text{row total} \times \text{column total}}{\text{grand total}}\]

This distributes the overall opinion proportions equally across all parties.

The Chi-Square Statistic

Measures total deviation of observed from expected:

\[\chi^2 = \sum_{\text{all cells}} \frac{(\text{Observed} - \text{Expected})^2}{\text{Expected}}\]

Why this formula?

• Squaring: Makes all contributions positive
• Dividing by Expected: Standardizes (a difference of 5 matters more when Expected is 10 than when Expected is 100)

What Does \(\chi^2\) Tell Us?

• \(\chi^2 = 0\) means observed = expected exactly (perfect independence)
• Larger \(\chi^2\) means greater deviation from independence
• Larger \(\chi^2\) → stronger evidence against \(H_0\)

But how large is “large enough” to reject \(H_0\)?

We need a reference distribution → chi-square distribution

Here is the original GSS data with 5 random permutations.

# A tibble: 149 × 8
      id natarms    party randPerm1 randPerm2 randPerm3 randPerm4 randPerm5
   <int> <fct>      <chr> <fct>     <fct>     <fct>     <fct>     <fct>    
 1     1 TOO LITTLE Ind   Rep       Ind       Ind       Ind       Dem      
 2     2 TOO MUCH   Ind   Dem       Dem       Ind       Rep       Ind      
 3     3 TOO MUCH   Dem   Dem       Rep       Rep       Rep       Dem      
 4     4 TOO MUCH   Ind   Rep       Ind       Dem       Rep       Rep      
 5     5 TOO MUCH   Ind   Ind       Dem       Rep       Ind       Ind      
 6     6 TOO MUCH   Ind   Dem       Rep       Rep       Dem       Ind      
 7     7 TOO MUCH   Ind   Dem       Dem       Ind       Ind       Dem      
 8     8 TOO MUCH   Dem   Ind       Ind       Ind       Dem       Rep      
 9     9 TOO LITTLE Dem   Ind       Dem       Ind       Rep       Ind      
10    10 TOO LITTLE Ind   Dem       Dem       Ind       Rep       Ind      
# ℹ 139 more rows

Here is the dotplot of the corresponding \(\chi^2\) statistics.

Here is the resulting histogram of 1000 simulations

Histogram of \(X^2\) statistics for 1,000 random permutations. Observed value (\(18.97\)) indicated by dashed vertical line.

• Note that the shape of the histogram is neither symmetric nor bell-shaped. In fact, it only uses non-negative values

Test for Independence Using a Mathematical Model

• \(r\) is the number of rows and \(c\) is the number of columns in the two-way table (no totals)

Degrees of Freedom

For contingency tables:

\[df = (\text{rows} - 1) \times (\text{columns} - 1)\]

For our 3×3 table:

\[df = (3-1) \times (3-1) = 2 \times 2 = 4\]

Intuition: After fixing row and column totals, only \(df\) cells can vary freely.

The Chi-Square Distribution

• Always right-skewed, starts at 0
• As df increases, distribution shifts right

The Null Distribution

When conditions are met, we can use the chi-square distribution as our reference:

• Use the chi-square distribution with \(df = (r-1) \times (c-1)\)
• This distribution describes what \(\chi^2\) values we’d expect if \(H_0\) is true

Computing the p-value:

• P-value = area to the right of observed \(\chi^2\) under the chi-square curve
• Larger \(\chi^2\) values are more extreme → stronger evidence against \(H_0\)

Conditions for Chi-Square Test

Independence:

• Random sample ✓
• Observations independent of each other ✓

Expected Counts (under \(H_0\)):

• At least 5 expected in each cell ✓

Check expected counts, not observed counts!

• Conclusion
  • We reject null hypothesis in this case
  • There is strong evidence that opinion on military spending and political affiliation are associated
  • We can generalize these results to a larger population since it was a representative sample
  • We cannot draw cause-and-effect conclusion since it was an observational study

Note: The chi-square test tells us there IS an association, but not the direction or which groups differ.

Goodness-of-Fit Test (Optional)

• The Chi-square goodness-of-fit test checks if observed categorical data fit an expected distribution.

• Formula: \[\chi^2 = \sum \frac{(Obs - Exp)^2}{Exp}\]

• Used for genetics, health studies, or marketing data.

• Example: Do observed blood type frequencies in a population match known distribution?

• Suppose we have a random sample of 350 people with the following observed blood types:

• Research Question: Is there significant evidence that the distribution of blood types in the sample is different from the population’s distribution?

Chi-square Calculation and p-value

# Observed data
observed <- c(A = 155, B = 40, AB = 15, O = 140)

# Assumed distribution of proportions
expected_prop <- c(A = 0.40, B = 0.11, AB = 0.04, O = 0.45)

# Expected counts
total <- sum(observed)
expected <- total * expected_prop

# Chi-square statistic
chi_sq <- sum((observed - expected)^2 / expected)
chi_sq

[1] 3.681457

# Degrees of freedom = (categories - 1)
df <- length(observed) - 1
df

[1] 3

# p-value
p_val <- pchisq(chi_sq, df, lower.tail = FALSE)

Chi-Square distribution and p-value

Interpretation

• If p-value < 0.05 → observed distribution significantly differs from expected.

• If p-value > 0.05 → we don’t have significant evidence that the observed distribution significantly differs from expected.

• Here, p-value indicates whether the sample matches the expected blood type proportions.

Summary

Chi-square test extends two-proportion test to larger tables:

Conditions: Independence + Expected counts ≥ 5 in each cell

Connection to Topic 10

Chi-square generalizes the two-proportion test to larger tables!

(For 2×2 tables, both methods give equivalent results)

References

• Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
• Section 18.2