4. Introduction to Confidence Intervals

One Proportion
Math 115

Yurk

Two Types of Questions

Hypothesis Testing: Is a specific value plausible?

  • Medical consultant: “Is her complication rate different from 10%?”
  • We tested \(H_0: p = 0.10\)
  • Answer: Reject or fail to reject \(H_0\)

Today (Confidence Intervals): What values are plausible?

  • “What IS the complication rate?” (not just testing one value)
  • Answer: An interval estimate

Point Estimates and Their Limitations

From hypothesis testing, we know:

  • \(\hat{p}\) is our best single guess for \(p\)
  • But different samples give different values of \(\hat{p}\) (sampling variability)
  • A single number doesn’t express our uncertainty

Solution: Use an interval estimate instead of just a point estimate

  • Expresses a range of plausible values for \(p\)
  • This is called a confidence interval

Candidate X

Context: Candidate X is running for mayor. Her campaign wants to estimate the proportion of all voters who support her.

Data: Campaign polls a random sample of 30 voters

  • 21 people support Candidate X
  • \(\hat{p} = \frac{21}{30} = 0.7\) (70%)

Point estimate: About 70% of all voters support her

Question: How confident should we be in this estimate?

The Challenge

We want to know how much \(\hat{p}\) varies from sample to sample.

In hypothesis testing:

  • We simulated from a spinner assuming \(p = 0.10\) (the null hypothesis)
  • This gave us the null distribution
  • Showed what we’d see if \(H_0\) were true

Now we have a problem:

  • We don’t know \(p\) (that’s what we’re trying to estimate!)
  • Can’t build a spinner without knowing \(p\)
  • Need a different approach…

Our Best Information

Key insight: Our sample is the best approximation we have of the population

  • If 70% of our sample supports Candidate X, that’s our best guess about the population
  • The sample reflects the population (at least approximately)

The bootstrap idea: Use our sample as a stand-in for the population

Bootstrap Sampling

Bootstrap sample: Sample WITH replacement from the original sample

  • Same size as original (n = 30)
  • “With replacement” = same observation can appear multiple times
  • Each bootstrap sample is slightly different from original

Example bootstrap sample:

  • Might select ID #5 three times, ID #12 twice, ID #18 zero times
  • Creates natural variation

The bootstrapped proportion is estimated for each bootstrap sample. The resulting bootstrap distribution (dotplot) provides a measure for how the proportions vary from sample to sample. From IMS2 Figure 12.3.

Why Does Resampling Work?

The logic:

  1. Our sample reflects the population structure
  2. Resampling with replacement recreates the “luck of the draw”
  3. This variation approximates how \(\hat{p}\) varies across samples
  4. Key: We’re NOT assuming any particular value of \(p\)

Compare to hypothesis testing:

  • Hypothesis testing: assumed \(p = 0.10\), simulated what that would look like
  • Confidence intervals: use actual data, no assumption about \(p\)

Bootstrap vs Randomization

Hypothesis Testing

  • Simulated from spinner at \(p_0 = 0.10\)
  • Assumed \(H_0\) is TRUE
  • Distribution centered at 0.10
  • Answers: “Is 0.10 plausible?”

Confidence Intervals

  • Resample from actual data
  • No assumption about \(p\)
  • Distribution centered at \(\hat{p} = 0.7\)
  • Answers: “What values are plausible?”

Same concept (simulation for variability), different purpose

Let’s Try It!

Activity: Create your own bootstrap sample

Original sample data: 30 voters (next slide shows the data)

Instructions:

  1. Go to random.org/integers
  2. Generate 30 random integers between 1 and 30
  3. IMPORTANT: This allows duplicates (this is “with replacement”)
  4. Look up each ID in the table → find their vote
  5. Count how many “yes” votes in your bootstrap sample
  6. Calculate \(\hat{p}_{boot} = \frac{\text{yes votes}}{30}\)

Original Sample Data

ID Vote
1 yes
2 yes
3 yes
4 no
5 yes
6 yes
7 yes
8 yes
9 no
10 yes
11 yes
12 yes
13 no
14 yes
15 yes
ID Vote
16 yes
17 yes
18 no
19 yes
20 yes
21 yes
22 no
23 no
24 no
25 yes
26 yes
27 yes
28 no
29 yes
30 no

Summary: 21 voters said “yes”, 9 voters said “no”

Class Bootstrap Distribution

Activity: Let’s build the bootstrap distribution together!

Instructions: Draw a dot above your bootstrap proportion

Question: Where is this distribution centered? Compare to hypothesis testing (centered at 0.10).

Interactive Bootstrap Demo

Try the interactive simulation:

🎯 Open Bootstrap Demo

  • Run 1 sample at a time and watch the resampling animation
  • Run 100 samples to quickly build the bootstrap distribution
  • See how bootstrap proportions stack up in the dotplot

Building the Bootstrap Distribution

Histogram of 1000 bootstrap proportions. Distribution centered near 0.7 (our observed \(\hat{p}\)).

Bootstrap Standard Error

Standard Error (SE): Standard deviation of a statistic (in this case \(\hat{p}\))

  • Measures uncertainty in our estimate
  • For Candidate X: \(SE_{boot} = 0.082\)

Interpretation: \(\hat{p}\) typically varies by about \(\pm 0.082\) from sample to sample

The Percentile Method

To create a 95% confidence interval:

  1. Take the bootstrap distribution
  2. Find the middle 95% of values
  3. Cutoffs: 2.5th percentile and 97.5th percentile

Why these percentiles?

  • 2.5% below, 95% in the middle, 2.5% above
  • These values “fence in” the middle 95%

Computing the 95% CI

95% Confidence Interval: (0.533, 0.834)

Interpreting Confidence Intervals

Our 95% CI for Candidate X: (0.533, 0.834)

Correct interpretation:

“We are 95% confident that the true proportion of voters who support Candidate X is between 0.533 and 0.834.”

What this means:

  • Values inside the interval are plausible for \(p\)
  • Values outside the interval are implausible for \(p\)

What Does “95% Confident” Mean?

Common Misinterpretation

❌ WRONG: “There is a 95% probability that \(p\) is in this interval”

✅ CORRECT: “We are 95% confident that this interval contains \(p\)”

Why the distinction?

  • The parameter \(p\) is fixed (but unknown)
  • It either is or isn’t in our interval
  • The 95% refers to our confidence in the method, not probability about this specific interval
  • If we repeated this process many times using new samples of 30, about 95% of intervals would contain \(p\)

Properties of Confidence Intervals

Three key properties:

  1. CI contains the observed statistic (usually near the center)
    • Our CI for Candidate X is centered near \(\hat{p} = 0.7\)
  2. Larger sample → narrower CI (more precision)
    • n = 30 gives wider interval than n = 300 would
  3. Higher confidence level → wider CI (more conservative)
    • 99% CI is wider than 95% CI

Other Confidence Levels

For Candidate X bootstrap distribution:

Confidence Level Percentiles Interval
90% 5th to 95th (0.567, 0.833)
95% 2.5th to 97.5th (0.533, 0.834)
99% 0.5th to 99.5th (0.5, 0.9)

Notice: Higher confidence → wider interval

Trade-off: Confidence vs. Precision

Connecting to Hypothesis Testing

Let’s revisit the medical consultant from hypothesis testing:

Data: 3 complications in 62 surgeries, \(\hat{p} = 0.048\)

Bootstrap 95% CI: (0, 0.113)

Key observation: The national rate of 0.10 IS in this interval

Connection to hypothesis testing: We failed to reject \(H_0: p = 0.10\) (p-value = 0.11)

Insight: Values in the CI are exactly those we would NOT reject in a (two-sided) hypothesis test!

The national rate (0.10) falls within the 95% CI—consistent with our hypothesis test result.

The Confidence Interval Process

Summary of steps:

  1. Collect sample and calculate \(\hat{p}\)

  2. Generate bootstrap samples: Resample with replacement (at least 1000 times)

  3. Calculate \(\hat{p}_{boot}\) for each bootstrap sample

  4. Build bootstrap distribution from all bootstrap proportions

  5. Find percentiles: 2.5th and 97.5th for 95% CI

  6. Interpret in context: “We are 95% confident that \(p\) is between…”

Bootstrap vs H-Test Randomization Summary

Feature Hypothesis Testing Confidence Intervals
Question Is \(p_0\) plausible? What values of \(p\) are plausible?
Method Simulate \(H_0\) Bootstrap resampling from original data with replacement
Assumes \(H_0\) is true (\(p = p_0\)) Nothing about \(p\)
Distribution centered at \(p_0\) (null value) \(\hat{p}\) (observed value)
Output P-value Confidence interval
Decision Reject / Fail to reject \(H_0\) Values in / out of CI

Both methods: Use simulation to understand sampling variability, but for different purposes

References