9. Inference for Two Proportions (Randomization)

b. Bootstrap CI
Math 115

Yurk

Recap: CPR Study

Randomized trial: 90 cardiac-arrest patients receiving CPR, assigned to blood thinner (treatment, n = 40) or standard care (control, n = 50). Response: survived at 24 hours?

Group Died Survived Total
Control 39 11 50
Treatment 26 14 40
Total 65 25 90
  • Observed difference: \hat{p}_T - \hat{p}_C = 0.35 - 0.22 = 0.13
  • 9a permutation test: p-value \approx 0.27 → failed to reject H_0

From Testing to Estimation

The permutation test answers: “Is there a difference?”

But we also want to know: “How big is the difference?”

Solution: Bootstrap confidence interval for p_1 - p_2

  • Gives a range of plausible values for the true difference
  • More informative than just a yes/no answer

Bootstrap Approach

Same idea as the single-proportion bootstrap from topic 4:

  1. Throw each case (group + outcome) into a hat
  2. Draw 90 cases with replacement — some come up more than once, some not at all
  3. Compute the new \hat{p}_T - \hat{p}_C
  4. Repeat 1000 times → bootstrap distribution

Unlike permutation: each case keeps its group and its outcome. No shuffling to break the association.

Why Does Bootstrap Work?

Goal: Know how \hat{p}_T - \hat{p}_C would vary across repeated samples from the population.

Problem: We only have one sample — the population is out of reach.

Bootstrap idea: The sample is our best stand-in for the population. Resampling from the sample mimics drawing repeated samples from the population.

→ Spread of bootstrap statistics ≈ spread of \hat{p}_T - \hat{p}_C across real repeated samples → use that spread to build a CI.

Row Mechanics

Permutation

Group stays · outcomes shuffled
id group outcome
1 Tx SD
2 Tx DD
3 Tx DD
4 Tx SD
5 Tx SD
6 Tx DS
7 Ctl DS
8 Ctl DS
9 Ctl SD
10 Ctl DS
11 Ctl DS
12 Ctl SD

Bootstrap

Rows drawn with replacement
id group outcome drawn
1 Tx S —
2 Tx D —
3 Tx D —
4 Tx S —
5 Tx S —
6 Tx D —
7 Ctl D —
8 Ctl D —
9 Ctl S —
10 Ctl D —
11 Ctl D —
12 Ctl S —

shuffle outcomes

draw rows with replacement

CPR Bootstrap Demo

View interactive bootstrap demo

The demo shows:

  1. Original 90 patients — shapes = group (○ treatment, □ control), colors = outcome (green survived, gray died)
  2. Resampling WITH replacement from entire sample
  3. Group sizes vary in each bootstrap sample
  4. Building the bootstrap distribution

CPR Bootstrap Distribution

1000 bootstraps. Centered near observed difference (0.13), NOT at 0

CPR Confidence Interval

95% Bootstrap Percentile CI:

Find the 2.5th and 97.5th percentiles of the bootstrap distribution:

(-0.064, 0.325)

Interpretation: We are 95% confident that the true difference in survival rates (treatment − control) is between -0.064 and 0.325.

CI and Test Agreement

The confidence interval and hypothesis test give consistent results:

For CPR study:

  • 95% CI: (-0.064, 0.325) includes 0
  • P-value \approx 0.27 > 0.05 \rightarrow Don’t reject H_0
  • Both results indicate that 0 is a plausible value for p_T-p_C

ci

p

both

Estimate the CI Bounds

The next slide shows the bootstrap distribution for the active-learning study from 9a (1000 resamples; observed difference \approx 0.20).

Estimate the 95% CI bounds by counting bars from each tail.

Hint: a 95% CI excludes about 25 resamples from each tail (2.5\% \times 1000 = 25). Advance for the walkthrough.

Reading the Bounds

Left tail counts placeholder

Left tail counts: 1 + 2 + 4 + 3 + 2 + 13 = 25

Left 95% bound \approx -0.015

Right tail counts (from right): 1 + 5 + 7 + 10 = 23

95% CI: (-0.015,\ 0.411) — includes 0; fail to reject (matches 9a).

Stents — Interpret a Bootstrap CI

For the stents-and-stroke study from 9a (treatment vs. control, stroke within 30 days), a bootstrap procedure gives a 95% confidence interval for p_T - p_C (stroke rate):

(0.040, \; 0.150)

  1. Does the CI include 0? No. Both endpoints are positive.

  2. Interpret the CI in context. We are 95% confident that the true difference in stroke rates (treatment − control) is between 0.040 and 0.150 — stents are associated with a 4 to 15 percentage-point higher stroke rate.

  3. Is this consistent with the H-test result (p-value \approx 0.004)? Yes. Both reject H_0: the CI excludes 0, and the p-value is below \alpha = 0.05. They agree that 0 is not a plausible value for the true difference.

Permutation vs Bootstrap

Permutation Test Bootstrap CI
Purpose Hypothesis testing Estimation
Question Is there a difference? How big is the difference?
Assumes Null is true Nothing about null
Method Shuffle response values Resample with replacement
Distribution center 0 Observed difference
Result P-value Confidence interval

References

Course Home