3. Introduction to Hypothesis Testing
a. Setup and Simulation
The Big Picture
So far we’ve described data (e.g., center, spread, proportions)
Now we ask harder questions:
- Does this drug actually work?
- Are college graduation rates accurate?
- Are MLB pitchers really more left-handed than the general population?
Each of these requires an inference — a statement about an unknown population, supported by data from a sample. Hypothesis testing is the framework; we’ll start with studies involving a single proportion.
One Proportion
These studies involve a single binary categorical variable — two levels, one labeled the success. The statistic of interest is a single proportion, denoted \hat{p}.
Examples:
| vote choice |
A / B |
voters choosing A |
| side effect |
yes / no |
patients with side effects |
| product condition |
defective / OK |
defective products |
Note: “success” just means the outcome we’re counting — it doesn’t have to be a positive outcome.
Statistical Notation: The “Hat”
The hat (as in \hat{p}) denotes an estimate from our sample:
p — population parameter
The TRUE proportion (unknown).
\hat{p} — sample statistic
Our ESTIMATE from data.
The true population proportion p is unknown — that’s what we’re trying to learn about.
Computing p̂
Count the successes (x) and divide by the sample size (n):
\hat{p} = \frac{x}{n}
Example: 12 of 50 students sampled prefer online classes:
\hat{p} = \frac{12}{50} = 0.24
Identify the Pieces
Scenario: a streaming service surveys 80 of its users. 56 of them watch more than 10 hours of programming per week.
Identify each piece — advance for answers:
| Cases? |
the 80 surveyed users |
| Variable? |
does the user watch more than 10 hours per week? |
| Two levels? |
yes and no. Below we’ll treat “yes” as the success. |
| Single-proportion study? |
Yes — one binary variable |
| Parameter p? |
the true proportion of all users who watch more than 10 hours per week (unknown) |
| Sample size n? |
80 |
| Count of successes x? |
56 |
| Sample statistic \hat{p}? |
\hat{p} = x/n = 56/80 = 0.70 |
Sampling Variability
Imagine we knew the true left-handedness rate was p = 0.10. Three samples of 100 people might yield \hat{p} = 0.08, 0.13, 0.11. This natural sample-to-sample variation is sampling variability.
In real life we don’t know p. We see one sample and have to reason backward: when our \hat{p} looks unusual,
is something real going on in the population we sampled from, or is it just due to sampling variability?
Medical Consultant
Setup: a consultant claims her liver-donor patients have a lower complication rate than the 10% national average.
Data: 3 complications in 62 surgeries.
\hat{p} = \frac{3}{62} = 0.048 \quad (\approx 4.8\%)
Is \hat{p} = 0.048 evidence that the consultant’s true rate p < 0.10?
Source: Çetinkaya-Rundel & Hardin, Introduction to Modern Statistics.
Innocent Until Proven Guilty
We assume the consultant is no different from the national rate — and reject that assumption only if our data look very unusual.
| 1. |
Presume innocence |
Assume p = 0.10 |
| 2. |
Look for evidence of guilt |
Compare data to chance |
| 3. |
Convict only if evidence is strong |
Reject only if data look unusual |
Null Hypothesis (H_0)
The null hypothesis is the skeptical position — what we assume is true and look for evidence against.
H_0: p = 0.10
“The consultant’s rate equals the national average.”
Alternative Hypothesis (H_A)
The alternative hypothesis is the claim we want to test:
H_A: p < 0.10
“The consultant’s rate is lower than the national average.”
Important: hypotheses are about the population parameter p, not the sample statistic \hat{p}. We already know \hat{p} = 0.048 — the question is what p is.
State the Hypotheses
Scenario: a college claims its 4-year graduation rate is 70%, but a consumer-research organization suspects the actual rate is lower. They follow a sample of 80 students from a recent cohort and find that 48 graduated within 4 years.
Set up the test — advance for answers:
| Parameter of interest? |
p = the college’s true 4-year graduation rate |
| H_0? |
H_0: p = 0.70 |
| H_A? |
H_A: p < 0.70 |
| \hat{p}? |
48 / 80 = 0.60 |
The Testing Strategy
- Assume H_0 is true (consultant rate = 0.10).
- Simulate what we’d expect to see under that assumption.
- Compare our observed \hat{p} to the simulations.
- Decide:
- If our data look very unusual under H_0 → reject H_0.
- If they look plausible → cannot reject H_0.
We’ll spend the rest of this lecture seeing what “simulate” looks like. Making “unusual” precise comes next class.
Simulating One Sample
To test the hypothesis, we simulate samples we’d expect to see if H_0 (p = 0.10) were true. Two ways to simulate one sample:
Spinner 
10% orange (complication), 90% blue (no complication). Spin 62 times, count the complications, and divide by 62 to get a simulated \hat{p}.
Random number generator
Draw 62 numbers from 1 to 10 (e.g. via random.org). Each “1” = complication; “2–10” = no complication. Count the 1’s and divide by 62 to get a simulated \hat{p}.
Try It Yourself
Activity: simulate one sample under H_0. Go to random.org/integers and generate 62 random integers from 1 to 10 (each “1” = complication).
Identify each piece — advance for answers:
| Count of successes x? |
the number of 1’s you observed |
| Sample size n? |
62 |
| Simulated \hat{p}? |
\hat{p} = x/n |
Bonus: try the interactive spinner demo for an animated version (Run 1 Sample).
Repeat 100 Times
- How did your simulated \hat{p} compare to the consultant’s observed \hat{p} = 0.048?
- Each simulated sample (62 spins, or 62 random integers) gives one \hat{p} — the proportion of times the spinner landed on orange, or the proportion of 1’s in the draw.
- Imagine doing this 100 times. The collection of those 100 simulated \hat{p} values is the null distribution — what we’d expect to see when H_0 is true.
100 Simulations: The Null Distribution
![]()
This collection of 100 simulated \hat{p} values is the null distribution — what’s typical when H_0 is true. It’s centered around p = 0.10 (the rate we assumed), not around our observed \hat{p} = 0.048 — because each simulation came from a population where the true rate really is 0.10.
Where Does Our p̂ Land?
![]()
The red dashed line marks the consultant’s observed \hat{p} = 0.048. The 9 pink dots are simulations at or below it — putting our observed value in the left tail. Unusual enough to reject H_0? With only 100 simulations, hard to tell — let’s do more.
1000 Simulations: A Sharper Picture
![]()
Same null distribution, sharper picture — still centered around p = 0.10. With 1000 simulations the dots overlap, so we switch to a histogram. 111 of 1000 simulated proportions are at or below 0.048.
Surprising Enough?
So if H_0 is really true, we’d only expect to see a result like the consultant’s — or more extreme — about 11.1% of the time.
Surprising enough to reject H_0? Next class, we’ll dig in.
