3. Introduction to Hypothesis Testing

One Proportion
Math 115

Yurk and Martin

The Big Picture

So far we’ve learned to describe data:

  • Summarize with statistics (mean, proportion, etc.)
  • Visualize distributions

Now we learn to make inferences:

  • Draw conclusions about populations from samples

One Proportion - A Simple Starting Point

  • As we begin to explore statistical inference we will focus on studies that involve a single binary, categorical variable
  • Binary = two levels (one identified as a success)
  • The statistic of interest in these studies is a single proportion (of successes)

Examples:

  • Proportion of voters supporting a candidate
  • Proportion of patients experiencing side effects
  • Proportion of students passing an exam
  • Proportion of products that are defective

Statistical Notation: The “Hat”

In statistics, we use \(\hat{}\) (called a “hat”) to indicate an estimate from our sample:

\(p\) = Population parameter (what we want to know)

  • The TRUE proportion

\(\hat{p}\) = Sample statistic (what we can calculate)

  • Our ESTIMATE from data

Example: If 12 students prefer online classes in a sample of 50:

  • \(\hat{p} = \frac{12}{50} = 0.24\)
  • what is \(p\)?

Sampling Variability

Even when the population proportion \(p\) is fixed, different samples give different proportions (different values of \(\hat{p}\)).

Example: If the true proportion of left-handed people is \(p=0.13\) (13%)…

  • One sample of 100 people might have 11 left-handers: \(\hat{p}=0.11\) (11%)
  • Another sample might have 16: \(\hat{p}=0.16\) (16%)
  • Another sample might have 12: \(\hat{p}=0.12\) (12%)

This natural variation is called sampling variability

The Key Question

When we observe something unusual in our sample:

Is it a real effect, or just due to chance?

  • Could be due to sampling variability (“luck of the draw”)
  • Or could indicate a real difference in the population

Hypothesis testing helps us answer this question

Medical Consultant

Context: Some organ donors work with a medical consultant who helps them through the surgery process.

Known fact: The national complication rate for liver donor surgeries is 10%

Claim: One consultant claims her patients have a lower complication rate

Data: Out of 62 surgeries she assisted,3 resulted in complications:

  • \(\hat{p} = \frac{3}{62} = 0.048\) (4.8%)

Question: Is this evidence of a truly lower rate?

Notation Check

\(p\) = Population parameter (what we want)

  • The consultant’s true, long-run proportion of complications (\(p=???\))

\(\hat{p}\) = Sample statistic (what we can calculate)

  • proportion of complications observed in the sample of 62 surgeries (\(\hat{p} = 0.048\))

Question: Is this evidence that \(p<0.1\)?

Two Competing Explanations

How do we explain the consultant’s low complication rate?

Explanation 1: The consultant is no different from the national average

  • True rate is 10% (\(p=0.1\))
  • She just happened to get a “lucky” sample

Explanation 2: The consultant truly has a lower complication rate

  • True rate is less than 10% (\(p<0.1\))
  • The sample reflects a real difference

How do we decide which is more plausible?

The Skeptical Approach

Strategy: Start by assuming there’s nothing special going on

  1. Assume the consultant is no different (10% rate)
  2. Ask: How surprising is our data under this assumption?
  3. Decide: If very surprising → evidence against our assumption

Innocent Until Proven Guilty

This is like a courtroom:

  • Start with “innocent (\(p=0.1\)) until proven guilty (\(p<0.1\))”
  • Look for evidence against innocence
  • Need strong evidence to convict

Null Hypothesis (\(H_0\))

The null hypothesis represents the skeptical position:

\[H_0: p = 0.10\]

  • “The consultant’s complication rate equals the national average”
  • This is what we assume to be true
  • We look for evidence against it

Key idea: The null hypothesis typically represents “no effect” or “no difference”

Courtroom parallel: \(H_0\) = presumption of innocence

Alternative Hypothesis (\(H_A\))

The alternative hypothesis represents the claim we want to test:

\[H_A: p < 0.10\]

  • “The consultant’s complication rate is lower than the national average”
  • This is what we conclude if we reject \(H_0\)

Stating Hypotheses

Medical Consultant Example:

In words In symbols
\(H_0\) Consultant’s rate equals national average \(p = 0.10\)
\(H_A\) Consultant’s rate is lower than national average \(p < 0.10\)

Important:

  • Hypotheses are about the population parameter (\(p\)), not the sample statistic (\(\hat{p}\))
  • We already know \(\hat{p} = 0.048\); we’re asking about \(p\)

The Testing Strategy

  1. Assume \(H_0\) is true (consultant rate = 10%)

  2. Simulate what we’d expect to see under this assumption

  3. Compare our observed data to these simulations

  4. Decide:

    • If our data is very unusual → reject \(H_0\)
    • If our data is plausible → cannot reject \(H_0\)

What We Need: A Null Distribution

Question: If \(H_0\) is true, what values of \(\hat{p}\) should we expect?

Answer: We need the null distribution

  • The distribution of \(\hat{p}\) values we’d see if \(H_0\) were true
  • Shows us what’s “typical” under the null hypothesis
  • Lets us judge whether our observed \(\hat{p}\) is unusual

How do we get it? Simulation!

The Spinner Model

Imagine a spinner:

  • 10% of the spinner is orange: “complication”
  • 90% of the spinner is blue: “no complication”

To simulate one sample under \(H_0\):

  1. Spin the spinner 62 times (once per surgery)
  2. Count how many land on “complication”
  3. Calculate \(\hat{p} = \frac{\text{number of complications}}{62}\)

Interactive Spinner Demo

Try the interactive simulation:

🎯 Open Spinner Demo

  • Run 1 sample at a time and watch the spinner animation
  • Run 100 samples to quickly build the null distribution
  • See how sample proportions stack up in the dotplot

100 Simulations

Each dot = one simulated sample proportion under \(H_0\)

Red dashed line = our observed value (\(\hat{p} = 0.048\))

What Do We See?

From our 100 simulations:

  • Sample proportions vary from sample to sample
  • Most simulated proportions cluster around 0.10 (as expected under \(H_0\))
  • Our observed value (0.048) is in the left tail

Key Question

Our question: Is \(\hat{p} = 0.048\) unusually low, or just normal variation?

  • 9 out of 100 simulations were at or below 0.048

Is this unusual enough to reject \(H_0\)? Let’s do more simulations for a clearer picture…

1000 Simulations

Histogram of 1000 simulated proportions under \(H_0\)

This time 111 out of 1000 simulated values are at or below 0.048.

Measuring Surprise: The P-Value

P-value = Probability of observing our result (or more extreme) if \(H_0\) is true

Interpretation: How likely is it to see a sample proportion of 0.048 or less if the true rate is 0.10?

Courtroom parallel: P-value = strength of prosecution’s evidence

Understanding “More Extreme”

“More extreme” means in the direction of \(H_A\):

  • Since \(H_A: p < 0.10\) (consultant is BETTER)
  • “More extreme” = LOWER proportions
  • We count simulations where \(\hat{p} \leq 0.048\)

Computing the P-Value

From our 1000 simulations:

  • Count simulations where \(\hat{p} \leq 0.048\)
  • Result: 111 out of 1000

\[\text{P-value} = \frac{111}{1000} = 0.111\]

About 11.1% of simulations produced a result as extreme as (or more extreme than) what we observed

Watch Out: Three Different “p” Terms!

Don’t confuse these three:

  • \(p\) = population parameter (the true proportion)
  • \(\hat{p}\) = sample statistic (our calculated estimate)
  • p-value = probability of observing our result (or more extreme) if \(H_0\) is true

They all use the letter “p” but mean completely different things!

Building Intuition: What If?

How would the p-value change with different observed results?

Suppose the consultant still had 62 surgeries, but…

Observed complications \(\hat{p}\) Expected p-value
1 complication 0.016 Very small (≈0.01)
3 complications 0.048 0.11 (our actual result)
6 complications 0.097 Large (≈0.57)
7 complications 0.113 Large (≈0.72)

Pattern: Lower \(\hat{p}\) → smaller p-value → stronger evidence against \(H_0\) (why?)

Interpreting the P-Value

P-value \(\approx\) 0.111 means:

“If the consultant truly has the same 10% complication rate as the national average, there is about an 11% chance of observing 3 or fewer complications in 62 surgeries.”

Is this surprising?

  • Not extremely rare, but not common either
  • We need a standard to make a decision

The Significance Level (\(\alpha\))

To make decisions, we set a significance level (\(\alpha\)) in advance

Common choice: \(\alpha = 0.05\) (5%)

Decision rule:

  • If p-value \(< \alpha\): Reject \(H_0\) (result is “statistically significant”)
  • If p-value \(\geq \alpha\): Fail to reject \(H_0\) (result is not statistically significant)

Courtroom parallel: \(\alpha\) = standard for “beyond reasonable doubt”

Making a Decision

Our results:

  • P-value \(\approx\) 0.111
  • Significance level \(\alpha = 0.05\)

Comparison:

  • Is 0.111 \(<\) 0.05? No

Conclusion:

We fail to reject \(H_0\) at the \(\alpha = 0.05\) level.

The data do not provide convincing evidence that the consultant has a lower complication rate than the national average.

What Does “Fail to Reject” Mean?

Important distinctions:

What we CAN say What we CANNOT say
Data don’t provide strong evidence against \(H_0\) \(H_0\) is true
10% is a plausible value for the consultant’s rate Consultant’s rate is exactly 10%
We can’t rule out that the consultant is average Consultant is definitely not better

“Fail to reject” ≠ “Accept”

The consultant might have a lower rate — we just don’t have enough evidence to conclude that

Courtroom parallel: “Not guilty” ≠ “innocent”

Your Turn (Guided Practice)

Scenario: Researchers want to know if dogs understand human cues.

Data: In 10 trials Harley the dog was presented with two bowls of food when a person participating in the research pointed toward one of those bowls,

  • Harley chose the correct bowl 9 times (90%)

Question: Is this evidence dogs understand human cues? That is did Harley understand which bowl was being pointed to or did he just get lucky?

Let’s work through this together…

Guided Practice: Set Up

Step 1: Identify the parameter

  • \(p\) = long-run (population) proportion of correct responses by Harley

Step 2: State the hypotheses

  • \(H_0: p = 0.50\) (Harley randomly goes to a bowl)
    • This can be called the “test value”
  • \(H_A: p > 0.50\) (Harley understands the cues)

Step 3: Calculate sample statistic

  • \(\hat{p} = \frac{9}{10} = 0.9\)

Guided Practice: Simulate

Step 5: Physical simulation

  • Flip a penny 10 times
  • Treat heads as a success (correct bowl)
  • Calculate proportion of successes
  • Plot your proportion on the dot plot on the board

Step 6: Simulate 1000 samples under \(H_0\)

  • Use the Randomize module in Jamovi

Step 7: Calculate the p-value

  • The Randomize module will do this for us

Guided Practice: Interpret Results

Your conclusions:

  1. Decision (using \(\alpha = 0.05\)): Decide whether to reject \(H_0\)

  2. Interpretation: Which is correct?

  • The data do not provide convincing evidence that Harley understands human cues
  • The data provide convincing evidence that Harley does understand human cues

The Hypothesis Testing Framework

Summary of the process:

  1. State hypotheses: \(H_0\) and \(H_A\)

  2. Collect data: Calculate observed statistic (\(\hat{p}\))

  3. Simulate null distribution: What would we see if \(H_0\) were true?

  4. Calculate p-value: How extreme is our observation?

  5. Make decision: Compare p-value to \(\alpha\)

  6. State conclusion: In context of the original question

References