Intro to Hypothesis Testing:Single Proportion

The Big Picture

So far we’ve learned to describe data:

• Summarize with statistics (mean, proportion, etc.)
• Visualize distributions

Now we learn to make inferences:

• Draw conclusions about populations from samples

One Proportion - A Simple Starting Point

Statistical Notation: The “Hat”

In statistics, we use \(\hat{}\) (called a “hat”) to indicate an estimate from our sample:

Example: If 12 students prefer online classes in a sample of 50:

• \(\hat{p} = \frac{12}{50} = 0.24\)
• \(p\) - true proportion of all students who prefer online classes

Sampling Variability

Even when the population proportion \(p\) is fixed, different samples give different proportions (different values of \(\hat{p}\)).

Example: If the true proportion of left-handed people is \(p=0.13\) (13%)…

• One sample of 100 people might have 11 left-handers: \(\hat{p}=0.11\) (11%)
• Another sample might have 16: \(\hat{p}=0.16\) (16%)
• Another sample might have 12: \(\hat{p}=0.12\) (12%)

This natural variation is called sampling variability

The Key Question

When we observe something unusual in our sample:

Is it a real effect, or just due to chance?

• Could be due to sampling variability (“luck of the draw”)
• Or could indicate a real difference in the population

Hypothesis testing helps us answer this question

Innocent Until Proven Guilty

This is like a courtroom:
Start with “innocent (\(p=0.1\)) until proven guilty (\(p<0.1\))” Look for evidence against innocence
Need strong evidence to convict

Null Hypothesis (\(H_0\))

The null hypothesis represents the skeptical position:

\[H_0: p = 0.10\]

• “The consultant’s complication rate equals the national average”
• This is what we assume to be true
• We look for evidence against it

Key idea: The null hypothesis typically represents “no effect” or “no difference”

Courtroom parallel: \(H_0\) = presumption of innocence

Alternative Hypothesis (\(H_A\))

The alternative hypothesis represents the claim we want to test:

\[H_A: p < 0.10\]

• “The consultant’s complication rate is lower than the national average”
• This is what we conclude if we reject \(H_0\)

The Testing Strategy

1. Assume \(H_0\) is true (consultant rate = 10%)

2. Simulate what we’d expect to see under this assumption

3. Compare our observed data to these simulations

4. Decide:

   • If our data is very unusual → reject \(H_0\)
   • If our data is plausible → cannot reject \(H_0\)

What We Need: A Null Distribution

Question: If \(H_0\) is true, what values of \(\hat{p}\) should we expect?

Answer: We need the null distribution

• The distribution of \(\hat{p}\) values we’d see if \(H_0\) were true
• Shows us what’s “typical” under the null hypothesis
• Lets us judge whether our observed \(\hat{p}\) is unusual

How do we get it? Simulation!

Hypothesis Test Using Randomization

• We can use simulation to approximate the null distribution
• We simulate the outcome by spinning a spinner with 10% of the area representing “complication” and 90% representing “no complication”
• For each simulation, spin the spinner 62 times (sample size) and record the proportion of complications in the sample
• Repeat to obtain proportions for 1,000 simulated samples

Key Question

Our question: Is \(\hat{p} = 0.048\) unusually low, or just normal variation?

• 9 out of 100 simulations were at or below 0.048

• Note that since the alternative hypothesis is in the form “less than”, p-value region is in the left tail

• So the proportion of times when the simulated value was at 0.048 or below is 0.09

Is this unusual enough to reject \(H_0\)? Let’s do more simulations for a clearer picture…

Measuring Surprise: The P-Value

P-value = Probability of observing results “as extreme as” the observed result (i.e. 0.048) if \(H_0\) is true

Interpretation: How likely is it to see a sample proportion of 0.048 or less if the true rate is 0.10?

Courtroom parallel: P-value = strength of prosecution’s evidence

Interpreting the P-Value

P-value \(\approx\) 0.111 means:

“If the consultant truly has the same 10% complication rate as the national average, there is about an 11% chance of observing 3 or fewer complications in 62 surgeries.”

Is this surprising?

• Not extremely rare, but not common either
• We need a standard to make a decision

The Significance Level (\(\alpha\))

To make decisions, we set a significance level (\(\alpha\)) in advance

Common choice: \(\alpha = 0.05\) (5%)

Decision rule:

• If p-value \(< \alpha\): Reject \(H_0\) (result is “statistically significant”)
• If p-value \(\geq \alpha\): Fail to reject \(H_0\) (result is not statistically significant)

Courtroom parallel: \(\alpha\) = standard for “beyond reasonable doubt”

What Does “Fail to Reject” Mean?

Important distinctions:

“Fail to reject” ≠ “Accept”

The consultant might have a lower rate — we just don’t have enough evidence to conclude that

Courtroom parallel: Failed to find enough evidence for the “guilty” verdict

Payday Loan Regulations

• Borrowers use payday loans to get a cash advance before their next payday
• Borrower writes a check for loan amount + service fee
• Lender holds check until borrower’s payday
• Very high APR equivalent (often over 300%)

Question: Does the majority of all payday borrowers in MI support additional regulation that would require payday lenders to do a credit check?

Data:

• payday data set is available here
• Researchers selected a random sample of 830 payday borrowers
  • 443 out 830 (or proportion of 0.534) said they would support a regulation
  • \(\hat{p}=0.534\)
  • Let \(p\) be the true long-run proportion of all MI payday borrowers who support additional regulation

Hypotheses

Guided Practice: Set Up

Step 1: Identify the parameter

• \(p\) = proportion of ALL payday borrowers who support regulations

Step 2: State the hypotheses

• \(H_0: p = 0.50\) (exactly half support)
• \(H_A: p > 0.50\) (more than half support)

Step 3: Calculate sample statistic

• \(\hat{p} = \frac{443}{830} = 0.534\)

Guided Practice: Simulate

Step 4: Simulate 1000 samples under \(H_0\)

• Use the Randomize module in Jamovi

Step 5: Calculate the p-value

• The Randomize module will do this for us

Using Randomize Module

• Open the data set payday.csv if it is not already open

• Go to “Randomize” -> “Single Proportion:Hypothesis Testing”

• Move variable legislation to “Variable” box

• Make sure that your “Test Value” is the same as the null hypothesis value (0.5 in our case)

• Choose the appropriate “Alternative hypothesis” (“Greater” in our case)

• Make the “Number of simulated samples” to be 1000 and press “Enter”

• Your scrren should look like the picture below
• Note that your p-value could be slightly different than on the picture

Guided Practice: Interpret Results

Your conclusions:

1. Decision (using \(\alpha = 0.05\)): Decide whether to reject \(H_0\)

2. Interpretation

• p-value of the test of significance is \(< 0.05\)
  • We reject null hypothesis and rule in the favor of the alternative hypothesis

  In the context of the problem:The data provide statistically significant evidence that MORE than half of all payday borrowers support new legislation

The Hypothesis Testing Framework

Summary of the process:

1. State hypotheses: \(H_0\) and \(H_A\)

2. Collect data: Calculate observed statistic (\(\hat{p}\))

3. Simulate null distribution: What would we see if \(H_0\) were true?

4. Calculate p-value: How extreme is our observation?

5. Make decision: Compare p-value to \(\alpha\)

6. State conclusion: In context of the original question

References

• Introduction to Modern Statistics (2e) textbook by Mine Cetinkaya-Rundel and Johanna Hardin
• Chapter 11, Section 16.1