3. Hypothesis Testing

Math 115 - Examples for Topic 3

Author

Yurk

These worked examples demonstrate the complete hypothesis testing process for a single proportion using randomization. Each example walks through all steps from stating hypotheses to making a conclusion.

Example 1: Reading P-Values from a Histogram

Scenario: A college claims that 80% of its students graduate within 4 years. A researcher suspects the true graduation rate is lower than 80%. She surveys a random sample of 100 students from the college and finds that 72 graduated within 4 years.

Question: Is there convincing evidence that the true 4-year graduation rate is less than 80%? Use a significance level of \(\alpha=0.05\)

Given: Below is a null distribution showing 1000 simulated proportions under the assumption that the true graduation rate is 80%.

Task: Use the histogram above to answer the following questions.

Step 1: Identify the parameter

Let \(p\) = the true proportion of students at this college who graduate within 4 years.

Step 2: State the hypotheses

Since the researcher suspects the rate is lower than 80%, this is a one-sided (lower-tail) test:

  • \(H_0: p = 0.80\) (the college’s claim is correct)
  • \(H_A: p < 0.80\) (the true rate is lower than claimed)

Step 3: Calculate the sample statistic

From the sample:

\[\hat{p} = \frac{72}{100} = 0.72\]

Step 4: Estimate the p-value

The p-value is the proportion of simulated samples (under \(H_0\)) that have a sample proportion at or below our observed value of 0.72.

Looking at the histogram:

  • The red shaded region shows simulations with \(\hat{p} \leq 0.72\)
  • The number above each bar is the count of simulations in that bar, so we can count the tail exactly: add up the counts on the shaded bars

Estimated p-value:

\[\text{P-value} \approx \frac{24}{1000} = 0.024\]

Step 5: Interpret the p-value

Interpretation: If the true graduation rate is 80%, there is approximately a 2.4% chance of observing a sample proportion of 0.72 or lower in a random sample of 100 students.

Step 6: Make a decision

Using the common significance level \(\alpha = 0.05\):

  • P-value = 0.024
  • Compare: Is 0.024 < 0.05? YES

Decision: We reject \(H_0\) at the \(\alpha = 0.05\) level.

Step 7: State the conclusion in context

The data provide convincing evidence that the true 4-year graduation rate at this college is less than 80%. The observed sample proportion of 0.72 is unusually low if the true rate were 80%, occurring in only about 2.4% of random samples under that assumption.

Example 2: Using Jamovi Output

Scenario: A pharmaceutical company claims that a new medication causes side effects in 15% of patients. A medical researcher suspects the true rate might be higher than 15%. She conducts a study with 80 randomly selected patients and finds that 18 experienced side effects.

Question: Is there convincing evidence that the true side effect rate is greater than 15%?

Task: Use Jamovi’s Randomize Module to conduct a hypothesis test at the \(\alpha = 0.05\) level.

Jamovi Setup

To conduct this test in Jamovi using the Randomize module:

Step 1: Prepare the data

Enter the data with a binary variable called side_effect with values yes (experienced side effects) and no (did not experience side effects). You should have 80 observations: 18 with yes and 62 with no.

Step 2: Access the Randomize Module

  1. Click on the Randomize icon in the Analyses tab (you must have the Randomize module installed)
  2. Select Single Proportion - Hypothesis Test from the list

Step 3: Configure the analysis

In the interface, set the following options:

  • Variable: Drag side_effect into the Variable box
  • Test value: Enter 0.15 (this is the null hypothesis value, \(p_0\))
  • Alternative hypothesis: Select Greater than (since \(H_A: p > 0.15\))
  • Number of simulated samples: Start with 100 to visualize the distribution
  • Plot type: Select Dot plot (for 100 samples)
  • Seed the random number generator: (Optional) Check this box and use seed 8675309 for reproducible results

Step 4: Understanding the simulation

The randomization test simulates the null distribution by repeatedly:

  1. Creating a “spinner” with 15% of the area representing “success” (side effects)
  2. Spinning 80 times (once for each patient in the sample)
  3. Calculating the proportion of successes in that simulated sample
  4. Repeating many times to build the null distribution

This shows how the sample proportion would vary if \(H_0\) were true (\(p = 0.15\)).

Step 5: Interpret the visualization

The dot plot shows:

  • Each dot represents the sample proportion from one simulated sample
  • The vertical red dashed line shows your observed proportion (0.225)
  • Red dots are simulated proportions at or above the observed value (counted in p-value for upper-tail test)
  • Black dots are simulated proportions below the observed value

Step 6: Increase precision

After examining the dot plot with 100 simulations:

  • Change Number of simulated samples to 1000 for a more precise p-value estimate
  • Change Plot type to Histogram (easier to visualize with many simulations)

Step 7: Example output with 1000 simulations

After running with 1000 simulations and a histogram plot, Jamovi produces something similar to the following null distribution:

Interpretation of the histogram:

  • Each bar represents a range of simulated sample proportions
  • The red-shaded region shows proportions at or above 0.225 (counted in the p-value)
  • The vertical red dashed line shows the observed proportion
  • Most simulations cluster around 0.15 (the null hypothesis value)

Step 8: Read the Simulation Results table

The Single Proportion - Hypothesis Test analysis produces a table of simulation results similar to the following:

Warning in attr(x, "align"): 'xfun::attr()' is deprecated.
Use 'xfun::attr2()' instead.
See help("Deprecated")
Simulation Results
Reps p-value
1000 0.045
Note: HA: p > 0.15

The table shows:

  • Reps: Number of simulations performed (1000)
  • p-value: Proportion of simulations with \(\hat{p} \geq 0.225\)
  • Note: The alternative hypothesis being tested

For an upper-tail test, the p-value is calculated as: \[\text{p-value} = \frac{\text{Number of simulations with } \hat{p} \geq \text{observed}}{\text{Total number of simulations}}\]

Step 1: Identify the parameter

Let \(p\) = the true proportion of patients who experience side effects from this medication.

Step 2: State the hypotheses

Since the researcher suspects the rate is higher than 15%, this is a one-sided (upper-tail) test:

  • \(H_0: p = 0.15\) (the company’s claim is correct)
  • \(H_A: p > 0.15\) (the true rate is higher than claimed)

Step 3: Calculate the sample statistic

From the sample:

\[\hat{p} = \frac{18}{80} = 0.225\]

Step 4: Obtain the p-value from Jamovi

From the Simulation Results table produced by the Single Proportion - Hypothesis Test analysis:

  • Number of simulations (Reps): 1000
  • P-value = 0.045

This p-value represents the proportion of simulated samples (under \(H_0: p = 0.15\)) that have a sample proportion at or above our observed value of 0.225. You may get a different value, but it will probably be close.

In other words, out of 1000 simulations where we assumed the true side effect rate is 15%, approximately 45 simulations produced a sample proportion as high or higher than what we observed (0.225).

Step 5: Interpret the p-value

Interpretation: If the true side effect rate is 15%, there is approximately a 4.5% chance of observing a sample proportion of 0.225 or higher in a random sample of 80 patients.

Step 6: Make a decision

Using the significance level \(\alpha = 0.05\):

  • P-value = 0.045
  • Compare: Is 0.045 < 0.05? YES

Decision: We reject \(H_0\) at the \(\alpha = 0.05\) level.

Step 7: State the conclusion in context

The data provide convincing evidence that the true side effect rate is greater than 15%. The observed sample proportion of 0.225 is unusually high if the true rate were 15%, occurring in only about 4.5% of random samples under that assumption.

Key Takeaways

Common Steps for All Hypothesis Tests

  1. Identify the parameter (\(p\) = the population proportion)
  2. State hypotheses (\(H_0\) and \(H_A\) based on research question)
  3. Calculate sample statistic (\(\hat{p}\))
  4. Obtain p-value (from simulation, histogram, or software)
  5. Interpret p-value (in context)
  6. Make decision (compare p-value to \(\alpha\))
  7. State conclusion (in context of the problem)

P-Value Interpretation

Remember: The p-value is NOT the probability that \(H_0\) is true.

The p-value is the probability of observing our result (or more extreme) if \(H_0\) were true.

  • Small p-value (< \(\alpha\)) → Reject \(H_0\) → Strong evidence against the null
  • Large p-value (≥ \(\alpha\)) → Fail to reject \(H_0\) → Insufficient evidence
Back to top