7. Two-Sided Tests and Decision Errors

a. Two-Sided Tests

Yurk

Where We’ve Been

Every hypothesis test we’ve done has had a specific direction in mind:

Direction Example From
H_A: p > p_0 (higher) payday loan support > 0.5 6a
H_A: p < p_0 (lower) medical consultant rate < 0.10 3

That’s a real commitment — we picked a direction before looking at the data, and the test only counts evidence in that direction.

Today: what if we don’t have a good reason to pick a direction in advance? (Often, we don’t — and a one-sided test in that case isn’t justified.)

When Direction Is Unknown

Some research questions have no natural direction — the question is simply “is this different from p_0?”

Quick example. A professor switches a course’s final from closed-book to open-book. Under closed-book, 70% of students passed. After the switch:

  • Could go up — students reason more, panic less without memorization stakes
  • Could go down — false security, less prep, time lost flipping pages

Both effects are documented in the pedagogy literature, with no strong prior either way. The honest move — run a two-sided test.

The AI image study coming up next lands in exactly this shape.

Two-Sided Hypotheses

A two-sided (or two-tailed) test:

  • H_0: p = p_0
  • H_A: p \neq p_0 (the parameter is different from p_0 — could be higher or lower)

Default to two-sided unless there’s a strong substantive reason to commit to a direction in advance. “Two-sided” is the more honest choice when both directions would be scientifically interesting.

AI Image Study


Setup: 80 students each view this image and decide — real or AI-generated?

Result: 48 correctly identify it as AI (\hat{p} = 0.6).

Hypotheses: \;H_0: p = 0.5 (pure guessing on a binary choice)  vs.  H_A: p \neq 0.5 (not pure guessing)

AI Image — Check Conditions

Conditions (under H_0, with p_0 = 0.5):

  • Independence: each student responds independently ✓
  • Success-failure (using p_0):
    • Expected successes: n \cdot p_0 = 80 \times 0.5 = 40 \geq 10 ✓
    • Expected failures: n \cdot (1-p_0) = 80 \times 0.5 = 40 \geq 10 ✓

→ Normal model is appropriate. We can compute a Z-score and use tail areas of the standard normal.

AI Image — Compute Z

Standard error (uses p_0 = 0.5):

SE = \sqrt{\frac{p_0(1 - p_0)}{n}} = \sqrt{\frac{0.5 \times 0.5}{80}} \approx 0.0559

Test statistic:

Z = \frac{\hat{p} - p_0}{SE} = \frac{0.6 - 0.5}{0.0559} \approx 1.79

Our observed proportion sits about 1.79 standard errors above p_0.

The Problem with One Tail

If we used a one-sided test (H_A: p > 0.5):

  • p-value = area to the right of Z = 1.79
  • This counts evidence only in the upper direction

But what if students had been worse than guessing — say 32 out of 80 (40%)? That would also be evidence against H_0 — a one-sided test would miss it.

For a two-sided H_A, both directions count as evidence against the null.

Both Tails Count

Z = 1.79 lands in the right tail — with area \approx 0.037.

Under H_0, there are observations in the left tail that are equally unlikely — same area \approx 0.037.

A two-sided p-value counts both unlikely regions. Doubling our tail captures the matching one.

Computing the Two-Sided P-Value

Your observed \hat{p} (or Z) splits the null distribution into two pieces. Take the smaller area, and double it.

\text{two-sided p-value} \;=\; 2 \cdot (\text{smaller area})

AI Image — Conclusion

The smaller area beyond Z = 1.79 is \approx 0.037, so:

\text{two-sided p-value} \;=\; 2 \times 0.037 \;=\; 0.074

Decision: 0.074 > \alpha = 0.05 → fail to reject H_0.

We don’t have convincing evidence that students identify this AI-generated image at a rate different from fifty percent.

The observed 60% accuracy looks higher than 50%, but on a sample of just 80 students it could plausibly happen by chance alone.

One-Sided or Two-Sided?

For each research question, write the hypotheses. One-sided (p > p_0 or p < p_0) or two-sided (p \neq p_0)?

  1. “Is the proportion of Hope students who eat breakfast different from the national rate of 60%?” Two-sided. H_0: p = 0.60  vs  H_A: p \neq 0.60. The word “different” — no direction specified.

  2. “Has the new study-skills program improved the proportion of students passing Calc I above the historical 75%?” One-sided. H_0: p = 0.75  vs  H_A: p > 0.75. We’re only interested if the rate went up.

  3. “Is the coin unfair?” Two-sided. H_0: p = 0.5  vs  H_A: p \neq 0.5. “Unfair” could mean either too many heads or too few.

Compute a Two-Sided P-Value

A national poll claims 40% of likely voters are still undecided about a major upcoming election. In a fresh random sample of n = 200 voters in your local district, 68 say they’re undecided (\hat{p} = 0.34). Does the local rate differ from the national figure? Use \alpha = 0.05

  1. Hypotheses? H_0: p = 0.40   vs   H_A: p \neq 0.40

  2. SE and Z. Using p_0 = 0.40:   SE = \sqrt{\frac{0.40 \times 0.60}{200}} \approx 0.0346.   Z = \frac{0.34 - 0.40}{0.0346} \approx -1.73.

  3. Two-sided p-value. Left-tail area beyond Z = -1.73 is about 0.042, so p-value = 2 \times 0.042 = 0.084.

  4. Decision and conclusion. 0.084 > 0.05 → fail to reject H_0. We don’t have strong evidence the true undecided rate differs from 40%. (A one-sided test in the lower direction would have been significant — that’s exactly why we don’t get to pick the side after seeing the data.)

Looking Ahead

We’ve now seen one-sided and two-sided tests, and we’ve used \alpha = 0.05 as our cutoff for “reject vs fail to reject.”

But what does that 0.05 actually cost us when we’re wrong?

Next time (7b): the two ways a hypothesis test can be wrong, the trade-off between them, and how confidence intervals connect to two-sided tests.

References

AI Image Detection Research:

Course Home