Math 115: Introduction to Statistics – 7. Two-Sided Tests and Decision Errors

Where We’ve Been

Every hypothesis test we’ve done has had a specific direction in mind:

Direction	Example	From
\(H_A: p > p_0\) (higher)	payday loan support \(> 0.5\)	6a
\(H_A: p < p_0\) (lower)	medical consultant rate \(< 0.10\)	3

That’s a real commitment — we picked a direction before looking at the data, and the test only counts evidence in that direction.

Today: what if we don’t have a good reason to pick a direction in advance? (Often, we don’t — and a one-sided test in that case isn’t justified.)

When Direction Is Unknown

Some research questions have no natural direction — the question is simply “is this different from \(p_0\)?”

Quick example. A professor switches a course’s final from closed-book to open-book. Under closed-book, 70% of students passed. After the switch:

Could go up — students reason more, panic less without memorization stakes
Could go down — false security, less prep, time lost flipping pages

Both effects are documented in the pedagogy literature, with no strong prior either way. The honest move — run a two-sided test.

The AI image study coming up next lands in exactly this shape.

Two-Sided Hypotheses

A two-sided (or two-tailed) test:

\(H_0: p = p_0\)
\(H_A: p \neq p_0\) (the parameter is different from \(p_0\) — could be higher or lower)

Default to two-sided unless there’s a strong substantive reason to commit to a direction in advance. “Two-sided” is the more honest choice when both directions would be scientifically interesting.

AI Image Study

Setup: 80 students each view this image and decide — real or AI-generated?

Result: 48 correctly identify it as AI (\(\hat{p} = 0.6\)).

Hypotheses: \(\;H_0: p = 0.5\) (pure guessing on a binary choice) vs. \(H_A: p \neq 0.5\) (not pure guessing)

Can people tell an AI-generated image from a real one? In our worked example, eighty students each see this image and decide — real, or AI? Three things could happen. If they truly can’t tell, they’re guessing — and on a binary choice that puts the rate around fifty percent. If the image is convincing enough to fool them, the rate drops below fifty. If it has tells they can spot, the rate climbs above fifty. So a departure in either direction is informative — which makes this a two-sided test. The null says the true detection rate equals fifty percent — what pure guessing on a binary choice would produce. The alternative says p is not equal to zero point five, with no committed direction. In our data, forty-eight got it right — sixty percent. Is that meaningfully different from fifty percent, or just luck on a sample this small? That’s what we’ll test.

AI Image — Check Conditions

Conditions (under \(H_0\), with \(p_0 = 0.5\)):

Independence: each student responds independently ✓
Success-failure (using \(p_0\)):
- Expected successes: \(n \cdot p_0 = 80 \times 0.5 = 40 \geq 10\) ✓
- Expected failures: \(n \cdot (1-p_0) = 80 \times 0.5 = 40 \geq 10\) ✓

→ Normal model is appropriate. We can compute a Z-score and use tail areas of the standard normal.

AI Image — Compute Z

Standard error (uses \(p_0 = 0.5\)):

\[SE = \sqrt{\frac{p_0(1 - p_0)}{n}} = \sqrt{\frac{0.5 \times 0.5}{80}} \approx 0.0559\]

Test statistic:

\[Z = \frac{\hat{p} - p_0}{SE} = \frac{0.6 - 0.5}{0.0559} \approx 1.79\]

Our observed proportion sits about \(1.79\) standard errors above \(p_0\).

The Problem with One Tail

If we used a one-sided test (\(H_A: p > 0.5\)):

p-value = area to the right of \(Z = 1.79\)
This counts evidence only in the upper direction

But what if students had been worse than guessing — say 32 out of 80 (40%)? That would also be evidence against \(H_0\) — a one-sided test would miss it.

For a two-sided \(H_A\), both directions count as evidence against the null.

Both Tails Count

\(Z = 1.79\) lands in the right tail — with area \(\approx 0.037\).

Under \(H_0\), there are observations in the left tail that are equally unlikely — same area \(\approx 0.037\).

A two-sided p-value counts both unlikely regions. Doubling our tail captures the matching one.

Computing the Two-Sided P-Value

Your observed \(\hat{p}\) (or \(Z\)) splits the null distribution into two pieces. Take the smaller area, and double it.

\[\text{two-sided p-value} \;=\; 2 \cdot (\text{smaller area})\]

AI Image — Conclusion

The smaller area beyond \(Z = 1.79\) is \(\approx 0.037\), so:

\[\text{two-sided p-value} \;=\; 2 \times 0.037 \;=\; 0.074\]

Decision: \(0.074 > \alpha = 0.05\) → fail to reject \(H_0\).

We don’t have convincing evidence that students identify this AI-generated image at a rate different from fifty percent.

The observed 60% accuracy looks higher than 50%, but on a sample of just 80 students it could plausibly happen by chance alone.

One-Sided or Two-Sided?

For each research question, write the hypotheses. One-sided (\(p > p_0\) or \(p < p_0\)) or two-sided (\(p \neq p_0\))?

“Is the proportion of Hope students who eat breakfast different from the national rate of 60%?” Two-sided. \(H_0: p = 0.60\) vs \(H_A: p \neq 0.60\). The word “different” — no direction specified.
“Has the new study-skills program improved the proportion of students passing Calc I above the historical 75%?” One-sided. \(H_0: p = 0.75\) vs \(H_A: p > 0.75\). We’re only interested if the rate went up.
“Is the coin unfair?” Two-sided. \(H_0: p = 0.5\) vs \(H_A: p \neq 0.5\). “Unfair” could mean either too many heads or too few.

Your turn. For each research question, you decide — is the alternative one-sided or two-sided? Pause and write your hypotheses down before clicking through. First — Hope students eating breakfast, different from the national rate of sixty percent. The keyword “different” doesn’t pick a direction — so this one is two-sided. Alternative is p differs from zero point six. Next — has the new study-skills program improved the proportion passing Calculus One above seventy-five percent? “Improved” is directional — we only care if the rate went up — so this is one-sided. Alternative is p greater than zero point seven five. Last — is the coin unfair? Unfair could mean too many heads or too few — both directions are scientifically interesting — so two-sided. Alternative is p differs from zero point five.

Compute a Two-Sided P-Value

A national poll claims 40% of likely voters are still undecided about a major upcoming election. In a fresh random sample of \(n = 200\) voters in your local district, 68 say they’re undecided (\(\hat{p} = 0.34\)). Does the local rate differ from the national figure? Use \(\alpha = 0.05\)

Hypotheses? \(H_0: p = 0.40\) vs \(H_A: p \neq 0.40\)
SE and \(Z\). Using \(p_0 = 0.40\): \(SE = \sqrt{\frac{0.40 \times 0.60}{200}} \approx 0.0346\). \(Z = \frac{0.34 - 0.40}{0.0346} \approx -1.73\).
Two-sided p-value. Left-tail area beyond \(Z = -1.73\) is about \(0.042\), so p-value \(= 2 \times 0.042 = 0.084\).
Decision and conclusion. \(0.084 > 0.05\) → fail to reject \(H_0\). We don’t have strong evidence the true undecided rate differs from 40%. (A one-sided test in the lower direction would have been significant — that’s exactly why we don’t get to pick the side after seeing the data.)

Your turn — a two-sided test from scratch. The setup — a national poll claims forty percent of voters are still undecided about an upcoming election. You take a fresh random sample in your local district. Out of two hundred voters, sixty-eight say they’re undecided. That gives a p hat of zero point three four. The question — does your local rate differ from the national forty percent, at alpha equal to zero point zero five? Try all four parts on your own before clicking through. Hypotheses first. Two-sided — null is p equals zero point four, alternative is p differs from zero point four. Standard error and Z. SE uses p naught — square root of zero point four times zero point six over two hundred — which is about zero point zero three five. Z is the difference between zero point three four and zero point four, divided by that SE — and that comes out to negative one point seven three. Below the null by about one point seven standard errors. Two-sided p-value. Z splits the standard normal — the smaller piece is the left tail beyond negative one point seven three, with area about zero point zero four two. Double the smaller area — about zero point zero eight four. Decision and conclusion. Zero point zero eight four is bigger than zero point zero five — so we fail to reject. We don’t have strong evidence the local undecided rate differs from forty percent. Now here’s the kicker — if we’d peeked at the data first and run a one-sided test pointed down, the p-value would have been zero point zero four two, which is significant. That’s exactly why we commit to direction in advance — picking the side after seeing the data unfairly inflates our chance of declaring a finding.

Looking Ahead

We’ve now seen one-sided and two-sided tests, and we’ve used \(\alpha = 0.05\) as our cutoff for “reject vs fail to reject.”

But what does that 0.05 actually cost us when we’re wrong?

Next time (7b): the two ways a hypothesis test can be wrong, the trade-off between them, and how confidence intervals connect to two-sided tests.

References

Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
Chapter 14

AI Image Detection Research:

How good are humans at detecting AI-generated images? - arXiv, July 2025
As Good as a Coin Toss - Communications of the ACM, September 2025