5. Scope of Inference

b. Random Sampling

Yurk

Random Sampling Enables Generalization

Topics 3 and 4 used samples to learn about populations — that’s inference. This lecture asks: when does inference actually work?

A representative sample mirrors the population. Random sampling is the reliable way to get there: it removes systematic differences between sample and population (on average).

Convenience samples, by contrast, typically don’t: easy-to-reach individuals may differ systematically from the rest, introducing sampling bias.

Result: random sampling enables generalization; convenience sampling does not.

Hemlock Woolly Adelgid

Two groups study adelgid presence on hemlock trees at Hope College Nature Preserve.

Group A (Convenience): walk the trails, inspect hemlocks visible from the trail.

Group B (Random): random number generator selects 50 from a list of all hemlocks.

Which group can generalize their findings to all hemlocks?

Group B. Random selection gives a sample that mirrors the population’s mix of locations, ages, and conditions. Group A only sees trail-side trees — not a representative slice of the forest.

Simple Random Sample (SRS)

What Group B did has a name.

Simple Random Sample (SRS): Every individual in the population has an equal and independent chance of being selected.

  • Equal: same selection probability for every individual
  • Independent: picking one doesn’t change another’s chances

Implementing an SRS (Hemlock study example):

  1. Create list of all tree IDs (sampling frame)
  2. Decide sample size (e.g., 50 trees)
  3. Use random number generator to select sample from list

Quality Control

A vitamin manufacturer’s standard: if the true defect rate exceeds 4%, the batch is out of spec and shouldn’t ship.

A QC inspector takes an SRS of 50 tablets from a recent batch of ~50,000. They find 6 defective (\hat{p} = 6/50 = 0.12).

Question: Is there evidence the batch’s true defect rate exceeds 4%?

H_0: p = 0.04

H_a: p > 0.04

Hypothesis Test

If H_0: p = 0.04 were true, what would \hat{p} from a random sample of 50 typically look like? Simulate 1,000 such samples.

Of 1,000 simulated samples, only 15 had \hat{p} \geq 0.12. Simulation p-value ≈ 0.015 — the probability of seeing data this extreme (or more) if H_0 were true.

QC: Does the Conclusion Generalize?

Conclusion. Reject H_0 at \alpha = 0.05. There is statistically significant evidence that the batch’s true defect rate exceeds 4% (\hat{p} = 0.12, p-value \approx 0.015). The factory should investigate.

Where does this conclusion apply?

  • The 50 tablets were an SRS from this batch (~50,000 tablets) → conclusion generalizes to the entire batch.
  • Does not generalize to: future batches, batches from other facilities, or batches under different operating conditions.

Identify the Sampling Design

For each scenario, decide: Is this a simple random sample? To what population (if any) can findings generalize?

(a) A nutritionist surveys the first 100 students entering the dining hall at lunch about their breakfast habits. (a) Not an SRS — convenience sample (whoever shows up at lunch). Generalizes at best to lunchtime dining-hall visitors, not all students; likely sampling bias (skips students who skip lunch, eat off-campus, etc.).

(b) A forestry team numbers every tree in a 40-acre plot 1–8,420 and uses a random number generator to pick 100 trees to measure for height. (b) SRS. Sampling frame is all 8,420 trees; every tree has equal and independent chance. Generalizes to the entire 40-acre plot.

(c) A study posts a link on Reddit asking sleep-deprived adults to fill out a survey about caffeine use. (c) Not an SRS — voluntary response from self-selected Reddit users. Doesn’t generalize to “sleep-deprived adults” broadly; those who chose to respond likely differ systematically.

Invasive Plant Survey

A botanist wants to estimate what fraction of survey patches in a nature preserve contain garlic mustard, an invasive plant.

  • The preserve is divided into 1,500 patches of equal area
  • The team uses an SRS to select 30 patches
  • Each patch is walked thoroughly and checked for garlic mustard
  • After surveys: garlic mustard found in 8 of the 30 patches (\hat{p} = 8/30 \approx 0.267)

Question: What’s a 95% CI for the true proportion of patches with garlic mustard?

95% Confidence Interval

Treat the 30-patch sample as a stand-in for the population (all 1,500 patches). Resample 30 patches with replacement 1,000 times — each resample gives a \hat{p}_{boot}.

95% bootstrap CI: the 2.5th and 97.5th percentiles → (0.13, 0.43).

Survey: Does the Conclusion Generalize?

Conclusion. \hat{p} \approx 0.27, 95% CI [0.13, 0.43]. We are 95% confident the true proportion of patches with garlic mustard is between roughly 13% and 43%.

Where does this conclusion apply?

  • The 30 patches were an SRS from the preserve’s 1,500 patches → CI generalizes to the entire preserve at the time of the survey.
  • Does not generalize to: other preserves, other invasive species, or other seasons.

Random Samples Are Representative

A representative sample mirrors the population. SRS gets us there.

Why not convenience sampling?

  • QC. Sampling the first 50 tablets could miss an early-run process glitch.
  • Plant survey. Sampling patches near the road could mean atypical light conditions.

SRS sidesteps both by giving every individual equal chance — no subgroup systematically favored. That’s how we get from sample to population.

References

Course Home