Math 115: Introduction to Statistics – 5. Scope of Inference

Random Sampling Enables Generalization

Topics 3 and 4 used samples to learn about populations — that’s inference. This lecture asks: when does inference actually work?

A representative sample mirrors the population. Random sampling is the reliable way to get there: it removes systematic differences between sample and population (on average).

Convenience samples, by contrast, typically don’t: easy-to-reach individuals may differ systematically from the rest, introducing sampling bias.

Result: random sampling enables generalization; convenience sampling does not.

Hemlock Woolly Adelgid

Two groups study adelgid presence on hemlock trees at Hope College Nature Preserve.

Group A (Convenience): walk the trails, inspect hemlocks visible from the trail.

Group B (Random): random number generator selects 50 from a list of all hemlocks.

Which group can generalize their findings to all hemlocks?

Group B. Random selection gives a sample that mirrors the population’s mix of locations, ages, and conditions. Group A only sees trail-side trees — not a representative slice of the forest.

The cleanest way to see why sampling design matters is to set two reasonable-looking studies side by side and ask which one’s findings actually apply to the whole forest. Two student groups want to study adelgid presence on hemlocks at the Hope College Nature Preserve. Group A walks the trails and inspects the hemlocks visible from the path. Group B uses a random number generator to pick fifty trees from a complete list of every hemlock in the preserve. Take a moment to decide which group’s findings can generalize to all the hemlocks in the preserve before advancing. Group B is the one whose findings you can extend to the whole preserve. Random selection gives a sample that mirrors the population’s mix of locations, ages, and tree conditions — a representative slice. Group A only sees the trees visible from the trail, leaving out everything growing deeper in the forest — and trail-side trees could very well differ systematically from off-trail trees in light, moisture, or how recently the trail was cleared. So Group A’s findings describe trail-visible hemlocks, not the whole preserve — even though both groups walked the same forest with equal effort.

Simple Random Sample (SRS)

What Group B did has a name.

Simple Random Sample (SRS): Every individual in the population has an equal and independent chance of being selected.

Equal: same selection probability for every individual
Independent: picking one doesn’t change another’s chances

Implementing an SRS (Hemlock study example):

Create list of all tree IDs (sampling frame)
Decide sample size (e.g., 50 trees)
Use random number generator to select sample from list

Quality Control

A vitamin manufacturer’s standard: if the true defect rate exceeds 4%, the batch is out of spec and shouldn’t ship.

A QC inspector takes an SRS of 50 tablets from a recent batch of ~50,000. They find 6 defective (\(\hat{p} = 6/50 = 0.12\)).

Question: Is there evidence the batch’s true defect rate exceeds 4%?

\(H_0: p = 0.04\)

\(H_a: p > 0.04\)

Hypothesis Test

If \(H_0: p = 0.04\) were true, what would \(\hat{p}\) from a random sample of 50 typically look like? Simulate 1,000 such samples.

Of 1,000 simulated samples, only 15 had \(\hat{p} \geq 0.12\). Simulation \(p\)-value ≈ 0.015 — the probability of seeing data this extreme (or more) if \(H_0\) were true.

How rare would a sample proportion of 12 percent be if the batch’s true defect rate really were the 4 percent threshold? That’s what the null distribution shows us — one thousand simulated samples of fifty tablets, each drawn from a population where the defect rate is exactly four percent. Back in three A we did this kind of simulation with a spinner showing four percent orange; here we use a random number generator. The distribution is centered near four percent because that’s the value we forced into the simulation — every sample is drawn from a population with p equals four percent, so on average that’s what each p hat lands near. Our observed twelve percent sits well into the right tail: out of those one thousand simulated samples, only about fifteen produced p hat that extreme or higher — giving a simulation p-value of about zero point zero one five.

QC: Does the Conclusion Generalize?

Conclusion. Reject \(H_0\) at \(\alpha = 0.05\). There is statistically significant evidence that the batch’s true defect rate exceeds 4% (\(\hat{p} = 0.12\), p-value \(\approx\) 0.015). The factory should investigate.

Where does this conclusion apply?

The 50 tablets were an SRS from this batch (~50,000 tablets) → conclusion generalizes to the entire batch.
Does not generalize to: future batches, batches from other facilities, or batches under different operating conditions.

Identify the Sampling Design

For each scenario, decide: Is this a simple random sample? To what population (if any) can findings generalize?

(a) A nutritionist surveys the first 100 students entering the dining hall at lunch about their breakfast habits. (a) Not an SRS — convenience sample (whoever shows up at lunch). Generalizes at best to lunchtime dining-hall visitors, not all students; likely sampling bias (skips students who skip lunch, eat off-campus, etc.).

(b) A forestry team numbers every tree in a 40-acre plot 1–8,420 and uses a random number generator to pick 100 trees to measure for height. (b) SRS. Sampling frame is all 8,420 trees; every tree has equal and independent chance. Generalizes to the entire 40-acre plot.

(c) A study posts a link on Reddit asking sleep-deprived adults to fill out a survey about caffeine use. (c) Not an SRS — voluntary response from self-selected Reddit users. Doesn’t generalize to “sleep-deprived adults” broadly; those who chose to respond likely differ systematically.

Your turn. Three sampling scenarios — for each one, decide whether it is a simple random sample, and to what population (if any) findings could generalize. Take a moment with each before advancing. Advance when you are ready to see the answers. Scenario A — the nutritionist surveying the first one hundred students entering the dining hall at lunch. This is not an SRS — it is a convenience sample of whoever happened to walk in. It might generalize to lunchtime dining-hall visitors, but not to all students; students who skip lunch or eat off-campus are systematically left out. Scenario B — the forestry team numbering every tree and using a random number generator to pick one hundred. This is a simple random sample. The sampling frame is all eight thousand four hundred twenty trees, and each one has an equal and independent chance of being selected. Findings generalize to the entire forty-acre plot. Scenario C — the Reddit-recruited survey on caffeine use. This is not an SRS — it is voluntary response from a self-selected group. Findings will not generalize to sleep-deprived adults broadly; people who choose to respond to a Reddit survey likely differ from those who do not.

Invasive Plant Survey

A botanist wants to estimate what fraction of survey patches in a nature preserve contain garlic mustard, an invasive plant.

The preserve is divided into 1,500 patches of equal area
The team uses an SRS to select 30 patches
Each patch is walked thoroughly and checked for garlic mustard
After surveys: garlic mustard found in 8 of the 30 patches (\(\hat{p} = 8/30 \approx 0.267\))

Question: What’s a 95% CI for the true proportion of patches with garlic mustard?

95% Confidence Interval

Treat the 30-patch sample as a stand-in for the population (all 1,500 patches). Resample 30 patches with replacement 1,000 times — each resample gives a \(\hat{p}_{boot}\).

95% bootstrap CI: the 2.5th and 97.5th percentiles → (0.13, 0.43).

Survey: Does the Conclusion Generalize?

Conclusion. \(\hat{p} \approx 0.27\), 95% CI [0.13, 0.43]. We are 95% confident the true proportion of patches with garlic mustard is between roughly 13% and 43%.

Where does this conclusion apply?

The 30 patches were an SRS from the preserve’s 1,500 patches → CI generalizes to the entire preserve at the time of the survey.
Does not generalize to: other preserves, other invasive species, or other seasons.

Random Samples Are Representative

A representative sample mirrors the population. SRS gets us there.

Why not convenience sampling?

QC. Sampling the first 50 tablets could miss an early-run process glitch.
Plant survey. Sampling patches near the road could mean atypical light conditions.

SRS sidesteps both by giving every individual equal chance — no subgroup systematically favored. That’s how we get from sample to population.

References

Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin (Section 2.1.4)