17. Simple Linear Regression

a. The Linear Model
Math 115

Yurk

A New Kind of Question

So far, our explanatory variables have always been categorical — they sorted observations into groups:

  • Topic 12: birth weight by smoker / nonsmoker
  • Topic 15: vocabulary score by social class (4 groups)

Today both variables are numerical — height (cm) and weight (kg). No groups to compare, and two new questions open up:

  • How does one numerical variable relate to another?
  • Can we predict one from the other?

Scatterplots

A scatterplot plots each observation as a point in the (x, y) plane.

Data: body measurements from 507 physically active adults (Heinz et al., 2003; bdims in openintro).

Each point is a single case — here, one person.

Positive and Negative Association

  • Positive association: higher x goes with higher y (taller → heavier in this sample).
  • Negative association: higher x goes with lower y.

Height and weight show a clear positive association.

Fitting a Line

The cloud of points sits roughly along a straight line. The line of best fit summarizes the trend.

For these data, the blue line is

\widehat{wgt} = -105 + 1.02 \times hgt

That one equation lets us describe the trend and predict y from x. How the “best” line is chosen (least squares) is for 17b. Today: how to read the line.

The Linear Model Equation

Our fitted line

\widehat{wgt} = -105 + 1.02 \times hgt

is one instance of a general form:

\hat{y} = b_0 + b_1 \, x

  • \hat{y} — the predicted value of y at this x (the “hat” marks a prediction)
  • b_0 — the intercept (predicted y when x = 0)
  • b_1 — the slope (change in \hat{y} for each one-unit increase in x)

Try it — why write \hat{y} rather than plain y?

y is the weight a person in the data actually has. \hat{y} is the weight the line predicts at that height. For almost everyone the two differ.

Statistics and Parameters

b_0 and b_1 are computed from the sample — they’re statistics.

Every statistic we’ve met has a population counterpart:

Sample statistic Population parameter
\hat{p} p
\bar{x} \mu
b_0, b_1 \beta_0, \beta_1

\beta_0 and \beta_1 are the true intercept and slope we’d get if we could see the whole population. We’ll do inference on \beta_1 in topic 18.

Variable Roles

Role Symbol Other names Example
Response y outcome, dependent weight
Predictor x explanatory, independent height

The response is what we’re trying to predict; the predictor is what we use to predict.

Making Predictions

Plug a value of x into \hat{y} = -105 + 1.02 \, x:

Predict weight for someone who is 170 cm tall:

\widehat{wgt} = -105 + 1.02 \times 170 = 68.4 \text{ kg}

Now you: predict weight for someone who is 180 cm tall.

\widehat{wgt} = -105 + 1.02 \times 180 = 78.6 kg

The 10 cm difference in x produced a 10.2 kg difference in \hat{y} — that’s the slope times 10, by design.

Interpreting the Slope

Template (memorize this):

For each additional [unit of x], we expect [y] to [increase / decrease] by [|b_1|] [units of y], on average.

Try it — fill in the template for our model (b_1 = 1.02):

For each additional centimeter of height, we expect weight to increase by 1.02 kg, on average.

Two cautions:

  • “on average” — individuals scatter around the line; the slope describes the trend, not any one person.
  • “we expect weight to increase” describes what the data do, not what height does to weight. These are observational data — heights were measured, not assigned. Claiming that height causes the extra weight would take a randomized experiment.

Interpreting the Intercept — and When Not To

Intercept (b_0 = -105):

The predicted weight for someone whose height is 0 cm is about -105 kg.

That’s not a meaningful person. A 0 cm height is nonsensical, and the model predicts negative weight there. So the intercept here is not interpretable as a prediction — it’s mostly a positioning parameter: it sets the line’s vertical placement so it passes through the data cloud at realistic heights.

Intercept is interpretable only when x = 0 is meaningful — for example, if x were “hours studied,” b_0 would be the predicted score with no studying.

Extrapolation

Extrapolating — predicting outside the range of observed x — can be misleading.

  • Our heights span \approx 147 to 198 cm. At 210 cm the line still returns a number — nothing in the data backs it up.
  • At 5 cm the line says -99.9 kg, which is nonsense.

Rule of thumb: only trust predictions within (or very near) the range of observed x values.

Predict and Interpret

Nutrition facts for 77 Starbucks food items — carbohydrates (g) predicting calories.

\widehat{calories} = 146 + 4.3 \times carb

  1. Predict the calories for an item with 40 g of carbohydrate. 146 + 4.3 \times 40 = 318 calories.

  2. Interpret the slope in context. For each additional gram of carbohydrate, we expect calories to increase by about 4.3, on average.

Is This Prediction Safe?

  1. A friend uses the same line for an item with 100 g of carbohydrate and gets 576 calories. What’s the problem? The calculation is right, but the largest carbohydrate value in the data is 80 g — 100 g is extrapolation. The line keeps going, but nothing out there tells us the pattern holds.

Looking Ahead

We’ve built the linear model and learned to read it — slope, intercept, predictions, and the limits of extrapolation.

Next time (17b): how the line is actually chosen (least squares), how to measure how strong the linear relationship is (the correlation coefficient r and R^2), and how to check whether a line is even the right tool (residual plots).

References

  • Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
  • Chapter 7
  • Heinz, G. et al. (2003). Exploring relationships in body dimensions. Journal of Statistics Education, 11(2). (bdims in openintro.)
Course Home