Math 115: Introduction to Statistics – 17. Simple Linear Regression

A New Kind of Question

So far, our explanatory variables have always been categorical — they sorted observations into groups:

Topic 12: birth weight by smoker / nonsmoker
Topic 15: vocabulary score by social class (4 groups)

Today both variables are numerical — height (cm) and weight (kg). No groups to compare, and two new questions open up:

How does one numerical variable relate to another?
Can we predict one from the other?

Scatterplots

A scatterplot plots each observation as a point in the \((x, y)\) plane.

Data: body measurements from 507 physically active adults (Heinz et al., 2003; bdims in openintro).

Each point is a single case — here, one person.

Positive and Negative Association

Positive association: higher \(x\) goes with higher \(y\) (taller → heavier in this sample).
Negative association: higher \(x\) goes with lower \(y\).

Height and weight show a clear positive association.

Fitting a Line

The cloud of points sits roughly along a straight line. The line of best fit summarizes the trend.

For these data, the blue line is

\[\widehat{wgt} = -105 + 1.02 \times hgt\]

That one equation lets us describe the trend and predict \(y\) from \(x\). How the “best” line is chosen (least squares) is for 17b. Today: how to read the line.

The Linear Model Equation

Our fitted line

\[\widehat{wgt} = -105 + 1.02 \times hgt\]

is one instance of a general form:

\[\hat{y} = b_0 + b_1 \, x\]

\(\hat{y}\) — the predicted value of \(y\) at this \(x\) (the “hat” marks a prediction)
\(b_0\) — the intercept (predicted \(y\) when \(x = 0\))
\(b_1\) — the slope (change in \(\hat{y}\) for each one-unit increase in \(x\))

Try it — why write \(\hat{y}\) rather than plain \(y\)?

\(y\) is the weight a person in the data actually has. \(\hat{y}\) is the weight the line predicts at that height. For almost everyone the two differ.

The line we just fit — predicted weight equals minus one hundred five plus one point oh two times height — is one instance of a general form that every simple linear model shares: y hat equals b naught plus b one times x. There are three pieces of notation to keep track of. Y hat — the “hat” marks it as a prediction. B naught is the intercept — the predicted y-value when x equals zero, geometrically where the line crosses the y-axis. B one is the slope — the change in predicted y for each one-unit increase in x. Before we go on, answer the question on the slide: why bother writing y hat instead of plain y? Because they are different numbers. Plain y is the weight a person in our data actually has. Y hat is the weight the line predicts for someone at that height. For almost every person in the dataset the two differ.

Statistics and Parameters

\(b_0\) and \(b_1\) are computed from the sample — they’re statistics.

Every statistic we’ve met has a population counterpart:

Sample statistic	Population parameter
\(\hat{p}\)	\(p\)
\(\bar{x}\)	\(\mu\)
\(b_0\), \(b_1\)	\(\beta_0\), \(\beta_1\)

\(\beta_0\) and \(\beta_1\) are the true intercept and slope we’d get if we could see the whole population. We’ll do inference on \(\beta_1\) in topic 18.

Variable Roles

Role	Symbol	Other names	Example
Response	\(y\)	outcome, dependent	weight
Predictor	\(x\)	explanatory, independent	height

The response is what we’re trying to predict; the predictor is what we use to predict.

Making Predictions

Plug a value of \(x\) into \(\hat{y} = -105 + 1.02 \, x\):

Predict weight for someone who is 170 cm tall:

\[\widehat{wgt} = -105 + 1.02 \times 170 = 68.4 \text{ kg}\]

Now you: predict weight for someone who is 180 cm tall.

\(\widehat{wgt} = -105 + 1.02 \times 180 = 78.6\) kg

The 10 cm difference in \(x\) produced a \(10.2\) kg difference in \(\hat{y}\) — that’s the slope times 10, by design.

Making a prediction is just plugging x into the equation. For someone one hundred seventy centimeters tall — about five foot seven — the predicted weight is minus one hundred five plus one point oh two times one seventy, which works out to about sixty-eight point four kilograms. Now take the second one yourself: what does the model predict for someone one hundred eighty centimeters tall, about five foot eleven? Minus one hundred five plus one point oh two times one eighty is about seventy-eight point six kilograms. Two things are worth noticing. First, the ten-centimeter difference in height showed up as a ten-point-two kilogram difference in predicted weight — that’s ten times the slope of one point oh two. So the slope tells you the rate of change of the prediction. Second, these are predictions for an average person at that height, not predictions about any specific individual — individuals scatter around the line.

Interpreting the Slope

Template (memorize this):

For each additional [unit of \(x\)], we expect [\(y\)] to [increase / decrease] by [\(|b_1|\)] [units of \(y\)], on average.

Try it — fill in the template for our model (\(b_1 = 1.02\)):

For each additional centimeter of height, we expect weight to increase by 1.02 kg, on average.

Two cautions:

“on average” — individuals scatter around the line; the slope describes the trend, not any one person.
“we expect weight to increase” describes what the data do, not what height does to weight. These are observational data — heights were measured, not assigned. Claiming that height causes the extra weight would take a randomized experiment.

The slope is the most important number in the linear model. Memorize this template for the interpretation: for each additional unit of x, we expect y to increase or decrease by the slope amount, on average. Pause and fill it in for our model, where the slope is about one point oh two. For each additional centimeter of height, we expect weight to increase by about one point oh two kilograms, on average. Two cautions come with that sentence. First, “on average” — the slope is about the typical case, not any individual. Two people the same height can weigh very different amounts; the slope describes the trend across the dataset, not a deterministic rule. Second, the phrase “we expect weight to increase” describes what the data do, not what height does to weight. These are observational data — the people in this study had their heights measured, not assigned. To claim that height causes the extra weight, we would need a randomized experiment.

Interpreting the Intercept — and When Not To

Intercept (\(b_0 = -105\)):

The predicted weight for someone whose height is 0 cm is about \(-105\) kg.

That’s not a meaningful person. A 0 cm height is nonsensical, and the model predicts negative weight there. So the intercept here is not interpretable as a prediction — it’s mostly a positioning parameter: it sets the line’s vertical placement so it passes through the data cloud at realistic heights.

Intercept is interpretable only when \(x = 0\) is meaningful — for example, if \(x\) were “hours studied,” \(b_0\) would be the predicted score with no studying.

The intercept is the predicted y when x equals zero. For our model, that’s minus one hundred five kilograms — a negative weight, predicted for someone zero centimeters tall. Both pieces are nonsensical: no one is zero centimeters tall, and weight can’t be negative. So the intercept here doesn’t carry a meaningful real-world interpretation. What does it do, then? It positions the line vertically — it sets where the line passes through the y-axis so the overall fit lines up with the data cloud at realistic heights. The intercept is interpretable as a prediction only when x equals zero is a meaningful value. For instance, if x were hours studied and y were exam score, the intercept would meaningfully predict the score for someone who studied zero hours — that’s a real possibility.

Extrapolation

Extrapolating — predicting outside the range of observed \(x\) — can be misleading.

Our heights span \(\approx 147\) to \(198\) cm. At 210 cm the line still returns a number — nothing in the data backs it up.
At 5 cm the line says \(-99.9\) kg, which is nonsense.

Rule of thumb: only trust predictions within (or very near) the range of observed \(x\) values.

The line is fit to data with x-values in a specific range — for our body data, heights from about one hundred forty-seven to one hundred ninety-eight centimeters. Within that range, the line is a sensible summary of the trend. The shaded band on the right of the plot is where we have no data at all. Extrapolating into it — predicting at heights we never observed — is risky. The linear pattern might continue, or might bend, or might disappear. We don’t know, because there’s nothing out there to check against. The orange point marks a prediction at two hundred ten centimeters. It sits only a little past the data, so the line may still be roughly right — but the data can’t confirm it. A five centimeter prediction is absurd: the line returns a negative weight, and no adult is anywhere near that height. The honest rule of thumb: trust predictions only within the range of observed x-values, or very close to that range.

Predict and Interpret

Nutrition facts for 77 Starbucks food items — carbohydrates (g) predicting calories.

\[\widehat{calories} = 146 + 4.3 \times carb\]

Predict the calories for an item with 40 g of carbohydrate. \(146 + 4.3 \times 40 = 318\) calories.
Interpret the slope in context. For each additional gram of carbohydrate, we expect calories to increase by about 4.3, on average.

Here’s a different setting to practice on. These are nutrition facts for seventy-seven Starbucks food items, and we’re using grams of carbohydrate to predict calories. The fitted line is drawn on the plot, and its equation is below: predicted calories equals one hundred forty-six plus four point three times carbs. Two questions. Predict the calories for an item with forty grams of carbohydrate, then interpret the slope in context. Pause here and work them out. For forty grams: one hundred forty-six plus four point three times forty. Four point three times forty is one hundred seventy-two, so the prediction is three hundred eighteen calories. And the slope: for each additional gram of carbohydrate, we expect calories to increase by about four point three, on average. That number is worth a second look — a gram of carbohydrate carries about four calories, so the slope our data produced lands close to the nutritional constant. Not exactly, though, and it shouldn’t: items with more carbohydrate tend to be bigger overall, so they carry more fat and protein too. The slope picks up all of that, not just the carbohydrate.

Is This Prediction Safe?

A friend uses the same line for an item with 100 g of carbohydrate and gets 576 calories. What’s the problem? The calculation is right, but the largest carbohydrate value in the data is 80 g — 100 g is extrapolation. The line keeps going, but nothing out there tells us the pattern holds.

Looking Ahead

We’ve built the linear model and learned to read it — slope, intercept, predictions, and the limits of extrapolation.

Next time (17b): how the line is actually chosen (least squares), how to measure how strong the linear relationship is (the correlation coefficient \(r\) and \(R^2\)), and how to check whether a line is even the right tool (residual plots).

That sets up the linear model. We’ve covered how to read a line — slope, intercept, predictions, and the limits of extrapolation. Three things are still missing. First, how did we get the specific line of best fit out of all the lines we could have drawn? That’s least squares — the rule that picks the best line. Second, how strong is the linear pattern in the data? Some scatterplots cluster tightly around their line; others scatter widely. We need numbers to quantify that — the correlation coefficient r and the coefficient of determination R squared. Third, should we even be using a line? The relationship might be curved, or the spread might change with x — situations where a line isn’t the right summary. Residual plots help us check. All three are in seventeen B, coming up Wednesday.

References

Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
Chapter 7
Heinz, G. et al. (2003). Exploring relationships in body dimensions. Journal of Statistics Education, 11(2). (bdims in openintro.)