Math 115: Introduction to Statistics – 18. Inference for Linear Regression

What Is the Line Estimating?

Topic 17 fit a line to a sample: $\hat{y} = b_0 + b_1 x$. Behind that sample sits a population, where every individual follows

\[y = \beta_0 + \beta_1 x + \varepsilon\]

$\beta_0$ and $\beta_1$ are the population intercept and slope — fixed, unknown, never observed. $\varepsilon$ is how far one individual falls above or below that line.

$b_0$ and $b_1$ estimate $\beta_0$ and $\beta_1$. A different sample gives different estimates — so what can one sample’s $b_1$ tell us about $\beta_1$? That question is inference for the slope.

In topic seventeen we fit a line to a sample: y hat equals b naught plus b one times x. Behind that sample sits a population, and the model says every individual in it follows one equation: y equals beta naught, plus beta one times x, plus epsilon.

The dark line in the picture is the population line. Beta naught and beta one are its intercept and slope — fixed numbers, unknown to us, and never observed. Epsilon is the last piece: how far one individual falls above or below that line. The red point sits about fifteen above the line, and that gap is its epsilon.

The population slope here is three. Population lines can have any slope at all; this one happens to be a fairly strong positive.

B naught and b one, computed from a sample, are estimates of beta naught and beta one. Draw a different sample and you get different estimates. So the question for this topic is what one sample’s b one can tell us about the population slope, beta one. That is inference for the slope.

What Does “No Relationship” Look Like?

A different population — this time $x$ tells you nothing about $y$:

Try it — in $y = \beta_0 + \beta_1 x + \varepsilon$, what must $\beta_1$ be?

$\beta_1 = 0$, and the model collapses to $y = \beta_0 + \varepsilon$. $x$ is gone from the equation. Every $x$ gives the same predicted $y$, so the population line is flat. Here $\beta_0 = 50$.

Do Samples Come Out Flat?

Draw a sample of 20 from that population — the one where $\beta_1 = 0$.

Same population every time, with $\beta_1 = 0$. But the sample slope $b_1$ never comes out zero.
So the question: is $b_1$ far enough from zero to rule out $\beta_1 = 0$?

NYC Italian Restaurants

Research question: Is the price of dinner at an NYC Italian restaurant related to its food-quality rating?

Data: customer surveys from 168 Italian restaurants in NYC (Zagat, via Sheather, 2009).
Response ($y$): average price of dinner in dollars (includes one drink and tip).
Predictor ($x$): food-quality rating on Zagat’s 0–30 scale. These restaurants span only 16–25.

Fitted line: $\widehat{Price} = -17.8 + 2.94 \times Food$ — higher ratings track higher prices. But is there a linear relationship between food rating and price in the population, or is $b_1$ just sampling noise?

Here is today’s data, so we can work with a concrete example. Is the price of dinner at a New York City Italian restaurant related to its food-quality rating? The data are Zagat customer surveys for one hundred sixty-eight Italian restaurants. The response variable is the average price of dinner in dollars, including one drink and tip. The predictor is the food-quality rating, on Zagat’s zero-to-thirty scale. The scatterplot shows a clear positive trend — higher food ratings come with higher prices. Software fit the least-squares line: predicted price equals about minus eighteen plus two point nine four times food rating. That sample slope — about three dollars per rating point — is our b one. The rest of the lecture asks whether there is a linear relationship between food rating and price in the population, starting with the formal hypotheses.

Hypotheses for the Slope

Try it — the research question is whether food rating is related to price. Write $H_0$ and $H_A$ in terms of $\beta_1$.

$H_0: \beta_1 = 0$ — the true slope is zero; a restaurant’s food rating tells you nothing about its price, linearly.
$H_A: \beta_1 \neq 0$ — there is a linear relationship; food rating and price move together.

Two-sided is the default. One-sided versions ($\beta_1 > 0$ or $\beta_1 < 0$) exist when the research question commits to a direction.

Before the hypotheses appear, write them yourself. The research question is whether food rating is related to price, and the parameter is beta one, the population slope. What are the null and the alternative? The null hypothesis says beta one equals zero — the true slope is zero, which for our restaurants means a food rating tells you nothing about price in a linear sense. There’s no linear relationship in the population. The alternative says beta one is not equal to zero — food rating and price are linearly related in the population. Two-sided is the default; the slope could be positive or negative. One-sided versions exist when the research question commits to a direction — beta one greater than zero if we’re testing for a positive relationship, less than zero for a negative one.

Test Statistic and Degrees of Freedom

Same T-statistic shape we’ve used since topic 11:

\[T = \frac{\text{statistic} - \text{null value}}{SE} = \frac{b_1 - 0}{SE_{b_1}} = \frac{b_1}{SE_{b_1}}\]

Software reports $b_1$ and $SE_{b_1}$ — you can always verify $T$ by hand.

Reference distribution: under $H_0$, $T$ follows a t-distribution with

\[df = n - 2\]

Why $n - 2$? Fitting the line uses up two degrees of freedom — one for the intercept $b_0$, one for the slope $b_1$.

Conditions for Inference

Four conditions for the t-distribution to be a valid reference for $T$:

Condition	Rule	Check with
Linearity	True relationship between $x$ and $y$ is approximately linear.	Residual plot
Constant variance	Residual spread is about the same across all $\hat{y}$.	Residual plot
Normality	$n \geq 30$: mild skew is fine, but no extreme outliers. $n < 30$: residuals should look roughly normal.	Residual plot
Independence	Observations are independent of each other.	Study design

Four conditions have to hold for the t-distribution to be a valid reference for the test statistic. Linearity. The true underlying relationship between x and y is approximately linear. If it’s curved, a straight line doesn’t fit, and that curve shows up in the residual plot. Constant variance. The residual spread is about the same across all predicted values — the model’s error doesn’t get bigger or smaller as x changes. A fan shape in the residual plot is the failure to watch for. Normality. With at least thirty observations, the residuals can show mild skew, as long as there are no extreme outliers. Under thirty, they should look roughly normal. On the plot, scan for a lone point sitting far off from the rest, or a spread that’s clearly lopsided, with a long tail on one side of the zero line. Independence. Observations are independent of each other. This is the one condition no plot will ever settle — it comes from how the study was run, from the design.

NYC Residual Plot

Linearity: no obvious curve — the residuals scatter randomly around zero ✓.
Constant variance: spread looks roughly the same across $\hat{y}$ ✓.
Normality: no lone point far off from the rest, and no long tail on one side of zero ✓. (Additionally, $n = 168$ is large, so the CLT covers $b_1$.)
Independence: restaurants surveyed separately ✓.

Conditions met — proceed with the t-test.

Checking the four conditions for the NYC restaurant model. Linearity: the residual plot doesn’t show an obvious curve, and the points center on zero across the range of predicted prices, so that holds. Constant variance holds too — the spread is about the same on the left as on the right, with no fan shape. For normality, scan for gross problems: no single point sits far off from the rest, and the residuals are balanced above and below zero, with no long tail on one side. Additionally, with n equal to one hundred sixty-eight, the Central Limit Theorem covers the sampling distribution of b one. And independence holds because each restaurant was surveyed separately, with no reason to expect dependence between restaurants. All four conditions are met, so we proceed with the t-test using the t-distribution as the reference.

Software Output

The regression output (Jamovi):

Term	Estimate (b)	Std. Error (SE)	T	p-value
(Intercept)	-17.83	5.863	-3.04	0.003
Food	2.94	0.283	10.37	< 0.001

Read the Food row (the slope):

$b_1 = 2.94$ — predicted price rises about $3 per point of food rating.
$SE_{b_1} = 0.283$.
$T = 10.37$, p-value $< 0.001$.

The regression output table — what Jamovi shows you. Two rows: one for the intercept, one for the slope. Read the slope row, labeled “Food,” for the test. The estimate column shows b one equals about two point nine four — predicted price increases by about three dollars per food-rating point. The standard error column shows about zero point two eight. The T column shows about ten point four. The p-value is less than zero point zero zero one — far below any conventional threshold. The intercept row also has its own t-statistic and p-value, but we usually focus on the slope when testing for a relationship. And the intercept here is worth a note: it’s the predicted price at a food rating of zero, which is far below the sixteen-to-twenty-five range these restaurants actually span. It positions the line; it isn’t a prediction anyone should use.

Verify the T-Statistic

The output gives you $b_1$ and $SE_{b_1}$. You should always be able to compute $T$ yourself — the software is just doing the arithmetic.

Recall: $b_1 = 2.94$, $SE_{b_1} = 0.283$. Try it: compute $T$ and compare to the output.

$T = \dfrac{b_1}{SE_{b_1}} = \dfrac{2.94}{0.283} \approx 10.39$ — matches the output’s $T = 10.37$ (the tiny difference is rounding) ✓. The slope sits more than 10 standard errors above zero.

The P-Value Picture

Two-sided p-value = 2 × (smaller-tail area) ≈ < 0.0001 — software reports < 0.001.

Conclusion

Results: $T = 10.37$, $df = 166$, p-value $< 0.001$ → reject $H_0$.

In context: there is convincing evidence that food-quality rating is associated with meal price at NYC Italian restaurants. Higher food ratings are associated with higher prices.

Scope of inference.

Generalization: these are Zagat-guide restaurants, listed because they were selected for the guide and drew enough diner reviews — not a random sample of NYC Italian restaurants. Strictly, the result describes the restaurants studied; extending it further is a judgment call, not something the sampling justifies.
Causation: nobody assigned food ratings to restaurants, so association only — not that higher food quality causes higher prices. (Pricier restaurants may attract better-trained chefs; nicer décor may lift ratings.)

Let’s put the result together. T is about ten point four with one hundred sixty-six degrees of freedom, and p is well under zero point zero zero one, so we reject the null. In context, there is convincing evidence that food-quality rating is associated with meal price at NYC Italian restaurants, with higher food ratings linked to higher prices.

Now consider the scope of inference. On generalization: these restaurants are in the data because they made the Zagat guide and drew enough diner reviews. Nothing random selected them, so this is not a random sample of NYC Italian restaurants. The result describes the restaurants studied; extending it further is a judgment call.

On causation: nobody assigned food ratings, so we have association only. Pricier restaurants can afford better-trained chefs, and nice décor may lift ratings — there are plenty of explanations besides food quality driving price.

Looking Ahead

The slope is statistically distinguishable from zero, but how big is the population slope? “More than zero” is a weak summary — we’d like a range of plausible values.

Next (18b): the matching confidence interval for $\beta_1$ — same $b_1$, same $SE_{b_1}$, same $t$-critical-value machinery as topic 11b, applied to the regression slope.

References

Introduction to Modern Statistics (2e) textbook by Mine Çetinkaya-Rundel and Johanna Hardin
Sections 24.4, 24.6
Sheather, S. J. (2009). A Modern Approach to Regression with R. Springer. (restNYC data.)

Condition	Rule	Check with
Linearity	True relationship between \(x\) and \(y\) is approximately linear.	Residual plot
Constant variance	Residual spread is about the same across all \(\hat{y}\).	Residual plot
Normality	\(n \geq 30\): mild skew is fine, but no extreme outliers. \(n < 30\): residuals should look roughly normal.	Residual plot
Independence	Observations are independent of each other.	Study design