Chapter 10: Two quantitative variables
Section 10.3: Least squares regression
In Sections 10.1 and 10.2, we used graphical and numerical methods to
describe the observed association between two quantitative variables and
used correlation coefficient simulation for drawing inference. For a
linear association, it is helpful to develop a mathematical model of it.
This allows us to make predictions about the response variable from the
explanatory variable. The most common way to do this is with a least
squares regression line.
Example 10.3: Height and winning at tennis (continued from
Exploration 10.1)
Open the Correlation/Regression
applet and select the “WimbledonMF” data set in the applet, which we
looked at in Exploration 10.1.
The scatterplot shows a weak positive association because of the
trend of increasing height in more recent years. This is supported by
the correlation coefficient of r = 0.189.
The association appears to be roughly linear, and the regression line
is \[
\widehat{height} = 104.7 + 0.0378 * year.
\] This equation has the form \[
\widehat{y} = a + b x
\] where
- \(a\) is the y-intercept,
- \(b\) is the slope,
- \(x\) is the value of the
explanatory variable, and
- \(\widehat{y}\) is a point on the
line that represents the predicted value of the response variable.
Questions
- What are the units on \(\widehat{height}\)?
- What are the units on 104.7?
- What are the units on 0.0378? What is the practical significance of
this slope?
- What are the units on \(year\)?
Key ideas
- The slope of a least squares regression line is interpreted as the
predicted change in the average response variable for a one-unit change
in the explanatory variable.
- For a given data set, the signs (positive, negative, or zero) for
the slope of the regression line, b, and the correlation coefficient, r,
and must be the same.
- Predicting values for the response variable for values of the
explanatory variable that are outside of the range of the original data
is known as extrapolation, and can give very misleading
predictions.
- The y-intercept, a, of a regression line is interpreted as the
predicted value of the response variable when the explanatory variable
has a value of zero.
- Be wary of extrapolation in interpreting the intercept or other
values outside the original data range.
Key idea residuals and coefficient of determination
- A residual is a difference between an observed response and the
predicted response from the least square regression line: \(residual = observed - predicted\).
- The least squares regression line minimizes the sum of the squared
residuals.
- The coefficient of determination (\(r^2\)) is the percentage of variation in
the response variable that is explained by the linear relationship with
the explanatory variable. \[
r^2 = \frac{SSE(\bar{y}) - SSE(regression line)}{SSE(\bar{y})}.
\]
- In the Wimbledon example, because correlation coefficient is \(r = 0.189\), the coefficient of
determination is \(r^2 = 0.189^2 =
0.0357\). This means that 3.5721% of the observed variation in
height of Wimbledon championship winners is explained by knowing the
year that they won.
Exploration 10.3: Arctic sea ice extent