Welcome to the "Best-Fit" Line: Least-Squares Regression

In previous chapters, we looked at scatterplots and learned how to describe the relationship between two variables. Now, we are going to find the specific mathematical "ruler" that lets us make predictions: the Least-Squares Regression Line (LSRL). Don't worry if the name sounds intimidating—it’s just the formal name for the "line of best fit" that you’ve likely seen before!

In this chapter, we will dive into why we use this specific line, how to interpret its parts, and how to measure how well it actually works using a special number called \(r^2\).

What Makes it "Least-Squares"?

There are infinitely many lines you could draw through a cloud of data points. Why do we pick the LSRL? The goal of regression is to make our residuals (the vertical distance between the actual data points and the line) as small as possible.

The "Least-Squares" line is the one specific line that minimizes the sum of the squared residuals. We square them so that positive and negative distances don't cancel each other out. Think of it like a game: the line is trying to position itself so that the total "penalty" (the squared distances) is the smallest it can possibly be.

Quick Review: Remember from our chapter on residuals that a residual is calculated as \(y - \hat{y}\) (Actual value minus Predicted value).

The Equation of the Line

The formula for the least-squares regression line is written as:

\(\hat{y} = a + bx\)

  • \(\hat{y}\) (y-hat): This is the predicted response. Always remember the "hat"—it tells the reader we are making a prediction, not stating an actual data point!
  • \(a\): The y-intercept. This is the predicted value of \(y\) when \(x = 0\).
  • \(b\): The slope. This tells us how much we expect \(\hat{y}\) to change for every one-unit increase in \(x\).
  • \(x\): The explanatory variable.

Note: According to the Fall 2026 CED, you will obtain the values for \(a\) and \(b\) using technology (like your graphing calculator) rather than calculating them by hand.

The Special Mean Point

Did you know? The least-squares regression line always passes through the point \((\bar{x}, \bar{y})\). That is, it passes through the mean of the x-values and the mean of the y-values. It’s the "anchor" of the line!

Key Takeaway: The LSRL is the unique line that makes the "squared errors" as small as possible, ensuring our predictions are as accurate as they can be for the whole dataset.

Interpreting Slope and Intercept in Context

On the AP Exam, you will often be asked to "interpret" these values. Use these fill-in-the-blank templates to make sure you get full credit!

1. Interpreting the Slope (\(b\))

"For every 1 [unit] increase in [explanatory variable \(x\)], the predicted [response variable \(y\)] is expected to increase/decrease by [\(b\) units]."

Example: If the slope for a model predicting test scores (\(y\)) from hours studied (\(x\)) is \(b = 5\), we would say: "For every 1 additional hour studied, the predicted test score is expected to increase by 5 points."

2. Interpreting the y-intercept (\(a\))

"When the [explanatory variable \(x\)] is 0 [units], the predicted [response variable \(y\)] is [\(a\) units]."

A Warning on the Intercept: Sometimes the intercept doesn't make physical sense. For example, if you are predicting weight based on height, the intercept would be the predicted weight of a person who is 0 inches tall! If the value \(x=0\) is far outside your data range, the intercept is just a starting point for the math and has no practical meaning.

Measuring Success: The Coefficient of Determination (\(r^2\))

Once we have our line, we need to know: How good is this line at explaining what’s happening? That’s where \(r^2\) (the coefficient of determination) comes in.

As the name suggests, \(r^2\) is literally the square of the correlation coefficient (\(r\)). While \(r\) tells us direction and strength, \(r^2\) tells us about variation.

The Official Interpretation of \(r^2\)

You should memorize this specific sentence structure for the exam:

"[\(r^2\) as a percentage]% of the variation in [response variable \(y\)] can be explained by the linear relationship with [explanatory variable \(x\)]."

Analogies to help you understand:
Imagine you are trying to predict electricity bills. Some of the variation in bills is because of the outside temperature (explanatory variable). If \(r^2 = 0.80\), it means \(80\%\) of the reason bills go up and down is explained by the temperature. The other \(20\%\) is "unexplained"—it might be due to how many lights you left on or the age of your appliances.

Key Takeaway: \(r^2\) is always between \(0\) and \(1\) (or \(0\%\) and \(100\%\)). A higher \(r^2\) means the line fits the data better and leaves less "unexplained" variation.

Common Mistakes and Tips

  • Don't Forget the "Predicted": When interpreting slope or the intercept, always use words like "predicted," "estimated," or "average." If you say "the score will be 80," you are implying it's a fact, which is statistically incorrect. Use "the predicted score is 80."
  • Extrapolation Danger: Avoid predicting \(y\) values for \(x\) values that are far outside the range of your original data. This is called extrapolation, and it's risky because the linear pattern might not continue there!
  • Correlation is not Causation: Just because a least-squares line has a high \(r^2\), it doesn't mean that \(x\) causes \(y\) to change. It only means they are associated.
  • Calculator Output: When looking at computer or calculator output, "intercept" or "constant" is your \(a\), and the name of your explanatory variable is usually next to your slope \(b\).

Quick Review Box

LSRL Equation: \(\hat{y} = a + bx\)
Residual: \(y - \hat{y}\)
Goal: Minimize \(\sum (y - \hat{y})^2\)
Slope (\(b\)): Predicted change in \(y\) for each \(+1\) change in \(x\).
\(r^2\): % of variation in \(y\) explained by the linear model with \(x\).