Introduction to Correlation and Regression

In the world of statistics, we often want to know if two things are related. For example, does the amount of time you spend studying correlate with your exam scores? Does the outside temperature affect ice cream sales? This chapter focuses on bivariate data—data involving two variables—and how we can measure and model the relationship between them. By the end of these notes, you will be able to describe how two variables "move" together and use mathematical models to make predictions.

1. Bivariate Data and Scatter Diagrams

Bivariate data is simply a set of data that has two variables, usually represented as \((x, y)\) pairs.

  • Independent variable (\(x\)): The variable we think might be causing a change.
  • Dependent variable (\(y\)): The variable we are measuring or predicting.

The first step in analyzing this data is to create a scatter diagram (also called a scatter plot). This is a graph where each data point is plotted on a coordinate plane. Looking at the "cloud" of points helps us see if there is a pattern.

Quick Review:

  • If the points go "up" from left to right, there is a positive correlation.
  • If the points go "down" from left to right, there is a negative correlation.
  • If the points are scattered randomly, there is no correlation.

2. Pearson’s Product-Moment Correlation Coefficient \(r\)

While a graph is great, mathematicians like numbers! The Pearson’s product-moment correlation coefficient, denoted by the letter \(r\), is a numerical value that measures the strength and direction of the linear relationship between two variables.

Key properties of \(r\):

  • The value of \(r\) is always between \(-1\) and \(1\): \(-1 \leq r \leq 1\).
  • \(r = 1\): Perfect positive linear correlation (a perfect straight line pointing up).
  • \(r = -1\): Perfect negative linear correlation (a perfect straight line pointing down).
  • \(r = 0\): No linear correlation at all.

Interpreting the strength: Don't worry if this feels subjective; usually, we follow these general guidelines:

  • \(|r| > 0.75\): Strong correlation
  • \(0.5 < |r| < 0.75\): Moderate correlation
  • \(0.25 < |r| < 0.5\): Weak correlation

Did you know? You will almost always use your Graphic Display Calculator (GDC) to find the value of \(r\). Just enter your \(x\) and \(y\) values into the lists and perform a "Linear Regression" calculation.

3. The Line of Best Fit "By Eye"

Before using a calculator to find the exact regression line, you might be asked to draw a line of best fit "by eye." For this to be accurate in an IB exam, you must follow two rules:

  1. The line must pass through the mean point \((\bar{x}, \bar{y})\).
  2. The line should follow the general trend of the data, with roughly half the points above the line and half below.

How to find the mean point: Calculate the average of all \(x\) values (\(\bar{x}\)) and the average of all \(y\) values (\(\bar{y}\)).
For example, if \(\bar{x} = 10\) and \(\bar{y} = 20\), you must mark the point \((10, 20)\) on your scatter diagram and ensure your line passes exactly through it.

4. The Regression Line of \(y\) on \(x\)

The least squares regression line is the "mathematically perfect" line of best fit. It minimizes the sum of the squares of the vertical distances between the data points and the line. The equation is usually written in the form:
\(y = ax + b\)

Interpreting the coefficients:

  • \(a\) (the gradient): This tells us how much \(y\) is expected to change for every one unit increase in \(x\). If \(a = 2.5\), then for every 1 unit increase in \(x\), \(y\) increases by 2.5.
  • \(b\) (the \(y\)-intercept): This is the predicted value of \(y\) when \(x = 0\). Note: In many real-world contexts, this value might not make sense (e.g., predicting the height of a tree when it is 0 days old).

Key Takeaway: Use the \(y\) on \(x\) line when you want to predict \(y\) given a known value of \(x\).

5. The Regression Line of \(x\) on \(y\)

Sometimes, we want to do the opposite: we have a value for \(y\) and we want to predict what \(x\) would be. In this case, we use the regression line of \(x\) on \(y\).
Its equation looks like this:
\(x = cy + d\)

Important Note: This is not just the \(y = ax + b\) equation rearranged! It is a different calculation that minimizes horizontal distances. Always use the specific line designed for the variable you are trying to predict.

6. Prediction and Reliability

Once you have your regression equation, you can plug in values to make predictions. However, you must be careful about how reliable those predictions are.

Interpolation vs. Extrapolation:

  • Interpolation: Making a prediction within the range of the original data \(x\)-values. This is generally considered reliable.
  • Extrapolation: Making a prediction outside the range of the original data. This is unreliable because we don't know if the linear trend continues forever.

The role of \(r\): The closer \(|r|\) is to 1, the stronger the linear relationship, and the more reliable your predictions will be. If \(r = 0.2\), even an interpolation might be quite inaccurate.

Summary Checklist for Reliability: A prediction is most reliable when:

  1. The value of \(|r|\) is close to 1.
  2. You are using interpolation (within the data range).
  3. There is a large sample size of data.

Common Mistakes to Avoid

  • Mixing up the lines: Don't use the \(y\) on \(x\) line to predict \(x\). Your GDC can calculate both—make sure you pick the right one!
  • Ignoring the mean point: When sketching, always plot \((\bar{x}, \bar{y})\) first. If your line doesn't pass through it, you will lose marks.
  • Correlation vs. Causation: Just because two variables have a high \(r\) value doesn't mean one causes the other. For example, ice cream sales and shark attacks are positively correlated, but that's because they both increase in the summer, not because ice cream causes shark attacks!

Final Tip: Practice using your GDC to find \(a\), \(b\), and \(r\). In Paper 2, these are easy marks if you know which buttons to press!