Introduction to Correlation and Regression

Welcome to one of the most practical chapters in the IB Math AI course! In previous chapters, we looked at one variable at a time (like the heights of students). Now, we are looking at bivariate data, which means we are looking at two variables simultaneously to see if there is a relationship between them. For example, does the time you spend studying correlate with your exam scores? Does the temperature outside affect ice cream sales?

In this chapter, we will learn how to visualize these relationships, measure their strength, and use mathematical models to make predictions.

1. Visualizing Data: Scatter Diagrams

The first step in analyzing bivariate data is to plot it on a scatter diagram. We plot the independent variable (usually \(x\)) on the horizontal axis and the dependent variable (usually \(y\)) on the vertical axis.

Key features to look for:

  • Direction: Is it positive (as \(x\) increases, \(y\) increases) or negative (as \(x\) increases, \(y\) decreases)?
  • Shape: Does the data look like a straight line (linear) or a curve?
  • Strength: How closely do the points follow a pattern?
  • Outliers: Are there any points that don't seem to fit the general trend?

The Mean Point: A very important point in this chapter is the mean point, denoted as \((\bar{x}, \bar{y})\). This represents the average of all \(x\)-values and the average of all \(y\)-values. Every "line of best fit" must pass through this point.

Quick Tip: If a question asks you to "comment on the scatter diagram," mention the direction, strength, and linearity (e.g., "There is a strong, positive, linear correlation").

2. Pearson’s Product-Moment Correlation Coefficient (\(r\))

While looking at a graph is helpful, mathematicians like numbers. The Pearson’s correlation coefficient, \(r\), gives us a numerical value for the strength and direction of a linear relationship.

The Scale of \(r\):

  • \(r = 1\): Perfect positive linear correlation.
  • \(0.75 \le r < 1\): Strong positive correlation.
  • \(0.5 \le r < 0.75\): Moderate positive correlation.
  • \(0 < r < 0.5\): Weak positive correlation.
  • \(r = 0\): No linear correlation.
  • Negative values follow the same strength scale but indicate a negative direction (e.g., \(r = -0.9\) is a strong negative correlation).

Important Limitations:

  • Pearson’s \(r\) only measures linear relationships. Data could have a perfect curved relationship, and \(r\) might still be close to \(0\).
  • Pearson’s \(r\) is very sensitive to outliers. A single point far away from the trend can significantly change the value of \(r\).

3. The Regression Line of \(y\) on \(x\)

If we find a linear correlation, we can create an equation to model it. In this course, we focus on the least squares regression line, which is written in the form:

\(y = ax + b\)

Interpreting the equation:

  • \(a\) (the gradient): This tells us the predicted change in \(y\) for every 1-unit increase in \(x\). If \(a = 2.5\), then for every extra hour of study, the score is predicted to increase by 2.5 points.
  • \(b\) (the y-intercept): This is the predicted value of \(y\) when \(x = 0\). Note: This value only makes sense if \(x = 0\) is within or near the range of our data!

Making Predictions:

  • Interpolation: Predicting a \(y\)-value for an \(x\)-value within the range of your data. This is generally considered reliable.
  • Extrapolation: Predicting a \(y\)-value for an \(x\)-value outside the range of your data. This is dangerous and often unreliable because we don't know if the trend continues forever!

Quick Review: To find \(a\), \(b\), and \(r\), you will almost always use your Graphic Display Calculator (GDC). Enter your data into lists and use the "Linear Regression" function.

4. Spearman’s Rank Correlation Coefficient (\(r_s\))

Sometimes Pearson's \(r\) isn't the best tool. This is where Spearman's Rank (\(r_s\)) comes in. Instead of using the raw data, we rank the data (1st, 2nd, 3rd, etc.) and calculate the correlation between the ranks.

When to use Spearman's Rank:

  • When the relationship is non-linear but monotonic (it always goes up or always goes down, but not in a straight line).
  • When there are outliers (Spearman's is much less affected by them than Pearson's).
  • When the data is qualitative but ordinal (e.g., ranking students by "effort" from 1 to 10).

Key comparison: If Pearson's \(r\) is low but the scatter diagram shows a clear curve, Spearman's \(r_s\) will likely be much higher, indicating a consistent relationship even if it isn't a straight line.

5. (HL Only) Non-Linear Regression and Residuals

For Higher Level students, we explore relationships that aren't straight lines. Sometimes a power function, an exponential function, or a polynomial might fit the data better.

How do we know which model is best?

  • The Coefficient of Determination (\(R^2\)): This value (between 0 and 1) tells us the proportion of the variance in \(y\) that is predictable from \(x\). If \(R^2 = 0.85\), we say "85% of the variation in \(y\) can be explained by the model." The closer to 1, the better the fit.
  • Residuals: A residual is the vertical distance between an observed data point and the regression curve. \(Residual = y_{observed} - y_{predicted}\).
  • Sum of Square Residuals: The "best" model is the one that minimizes the sum of these squared distances.

Linearizing Data: You may be asked to use logarithms to transform a non-linear relationship (like \(y = ka^x\)) into a linear one so that you can apply linear regression techniques.

6. (HL Only) Testing for Correlation

Just because we find a correlation in a sample doesn't mean the whole population has that correlation. We use a hypothesis test to check if the population correlation coefficient (denoted by the Greek letter \(\rho\), "rho") is actually zero.

  • \(H_0\): \(\rho = 0\) (There is no correlation in the population).
  • \(H_1\): \(\rho \neq 0\) (There is a significant correlation) — this is a two-tailed test. Or \(\rho > 0\) or \(\rho < 0\) for one-tailed tests.

We use the p-value from our GDC to make a decision. If \(p < \text{significance level}\), we reject \(H_0\) and conclude there is a significant relationship.

Common Mistake to Avoid: Correlation does not imply causation! Just because two things are correlated (e.g., shark attacks and ice cream sales) doesn't mean one causes the other (both are caused by a third variable: hot weather).

Key Takeaways Summary

1. Visualize: Always start with a scatter diagram to see the trend.
2. Mean Point: The regression line must pass through \((\bar{x}, \bar{y})\).
3. Pearson’s \(r\): Measures the strength/direction of linear relationships only.
4. Spearman’s \(r_s\): Used for curved (monotonic) data or when outliers are present.
5. Regression Line: Use \(y = ax + b\) for predictions, but beware of extrapolation.
6. (HL) \(R^2\): Use this to compare different models; the closer to 1, the better the explanation of the data.