Welcome to Bivariate Distributions!
In single-variable (univariate) statistics, we look at one feature at a time, such as the heights of students or their test scores. But what happens when we want to know if two variables are connected? For example: Does spending more hours revising lead to higher exam scores? or Does engine size affect fuel consumption?
This chapter covers Bivariate Distributions for CCEA AS Further Mathematics (Unit AS 2: Section C). You will learn how to measure the strength and direction of linear relationships between two variables and how to build mathematical models (regression lines) to make reliable predictions.
Don't worry if the formulae look a bit long at first glance! Once broken down into standard summary statistics, the calculations follow a simple, step-by-step routine.
1. Understanding Bivariate Data & Scatter Diagrams
Variables: Explanatory vs. Response
Bivariate data consists of linked pairs of observations \((x_i, y_i)\) taken from the same individual or item:
1. Independent / Explanatory Variable (\(x\)): The variable that is set, controlled, or thought to explain changes in the other variable. It is plotted on the horizontal (\(x\)) axis.
2. Dependent / Response Variable (\(y\)): The variable that is measured or responds to changes in the explanatory variable. It is plotted on the vertical (\(y\)) axis.
Example: If you are investigating how the amount of fertiliser (\(x\)) affects crop yield (\(y\)), fertiliser amount is the explanatory variable and crop yield is the response variable.
Scatter Diagrams
A scatter diagram is the visual starting point for bivariate analysis. Plotting the pairs \((x, y)\) on Cartesian axes allows us to check for:
• Pattern & Linearity: Do the points roughly lie along a straight line, or do they follow a curved (non-linear) trend?
• Direction: Does \(y\) increase as \(x\) increases (positive association), or does \(y\) decrease as \(x\) increases (negative association)?
• Outliers: Are there unusual data points that sit far away from the main cluster?
Section 1 Key Takeaways
Key Takeaway: Always identify which variable is explanatory (\(x\)) and which is response (\(y\)) before drawing a scatter diagram or performing calculations. \(x\) explains \(y\).
2. Pearson’s Product Moment Correlation Coefficient (\(r\))
What Does \(r\) Measure?
Pearson’s Product Moment Correlation Coefficient, denoted by \(r\), is a numerical measure of the strength and direction of a linear association between two quantitative variables.
The value of \(r\) is always bounded between \(-1\) and \(+1\):
• \(r = +1\): Perfect positive linear correlation (all points lie exactly on a straight line with a positive slope).
• \(0.8 \le r < 1\): Strong positive linear correlation.
• \(r = 0\): No linear correlation (points are randomly scattered, or they may have a non-linear relationship like a quadratic curve).
• \(-1 < r \le -0.8\): Strong negative linear correlation.
• \(r = -1\): Perfect negative linear correlation (all points lie exactly on a straight line with a negative slope).
Crucial Concept: Correlation vs. Causation
A high correlation does NOT imply cause-and-effect!
Just because two variables have a high value of \(r\) does not mean that changes in \(x\) directly cause changes in \(y\). The observed association might be coincidental or driven by a third, unseen factor called a confounding or lurking variable.
Classic Analogy: Sales of ice cream and the number of people suffering from sunburn are strongly positively correlated. Eating ice cream does not cause sunburn; both are driven by hot, sunny weather (the confounding variable).
The Three Summary Statistics (\(S_{xx}\), \(S_{yy}\), \(S_{xy}\))
To calculate \(r\) and the lines of regression, we first compute three fundamental summary statistics from the data totals (\(n\), \(\sum x\), \(\sum y\), \(\sum x^2\), \(\sum y^2\), and \(\sum xy\)):
Sum of squares for \(x\):
\(S_{xx} = \sum x^2 - \frac{(\sum x)^2}{n} = \sum (x - \bar{x})^2\)
Sum of squares for \(y\):
\(S_{yy} = \sum y^2 - \frac{(\sum y)^2}{n} = \sum (y - \bar{y})^2\)
Sum of products of \(x\) and \(y\):
\(S_{xy} = \sum xy - \frac{(\sum x)(\sum y)}{n} = \sum (x - \bar{x})(y - \bar{y})\)
Note: \(S_{xx}\) and \(S_{yy}\) are measures of variance and must always be positive. \(S_{xy}\) measures covariance and can be positive, negative, or zero, which determines the sign of \(r\).
The Formula for \(r\)
Using the summary statistics, the correlation coefficient is given by:
\(r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} = \frac{\sum xy - \frac{\sum x \sum y}{n}}{\sqrt{\left(\sum x^2 - \frac{(\sum x)^2}{n}\right)\left(\sum y^2 - \frac{(\sum y)^2}{n}\right)}}\)
Linear Transformations (Coding) and \(r\)
Did you know? The correlation coefficient \(r\) is a dimensionless scale-free quantity. This means it is unaffected by linear coding of the variables.
If we transform \(x\) and \(y\) into coded variables \(u\) and \(v\) using linear shifts and scalings:
\(u = \frac{x - a}{b} \quad \text{and} \quad v = \frac{y - c}{d} \quad (\text{where } b > 0, d > 0)\)
Then the correlation coefficient between \(u\) and \(v\) is identical to that between \(x\) and \(y\):
\(r_{uv} = r_{xy}\)
Important Exception: If one of the scaling factors is negative (for example, if \(u = -x\)), the direction of the relationship is flipped, so \(r\) changes sign: \(r_{uv} = -r_{xy}\).
Section 2 Key Takeaways
Key Takeaway: \(r\) only measures linear association and lies in \([-1, 1]\). Always calculate \(S_{xx}\), \(S_{yy}\), and \(S_{xy}\) first with full precision before computing \(r\).
3. Least Squares Regression Analysis
When two variables exhibit a linear correlation, we can construct a line of best fit called the least squares regression line. This line minimises the sum of the squared vertical distances (residuals) from the data points to the line.
1. Regression Line of \(y\) on \(x\)
This is the standard regression line used when \(x\) is the independent/explanatory variable and we wish to model or predict the dependent/response variable \(y\).
Equation:
\(y = a + bx\)
Gradient (Slope) \(b\):
\(b = \frac{S_{xy}}{S_{xx}}\)
Intercept \(a\):
\(a = \bar{y} - b\bar{x} \quad \text{where } \bar{x} = \frac{\sum x}{n} \text{ and } \bar{y} = \frac{\sum y}{n}\)
Golden Property: The regression line of \(y\) on \(x\) always passes through the mean point \((\bar{x}, \bar{y})\), known as the centroid of the data.
2. Regression Line of \(x\) on \(y\)
If \(y\) is the independent variable and \(x\) is the dependent variable, or if we need to predict \(x\) from a known value of \(y\), we use the regression line of \(x\) on \(y\).
Equation:
\(x = a' + b'y\)
Gradient \(b'\):
\(b' = \frac{S_{xy}}{S_{yy}}\)
Intercept \(a'\):
\(a' = \bar{x} - b'\bar{y}\)
Exam Warning: Do not simply rearrange the equation \(y = a + bx\) to make \(x\) the subject! Minimising horizontal residuals produces a mathematically different line than minimising vertical residuals, so \(b' \ne \frac{1}{b}\).
Step-by-Step Worked Example
A researcher measures the study time in hours (\(x\)) and test score (\(y\)) for \(n = 5\) students, obtaining the following summary values:
\(n = 5\), \(\sum x = 20\), \(\sum y = 150\), \(\sum x^2 = 90\), \(\sum y^2 = 4620\), \(\sum xy = 640\).
Step 1: Calculate the means
\(\bar{x} = \frac{\sum x}{n} = \frac{20}{5} = 4\)
\(\bar{y} = \frac{\sum y}{n} = \frac{150}{5} = 30\)
Step 2: Calculate the summary statistics
\(S_{xx} = \sum x^2 - \frac{(\sum x)^2}{n} = 90 - \frac{20^2}{5} = 90 - \frac{400}{5} = 90 - 80 = 10\)
\(S_{yy} = \sum y^2 - \frac{(\sum y)^2}{n} = 4620 - \frac{150^2}{5} = 4620 - \frac{22500}{5} = 4620 - 4500 = 120\)
\(S_{xy} = \sum xy - \frac{(\sum x)(\sum y)}{n} = 640 - \frac{20 \times 150}{5} = 640 - \frac{3000}{5} = 640 - 600 = 40\)
Step 3: Calculate the correlation coefficient \(r\)
\(r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} = \frac{40}{\sqrt{10 \times 120}} = \frac{40}{\sqrt{1200}} \approx 1.1547... \text{ (wait, }\sqrt{1200} \approx 34.641\text{)}\)
\(r = \frac{40}{34.641} \approx 0.926\)
Interpretation: There is a strong positive linear correlation between study hours and test scores.
Step 4: Find the regression line of \(y\) on \(x\)
\(b = \frac{S_{xy}}{S_{xx}} = \frac{40}{10} = 4\)
\(a = \bar{y} - b\bar{x} = 30 - (4)(4) = 30 - 16 = 14\)
Equation: \(y = 14 + 4x\)
Contextual Interpretation: The slope \(b = 4\) means that for each additional hour spent studying, the test score increases by an estimated \(4\) marks. The intercept \(a = 14\) is the estimated score for a student studying \(0\) hours.
Section 3 Key Takeaways
Key Takeaway: For the regression line of \(y\) on \(x\), \(b = \frac{S_{xy}}{S_{xx}}\). For \(x\) on \(y\), \(b' = \frac{S_{xy}}{S_{yy}}\). Both lines pass through \((\bar{x}, \bar{y})\).
4. Predictions: Interpolation vs. Extrapolation
Once you have a regression equation, you can use it to estimate values. However, examiners frequently test your understanding of how reliable these estimates are.
Interpolation (Generally Reliable)
Interpolation means estimating an unknown value that lies within the range of the original sample data.
• Reliability: If the scatter diagram shows a strong linear pattern (\(|r|\) is close to \(1\)), interpolation is considered reliable because the linear model has been demonstrated to hold in this region.
Extrapolation (Unreliable & Risky)
Extrapolation means estimating a value that lies outside the range of the original sample data.
• Reliability: Extrapolation is dangerous and unreliable. We have no evidence that the linear relationship continues beyond the maximum or minimum observed values of \(x\). The trend might flatten, curve, or break down completely.
Real-world Example: A child’s height increases linearly between ages 4 and 10. If you use this regression line to extrapolate their height at age 35, the model will predict an impossibly large height!
The Golden Rule for Choosing the Right Line
Always select your regression line based on what you are trying to predict:
• To predict \(y\) from a known value of \(x\) → Use the regression line of \(y\) on \(x\) (\(y = a + bx\)).
• To predict \(x\) from a known value of \(y\) → Use the regression line of \(x\) on \(y\) (\(x = a' + b'y\)).
Section 4 Key Takeaways
Key Takeaway: Predictions within the sample domain (interpolation) are reliable; predictions outside the domain (extrapolation) are unreliable. Never use \(y = a + bx\) to predict \(x\).
5. Common Pitfalls & Examiner Checklist
Make sure you avoid these classic mistakes highlighted in CCEA examiner reports:
1. Premature Rounding: Keep full calculator accuracy when finding \(S_{xx}\), \(S_{yy}\), and \(S_{xy}\). Rounding these intermediate values leads to compounding errors in \(r\), \(a\), and \(b\).
2. "No Relationship" vs "No Linear Relationship": If \(r \approx 0\), never write that there is "no relationship". Always state there is no linear correlation (a strong quadratic or non-linear relationship could still exist).
3. Forgetting the Centroid: Remember that \((\bar{x}, \bar{y})\) lies on the regression line. This is a great way to check your calculated intercept: verify that \(\bar{y} = a + b\bar{x}\).
4. Dividing by \(n\) in Summary Formulas: A common algebraic slip is forgetting to divide \(\sum x \sum y\) or \((\sum x)^2\) by \(n\). Double-check your formula sheet before computing.
5. Claims of Causality: In written explanations, never claim that one variable causes the other based solely on a high value of \(r\). Always mention potential confounding or lurking variables.
Quick Review Summary Box
• \(S_{xx} = \sum x^2 - \frac{(\sum x)^2}{n}\), \quad \(S_{yy} = \sum y^2 - \frac{(\sum y)^2}{n}\), \quad \(S_{xy} = \sum xy - \frac{\sum x \sum y}{n}\)
• \(r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}}\) where \(-1 \le r \le 1\)
• Line of \(y\) on \(x\): \(y = a + bx\) where \(b = \frac{S_{xy}}{S_{xx}}\) and \(a = \bar{y} - b\bar{x}\)
• Line of \(x\) on \(y\): \(x = a' + b'y\) where \(b' = \frac{S_{xy}}{S_{yy}}\) and \(a' = \bar{x} - b'\bar{y}\)
• Coding \(u = \frac{x-a}{b}, v = \frac{y-c}{d}\) (\(b, d > 0\)) leaves \(r\) invariant (\(r_{uv} = r_{xy}\)).