Introduction to Multiple Linear Regression
Welcome to one of the most practical chapters in the CS1 syllabus! In the previous chapter, we looked at Simple Linear Regression, where we predicted a response variable using just one explanatory variable (like predicting claim size based only on the age of the policyholder). In the real world, things are rarely that simple.
Multiple Linear Regression (MLR) allows us to use several explanatory variables at once. Imagine you are pricing motor insurance: you wouldn't just look at the driver's age. You would also consider the engine size, the location, and how many years of no-claims discount they have. This chapter teaches you how to handle multiple inputs and, crucially, how to decide which ones actually belong in your model.
1. The Multiple Linear Regression Model
In MLR, we assume that a response variable \( Y \) is a linear combination of several explanatory variables \( x_1, x_2, \dots, x_k \), plus some random error.
The model for the \( i \)-th observation is written as:
\( Y_i = \beta_0 + \beta_1 x_{i1} + \beta_2 x_{i2} + \dots + \beta_k x_{ik} + \epsilon_i \)
Where:
- \( Y_i \) is the response variable (e.g., total claim amount).
- \( x_{i1}, x_{i2}, \dots, x_{ik} \) are the explanatory variables (e.g., age, car power, mileage).
- \( \beta_0 \) is the intercept.
- \( \beta_1, \dots, \beta_k \) are the slope parameters (coefficients) for each variable.
- \( \epsilon_i \) is the random error term, usually assumed to be independent and normally distributed: \( \epsilon_i \sim N(0, \sigma^2) \).
Key Assumption: Linearity
Don't be fooled by the word "linear." The "linear" in MLR refers to the parameters (\( \beta \)), not the variables. You could have a variable like \( x^2 \) in your model, and it would still be a linear regression model as long as it looks like \( \beta_j x_j^2 \).
2. Interpreting the Coefficients
One of the most common exam tasks is explaining what a specific \( \beta \) means. In MLR, we use the "all else being equal" rule.
Interpretation: The coefficient \( \beta_j \) represents the expected change in the response variable \( Y \) for a one-unit increase in \( x_j \), assuming all other explanatory variables in the model are held constant.
Example: If the coefficient for "Years of Driving Experience" is \( -50 \), it means that for every extra year of experience, the predicted insurance premium decreases by \$50, provided the car type and location remain the same.
Quick Review: In Paper B (the R exam), you will use the lm() function to find these estimates. The output will provide an Estimate, a Standard Error, a t-value, and a p-value for each coefficient.
3. Assessing Model Fit
Once we have a model, we need to know if it's actually any good. We use two main measures:
Coefficient of Determination (\( R^2 \))
\( R^2 = \frac{SS_{Reg}}{SS_{Tot}} = 1 - \frac{SS_{Res}}{SS_{Tot}} \)
This tells us the proportion of the total variation in \( Y \) that is explained by the model. It ranges from 0 to 1 (or 0% to 100%).
The Problem with \( R^2 \): Overfitting
Here is a trap many students fall into: Adding more variables will always increase \( R^2 \) (or at least keep it the same), even if those variables are completely useless! If you keep adding variables, you might "overfit" the model, meaning it explains the random noise in your specific data set rather than the actual underlying trend.
Adjusted \( R^2 \)
To fix the overfitting problem, we use Adjusted \( R^2 \). This measure penalizes you for adding unnecessary variables. It only increases if the new variable improves the model more than would be expected by chance.
\( Adjusted \ R^2 = 1 - (1 - R^2) \frac{n - 1}{n - k - 1} \)
Where \( n \) is the number of observations and \( k \) is the number of explanatory variables.
4. Choice of Explanatory Variables
How do we decide which variables to keep? Actuaries use several strategies to find the "parsimonious" model (the simplest model that does a good job).
Partial F-test (Comparing Nested Models)
If you have a "Big Model" (with many variables) and a "Small Model" (with only some of those variables), you can use an F-test to see if the extra variables in the Big Model are actually significant.
The null hypothesis \( H_0 \) is that the extra coefficients are all zero (i.e., the extra variables are useless).
Akaike Information Criterion (AIC)
The AIC is a very popular measure for model selection in actuarial practice. It balances the "likelihood" (how well the model fits) with "simplicity" (how many variables it uses).
Rule: When comparing models, the one with the lowest AIC is generally preferred.
Stepwise Selection Procedures
Manual selection is slow, so we often use these algorithms:
- Backward Elimination: Start with all possible variables. Remove the one with the highest p-value (the least significant). Repeat until all remaining variables are significant (usually p < 0.05).
- Forward Selection: Start with no variables. Add the one that provides the most significant improvement. Repeat until no more significant variables can be added.
- Stepwise: A combination of both, where you can add or remove variables at each step.
Common Mistake to Avoid: Don't just look at p-values in isolation. If two variables are highly correlated (e.g., "Age of Driver" and "Years since License obtained"), including both might make them both look insignificant! This is called multicollinearity.
5. Prediction in Multiple Regression
Just like in simple regression, we can use our fitted model to predict future values. There are two types of intervals you need to know:
- Confidence Interval: For the mean response (the average \( Y \) for a given set of \( x \)'s).
- Prediction Interval: For an individual future observation.
Key Takeaway: The Prediction Interval is always wider than the Confidence Interval because an individual observation has more uncertainty (it includes the variance of the mean PLUS the random error \( \sigma^2 \)).
6. Summary of Model Selection Criteria
When choosing the best set of explanatory variables, look for:
- Low AIC: Indicates a good balance of fit and simplicity.
- High Adjusted \( R^2 \): Indicates the variables are truly adding explanatory power.
- Significant p-values: Usually \( < 0.05 \) for the individual coefficients.
- Residual Analysis: Always check that your residuals look like random noise (no patterns). If the residuals show a pattern, you might be missing a variable or need a different model type.
Note: Residual analysis is covered in detail in the "Residual analysis and model validation" chapter. For now, just remember it is a vital step in checking if your choice of variables was successful!
Did you know? In Paper B, if you are asked to select the best model, you should usually fit a few variations, compare their AIC values, and check the p-values of the coefficients. If the AIC values are very close, the simpler model is usually preferred (the Principle of Parsimony)!