Diagnostic tests of model fit and assumption checking

Welcome to the "Check-Up": Model Diagnostics and Assumption Checking

You’ve done the hard work of building a linear model, but how do you know if you can actually trust its results? In the world of actuarial modeling, building a model is only half the battle. The other half is making sure the model's assumptions hold true. Think of this chapter as a "medical check-up" for your model. We are looking for symptoms of trouble that might make our predictions unreliable.

Don’t worry if this seems tricky at first! We are going to break down these statistical "vital signs" using simple analogies and clear steps.

1. The "Big Four" Assumptions (LINE)

For a standard linear regression model to be valid, we usually look for four specific conditions. A great way to remember these is the mnemonic LINE:

L – Linearity: The relationship between the predictors and the response is a straight line.
I – Independence: The errors (residuals) are independent of each other.
N – Normality: The errors follow a normal distribution.
E – Equal Variance: The spread of the errors is constant across all levels of the predictors (this is fancy-talk for Homoscedasticity).

Why do these matter?

If these assumptions are violated, your p-values might be wrong, your confidence intervals might be misleading, and your predictions could be way off. It’s like trying to bake a cake with a broken thermometer—you’re following the recipe, but the results won't be what you expect!

Key Takeaway:

Before trusting your model's output, always check the LINE assumptions. Most of these are checked by looking at Residuals (the difference between the actual value and what the model predicted).

2. Residual Plots: The Model’s X-Ray

The most common tool for diagnostics is the Residual Plot, where we plot the residuals (\(e_i\)) on the y-axis and the fitted values (\(\hat{y}_i\)) on the x-axis.

Non-Linearity (The Curve)

If you see a clear U-shape or a curve in your residual plot, it means your model is missing a non-linear relationship.
The Fix: You might need to transform your predictors (like using \(log(x)\) or \(x^2\)).

Heteroscedasticity (The Funnel)

If the residuals get wider or narrower as you move across the plot (it looks like a fan or a megaphone), you have "Heteroscedasticity." This means the "Equal Variance" assumption is broken.
The Fix: Often, taking the log of the response variable (\(log(y)\)) can help shrink that funnel shape.

Quick Review: What are we looking for?

In a perfect world, a residual plot should look like a random cloud of dots with no discernible pattern, centered around zero.

3. Outliers and High Leverage Points

Not all data points are created equal. Some points have a massive impact on your model, while others don't.

Outliers

An outlier is a point where the actual \(y\) value is very far from the predicted value. It has a large residual.
Analogy: A student who gets a 0% on an exam when everyone else got 90%.

High Leverage Points

A point has high leverage if it has an unusual \(x\) value (predictor value). These points are "stuck out" far from the average of the other data points. They have the power to "pull" the regression line toward them.
Analogy: A person who is 8 feet tall in a study of average heights.

Cook’s Distance: The "Drama Score"

How do we know if a point is actually harming the model? We use Cook's Distance. It measures how much the predicted values change if we remove that specific point.
Simple Rule: If Cook’s Distance is high (usually greater than 1), that point is influential and deserves a closer look.

Key Takeaway:

An outlier is weird in the \(y\)-direction. A leverage point is weird in the \(x\)-direction. An influential point is one that actually changes the slope of your line significantly.

4. Collinearity: When Predictors Fight

Collinearity (or Multicollinearity) happens when two or more of your predictor variables are highly correlated with each other.

Did you know? If two variables tell the same story (like "Height in Inches" and "Height in Centimeters"), the model gets confused and doesn't know which variable should get the "credit" for the prediction.

The Symptom: VIF (Variance Inflation Factor)

We use the VIF to detect collinearity.
1. A VIF of 1 means no correlation.
2. A VIF exceeding 5 or 10 indicates problematic collinearity.

Common Mistakes to Avoid:

Students often think collinearity makes the model less accurate at predicting. That's not true! Collinearity makes the coefficients (the \(\beta\) values) unstable and difficult to interpret, but the overall predictions can still be okay. However, for the SRM exam, we generally want to remove variables with high VIFs to keep the model stable.

5. Summary of Diagnostic Steps

If you're stuck on an exam question, follow this mental checklist:

1. Check the Residual Plot: Look for patterns (non-linearity) or funnels (non-constant variance).
2. Check the QQ-Plot: This is used for the Normality assumption. Points should fall along a straight diagonal line.
3. Check VIF: If a VIF is > 10, you have a collinearity problem.
4. Check Cook’s Distance: Look for influential points that are "bossing the model around."

Quick Tip for the Exam: If the question mentions that the variance of the error terms increases as the predictor increases, they are talking about Heteroscedasticity. The answer will often involve a log-transformation!

Final Key Takeaways

• Residuals are the primary tool for checking model assumptions.
• Linearity is checked with Residual vs. Fitted plots.
• Normality is checked with a Normal QQ-plot.
• Homoscedasticity means "constant variance."
• VIF measures how much your variables overlap (Collinearity).
• Cook’s Distance identifies points that have a large influence on the model results.