Comparing results with linear and tree-based methods

Exam ATPA – Advanced Topics in Predictive Analytics · SOA (Society of Actuaries) · 5 min read

Welcome to the Showdown: Linear vs. Tree-Based Models!

Welcome, future actuaries! In this chapter, we are going to dive into one of the most important practical skills for Exam ATPA: comparing results between linear and tree-based methods. Think of this as a "clash of the titans." On one side, we have the classic, reliable Linear Models (like GLMs). On the other, we have the modern, flexible Tree-Based Models (like Random Forests and XGBoost).

Choosing the "best" model isn't just about picking the one with the lowest error. It’s about understanding why one model outperforms the other and how their predictions differ. Don't worry if this seems a bit abstract right now—we’re going to break it down piece by piece!

1. Understanding the Fundamental Difference

Before we compare their results, we need to remember how these two "think" differently.

Linear Models (The Straight-Line Approach): These models assume that the relationship between your inputs and your target is a straight line (or a smooth curve). They are additive.
Analogy: Imagine you are baking a cake. A linear model thinks: "Every extra cup of sugar adds exactly 100 calories, regardless of how much flour you use."

Tree-Based Models (The Flowchart Approach): These models use a series of "if-then" splits. They are excellent at finding interactions and non-linear patterns.
Analogy: A tree model thinks: "If you have more than 2 cups of sugar AND the oven is over 400 degrees, the cake burns (a non-linear jump in 'badness'), but if the oven is cool, the sugar doesn't matter as much."

Quick Review: The Core Contrast

Linear Models: High interpretability, but struggle with complex, "wiggly" data relationships unless you manually add interaction terms.
Tree-Based Models: High flexibility, great at capturing complex patterns automatically, but can be harder to explain to a board of directors (the "black box" problem).

2. Comparing Predictive Performance

When you run both a GLM (Linear) and a GBM (Tree-based) for an ATPA project, you need to compare how well they actually predict. We usually look at these metrics:

Key Metrics to Watch:

1. RMSE (Root Mean Squared Error): Tells us how far off our predictions are on average. Lower is better!
2. MAE (Mean Absolute Error): Similar to RMSE, but less sensitive to big outliers.
3. Deviance: Often used in GLMs to see how well the model fits the data distribution.

The "Aha!" Moment: If your Tree-Based model has a much lower RMSE than your Linear model, it’s a huge hint that your data has non-linearities or complex interactions that the linear model is missing.

Common Mistake to Avoid:

The Overfitting Trap: Just because a Tree-Based model has a perfect score on your Training Data doesn't mean it's better. Always compare results on the Test Data (or via Cross-Validation). Trees are notorious for "memorizing" the noise in the training set!

3. Comparing the "Shape" of Predictions

One of the best ways to compare these models is to look at their Predicted vs. Actual plots or Residual plots.

Linear Model Residuals: If you see a "U-shape" in your residuals for a linear model, it means the model is missing a curve. It’s trying to fit a straight ruler to a round bowl.
Tree Model Predictions: Trees produce "step-functions." If you plot the predictions, they look like a staircase. If the true relationship is actually a very smooth, straight line, the tree model might actually perform worse because it’s trying to turn a smooth ramp into stairs.

Did you know? Tree-based models cannot extrapolate. If your training data has home prices up to \$1 million, a tree model will almost never predict a price of \$2 million for a new, larger house. It can only predict values within the range it has seen. A Linear model, however, will happily keep following the line upward!

4. Feature Importance vs. Coefficients

How do the models tell us what’s important? This is a major point of comparison in the ATPA exam.

Linear Models use Coefficients ($\beta$):
We look at the magnitude and the p-value.
Example: A coefficient of 0.5 means "for every 1 unit increase in X, the target increases by 0.5." It's very precise.

Tree-Based Models use Variable Importance:
This is usually based on Gain (how much the "purity" of the model improves when we split on that variable). It doesn't give you a simple "0.5 increase" rule; it just tells you which variables were the most useful for making splits.

Strategy Tip: If both models agree that "Age" is the most important variable, you can be very confident in that finding! If they disagree, investigate why. Perhaps "Age" only matters when "Income" is low (an interaction), which the Tree found but the Linear model missed.

5. The "Actuarial Choice": Why pick one over the other?

In your ATPA written report, you’ll often have to justify your model choice. Use this "Key Takeaway" table to help you decide:

Choose the Linear Model (GLM) if:

The relationship is mostly simple and additive.
The regulators require a clear, formula-based explanation (e.g., $Y = 2x_1 + 3x_2$).
You have a small dataset where complex trees might overfit.

Choose the Tree-Based Model (GBM/Random Forest) if:

Predictive accuracy is the #1 priority.
The data has complex interactions you don't want to manually code.
The data has lots of missing values or outliers (trees handle these much better!).

Summary and Key Takeaways

1. Logic: Linear models are additive and smooth; Tree models are split-based and "stair-like."
2. Interactions: Trees find interactions automatically; Linear models need you to tell them where the interactions are.
3. Extrapolation: Linear models can predict outside the training range; Trees cannot.
4. Validation: Always use Test/Holdout data to compare performance metrics like RMSE to avoid the overfitting trap.
5. Interpretability: Linear models provide clear coefficients; Trees provide relative importance rankings.

Final Encouragement: Don't feel pressured to always pick the most "complex" model. Sometimes the simplest linear model is the most robust. In ATPA, the explanation of why you chose a model is often just as important as the model's accuracy!

Quick check

Can you answer these now?

Open each question to check the key ideas from this chapter.

What is the fundamental difference in logic between linear models and tree-based models?

Linear models are additive and assume smooth relationships, while tree-based models use iterative if-then splits to capture non-linear patterns and interactions.

How do linear and tree-based models compare in terms of interpretability and flexibility?

Linear models provide high interpretability but low flexibility, while tree-based models offer high flexibility at the cost of being harder to explain (the 'black box' problem).

Which three metrics are commonly used to compare the performance of linear and tree-based models in ATPA?

The key metrics are Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Deviance.

What does a significantly lower RMSE in a tree-based model compared to a linear model usually indicate?

A significantly lower RMSE indicates that the dataset likely contains non-linear relationships or complex interactions that the linear model is failing to capture.

Why is it critical to compare model results on test data rather than training data?

Comparing on test data prevents the 'overfitting trap,' as tree-based models are prone to 'memorizing' noise in the training set which does not generalize to new data.

How do the 'shapes' of predictions differ between linear and tree-based models?

Linear models produce smooth, continuous predictions, whereas tree-based models produce 'step-functions' that look like a staircase when plotted.

What is the limitation of tree-based models regarding extrapolation?

Tree-based models cannot extrapolate; they can only predict values within the range of the target variables seen during training.

How does the interpretation of feature importance differ between coefficients and variable importance rankings?

Coefficients (̢) provide a precise unit-change impact, while variable importance provides a relative ranking of which features were most useful for making splits.

When is it professionally appropriate to choose a linear model over a more accurate tree-based model?

A linear model is preferred when regulatory transparency is required, the dataset is small, or the underlying relationships are known to be simple and additive.

What are the primary advantages of using a tree-based model like a GBM or Random Forest?

The primary advantages are superior predictive accuracy, automatic detection of complex interactions, and better handling of missing values or outliers.

Ready to test yourself?

Turn these notes into exam-style practice. Get unlimited AI questions on this topic with instant marking and explanations.

Practice This Topic

More Exam ATPA – Advanced Topics in Predictive Analytics chapters

* The content provided by thinka is generated by AI and may not always be accurate or up-to-date. Please use it as a supplementary resource and verify with official materials.

Put These Notes into Practice

Reading the notes is a great start. Now practise with unlimited AI-generated questions and get instant feedback. 100,000+ students are already improving their grades.

Start Practising Now View Pricing

Done reading? Test yourself with AI practice questions

Practice This Topic Now