Explaining why a model predicts certain values

Welcome to the World of Model Explainability!

In the past, many advanced predictive models were seen as "black boxes"—you put data in, and a prediction comes out, but nobody really knows how the model made its decision. As an actuary, "because the computer said so" is never an acceptable answer! In this chapter, we are going to learn how to peek inside that black box. We'll explore the tools that help us explain why a model predicts a certain value, both for the model as a whole and for individual cases.

Whether you are explaining a rate increase to a regulator or telling a customer why their insurance premium changed, these concepts are your best friends. Let’s dive in!

1. Global vs. Local Interpretability

Before we look at specific tools, we need to understand the two levels of "explaining" a model. Don't worry if these terms sound fancy; the concepts are quite simple!

Global Interpretability (The "Big Picture")

Global interpretability is about understanding how the model works across the entire dataset. It answers questions like: "Generally speaking, which variables are the most important for predicting the outcome?" or "How does the predicted value change as Age increases for everyone in the data?"

Local Interpretability (The "Individual Story")

Local interpretability focuses on a single, specific prediction. It answers questions like: "Why did the model predict a high risk for John Doe specifically?" Even if "Age" is important globally, for John Doe, a "Recent Accident" might be the reason his score is high.

Quick Analogy: Imagine a recipe for a cake. Global interpretability is knowing that, in general, sugar makes the cake sweet and flour gives it structure. Local interpretability is explaining why one specific cake turned out salty (maybe you accidentally used salt instead of sugar in that one batch!).

Summary: Global = The Whole Model. Local = One Specific Prediction.

2. Feature Importance

Feature Importance (or Variable Importance) is one of the most common ways to explain a model. It provides a score for each input variable (feature) based on how much it contributes to the model's predictive power.

In Tree-based models (like Random Forests or XGBoost), importance is often calculated by looking at how much each variable reduces "impurity" (like Gini impurity or Mean Squared Error) across all the trees.

Common Pitfall to Avoid: Just because a variable has high importance doesn't mean it causes the outcome. It just means the model relies on it heavily to make predictions. Also, if two variables are highly correlated, the model might split the importance between them, making both look less important than they actually are!

3. Partial Dependence Plots (PDP)

A Partial Dependence Plot (PDP) shows the marginal effect of one or two features on the predicted outcome. In simpler terms: if we change the value of Variable X while keeping everything else the same (on average), how does the prediction change?

How it works (Step-by-Step):

1. Pick a variable you want to study (e.g., Vehicle Age).
2. For every single row in your dataset, keep all other variables exactly as they are, but change Vehicle Age to "1 year old."
3. Record the average prediction for the whole dataset.
4. Repeat this for "2 years old," "3 years old," and so on.
5. Plot the averages on a graph.

Key Limitation: PDPs assume that the feature you are studying is not highly correlated with other features. If they are, the "average" scenarios the PDP creates might be physically impossible in the real world.

Summary: PDPs show the average relationship between a feature and the prediction.

4. Individual Conditional Expectation (ICE) Plots

If PDP is the "average," ICE plots are the "individual components." While a PDP shows one line (the average), an ICE plot shows a separate line for every single observation in your dataset.

Why use ICE? Sometimes, the "average" hides the truth. For example, a drug might help men but hurt women. A PDP would show a flat line (zero effect on average), but an ICE plot would show some lines going up and some lines going down, revealing the interaction.

Memory Aid: Think of ICE as "Individual." It breaks the "ice" of the average to show you what's happening to each person.

5. SHAP Values (Shapley Additive Explanations)

SHAP is currently the "gold standard" for model explainability in predictive analytics. It is based on Game Theory.

Imagine a group of players (Features) playing a game to win a prize (the Prediction). SHAP calculates how much "credit" each player deserves for the final prize. The mathematical foundation ensures that the total prediction is fairly distributed among the features.

The SHAP Equation (Simplified):

\( g(z') = \phi_0 + \sum_{i=1}^{M} \phi_i z'_i \)

Where:
- \( \phi_0 \) is the base value (the average prediction for all observations).
- \( \phi_i \) is the SHAP value for feature \( i \).
- The sum of all SHAP values plus the base value equals the actual prediction.

Why SHAP is great:

1. Local and Global: You can use SHAP for a single person (Local) or average them to see the whole model (Global).
2. Fairness: It accounts for the fact that features might work together (interactions).
3. Directional: Unlike simple importance, SHAP tells you if a feature is pushing the prediction up or down.

Quick Review: If an insurance applicant's score is 80 (where the average is 50), SHAP might tell you: +20 for "Previous Accidents," +15 for "Young Age," and -5 for "Safe Vehicle Features." (50 + 20 + 15 - 5 = 80).

6. LIME (Local Interpretable Model-agnostic Explanations)

LIME is another tool for local interpretability. The "Model-agnostic" part means it works on any model—Random Forests, Neural Networks, or even models you didn't build yourself!

How LIME works (The "Surrogate" Approach):

1. Pick a single observation you want to explain.
2. "Wiggle" the data slightly around that observation (create many similar but slightly different data points).
3. See how the complex black-box model predicts these new points.
4. Build a very simple, interpretable model (like a simple Linear Regression or a small Decision Tree) on this local "wiggled" data.
5. Use that simple model to explain the complex one.

Analogy: Imagine a giant, winding mountain range (the complex model). If you want to explain the slope at one specific spot, you don't need a map of the whole range. You just need a small, flat piece of plywood (the simple LIME model) to show the angle of the ground right under your feet.

7. Key Differences and When to Use What

Don't let the variety of tools overwhelm you! Here is a quick guide:

Use Feature Importance when:

You need a quick, high-level ranking of which variables matter most for the model's accuracy.

Use PDP/ICE when:

You want to see the shape of the relationship (e.g., is it a straight line? Does it level off after a certain age?).

Use SHAP when:

You need a theoretically sound, precise explanation of how much each variable contributed to a specific individual's prediction.

Use LIME when:

You have a very complex model (like deep learning for images or text) and SHAP is taking too long to calculate, or you need a simple "local" surrogate model.

Summary Checklist for the Exam

- Global vs. Local: Know that Global is the whole population; Local is the individual observation.
- PDP: Shows the average effect. Beware of correlations!
- ICE: Shows individual lines; useful for spotting interactions that the average (PDP) hides.
- SHAP: Uses Game Theory to "distribute" the prediction value among features. Sums up to the total prediction.
- LIME: Fits a simple, local model around a specific point to explain it.

Don't worry if these seem tricky at first! The key is to remember the goal: we are turning "The computer says so" into "The computer says so because Feature A pushed the result up and Feature B pushed it down." You've got this!