Welcome to Model Explainability!
Hello there! Welcome to one of the most important chapters in your ATPA journey: Aspects of Explainability. If you’ve ever built a complex model and felt like it was a "black box" that you couldn't quite explain to your boss or a client, you aren't alone. In this chapter, we’re going to open that box together.
As actuaries, we don't just need to be right; we need to be able to explain why we are right. Whether you are dealing with regulators, policyholders, or senior management, "the computer said so" is never an acceptable answer. Let’s dive in and learn how to make our models speak!
1. The Core Conflict: Accuracy vs. Interpretability
Before we look at specific tools, we need to understand the trade-off. Generally, there is a tug-of-war in predictive analytics:
1. Simple Models: Think Linear Regression or Decision Trees. They are easy to explain (high interpretability) but might not catch complex patterns (lower accuracy).
2. Complex Models: Think Random Forests, Gradient Boosting Machines (GBMs), or Neural Networks. They are incredibly powerful (high accuracy) but are hard to see inside of (low interpretability).
Explainability is the set of tools we use to bridge this gap, allowing us to use complex models while still providing clear reasons for their predictions.
2. Two Ways to Look at a Model: Global vs. Local
When we talk about explaining a model, we usually look at it from two different perspectives. Don't worry if these sound similar; the difference is just about zoom level.
Global Explainability (The Big Picture)
Global explainability focuses on how the model works as a whole. It answers the question: "Which features are most important across the entire dataset?"
Analogy: Imagine a recipe for a cake. Global explainability tells you that, in general, flour and sugar are the most important ingredients for any cake you bake.
Local Explainability (The Individual Case)
Local explainability focuses on a single, specific prediction. It answers the question: "Why did the model give THIS specific person a high risk score?"
Analogy: Now imagine a specific cake that tastes a bit salty. Local explainability tells you that for this specific cake, the salt was the most influential ingredient, even if flour is usually more important overall.
Quick Review:
- Global: How does the model behave overall?
- Local: Why did it make this specific prediction?
3. Model-Agnostic vs. Model-Specific Tools
There are two types of tools we can use to explain models:
1. Model-Specific: These tools only work for one type of model. For example, looking at the "coefficients" of a GLM or the "branch splits" of a single decision tree.
2. Model-Agnostic: These are the "Swiss Army Knives" of explainability. They can be used on any model, whether it’s a simple regression or a deep neural network. These are the focus of the ATPA curriculum.
4. Global Explainability Tools
Let's look at the two most common ways to explain a model globally.
Permutation Feature Importance
This is a clever way to see which variables the model relies on most. Here is how it works step-by-step:
1. Measure the model's original accuracy.
2. Pick one feature (like "Age") and randomly shuffle its values (this "breaks" the relationship between Age and the outcome).
3. Measure the accuracy again.
4. If the accuracy drops significantly, "Age" was very important. If the accuracy barely changes, "Age" wasn't doing much anyway.
Partial Dependence Plots (PDP)
A PDP shows the marginal effect of one or two features on the predicted outcome of a model. It helps us visualize the relationship (is it linear? curved? step-wise?).
Example: A PDP for "Vehicle Age" in an auto insurance model might show that as the car gets older, the predicted cost of claims goes down, but then levels off after 10 years.
Common Mistake to Avoid: PDPs assume that the feature you are looking at is not highly correlated with other features. If features are highly correlated, PDPs can show "impossible" data points (like a 10,000 sq ft house with only 1 bedroom).
5. Local Explainability Tools
When you need to justify a decision to a specific client, you use these tools.
Individual Conditional Expectation (ICE) Plots
Think of an ICE plot as the "local" version of a PDP. While a PDP shows the average effect, an ICE plot shows the effect for every single observation individually. If all the lines in an ICE plot follow the same shape, the PDP is a good representation. If the lines are going in all different directions, the model has complex interactions.
LIME (Local Interpretable Model-agnostic Explanations)
LIME works by taking a specific data point and "wiggling" the data around it to see how the predictions change. It then builds a very simple, "surrogate" model (like a straight line) that approximates the complex model just in that small neighborhood.
Mnemonic: LIME is like a Magnifying Glass. It ignores the whole map and just looks at one tiny spot very closely.
SHAP (Shapley Additive Explanations)
SHAP is currently the "gold standard" for explainability. It is based on Game Theory. It treats each feature as a "player" in a game and calculates how much "credit" each player deserves for the final score.
The math is complex, but the idea is simple: It calculates the difference between the prediction and the average prediction, then assigns pieces of that difference to each feature.
Key Concept: SHAP values are additive. This means:
\( \text{Baseline Value} + \text{SHAP Feature 1} + \text{SHAP Feature 2} + \dots = \text{Final Prediction} \)
6. Summary of Explainability Aspects
To help you study, here is a quick cheat sheet for the methods we discussed:
- Variable Importance: Global. Tells you "what" matters most.
- PDP: Global. Tells you "how" a feature affects the average prediction.
- ICE: Local. Tells you "how" a feature affects one specific instance.
- LIME: Local. Creates a simple model to explain a complex one locally.
- SHAP: Local (can be aggregated to Global). Mathematically fair way to distribute "blame" or "credit" to features.
Final Pro-Tip for the Exam
Don't worry if the math behind SHAP or LIME feels overwhelming. On the ATPA exam, you are more likely to be asked to interpret a chart or justify why you chose a specific explainability method. Always link your explanation back to the business problem. If the goal is to explain a rate increase to a policyholder, a SHAP waterfall chart is usually your best friend because it shows exactly how you got from the "average" price to "their" price!
Key Takeaway: Explainability turns a "Black Box" into a "Glass Box." It builds trust, ensures fairness, and helps you validate that your model is actually learning what you think it’s learning.