Variable transformations and interactions

Welcome to the World of Non-Linearity!

So far in your Exam SRM journey, you have likely been looking at Linear Models in their simplest form: "as \(X\) goes up, \(Y\) goes up by a fixed amount." While that is a great starting point, the real world (and actuarial data) is rarely that straightforward. Sometimes the relationship is curved, and sometimes the impact of one variable depends entirely on another variable.

In this chapter, we will explore Variable Transformations (changing the shape of our variables) and Interactions (looking at how variables work together). These tools allow our linear models to capture much more complex, real-world patterns without losing the mathematical "magic" of Ordinary Least Squares (OLS).

1. Variable Transformations

A transformation involves applying a mathematical function to a predictor \(X\), the response \(Y\), or both. We do this when the raw data doesn't fit a straight-line relationship.

A. Logarithmic Transformations

The log transformation (usually the natural log, \( \ln \)) is one of the most common tools in an actuary's toolkit. It is especially useful when data is highly skewed or when we expect changes to happen in percentages rather than absolute units.

• Log-Linear Model: \( \ln(Y) = \beta_0 + \beta_1 X + \epsilon \). Here, a 1-unit change in \(X\) is associated with a \( (e^{\beta_1} - 1) \times 100\% \) change in \(Y\).
• Log-Log Model: \( \ln(Y) = \beta_0 + \beta_1 \ln(X) + \epsilon \). Here, \( \beta_1 \) represents elasticity—a 1% change in \(X\) leads to a \( \beta_1 \)% change in \(Y\).

Quick Tip: If you see a plot where the points fan out like a trumpet (heteroscedasticity), a log transformation on \(Y\) often helps "stabilize" that variance!

B. Polynomial Regression

Sometimes the relationship between \(X\) and \(Y\) is curved (like a U-shape or an S-shape). To capture this, we can add polynomial terms like \(X^2\) or \(X^3\) to the model.

\( Y = \beta_0 + \beta_1 X + \beta_2 X^2 + \dots + \beta_p X^p + \epsilon \)

Even though the relationship with \(X\) is curved, this is still a linear model. Why? Because the model is "linear in the parameters" (\(\beta\)). We are just treating \(X^2\) as if it were a new, separate variable.

Common Mistake: Don't go overboard! Adding very high-degree polynomials (like \(X^8\)) can lead to overfitting, where the model follows the "noise" in the data rather than the true underlying pattern.

2. Interaction Terms

In a standard linear model, we assume the effect of \(X_1\) on \(Y\) is the same, regardless of the value of \(X_2\). But what if that isn't true?

Example: Think about the effect of Exercise (\(X_1\)) and Diet (\(X_2\)) on Weight Loss (\(Y\)). Exercise might be more effective at causing weight loss if the person is also on a healthy diet. This "synergy" is an interaction.

A. The Interaction Formula

We represent an interaction by multiplying two predictors together:

\( Y = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 (X_1 \times X_2) + \epsilon \)

In this model:

• \( \beta_1 \) and \( \beta_2 \) are the main effects.
• \( \beta_3 \) is the interaction effect.

B. Interpreting the Change

You can rewrite the equation to see how the "slope" of \(X_1\) changes:

\( Y = \beta_0 + (\beta_1 + \beta_3 X_2) X_1 + \beta_2 X_2 + \epsilon \)

The "effective slope" for \(X_1\) is now \( (\beta_1 + \beta_3 X_2) \). This means the impact of \(X_1\) depends on the value of \(X_2\).

C. Interactions with Qualitative (Dummy) Variables

This is a very common SRM exam topic. If you have a quantitative variable (e.g., Age) and a dummy variable (e.g., Gender, where Male = 1), adding an interaction term allows for different slopes for each group.

• No Interaction: Parallel lines (different intercepts, same slope).
• With Interaction: Non-parallel lines (different intercepts and different slopes).

Did you know? In actuarial pricing, we often use interactions between "territory" and "age" because young drivers might be riskier in cities than they are in rural areas.

3. The Hierarchical Principle

This is a "golden rule" for the SRM exam. The Hierarchical Principle states that if we include an interaction term (\(X_1 X_2\)) in our model, we should always include the main effects (\(X_1\) and \(X_2\)) as well.

Why? Even if the p-values for \(X_1\) or \(X_2\) are not significant on their own, the interaction term is difficult to interpret without them. The main effects provide the baseline from which the interaction deviates.

Example: If your software output shows that \(X_1 X_2\) has a tiny p-value (significant) but \(X_1\) has a large p-value (not significant), keep both in the model!

4. Model Selection Considerations

When you add transformations (like \(X^2\)) or interactions (\(X_1 X_2\)), you are adding complexity to your model. This increases the number of predictors, \(d\).

Recall from your study of model selection (covered in a separate chapter) that adding variables will always decrease the Residual Sum of Squares (RSS) on the training data, but it might hurt your performance on new data. To decide if a transformation or interaction is actually "worth it," we use:

• T-tests: To see if the coefficient of the new term is significantly different from zero.
• F-tests: To see if a group of new terms (like \(X^2\) and \(X^3\)) improves the model.
• AIC/BIC: To penalize the model for having too many parameters. (Refer to the Model Selection Criteria chapter for these specific formulas).

Key Takeaways

1. Transformations: Use logs for percentages/skewed data and polynomials for curves. They allow linear models to fit non-linear shapes.

2. Interactions: Use these when the effect of one variable changes based on another variable. They appear as product terms (e.g., \(X_1 \times X_2\)) in the regression equation.

3. Hierarchy: If the interaction is in, the main effects must stay in. No exceptions!

4. Complexity vs. Fit: Use statistical tests (t-tests, F-tests) and criteria (AIC, BIC) to ensure your transformations and interactions are actually adding value to the model rather than just creating noise.