Introduction to Principal Component Analysis (PCA)

Welcome! In this chapter, we are exploring one of the most powerful tools in an actuary's data toolkit: Principal Component Analysis (PCA). If you have ever felt overwhelmed by a dataset with dozens of columns, PCA is here to help. It is a technique used to "simplify" complex data without losing the important stories the data is trying to tell.

Think of PCA like taking a high-resolution, 3D scan of a complex object and finding the perfect 2D angle to photograph it so that you can still recognize almost everything about it. In actuarial terms, we use it for dimensionality reduction—turning many variables into a few essential ones.

What is Dimensionality Reduction?

In many actuarial problems, we deal with a high number of explanatory variables. For example, when pricing insurance, you might have data on a driver's age, years of experience, engine size, local crime rates, and even the color of the car. This is "high-dimensional" data.

Dimensionality reduction is the process of reducing the number of variables under consideration by obtaining a set of "principal" variables. The goal is to:

  • Simplify the dataset.
  • Remove noise and redundancy (correlated variables).
  • Make it easier to visualize and model the data.

Key Takeaway: PCA helps us compress data by focusing on the variables that provide the most "information" (which, in statistics, we measure as variance).

How PCA Works: The Conceptual Steps

Don't worry if the underlying linear algebra feels daunting; for CS1, the focus is on the purpose and application of the method. Here is the logical flow of how PCA transforms your data:

1. Standardizing the Data

Because variables are often measured in different units (e.g., age in years vs. income in thousands of dollars), we usually standardize them. This ensures that variables with larger absolute values don't unfairly dominate the analysis.

2. Identifying the Principal Components (PCs)

PCA looks for a new set of axes for the data.
The First Principal Component (\(PC_1\)) is the direction in the data that captures the maximum amount of variance.
The Second Principal Component (\(PC_2\)) is the direction that captures the next highest amount of variance, with one major rule: it must be uncorrelated (orthogonal) to \(PC_1\).

3. Transforming the Variables

Each Principal Component is a linear combination of the original variables. If your original variables are \(X_1, X_2, \dots, X_p\), then a Principal Component \(Z_i\) looks like this:
\(Z_i = a_{i1}X_1 + a_{i2}X_2 + \dots + a_{ip}X_p\)

The coefficients (\(a_{i1}, a_{i2}, \dots\)) are known as loadings. They tell us how much weight each original variable has in that specific component.

Key Features of Principal Components

There are a few "Golden Rules" about PCs that are very common in exam questions:

  • Maximum Variance: \(PC_1\) always explains the largest possible portion of the total variance in the data.
  • Uncorrelated: Every PC is completely uncorrelated with every other PC. This is a huge advantage for regression modelling later!
  • Descending Importance: The components are ordered. \(PC_1\) is more "informative" than \(PC_2\), which is more informative than \(PC_3\), and so on.
  • Total Variance: The sum of the variances of all PCs is equal to the sum of the variances of the original variables. We haven't "created" information; we've just rearranged it.

Did you know? In many actuarial datasets, the first two or three principal components often explain over 80% of the total variance, allowing you to discard the remaining dozens of variables with very little loss of information!

Interpreting the Output

In your Paper B exam, you will likely use software like R to run PCA. You will need to interpret two main things:

1. Proportion of Variance Explained (PVE)

This tells you what percentage of the total "information" each component carries.
If \(PC_1\) has a PVE of \(0.60\), it means it explains 60% of the variance in the entire dataset.

2. Cumulative Proportion

This is the running total of PVE. If you decide to keep the first three components, you look at the Cumulative Proportion of \(PC_3\) to see how much of the original data's story you are still telling.

Common Mistake: Students sometimes think they must keep all components. The whole point of PCA is to reduce dimensionality. If you have 10 original variables and you keep 10 PCs, you haven't simplified anything!

Choosing the Number of Components

How do we decide when to stop? How many PCs are "enough"? There are two common methods mentioned in the context of data analysis:

  • The Scree Plot: A simple line graph showing the variance explained by each PC. We look for the "elbow"—the point where the graph flattens out. We usually keep the components before the elbow.
  • The Threshold Method: We decide on a target, such as "explaining 80% or 90% of the total variance," and keep the minimum number of PCs required to hit that target.

Quick Review: Why use PCA in Actuarial Work?

1. Simplification: Dealing with 3 variables is much easier than dealing with 50.

2. Visualization: We can't visualize 10-dimensional data, but we can easily plot \(PC_1\) against \(PC_2\) on a 2D scatter plot to see if there are any natural clusters or patterns in our policyholder data.

3. Avoiding Multicollinearity: In regression (which you will study in Topic 4), highly correlated explanatory variables can cause problems. Since PCs are uncorrelated, using them as inputs into a regression model solves this issue perfectly.

Summary: PCA is a dimensionality reduction tool that transforms a large set of correlated variables into a smaller set of uncorrelated variables called Principal Components, ranked by the amount of variance they explain.