Welcome to the World of Graduation!
In actuarial work, we often deal with "messy" data. When we look at raw mortality rates, they often jump up and down randomly because of small sample sizes or unexpected events. But in the real world, we expect mortality to follow a relatively smooth pattern as people age.
Graduation is simply the process of smoothing these "bumpy" raw rates to produce a model that is both practical and realistic. Think of it like using a photo editing app to smooth out the digital noise in a grainy picture so you can see the clear image underneath.
In this chapter, we will learn why we graduate data, the methods we use to do it, and most importantly, the statistical tests we use to check if we’ve done a good job!
1. Why do we Graduate?
Imagine you are looking at the death rates for 50-year-olds and 51-year-olds. If your data shows that 50-year-olds are twice as likely to die as 51-year-olds just because of a few random accidents in your small sample, you wouldn't want to use those "crude" rates to price insurance. It wouldn't make sense!
We graduate data for three main reasons:
1. To remove random fluctuations: Raw data is "noisy." We want to see the underlying trend.
2. To ensure regularity: We expect rates to progress logically (e.g., mortality generally increases with age).
3. Practicality: It is much easier to calculate premiums and reserves using a smooth mathematical formula or a reliable table than using jagged, raw data points.
The Great Balancing Act: Fit vs. Smoothness
When graduating data, you are caught in a tug-of-war between two goals:
• Adherence to data (Goodness of Fit): How closely your smooth curve stays to the actual observed points.
• Smoothness: How "flat" or "curveless" the progression is.
Analogy: If you connect every single dot on a graph with a jagged line, you have 100% adherence but 0% smoothness. If you draw a perfectly straight horizontal line, you have 100% smoothness but probably a very poor fit to the data. We want the "Goldilocks" middle ground!
2. Methods of Graduation
There are three main ways the curriculum suggests we can smooth our data. Don't worry if these seem technical; focus on the logic of each one.
A. Graphic Method
This is the "old school" way. You literally plot the crude rates on a graph and draw a smooth line through them by hand (or using software).
• Pros: Very intuitive; you can use your expert judgment.
• Cons: It's subjective. Two different actuaries will draw two different lines!
B. Graduation by Mathematical Formula (Parametric)
We assume the mortality rates follow a specific formula, like the Gompertz-Makeham law:
\( \mu_x = A + Bc^x \)
• Pros: Very smooth; easy to program into a computer.
• Cons: If the population doesn't actually follow that specific formula, the "fit" will be terrible.
C. Graduation by Reference to a Standard Table
We take an existing, well-known table (like a national mortality table) and "tweak" it to fit our specific data. We usually use a linear transformation:
\( \dot{q}_x = a + b \cdot q_x^s \)
(Where \( q_x^s \) is the standard table rate and \( \dot{q}_x \) is our graduated rate).
• Pros: Great when you have a small dataset; you "borrow" the shape of a larger, more reliable population.
• Cons: You are limited by the shape of the standard table you choose.
Quick Summary: Graduation is the art of balancing smoothness and fit. We can do it by hand (Graphic), by formula (Parametric), or by using a "big brother" table (Reference).
3. Statistical Tests: Did we do it right?
Once we have our graduated rates, we need to prove they are good. We use several tests to check if our graduated rates (\( \dot{E}_x \)) match our observed data (\( d_x \)) without being biased.
In all these tests, we look at residuals (the difference between what we observed and what our model predicted).
The Chi-Square (\( \chi^2 \)) Test (The Overall Fit Test)
This test looks at the total difference across all ages. We want the overall "error" to be small.
The formula is: \( \sum \frac{(d_x - E_x \dot{q}_x)^2}{E_x \dot{q}_x(1 - \dot{q}_x)} \)
• Null Hypothesis (\( H_0 \)): The graduated rates are the "true" rates.
• Decision: If our calculated value is less than the critical value from the table, we "fail to reject" the graduation. It’s a good fit!
Standardised Deviations Test (The Outlier Test)
We check if the individual deviations (errors) at each age follow a Standard Normal Distribution \( N(0, 1) \).
• What to look for: We expect about 95% of our deviations to fall between -1.96 and +1.96. If we see a value like +4.0, our graduation is failing at that specific age.
Signs Test (The Bias Test)
This is a simple but powerful test. We count how many times our graduated rates were higher than the data (+) and how many times they were lower (-).
• The Logic: If our model is unbiased, it should be like flipping a coin. We expect roughly an equal number of pluses and minuses.
• Common Mistake: Students often confuse this with the Grouping of Signs test. Remember: The Signs test is about the total number of pluses vs minuses.
Grouping of Signs Test (The "Clump" Test)
This test looks for runs of the same sign. For example, if your model is lower than the data for 10 ages in a row, your curve is "cutting under" the data in a specific section.
• Analogy: If you flip a coin 20 times and get 10 heads and 10 tails, that's normal (Signs Test). But if you get 10 heads in a row and then 10 tails in a row, something weird is happening (Grouping of Signs Test)!
Serial Correlations Test (The Pattern Test)
This looks for correlations between deviations at age \( x \) and age \( x+1 \).
• Goal: We want the errors to be random. if a positive error today usually means a positive error tomorrow, our model is missing a trend.
Did you know? The Stevens-Dodd method is often used to calculate the distribution of runs for the Grouping of Signs test. It’s one of those specific actuarial tools that makes this test more precise!
4. Summary of Tests - At a Glance
1. Chi-Square: Checks the overall size of errors.
2. Signs Test: Checks if there are too many positives or negatives (overall bias).
3. Grouping of Signs: Checks if positives or negatives are "clumping" together (local bias).
4. Standardised Deviations: Checks if any single age has a massive error.
5. Serial Correlation: Checks for patterns in the sequence of errors.
5. Common Pitfalls to Avoid
• Degrees of Freedom: In the Chi-Square test, remember to subtract the number of parameters you estimated. If you used a formula with 3 constants (\( A, B, c \)), you must subtract 3 from your degrees of freedom!
• Small Samples: If the number of deaths is very small (less than 5), the Chi-Square test might not be valid. You might need to group ages together.
• Over-smoothing: If you pass all the tests for smoothness but your Chi-Square value is huge, you’ve smoothed away the actual features of the data!
Key Takeaway: Graduation is successful if the resulting rates are smooth, represent the underlying trend of the data, and pass the statistical tests for bias and goodness-of-fit. Don't just look for one test to pass; a good actuary looks at all of them together to tell the full story.
Keep practicing those Chi-Square calculations! They are a staple in CS2 exams. You've got this!