Select and carry out appropriate statistical tests of reasonableness

Welcome to the "Sanity Check" Phase!

Hello there! You’ve reached a vital part of the CP2 journey. In this chapter, we are focusing on Select and carry out appropriate statistical tests of reasonableness. This is part of the "Preparation and analysis of data" section. Think of this as being a data detective. Before you build a complex model or hand over results to a client, you need to ask yourself: "Does this actually make sense?"

Don't worry if statistics feels a bit intimidating. We aren't doing heavy theoretical proofs here. We are learning how to use simple tools to make sure our data isn't lying to us. Let’s dive in!

1. What is a "Test of Reasonableness"?

In the actuarial world, we often deal with massive spreadsheets. A "Test of Reasonableness" is a check to ensure that the data we are using (input) and the results we are producing (output) are plausible.

Analogy: Imagine you are checking your bank statement. If you see a charge for $10 for a coffee, that’s reasonable. If you see a charge for $10,000 for a coffee, your "reasonableness alarm" should go off! That’s exactly what we are doing with model data.

Why is this important?
If you put "garbage" data into a perfect model, you will get "garbage" results. This is often called GIGO (Garbage In, Garbage Out). By carrying out these tests, you protect the integrity of your work.

2. Selecting the Right Statistical Tests

Not every test fits every situation. You need to choose your tools based on the type of data you have. Here are the most common statistical measures you will use in CP2:

A. Measures of Location (The "Middle")

Mean (Average): The sum of all values divided by the number of values. \( \bar{x} = \frac{\sum x_i}{n} \).
Median: The middle value when data is sorted. It’s great because it isn't easily swayed by one or two crazy-high numbers.
Mode: The most frequent value. Useful for categorical data (like "Policy Type").

B. Measures of Spread (The "Range")

Minimum and Maximum: Always check the smallest and largest values. If you are looking at "Ages of Pensioners" and the Max is 150, you’ve found a data error!
Standard Deviation: This tells you how spread out the data is. A very high standard deviation might mean the data is very volatile or contains errors.
Range: The difference between the Max and Min.

C. Counts and Totals

Simply counting the number of records or summing a column (like "Total Sum Assured") is a powerful check. If you had 1,000 policies last year and only 50 this year, something might be missing.

Quick Review Box:
- Use Mean for a general average.
- Use Median if the data has outliers (very high or very low values).
- Use Min/Max to find impossible values (like negative ages).
- Use Counts to ensure no data was lost during a transfer.

3. Step-by-Step: Carrying Out the Tests

When you are in the CP2 exam, you should follow a systematic approach to check your data:

Step 1: High-Level Summary
Produce a summary table of the data. Calculate the count, mean, min, and max for all key numerical columns.

Step 2: Compare with External or Prior Data
If you have data from last year, compare it! If the average claim size was $500 last year and it is $5,000 this year, you need to investigate why.

Step 3: Internal Consistency Checks
Check if the data makes sense relative to itself.
Example: Is the "Date of Death" later than the "Date of Birth"? Is the "Retirement Age" greater than the "Current Age"?

Step 4: Spot Checks
Pick 2 or 3 individual rows of data at random and "walk" them through your calculations manually. This is a great way to catch logic errors in your formulas.

Did you know?
In the professional world, actuaries often use "reconciliation." This means matching the total of your processed data back to the original source provided by the client to ensure not a single penny is lost.

4. Dealing with Outliers

An outlier is a data point that is significantly different from the others. For example, if most claims are $100, but one claim is $1,000,000.

How to handle them:
1. Investigate: Is it a typo? (e.g., someone hit '0' too many times).
2. Verify: If it's a real but extreme value, it should stay in the data, but you should mention it in your report.
3. Correct/Exclude: If it's clearly a mistake and you can't fix it, you might need to exclude it (and document that you did so!).

Memory Aid: The "S.C.A.N." Method
Summarize (get the mean/min/max)
Compare (against last year or benchmarks)
Analyze (look for outliers)
Notes (document what you found and why it’s okay)

5. Common Pitfalls to Avoid

1. Ignoring the Context: A value might be mathematically possible but actuarially impossible. A 105-year-old person is possible, but a 105-year-old person with a 20-year term life insurance policy is very unlikely!
2. Forgetting Categorical Data: Don't just check numbers. Check if the categories (like "Gender" or "Region") have sensible codes. If the codes are "M" and "F", but you see an "X", that’s a data issue.
3. Not Documenting: In CP2, you get marks for showing your checks. If you check the data and it’s fine, write down: "I checked the Max Age and it was 85, which is reasonable."

Summary and Key Takeaways

Key Takeaway 1: Reasonableness tests are your first line of defense against model errors. Always perform them before starting your main analysis.

Key Takeaway 2: Use a mix of location measures (Mean/Median) and spread measures (Min/Max/Std Dev) to get a full picture of the data.

Key Takeaway 3: Always compare your data against something else—whether it's last year's data, a different source, or just common sense.

Key Takeaway 4: Documentation is king. If you found an error and fixed it, or found a weird value and confirmed it was correct, write it down!

Don't worry if this seems like extra work at first. With practice, these checks will become second nature, and they will save you from making big mistakes in your models!