Perform checks on the intermediate and final results of a model

Welcome to Model Testing: Ensuring Your Results Make Sense!

In the world of actuarial modelling, building a complex spreadsheet is only half the battle. The other half—and arguably the more important part for your CP2 exam—is proving that your model actually works. Think of this like baking a tiered wedding cake: you wouldn't wait until the very end to taste the frosting, right? You'd check the ingredients, the batter, and the sponge along the way. That is exactly what we do when we perform checks on intermediate and final results.

In this chapter, we will learn how to verify that our "ingredients" (inputs) and "batter" (intermediate formulas) lead to a "perfect cake" (the final output). This is a core part of Model Testing, and getting this right is the difference between a pass and a fail.

Don't worry if this seems tricky at first! Many students feel overwhelmed by how much there is to check. We will break it down into simple, manageable steps.

1. Intermediate vs. Final Results: What’s the Difference?

Before we dive into the "how," let’s clarify the "what."

Intermediate Results

These are the "stepping stones" inside your model. They are the calculations that happen between your raw data and your final answer.
Example: If you are calculating a pension projection, an intermediate result might be the salary at age 45 or the investment return in year 5.

Final Results

These are the "headlines"—the numbers the client actually asked for.
Example: The Total Net Present Value (NPV) of a project or the Total Premium to be charged.

Quick Tip: You must check both! A correct final result could sometimes be a "fluke" caused by two offsetting errors in your intermediate steps. Checking intermediate results ensures your logic is sound throughout.

2. Key Types of Checks

There are four main "flavors" of checks you should perform. You can remember them with the mnemonic S.A.N.E.:

1. Spot Checks (Manual Recalculation):
Pick one row of data and calculate it manually (using a calculator or a simple side-calculation). If your manual result is \( 105.50 \) and your model says \( 105.50 \), your formula is likely correct.
Analogy: Double-checking the math on a restaurant bill using your fingers.

2. Aggregate Checks (High-Level):
Look at the big picture. If you have \( 1,000 \) policies and the average premium is \( \$500 \), your total premium should be around \( \$500,000 \). If the model says \( \$5,000,000 \), you’ve probably got a decimal point error!
\nKey Term: Reasonableness Check.

\n\n

3. Negative/Extreme Checks:
\nAsk yourself: "Can this number be negative?" If you are calculating the "Number of Lives," and the result is \( -5 \), something is wrong! Also, check if percentages stay between \( 0\% \) and \( 100\% \).

\n\n

4. Evolution/Trend Checks:
\nLook at how results change over time. If you are projecting a population, it should generally grow or shrink smoothly. A sudden, massive jump in year 3 usually indicates a formula error in that specific cell.

\n\n

Key Takeaway: Never just trust the spreadsheet. Always apply a "sanity test" to the numbers you see.

\n\n

3. Step-by-Step: How to Perform a Check

\n

When you are in the CP2 exam, follow these steps to perform and document your checks effectively:

\n\n

Step 1: Identify the key metric. Pick a result that significantly impacts the final output (e.g., a cumulative discount factor).

\n

Step 2: Use an alternative method. Don’t just look at the formula again. Use a calculator, or a simplified version of the formula, to see if you get the same result.

\n

Step 3: Document it! In CP2, if you don't write it down, it didn't happen. Note: "I checked the calculation for Policy ID #123 manually by multiplying \( X \) by \( Y \), and it matched the model output of \( Z \)."

\n

Step 4: Fix or Explain. If the check fails, find the error. If the result looks weird but is actually correct, explain why (e.g., "The spike in year 5 is due to the one-off maturity payment").

\n\n

4. Common Pitfalls to Avoid

\n

Even the best students fall into these traps. Keep an eye out for them:

\n

- The "Total" Trap: Checking only the grand total. Always check a few individual rows as well!

\n

- Ignoring Units: Is the result in \( \$ \), \( \$000s \), or \( \% \)? Mixing these up is the most common cause of "unreasonable" results.

- Circular Logic: Don't check a formula by using the same formula. That's like checking a dictionary to see if a word is spelled right, when the dictionary itself has a typo.

- Forgetting the "Direction": If interest rates go up, the present value should go down. If your model does the opposite, your logic is flipped.

5. Did You Know?

Did you know? In professional actuarial practice, "Peer Review" is a formal requirement. Actuaries often have a completely different person "re-perform" their checks to ensure no bias was involved. In CP2, you are your own peer reviewer, so you have to be extra critical of your work!

Summary Quick Review

Checklist for your Model:
- Are the Intermediate Results logically sound? (e.g., intermediate tax values are positive)
- Do Spot Checks match manual calculations? (Pick one row and grab your calculator)
- Are Final Results reasonable? (Do they pass the "smell test"?)
- Are Trends smooth? (No unexplained "jumps" in the data)
- Is every check Documented in the audit trail?

Final Thought: Testing isn't a separate task you do at the end; it's a mindset you maintain while building. By performing these checks, you build confidence in your results and secure those crucial "Model Testing" marks!