Comment on the reasonableness of the results from a model under different scenarios

Introduction to Reasoning with Your Results

Welcome to one of the most critical stages of the CP2 exam! You’ve built your model, the formulas are linked, and the numbers are flowing. But here is the big question: Do the numbers actually make sense?

In the "Model Testing" phase, commenting on the reasonableness of results is where you prove you aren’t just a spreadsheet builder, but an actuary. This chapter will teach you how to look at your model’s output under different scenarios and explain whether those results are logical or if they suggest a hidden error. Don't worry if this seems a bit abstract at first—it’s essentially a "sanity check" for your work.

What does "Reasonableness" actually mean?

Reasonableness means checking if the model’s output behaves the way we expect it to when we change the inputs. Think of it like cooking: if you double the amount of salt in a soup, you expect it to taste much saltier. If it suddenly tastes like sugar, you know something went wrong in the kitchen!

In CP2, we test reasonableness by looking at Scenarios. A scenario is just a set of "What if?" conditions. For example, "What if interest rates rise by 1%?" or "What if the mortality rate doubles?"

The Two Golden Rules of Reasonableness

When you comment on results under different scenarios, you should always check two things:

1. Direction of Change: Did the result move up or down as expected?
2. Magnitude of Change: Did it move by a sensible amount?

1. Direction of Change

This is the most basic check. If you increase an expense assumption, your projected profit should decrease. If the profit increases instead, you likely have a sign error in your formula (e.g., you added expenses instead of subtracting them).

Example: If you are calculating the Present Value (PV) of a future payment:
\( PV = \frac{Future Value}{(1 + i)^n} \)
If the interest rate \( i \) increases, the PV must decrease. If your model shows the PV going up, the direction is wrong, and your model is unreasonable.

2. Magnitude of Change

This asks: "Is the size of the change sensible?" If you increase the inflation rate by 0.1%, and your total project cost jumps by 500%, that is likely unreasonable. The change is too large for such a small tweak in the input.

Quick Review Box:
• Reasonable: Direction is correct AND magnitude is proportional.
• Unreasonable: Wrong direction OR "explosion" in numbers (unjustified magnitude).

How to Step-by-Step: Testing a Scenario

When you are asked to comment on the reasonableness of a scenario in the exam, follow these steps:

Step 1: Identify the Driver. Look at the input you are changing. Is it a "cost" (bad for profit) or an "income" (good for profit)?
Step 2: Form an Expectation. Before looking at the numbers, say to yourself: "I expect the final result to decrease because costs are higher."
Step 3: Observe the Result. Look at your model output. Did it do what you expected?
Step 4: Quantify and Explain. Don't just say "it changed." Say "The result decreased by 5%, which is reasonable as it reflects the increased cost of capital over the 10-year term."

Testing Extreme Scenarios (Sensitivity Analysis)

Sometimes, the best way to see if a model is working is to push it to its limits. This is often called Stress Testing.

Analogy: To test if a bridge is strong, you don't just put one car on it; you imagine what happens if 100 trucks cross it at once.

Common Extreme Tests:
• Zero Test: What happens if the interest rate is 0%? The Present Value should simply equal the sum of the cashflows.
• Infinite Test (Theoretical): What happens if a growth rate is extremely high? The values should skyrocket.
• Null Test: If there are no members in a pension scheme, the total liability must be zero.

Did you know? Checking extreme scenarios is one of the fastest ways to find "hard-coded" numbers in your spreadsheet that should actually be formulas!

Common Mistakes to Avoid

1. Forgetting the "Why": Many students state what happened ("The profit went down") but forget to explain why it is reasonable ("This is expected because the commission paid to agents has increased").
2. Ignoring Interactions: In complex models, changing one variable might affect another. For example, higher inflation might lead to higher salary increases. If your scenario only changes one but ignores the logic of the other, the result might look "unreasonable" even if the formula is correct.
3. Mixing up Percentage Points and Percentages: Increasing an interest rate from 4% to 5% is a 1 percentage point increase, but a 25% relative increase. Be clear in your comments!

Memory Aid: The "S.A.N.E." Check

Use this mnemonic when writing your comments on reasonableness:
S - Sign: Is the change positive or negative as expected?
A - Amount: Is the size of the change realistic?
N - Nature: Does the change reflect the underlying physics/economics of the problem?
E - Explanation: Have I clearly linked the cause (input) to the effect (output)?

Summary and Key Takeaways

• Context is King: A result is only "reasonable" in the context of the specific scenario being tested.
• Consistency: Check if the results are consistent with the initial base case.
• The "Gut" Test: If a number looks "weird," investigate it. Don't just assume the model is right and you are wrong.
• Professionalism: In the exam, the markers want to see that you can interpret data, not just calculate it. Clear, logical comments on reasonableness earn high marks.

Final Tip: If your results look unreasonable and you can't find the error in time, admit it in your summary! Explain why you think the result is wrong (e.g., "I expected a decrease, but the model showed an increase; there may be a sign error in the discounting formula"). You can still get marks for demonstrating actuarial judgment even if your spreadsheet has a bug.