Introduction: Why Models Need a Reality Check

Imagine you have built a complex model to predict the cost of insurance claims over the next ten years. You have the best data, the smartest formulas, and the most modern software. But there is a problem: the future is uncertain. What if inflation is higher than you thought? What if people live longer than expected? What if your software crashes right before the deadline?

In this chapter, we look at two critical parts of the Developing the Solution phase of the Actuarial Control Cycle:

  1. Sensitivity Analysis: Testing how "sensitive" your results are to changes in your assumptions.
  2. Operational Model Issues: The practical, "real-world" challenges of actually building and running a model without it breaking or giving the wrong answers.

Don’t worry if these sound like technical IT topics; in CP1, we focus on the principles and how they help an actuary give sound financial advice.

Section 1: Sensitivity Analysis

What is Sensitivity Analysis?

Sensitivity analysis involves changing one input or assumption in your model at a time to see how much the final output changes. For example, if you increase the discount rate by \(1\%\), does the value of the provisions drop by \(2\%\) or \(20\%\)?

How it Helps Decision Making (Objective 4.1)

Decision-makers (like a Board of Directors) don't just want a single number; they want to know the risk around that number. Sensitivity analysis helps by:

  • Identifying Key Drivers: It shows which assumptions have the biggest impact. If a small change in mortality rates doubles the required capital, management needs to focus heavily on monitoring mortality.
  • Setting Risk Margins: It helps in deciding how much "extra" money (prudence) to keep in reserve to cover potential fluctuations.
  • Understanding "What If" Scenarios: It allows management to test the impact of different business strategies before committing to them.

Checking the Appropriateness of Values (Objective 4.5)

When valuing provisions (the money set aside to pay future benefits on contingent events), sensitivity analysis acts as a "sanity check."

If the results change wildly with a tiny, realistic change in an assumption, the model might be too unstable or the assumption might be too aggressive. It ensures the fair valuation of liabilities is robust and not just a "lucky" single estimate.

A Simple Mathematical View

If your provision \(P\) is a function of an assumption \(a\), we are looking at the change:

\( \Delta P = P(a + \Delta a) - P(a) \)

Where \( \Delta a \) is a small change in the input. If \( \Delta P \) is very large, the model is "highly sensitive" to assumption \(a\).

Quick Tip: In the exam, if you are asked how to "validate" a result, "perform sensitivity analysis" is almost always a valid point!

Section 2: Operational Model Issues – Designing the Model

Building a model isn't just about the math; it’s about making sure it works in a business environment. (Objective 4.1)

Operational Design Considerations

When designing a model, an actuary must balance several competing factors:

  • The Objective: What is the model for? A model used for a quick "back-of-the-envelope" estimate for a new product price needs less detail than a model used for regulatory capital reporting.
  • Granularity (Level of Detail): Should we model every single policyholder individually (stochastic/individual approach) or group them into "model points" (grouped approach)? Heterogeneity within the data might require more detail.
  • Flexibility: Can the model be easily updated if the regulatory environment changes?
  • Complexity vs. Speed: A highly complex model might be more "accurate," but if it takes three days to run, it might be useless for making quick commercial decisions.

Section 3: Operational Model Issues – Running the Model

Once the model is built, the process of running it introduces its own set of risks and issues.

1. Data Issues

As the saying goes, "Garbage In, Garbage Out." Operational issues include:

  • Ensuring data is available in the correct format.
  • Checks on data governance and accuracy (Objective 3.6).
  • Handling missing or "dirty" data.

2. Model Governance and Controls

To ensure the model results are reliable, several "living with the solution" controls are needed:

  • Version Control: Ensuring everyone is using the latest, approved version of the model.
  • Access Limits: Only authorized personnel should be able to change the formulas or assumptions.
  • Audit Trails: A record of who changed what, and why.

3. Documentation

This is a favorite IFoA exam topic! A model is only useful if someone else can understand it. Good documentation should cover:

  • The objectives and limitations of the model.
  • The source of all assumptions used.
  • The mathematical formulas and logic.
  • Instructions on how to run it and interpret the surplus/profit outputs.

Section 4: The Trade-offs (Actuarial Common Sense)

When discussing operational issues, you often need to discuss trade-offs. Don’t worry if this seems tricky; it’s just about finding the "middle ground."

The "Triangle of Conflict":

  1. Accuracy: Getting the "perfect" number.
  2. Time: Getting the number quickly.
  3. Cost: Using fewer resources (staff, computer power, expensive software).

You can usually only have two! A very accurate model (1) usually takes a lot of time (2) and money (3).

Did you know? Many actuarial firms are moving toward "Cloud Computing" to solve the "Time" issue, allowing complex stochastic modelling to run in minutes instead of days.

Key Takeaways & Quick Review

Sensitivity Analysis:

  • Tests the impact of changing one assumption at a time.
  • Helps identify "key risk drivers."
  • Essential for decision making and checking the appropriateness of provisions.

Operational Issues:

  • Design: Must match the objective and balance complexity vs. speed.
  • Running: Requires strong data governance and internal controls.
  • Documentation: Critical for transparency and professional standards.

Common Exam Mistake: Forgetting that models are used by people. When the exam asks about "operational issues," don't just talk about code—talk about staff training, clear reporting, and meeting the needs of stakeholders.

Next Chapter: "Setting assumptions: information sources and prudence" — where we learn where those inputs for our sensitivity analysis actually come from!