Assumptions underlying Markov multiple state models

Welcome to Markov Multiple State Models!

Hello there! If you’ve made it to Exam ALTAM, you already know that life isn’t always just "Alive" or "Dead." Real-world insurance products—like disability income or long-term care insurance—need to track people as they move through various stages of health. That is exactly what Markov Multiple State Models do! They are the engines that power complex actuarial calculations.

In this chapter, we are going to look at the "rules of the game"—the assumptions that make these models work. Don't worry if this seems a bit abstract at first; we will break it down piece by piece with plenty of analogies to keep things grounded.

1. The Core Assumption: The Markov Property

The most important thing to understand is the Markov Property. In simple terms, it means the model has "no memory."

The Markov Property assumes that the probability of moving to a future state depends only on your current state, not on how you got there or how long you've been there.

Analogy: The GPS vs. The Travel Diary
Imagine you are driving to a friend's house. A Markov Model is like a GPS: it only cares where you are right now to tell you the next turn. It doesn't care if you started your journey in New York or Los Angeles. A Non-Markov Model is like a travel diary: it looks at your whole history to decide where you might go next.

Quick Review:
In ALTAM, we specifically assume that if a person is in State \( i \) at age \( x \), their past history (which states they visited before or how long they stayed in State \( i \)) does not affect the probability of moving to State \( j \) in the future.

Key Term: The History \( \mathcal{H}_t \)

In mathematical notation, we say the probability is independent of \( \mathcal{H}_t \) (the history of the process up to time \( t \)). Formally:
\( Pr(S(t+s) = j | S(t) = i, \mathcal{H}_t) = Pr(S(t+s) = j | S(t) = i) \)

2. Transition Probabilities

We use specific notation to describe moving between states. For a life aged \( x \):

\( _{t}p_{x}^{ij} \) is the probability that a life in State \( i \) at age \( x \) will be in State \( j \) at age \( x+t \).

There are two main types of probabilities you'll see:
1. Transition Probabilities (\( i \neq j \)): The chance of ending up in a different state.
2. Stay Probabilities (\( i = j \)): The chance of being in the same state at the end of the period. (Note: This doesn't mean they never left; they just have to be back in State \( i \) by the time we check!)

Did you know?
The sum of all probabilities starting from State \( i \) must always equal 1. For any time \( t \):
\( \sum_{j} {_{t}p_{x}^{ij}} = 1 \)
This is because the person must be in one of the possible states!

3. Forces of Transition (Intensities)

While probabilities look at what happens over a period of time (like 1 year), Forces of Transition (denoted by \( \mu \)) look at what happens in an instant.

The force of transition from State \( i \) to State \( j \) at age \( x+t \) is defined as:
\( \mu_{x+t}^{ij} = \lim_{h \to 0} \frac{_{h}p_{x+t}^{ij}}{h} \)

Analogy: Speedometer vs. Odometer
Think of the Force of Transition (\( \mu \)) as your car's speedometer—it tells you how fast you are moving at a specific split second. Think of the Probability (\( p \)) as your odometer—it tells you how far you actually traveled over an hour.

Important Point: We assume these forces are functions of age. In ALTAM, we usually deal with non-homogeneous models, meaning the "speed" of transitioning changes as you get older.

4. The Chapman-Kolmogorov Equations

This sounds intimidating, but the concept is very simple: it's just "connecting the dots."

If you want to know the probability of moving from State \( i \) to State \( j \) over a long time, you can break it down into two shorter steps by stopping at an intermediate state \( k \).

The Formula:
\( _{t+u}p_{x}^{ij} = \sum_{all \: k} {_{t}p_{x}^{ik}} \cdot {_{u}p_{x+t}^{kj}} \)

Step-by-Step Understanding:
1. You start in State \( i \).
2. After time \( t \), you could be in any State \( k \) (Healthy, Sick, etc.).
3. From that State \( k \), you then move to the final State \( j \) over the remaining time \( u \).
4. You sum up all the possible "paths" (all possible \( k \)'s) to get the total probability.

Key Takeaway: This equation allows us to build long-term probabilities by multiplying and adding shorter-term probabilities.

5. Kolmogorov’s Forward Equations

These are differential equations that describe how the probability of being in a state changes over time. For ALTAM, you need to understand the intuition behind them.

The rate of change in the probability \( _{t}p_{x}^{ij} \) depends on:
1. Inflow: People coming into State \( j \) from other states \( k \).
2. Outflow: People leaving State \( j \) to go to other states.

The General Form:
\( \frac{d}{dt} {_{t}p_{x}^{ij}} = \sum_{k \neq j} (_{t}p_{x}^{ik} \cdot \mu_{x+t}^{kj} - {_{t}p_{x}^{ij} \cdot \mu_{x+t}^{jk}}) \)

Common Mistake to Avoid:
Students often swap the indices. Remember: \( \mu^{kj} \) means moving from \( k \) to \( j \). In the "Inflow" part of the equation, we want to end up in \( j \), so \( j \) should be the second superscript.

6. Special Case: The Occupancy Probability

Sometimes we want to know the probability that a person stays in a specific state \( i \) for the entire duration \( t \) without ever leaving. We denote this as \( _{t}\bar{p}_{x}^{ii} \).

This is calculated using the total force of leaving State \( i \):
\( _{t}\bar{p}_{x}^{ii} = \exp \left( -\int_{0}^{t} \sum_{j \neq i} \mu_{x+s}^{ij} ds \right) \)

Note: The bar over the p indicates they stayed there continuously.

Quick Summary & Tips for Success

Summary Box:
• Markov Property: The future depends only on the current state.
• Non-homogeneous: Transition rates (\( \mu \)) change with age.
• Chapman-Kolmogorov: Break long paths into shorter ones by summing all possible intermediate states.
• Forward Equations: The math of "Inflow minus Outflow."

Final Encouragement:
Markov models are the "skeleton" of ALTAM. If you can visualize the states as circles and the transitions as arrows, the math becomes much easier to manage. You’ve got this! Keep practicing those transition diagrams, and the formulas will start to feel like second nature.