Markov multiple state models for life and long-term health benefits and CCRCs

Exam ALTAM – Advanced Long-Term Actuarial Mathematics · SOA (Society of Actuaries) · 6 min read

Welcome to the World of Multiple State Models!

In your previous studies, you likely focused on a simple "Alive or Dead" model. While that's a great start, real life is often more complex. People get sick, recover, move into retirement communities, or require long-term care. In this chapter, we explore Markov Multiple State Models. Think of this as moving from a black-and-white photo to a full-color movie! We are going to track how individuals move between various health and living states and how we calculate the costs associated with those transitions.

Did you know? The "Markov" part of the name comes from Andrey Markov, a Russian mathematician. His theory essentially says that to predict the future, you only need to know where you are right now, not how you got there. It’s like a "no-baggage" approach to math!

1. Understanding States and Transitions

In a Multiple State Model, we represent an individual's status using States (the circles) and Transitions (the arrows between them).

The Markov Property: This is the Golden Rule of these models. It assumes that the probability of moving to a future state depends only on the state the person is currently in. It does not matter how long they’ve been in that state or what states they visited previously.

Common Models You'll See:
1. Disability Model (The "Sickness-Death" Model): States are Healthy, Sick, and Dead. Transitions can go from Healthy to Sick, Sick to Healthy (recovery), and both to Dead.
2. CCRC Model: Continuing Care Retirement Communities. States usually include Independent Living (IL), Assisted Living (AL), Skilled Nursing Facility (SNF), and Dead.

Analogy: Imagine a house with several rooms. The Markov property says that if I want to know the chance of you walking into the kitchen, I only need to know you are currently in the living room. I don't need to know if you just came from the backyard or the bedroom.

Quick Review:
- State: A condition (e.g., "Active").
- Transition: Moving from one state to another (e.g., "Retiring").
- Absorbing State: A state you can enter but never leave (usually Dead).

2. Transition Intensities: The "Speed" of Change

We use the symbol $\mu_{x+t}^{ij}$ to represent the transition intensity from state $i$ to state $j$ at age $x+t$. This is the instantaneous rate of leaving state $i$ for state $j$.

The total intensity of leaving state $i$ is the sum of all intensities leading away from it:
$\mu_{x+t}^{i \bullet} = \sum_{j \neq i} \mu_{x+t}^{ij}$

Important Note: Transition intensities are like "speeds" on a speedometer. They aren't probabilities themselves (they can even be greater than 1!), but we use them to calculate probabilities.

3. Transition Probabilities

We denote the probability that an individual in state $i$ at time 0 will be in state $j$ at time $t$ as $_tp_x^{ij}$.

There are two main types of probabilities to keep straight:
1. $_tp_x^{ij}$: The probability of being in state $j$ at time $t$, regardless of how many times you switched states in between.
2. $_t\bar{p}_x^{ii}$: The probability of staying in state $i$ continuously from time 0 to $t$. This is often called the "occupancy" or "staying" probability.

The formula for staying in state $i$ continuously is:
$_t\bar{p}_x^{ii} = \exp\left( -\int_0^t \mu_{x+s}^{i \bullet} ds \right)$

Memory Trick: Think of the bar over the $p$ ($\bar{p}$) as a "lid" that keeps the person inside the state so they can't leave!

4. Kolmogorov Forward Equations

Don't let the name scare you! These equations describe how the probability of being in a state changes over a tiny sliver of time. For a transition from state $i$ to state $j$:

$\frac{d}{dt} \, _tp_x^{ij} = \sum_{k \neq j} (_tp_x^{ik} \mu_{x+t}^{kj} - _tp_x^{ij} \mu_{x+t}^{jk})$

Step-by-Step Logic:
1. The "In" Flow: You look at all the ways you could enter state $j$ from some other state $k$.
2. The "Out" Flow: You look at the ways you could leave state $j$ to go somewhere else.
3. The Change: The rate of change in probability is simply (Sum of ways to get in) minus (Sum of ways to get out).

Key Takeaway: If you are in a state $j$, the probability increases when people move into $j$ and decreases when people move out of $j$.

5. Actuarial Present Values (APV) in Multiple States

This is why we're here! We need to value benefits. In these models, benefits can be paid in two ways:

A. Annuities (State Benefits)

Payments are made while the person is in a specific state (e.g., receiving \$2,000/month while in "Assisted Living").
\n$APV = \int_0^\infty v^t \cdot _tp_x^{ij} \cdot B_t dt$
\nWhere $B_t$ is the rate of payment.

\n\n

B. Transition Benefits (Lump Sums)

A lump sum is paid the moment a transition occurs (e.g., a \$10,000 "Death Benefit" paid when moving from "Healthy" to "Dead").
$APV = \int_0^\infty v^t \cdot _tp_x^{ik} \cdot \mu_{x+t}^{kj} \cdot S_t dt$
Where $S_t$ is the lump sum amount.

Analogy: An Annuity is like paying rent (you pay as long as you occupy the room). A Transition Benefit is like a cover charge (you pay it once at the door when you walk in).

6. Continuing Care Retirement Communities (CCRCs)

CCRCs are a major topic in ALTAM. They are specialized housing for seniors that provide different levels of care. The model usually moves in one direction (towards more care), but some models allow for "recovery" (moving back to a lower care level).

Common Fee Structures:
1. Entry Fee: A large lump sum paid when moving into Independent Living.
2. Monthly Fees: Regular payments that usually increase as the level of care increases.
3. Refundable Entry Fees: Sometimes a portion of the entry fee is paid back to the estate when the resident dies.

Common Pitfall: When calculating the APV for CCRCs, students often forget to include the probability of surviving in the specific state. Always double-check your $_tp_x^{ij}$ terms!

7. Summary and Quick Tips

Key Points to Remember:
- Markov = No Memory. Only the current state matters.
- $\mu$ is the rate, $p$ is the probability.
- Kolmogorov Forward Equations track the flow of probability between "rooms."
- Annuity benefits depend on being in a state; Lump sums depend on the act of moving between states.

Exam Tip: On the ALTAM exam, you will often be given a transition matrix or a diagram. Before you start any math, draw the paths. If the question asks for the probability of being in state 2 at time 10, ask yourself: "What are all the different ways I can get to state 2?" This helps you set up the integrals correctly.

Don't worry if this seems tricky at first! Multiple state models are just a series of "Alive-Dead" models linked together. If you can handle one transition, you can handle ten—it just takes a bit more bookkeeping!

Quick check

Can you answer these now?

Open each question to check the key ideas from this chapter.

What is the fundamental assumption of the Markov Property in multiple state models?

The Markov Property assumes that the probability of transitioning to a future state depends only on the current state, not on the history of previous states or the time spent in the current state.

In a multiple state model, what characterizes an absorbing state?

An absorbing state is a state that an individual can enter but can never leave, such as the "Dead" state.

What does the transition intensity $\mu_{x+t}^{ij}$ represent?

It represents the instantaneous rate of transition from state $i$ to state $j$ at age $x+t$.

How is the total intensity of leaving state $i$, denoted $\mu_{x+t}^{i \bullet}$, calculated?

It is the sum of all individual transition intensities leading away from state $i$ to any other state $j$: $\mu_{x+t}^{i \bullet} = \sum_{j \neq i} \mu_{x+t}^{ij}$.

What is the difference between the probability $_tp_x^{ij}$ and the occupancy probability $_t\bar{p}_x^{ii}$?

$_tp_x^{ij}$ is the probability of being in state $j$ at time $t$ regardless of intermediate transitions, whereas $_t\bar{p}_x^{ii}$ is the probability of staying in state $i$ continuously from time 0 to $t$.

What is the formula for the probability of remaining in state $i$ continuously from time 0 to $t$?

$_t\bar{p}_x^{ii} = \exp\left( -\int_0^t \mu_{x+s}^{i \bullet} ds \right)$.

What logic governs the Kolmogorov Forward Equations for the rate of change in state probabilities?

The rate of change in the probability of being in state $j$ is the sum of the "in-flow" (entering $j$ from other states) minus the "out-flow" (leaving $j$ for other states).

How is the Actuarial Present Value (APV) of an annuity benefit paid while in a specific state calculated?

$APV = \int_0^\infty v^t \cdot _tp_x^{ij} \cdot B_t dt$, where $B_t$ is the rate of payment.

How is the APV of a lump sum benefit paid upon transition between states calculated?

$APV = \int_0^\infty v^t \cdot _tp_x^{ik} \cdot \mu_{x+t}^{kj} \cdot S_t dt$, where $S_t$ is the lump sum amount.

What are the typical fee components in a Continuing Care Retirement Community (CCRC) model?

CCRCs typically involve an upfront Entry Fee, recurring Monthly Fees (which vary by care level), and potentially a Refundable Entry Fee paid upon the resident's death.

Ready to test yourself?

Turn these notes into exam-style practice. Get unlimited AI questions on this topic with instant marking and explanations.

Practice This Topic

More Exam ALTAM – Advanced Long-Term Actuarial Mathematics chapters

* The content provided by thinka is generated by AI and may not always be accurate or up-to-date. Please use it as a supplementary resource and verify with official materials.

Put These Notes into Practice

Reading the notes is a great start. Now practise with unlimited AI-generated questions and get instant feedback. 100,000+ students are already improving their grades.

Start Practising Now View Pricing

Done reading? Test yourself with AI practice questions

Practice This Topic Now