Confidence intervals for transition intensity estimators

Welcome to the World of Uncertainty: Confidence Intervals for Transition Intensities

Hello! If you’ve made it to this part of the ALTAM curriculum, you already know how to calculate transition intensities (the "forces" that push people from one state to another, like from "Healthy" to "Sick"). But in the real world, our data isn't perfect. We are usually working with a sample, not the entire population.

In this chapter, we learn how to express how "sure" we are about our estimates. We aren't just saying "The intensity is 0.05"; we are saying "We are 95% confident the true intensity lies between 0.04 and 0.06." This is crucial for risk management!

1. Quick Recap: The MLE for Transition Intensity

Before we build an interval, we need a starting point. For a constant transition intensity \(\mu_{ij}\) (the force of moving from state \(i\) to state \(j\)), we use the Maximum Likelihood Estimator (MLE).

The Formula:
\(\hat{\mu}_{ij} = \frac{d_{ij}}{V_i}\)

Where:
- \(d_{ij}\) is the total number of transitions observed from state \(i\) to state \(j\).
- \(V_i\) is the total time all individuals spent in state \(i\) (often called waiting time or exposure).

Analogy: Imagine you are watching a revolving door at a hotel. \(V_i\) is the total time people spent standing inside the lobby, and \(d_{ij}\) is the number of people who actually walked through the door to the street. The "intensity" is just the rate of people leaving per hour spent inside.

Quick Review Box:
Always remember: Intensity = (Number of Events) / (Total Time at Risk). Don't let the notation scare you!

2. The Variance of the Estimator

To build a confidence interval, we need to know the "wobble" or variance of our estimate. If we have very little data (small \(V_i\)), our estimate will wobble a lot. If we have a huge amount of data, our estimate is very stable.

For large samples, the estimated variance of our intensity is:
\(\text{Var}(\hat{\mu}_{ij}) \approx \frac{\hat{\mu}_{ij}}{V_i} = \frac{d_{ij}}{(V_i)^2}\)

The Standard Error (SE) is simply the square root of the variance:
\(SE(\hat{\mu}_{ij}) = \sqrt{\frac{d_{ij}}{(V_i)^2}} = \frac{\sqrt{d_{ij}}}{V_i}\)

Did you know? This variance formula comes from the properties of the Poisson distribution, which is often used to model the number of transitions in a fixed time interval.

3. The Standard (Linear) Confidence Interval

This is the simplest type of interval. We assume the estimator follows a Normal Distribution (thanks to the Central Limit Theorem).

The Formula:
\(\hat{\mu}_{ij} \pm z_{\alpha/2} \cdot SE(\hat{\mu}_{ij})\)

For a 95% confidence interval, \(z_{\alpha/2}\) is 1.96.

The Problem: Transition intensities must be positive. However, if your intensity is small and your variance is large, the "minus" part of this formula might give you a negative lower bound. Since you can't have a negative force of mortality, we need a better way!

4. The Log-Transformed Confidence Interval

To avoid the "negative number problem," we use a log-transformation. We calculate the confidence interval for \(\ln(\hat{\mu}_{ij})\) first, and then convert it back. Because \(e^x\) is always positive, our final interval will always be above zero!

Step-by-Step Process:
1. Calculate the standard error of the natural log of the estimator:
\(SE(\ln \hat{\mu}_{ij}) = \sqrt{\frac{1}{d_{ij}}}\)
(Notice how simple this is! It only depends on the number of transitions.)

2. Calculate the "Error Factor" (let's call it \(EF\)):
\(EF = \exp \left( z_{\alpha/2} \cdot \frac{1}{\sqrt{d_{ij}}} \right)\)

3. The Confidence Interval is then:
Lower Bound: \(\frac{\hat{\mu}_{ij}}{EF}\)
Upper Bound: \(\hat{\mu}_{ij} \cdot EF\)

Memory Aid: In the log-transformed version, you Multiply and Divide by the error factor instead of Adding and Subtracting. This ensures the result stays positive!

Example: If you observed 25 deaths (\(d_{ij} = 25\)) over 500 person-years (\(V_i = 500\)):
- \(\hat{\mu} = 25/500 = 0.05\)
- \(SE(\ln \hat{\mu}) = 1/\sqrt{25} = 0.2\)
- For 95% CI: \(EF = \exp(1.96 \cdot 0.2) = 1.479\)
- Lower: \(0.05 / 1.479 = 0.0338\)
- Upper: \(0.05 \cdot 1.479 = 0.0739\)

5. Summary and Key Takeaways

When you are asked to find a confidence interval for a transition intensity on the ALTAM exam, keep these points in mind:

Key Points:
- The MLE: Always start by calculating \(\hat{\mu}_{ij} = d_{ij} / V_i\).
- The Linear Interval: Use \(\hat{\mu} \pm z \cdot \frac{\sqrt{d}}{V}\). Use this only if the question doesn't specify log-transformed (though log-transformed is often preferred in practice).
- The Log-Transformed Interval: Use this to ensure bounds are positive. The key formula is \(SE(\ln \hat{\mu}) = 1/\sqrt{d}\).
- Sample Size: The more transitions (\(d\)) you observe, the narrower your confidence interval becomes.

Common Mistake to Avoid:
Don't confuse \(V_i\) (time spent in state \(i\)) with the total number of people in the study. Only the time spent in the starting state of the transition matters for that specific intensity!

Key Takeaway: Confidence intervals give us a range of plausible values for the true transition intensity. The log-transformation is our best friend because it respects the reality that these rates cannot be negative.

Don't worry if the formulas look intimidating at first. Just remember: it's all about finding the "best guess" and then figuring out how much that guess might be off by!