Introduction: Your Map to Bayesian Credible Intervals
Welcome! So far in your Bayesian statistics journey, you’ve learned how to combine what you already know (the Prior) with new data (the Likelihood) to get an updated view of the world (the Posterior). But once you have that posterior distribution, how do you actually use it to make a decision or give an estimate?
In this chapter, we explore Credible Intervals. Think of these as the Bayesian version of the "Confidence Intervals" you met in Frequentist statistics. However, as you’re about to see, Bayesian intervals are often much more intuitive and "user-friendly" in their interpretation. Don't worry if the math looks intimidating at first—we'll break it down step-by-step!
1. What is a Credible Interval?
A Credible Interval is a range of values within which an unknown parameter (let’s call it \( \theta \)) lies with a certain probability, given the data we have observed.
If we calculate a 95% Credible Interval for a claim frequency \( \lambda \) to be [0.02, 0.08], we can literally say: "There is a 95% probability that the true value of \( \lambda \) is between 0.02 and 0.08."
Wait, isn't that what a Confidence Interval is?
Actually, no! In Frequentist stats (Confidence Intervals), the parameter is fixed, and the interval is random. You have to say "In 95% of repeated samples, this interval will cover the true parameter."
In Bayesian stats, the parameter is random and the data is fixed. This allows us to make direct probability statements about the parameter itself. It’s much more natural for most people to understand!
Key Takeaway:
A \( 100(1-\alpha)\% \) Bayesian credible interval for \( \theta \) is any interval \((a, b)\) such that:
\( P(a < \theta < b | \text{data}) = 1 - \alpha \)
2. The "Simple Case": Central Credible Intervals
While there are many ways to pick an interval (since many different ranges could add up to 95%), the IFoA curriculum focuses on Central Credible Intervals (also called equal-tailed intervals).
To find a \( 100(1-\alpha)\% \) central credible interval, we simply put half of the "missing" probability (\( \alpha/2 \)) in the left tail and half in the right tail.
The Step-by-Step Process:
1. Find the Posterior Distribution of your parameter (e.g., is it Normal? Gamma? Beta?).
2. Identify the total probability you want (e.g., 95%, so \( \alpha = 0.05 \)).
3. Find the \( \alpha/2 \) percentile (the lower bound, \( a \)).
4. Find the \( 1 - \alpha/2 \) percentile (the upper bound, \( b \)).
Example: For a 95% interval, you find the 2.5th percentile and the 97.5th percentile of the posterior distribution.
3. Working with the Normal Distribution
The most common "simple case" involves a Normal Posterior. Because the Normal distribution is symmetric, finding the interval is very straightforward.
If your posterior distribution for \( \theta \) is \( N(\mu_{post}, \sigma^2_{post}) \), the \( 100(1-\alpha)\% \) credible interval is:
\( (\mu_{post} - z_{\alpha/2} \cdot \sigma_{post}, \mu_{post} + z_{\alpha/2} \cdot \sigma_{post}) \)
Did you know?
For a 95% interval, \( z_{\alpha/2} \) is 1.96. For a 90% interval, it is 1.6449. You can find these values in your Formulae and Tables (Gold Book)!
Example Walkthrough:
Imagine you have calculated a posterior distribution for an unknown mean loss \( \mu \) to be \( N(500, 100) \). Note that the variance is 100, so the standard deviation is 10.
To find a 95% credible interval:
Lower bound: \( 500 - (1.96 \times 10) = 480.4 \)
Upper bound: \( 500 + (1.96 \times 10) = 519.6 \)
Interpretation: There is a 95% probability that the true mean loss is between 480.4 and 519.6.
4. Other Distributions (Gamma and Beta)
Sometimes your posterior isn't Normal. For example:
- If you use a Poisson likelihood and a Gamma prior, your posterior is Gamma.
- If you use a Binomial likelihood and a Beta prior, your posterior is Beta.
In these cases, you can't just use the "plus or minus" shortcut because these distributions aren't usually symmetric. Instead, you must use the tables or the properties of the distributions to find the points where the cumulative distribution function (CDF) equals \( \alpha/2 \) and \( 1 - \alpha/2 \).
Trick for Gamma distributions:
The IFoA tables provide values for the \( \chi^2 \) (Chi-squared) distribution. Remember that a Gamma distribution can be transformed into a Chi-squared distribution!
If \( \theta \sim Gamma(\alpha, \lambda) \), then \( 2\lambda\theta \sim \chi^2_{2\alpha} \). This is a lifesaver in exams!
Quick Review:
- Prior: What we thought before the data.
- Posterior: Our updated belief after seeing the data.
- Credible Interval: A range of values from the Posterior that contains the parameter with a specific probability.
5. Common Pitfalls to Avoid
1. Using the Prior: Students often accidentally calculate the interval using the Prior parameters instead of the Posterior parameters. Always check your work! The interval should be based on your "updated" knowledge.
2. Mixing up Variance and Standard Deviation: In the Normal distribution formula, remember to use the standard deviation (\( \sigma \)), not the variance (\( \sigma^2 \)).
3. Frequentist Language: Be careful with your wording in written answers. Don't say "95% of samples..." say "The probability that the parameter lies in this range is 95%."
Summary of Steps for Exam Success
1. Identify the likelihood and prior to determine the Posterior distribution and its parameters.
2. Decide the significance level \( \alpha \) (e.g., for 90% credible interval, \( \alpha = 0.10 \)).
3. Find the values \( a \) and \( b \) such that the area to the left of \( a \) is \( \alpha/2 \) and the area to the right of \( b \) is \( \alpha/2 \).
4. State the interval clearly as \( (a, b) \) and provide a brief interpretation if asked.
Don't worry if this seems tricky at first! The math is usually just a bit of algebra once you identify which distribution you are working with. Practice finding the posterior parameters first, and the intervals will fall into place.