Applications of time series models

Welcome to Applications of Time Series!

In the previous chapters, we spent a lot of time looking at the "anatomy" of time series models like AR, MA, and ARMA. Now, it's time to put those models to work! Think of this chapter as the "Bridge to Reality." We are going to learn how to take a messy set of real-world data and find the best mathematical model to describe it, check if that model is actually any good, and finally, use it to peer into the future.

Don't worry if the previous chapters felt a bit heavy on the algebra. Here, we focus on the process of applying those models. Let's dive in!

1. The Three Steps of Box-Jenkins

When actuaries talk about "applying" time series, they usually follow the Box-Jenkins Methodology. It’s a simple three-step cycle:

1. Model Identification: Picking the right type of model (Is it an AR, an MA, or an ARMA? What are \(p\) and \(q\)?).
2. Parameter Estimation: Calculating the actual numbers (the coefficients) for the model.
3. Diagnostic Checking: Making sure the model actually fits the data well.

Analogy: It’s like buying a suit. First, you pick the style (Identification). Then, the tailor takes your measurements (Estimation). Finally, you try it on to make sure it doesn't pinch your shoulders (Diagnostic Checking)!

2. Step 1: Model Identification (Finding \(p\) and \(q\))

How do we know which model to use? We use two "fingerprints" of the data: the Autocorrelation Function (ACF) and the Partial Autocorrelation Function (PACF).

The Cheat Sheet for \(p\) and \(q\)

If you see the following patterns in the graphs, here is what they usually mean:

• AR(\(p\)) model: The PACF "cuts off" (drops to zero) after lag \(p\), while the ACF decays slowly (tails off).
• MA(\(q\)) model: The ACF "cuts off" after lag \(q\), while the PACF decays slowly (tails off).
• ARMA(\(p, q\)) model: Both the ACF and PACF "tail off" (decay slowly) without a sharp cut-off.

Quick Review Box:
ACF measures the correlation between \(X_t\) and \(X_{t-k}\).
PACF measures the correlation between \(X_t\) and \(X_{t-k}\) after removing the effect of the observations in between.

Information Criteria (AIC and BIC)

Sometimes, looking at graphs isn't enough. We use "penalty scores" to choose between models. We want a model that fits well but isn't overly complex (too many parameters).
- AIC (Akaike Information Criterion)
- BIC (Bayesian Information Criterion)

Rule of Thumb: The lower the AIC or BIC value, the better the model! These tools help us avoid overfitting (making a model so complex it explains random noise instead of the actual trend).

3. Step 2: Parameter Estimation

Once we’ve decided on, say, an AR(1) model, we need to find the value of \(\alpha\). There are three common ways to do this:

1. Method of Moments: We equate the sample moments (like the mean and variance we see in our data) to the theoretical moments of the model. It's the simplest method but often less accurate.
2. Least Squares: We choose parameters that minimize the sum of the squared differences between the actual data and the values predicted by the model.
3. Maximum Likelihood Estimation (MLE): This finds the parameter values that make the observed data "most likely" to have happened. In the IFoA exams, this is often considered the most robust method for large datasets.

Did you know? For a simple AR(1) model, the Method of Moments estimate for \(\alpha\) is just the first sample autocorrelation coefficient (\(\hat{\rho}_1\))!

4. Step 3: Diagnostic Checking

After we build our model, we look at the residuals (the errors). If the model is perfect, the residuals should be White Noise.

How to check residuals:
- Visual Inspection: Plot the residuals. They should look like random "static" on a TV screen with no patterns.
- ACF of Residuals: The autocorrelations of the residuals should all be close to zero.
- Portmanteau Tests (e.g., Ljung-Box test): This is a statistical test that checks if a whole group of residual autocorrelations are significantly different from zero.

Common Mistake to Avoid: Don't forget that if your residuals show a pattern (like a wave or a trend), your model is incomplete. You’ve left some information on the table!

Summary Takeaway: If the residuals are White Noise, your model has "sucked out" all the useful information from the data. Good job!

5. Forecasting: Predicting the Future

This is why we do time series analysis in the first place! We want to calculate the Lead-\(l\) forecast, which we write as \(\hat{x}_n(l)\).

How to calculate a forecast:

The best forecast is the conditional expectation. Here is the step-by-step logic:
1. Write out the model equation for the future time point (e.g., \(X_{n+1} = \alpha X_n + \epsilon_{n+1}\)).
2. Take the expectation of everything.
3. Replace future White Noise terms (\(\epsilon_{n+1}, \epsilon_{n+2}...\)) with 0 (because their expected value is zero).
4. Replace past White Noise terms with their calculated residuals if they are known.
5. Replace past observations (\(X_n, X_{n-1}...\)) with their actual known values.

Example (AR(1)):
If \(X_t = 0.8 X_{t-1} + \epsilon_t\) and we know \(X_n = 10\):
The 1-step ahead forecast is: \(\hat{x}_n(1) = 0.8(10) + 0 = 8\).
The 2-step ahead forecast is: \(\hat{x}_n(2) = 0.8(8) + 0 = 6.4\).

Confidence in our Predictions

A "point forecast" (a single number) is rarely perfect. We use Prediction Intervals to show a range where the future value is likely to fall. As we try to forecast further into the future, our uncertainty increases, and the forecast interval gets wider.

Key Takeaway: For a stationary ARMA model, as the lead time \(l\) goes to infinity, the forecast will eventually just settle at the mean of the process.

6. Summary of Applications

To wrap up this chapter, remember the flow:
1. Identify: Use ACF/PACF graphs or AIC/BIC to pick \(p\) and \(q\).
2. Estimate: Use MLE or Least Squares to find the coefficients.
3. Check: Ensure residuals look like White Noise (Ljung-Box test).
4. Forecast: Use the model equation and expectations to predict the future.

Final Encouragement: Time series application is as much an art as it is a science. Practice looking at ACF and PACF plots—the more you see, the easier it becomes to spot the patterns. You've got this!