Stochastic time series: random walks, stationarity, and autocorrelation

Welcome to Time Series: Predicting the Unpredictable!

Welcome, future actuaries! Today we are diving into the heart of Time Series Models. If you’ve ever looked at a stock chart, tracked the daily temperature, or wondered why your insurance claims fluctuate month to month, you’ve looked at a time series. In this chapter, we explore how to mathematically describe "movement" over time. Don't worry if you’re not a math wizard yet—we’ll break this down step-by-step using simple logic and real-world analogies.

1. The Foundation: White Noise

Before we can understand complex movements, we need to understand the simplest one: White Noise. Think of white noise as the "static" on an old television. It’s completely random, has no pattern, and you can't use the past to predict the future.

A sequence \( \epsilon_t \) is called white noise if:
1. The average (mean) is zero: \( E[\epsilon_t] = 0 \).
2. The volatility (variance) is constant: \( Var(\epsilon_t) = \sigma^2 \).
3. There is no correlation between different time points: \( Cor(\epsilon_t, \epsilon_s) = 0 \) for \( t \neq s \).

Quick Review: If you see a plot where the dots are just scattered randomly around zero with no visible trend or "clumping," you’re likely looking at white noise!

2. The Random Walk: The "Drunkard's Path"

Imagine a person stands on a line at position 0. They flip a coin: heads, they take a step right; tails, they take a step left. Their position tomorrow depends entirely on where they are today, plus one random step. This is a Random Walk.

The formula looks like this:
\( x_t = x_{t-1} + w_t \)
Where \( w_t \) is white noise.

Key Properties of a Random Walk:

1. It is NOT stationary: Because the person can wander further and further away from the start, the variance increases as time goes on (\( Var(x_t) = t\sigma^2 \)).
2. Memory: A random walk has "perfect memory." Every single shock from the past is permanently embedded in the current value.
3. Random Walk with Drift: Sometimes, there is a consistent "nudge" in one direction.
\( x_t = c + x_{t-1} + w_t \)
Think of this like our drunkard walking on a moving walkway at the airport. They are stepping randomly, but the walkway is slowly pushing them forward (\( c \)).

Did you know? Many efficient market theorists argue that stock prices behave like random walks. If they do, it means you can't predict tomorrow's price based on today's price—any change is just random noise!

3. Stationarity: The Rules of the Game

In statistics, we love it when the "rules" don't change. Stationarity basically means that the statistical properties of the series do not depend on when you observe them.

For Exam SRM, we focus on Weak Stationarity (also called Covariance Stationarity). For a series to be weakly stationary, it must meet three strict criteria:
1. Constant Mean: The average value \( E[x_t] \) is the same for all \( t \).
2. Constant Variance: The spread of the data \( Var(x_t) \) is finite and the same for all \( t \).
3. Constant Autocovariance: The correlation between \( x_t \) and \( x_{t-h} \) only depends on the distance between them (\( h \)), not the actual time (\( t \)).

The Analogy:

Imagine a game of basketball.
- Stationary: The hoop stays at 10 feet, the court size stays the same, and the ball's bounciness is constant throughout the game.
- Non-stationary: The hoop gets higher every minute, or the court gets longer as the game progresses. It’s hard to predict the score if the rules keep changing!

Key Takeaway: A Random Walk is the classic example of a non-stationary series because its variance grows with time.

4. Autocorrelation: Correlation with Yourself

Autocorrelation measures how a variable is related to its own past values. If today’s temperature is highly correlated with yesterday’s temperature, we say the series has high autocorrelation at Lag 1.

The Autocorrelation Function (ACF) at lag \( k \) is defined as:
\( \rho_k = \frac{\gamma_k}{\gamma_0} \)
Where \( \gamma_k \) is the covariance at lag \( k \), and \( \gamma_0 \) is the variance.

What does the ACF tell us?

1. For White Noise: The ACF will be nearly 0 for all lags \( k > 0 \). (No relationship with the past).
2. For a Random Walk: The ACF will stay very high and decrease very, very slowly. This is a huge red flag that the data is non-stationary!
3. For Stationary Series: The ACF usually drops toward zero relatively quickly as the lag increases.

Memory Aid: Think of "Auto-" as "Self." Autocorrelation is just "Self-Correlation."

5. Making it Stationary: Differencing

What do we do if we have a non-stationary Random Walk? We fix it using Differencing! Instead of looking at the level of the series (where the person is), we look at the change (the step they took).

If \( x_t = x_{t-1} + w_t \), then:
\( x_t - x_{t-1} = w_t \)
The "First Difference" of a random walk is simply white noise, which is stationary!

Common Mistake to Avoid: Students often forget that log-transforming data helps with non-constant variance, while differencing helps with non-constant means/trends. If the data is "exponentially" growing, take the log first, then difference it!

Summary Quick-Check Box

1. White Noise: Pure randomness. Mean 0, constant variance, no correlation.
2. Random Walk: Today = Yesterday + Noise. Non-stationary (variance grows).
3. Stationarity: Mean, variance, and ACF are constant over time. Essential for most modeling.
4. ACF: Measures "memory." If it stays high for a long time, the series is likely non-stationary.
5. Differencing: Subtracting the previous value to turn a non-stationary series into a stationary one.

Don't worry if this seems tricky at first! Time series is like learning a new language. Once you recognize the patterns in the ACF and understand why stationarity matters, the rest of the pieces will fall into place. Keep practicing those formulas!