Introduction: Your History Matters!

Welcome to the world of Autoregressive (AR) models! If you’ve ever thought, "I’m tired today because I stayed up late last night," you already understand the core logic of an AR model. In time series, what happened yesterday usually has a direct impact on what happens today.

In this chapter, we will learn how to turn that intuition into a mathematical formula. Don't worry if the math looks intimidating at first—we'll break it down piece by piece. By the end of these notes, you'll see that AR models are just a specialized version of the linear regression you already know!

1. What is an Autoregressive Model?

The term "autoregressive" sounds fancy, but let’s look at the roots: Auto (self) and Regressive (moving backward/regression). Essentially, it is a regression of a variable against itself at previous time steps.

The AR(1) Model: The Foundation

The simplest version is the AR(1) model, where the current value (\(x_t\)) depends only on the value immediately before it (\(x_{t-1}\)).

The formula for an AR(1) model is:
\(x_t = c + \phi x_{t-1} + \epsilon_t\)

Breaking down the components:

  • \(x_t\): The value of the series at time \(t\) (e.g., today's stock price).
  • \(c\): A constant (intercept).
  • \(\phi\) (phi): The autoregressive coefficient. This tells us how strongly "connected" today is to yesterday.
  • \(x_{t-1}\): The value of the series at the previous time step (yesterday's price).
  • \(\epsilon_t\): The "white noise" or error term. This represents random shocks that can't be predicted.

Analogy: Think of a thermostat. If the room was 70 degrees yesterday (\(x_{t-1}\)), it’s likely to be close to 70 today, plus or minus some random change (\(\epsilon_t\)) based on the weather.

Quick Review: The Error Term (\(\epsilon_t\))

We assume the error term follows a normal distribution with a mean of 0 and a constant variance (\(\sigma^2\)). This is our "unpredictable" part of the model.

2. Stationarity: The "Rubber Band" Rule

For an AR(1) model to be useful for long-term forecasting, it usually needs to be stationary. This means its statistical properties (like mean and variance) don't change over time.

The Golden Rule for AR(1) Stationarity:
The model is stationary if and only if \(|\phi| < 1\).

Why does this matter?

  • If \(|\phi| < 1\): The series is like a rubber band. If a random shock (\(\epsilon_t\)) pushes the value away, the series eventually pulls back toward its long-run average (mean).
  • If \(\phi = 1\): This is called a Random Walk. There is no "pull" back to the center. Every shock stays in the system forever.
  • If \(|\phi| > 1\): The series "explodes" and goes to infinity. We don't want this for risk modeling!

Did you know? A Random Walk (\(\phi = 1\)) is often used to describe stock prices because today's price is the best guess for tomorrow's price, and changes are totally random.

3. Key Statistics of the AR(1) Model

When you are working with a stationary AR(1) model (\(|\phi| < 1\)), there are three formulas you must memorize for the exam:

1. The Mean (Long-run Average):
\(\mu = \frac{c}{1 - \phi}\)
Common Mistake: Thinking the mean is just \(c\). It's not! You must divide by \((1 - \phi)\).

2. The Variance:
\(Var(x_t) = \frac{\sigma^2}{1 - \phi^2}\)
Note: \(\sigma^2\) is the variance of the error term.

3. Autocorrelation (ACF):
The correlation between \(x_t\) and \(x_{t-k}\) (where \(k\) is the number of steps back) is:
\(\rho_k = \phi^k\)
This means the correlation decays exponentially as the time gap (\(k\)) increases.

Key Takeaway: In an AR(1) model, the effect of the past fades away over time, but it never truly hits zero (mathematically).

4. Moving Up: The AR(p) Model

Sometimes yesterday isn't enough info. Maybe the last 3 days matter! This is an AR(p) model, where \(p\) is the number of "lags" we include.

The AR(p) Equation:
\(x_t = c + \phi_1 x_{t-1} + \phi_2 x_{t-2} + ... + \phi_p x_{t-p} + \epsilon_t\)

Stationarity for AR(p):
It's more complex than the AR(1). While the "sum of \(\phi\) coefficients < 1" is a helpful guideline, the formal requirement involves "roots of the characteristic equation" being outside the unit circle. For Exam SRM, remember that all \(\phi\) coefficients must be small enough so the process doesn't explode.

5. Identifying AR Models: The PACF Trick

How do we know if a dataset is an AR model or something else (like an MA model)? We look at two plots:

  1. ACF (Autocorrelation Function): For an AR model, the ACF tails off (decreases gradually).
  2. PACF (Partial Autocorrelation Function): This is the "magic" tool for AR models. The PACF cuts off after \(p\) lags.

Memory Aid:
AR model = PACF cuts off. (Think "AP" like the Advanced Placement classes).
If the PACF has 2 significant bars and then drops to zero, it is likely an AR(2) model.

6. Forecasting with AR Models

Predicting the future with AR models is a step-by-step process. To find the next value, we plug in the most recent known values.

Step-by-Step Example (AR(1)):
Suppose \(x_t = 5 + 0.5 x_{t-1} + \epsilon_t\). Today's value (\(x_t\)) is 10.
1. Forecast for \(t+1\): Since we don't know the future error (\(\epsilon_{t+1}\)), we assume it is 0.
\(\hat{x}_{t+1} = 5 + 0.5(10) = 10\)
2. Forecast for \(t+2\): Use your forecast for \(t+1\) as the new input.
\(\hat{x}_{t+2} = 5 + 0.5(10) = 10\)

Note: In this specific example, the forecast stayed at 10 because 10 is the long-run mean (\(5 / (1 - 0.5) = 10\)). If the starting value was different, the forecasts would slowly drift toward 10.

7. Summary & Key Takeaways

Don't let the notation scare you! Here is what you absolutely must remember for the exam:

  • AR(1) Formula: \(x_t = c + \phi x_{t-1} + \epsilon_t\).
  • Stationarity: Must have \(|\phi| < 1\). If \(\phi = 1\), it’s a non-stationary Random Walk.
  • The Mean: \(\mu = c / (1 - \phi)\).
  • Identification: AR models have an ACF that tails off and a PACF that cuts off at lag \(p\).
  • Forecasting: Just plug the previous value into the formula and set the error term to zero.

Final Encouragement: AR models are the "building blocks" of almost everything in time series. Master the AR(1) formulas first, and the rest of the chapter will fall into place easily. You’ve got this!