Introduction

Welcome to one of the most practical chapters in the CS1 curriculum! Up until now, you have likely been studying random variables as static "snapshots." In this chapter, we shift gears to look at how events unfold over time using the Poisson process. We will also learn the "magic trick" of statistics: Simulation by Inverse Transform. This technique allows us to take a simple random number between 0 and 1 and transform it into a value from almost any distribution we want. Whether you are a math whiz or find formulas a bit daunting, these notes will help you master these core principles.

1. The Poisson Process

Imagine you are sitting in a coffee shop, counting how many customers walk through the door. If the customers arrive at a constant average rate, and one person’s arrival doesn't influence another's, you are observing a Poisson process.

What defines a Poisson Process?

A Poisson process with rate \(\lambda\) (where \(\lambda > 0\)) is a model for the number of events occurring in a fixed interval of time. The key connection you need to know for the CS1 syllabus is how this relates to the Poisson distribution.

If we define \(N(t)\) as the number of events that occur up to time \(t\), then:

\(N(t) \sim \text{Poisson}(\lambda t)\)

The probability of observing exactly \(k\) events in time \(t\) is given by the formula:
\(P(N(t) = k) = \frac{e^{-\lambda t} (\lambda t)^k}{k!}\) for \(k = 0, 1, 2, ...\)

Key Characteristics

  • Independence: The number of events in one time interval does not affect the number of events in a different, non-overlapping time interval.
  • Constant Rate: The average number of events per unit of time, \(\lambda\), stays the same throughout the process.
  • Proportionality: The expected number of events in an interval of length \(t\) is simply \(E[N(t)] = \lambda t\).

Quick Tip: Always check your units! If \(\lambda\) is "3 arrivals per hour" and you are asked for the probability of arrivals over 20 minutes, remember that \(t\) should be expressed in hours (\(t = 1/3\)) or \(\lambda\) should be converted to "1 arrival per 20 minutes."

Summary Table: Process vs. Distribution

Poisson Process: The ongoing "story" of events happening over time.
Poisson Distribution: The "count" of events that happened in a specific window of that time.

2. Simulation: The Inverse Transform Method

How do computers "generate" random numbers for distributions like the Normal or Exponential? They usually start with a Standard Uniform random variable, \(U \sim \text{Uniform}(0, 1)\), and then transform it. The most fundamental way to do this is the Inverse Transform Method.

The Big Idea

Every probability distribution has a Cumulative Distribution Function (CDF), denoted as \(F(x)\). The value of \(F(x)\) always lies between 0 and 1. The inverse transform method works by "reversing" this: we pick a random number \(u\) between 0 and 1 and find the value \(x\) such that \(F(x) = u\).

Step-by-Step: Continuous Distributions

Don't worry if this seems tricky; just follow these three steps:

  1. Find the CDF: Determine the expression for \(F(x) = P(X \le x)\).
  2. Set it equal to \(u\): Write the equation \(F(x) = u\).
  3. Solve for \(x\): Rearrange the equation so \(x\) is on one side. This is your "generating formula" \(x = F^{-1}(u)\).

Example: Generating an Exponential(\(\lambda\)) variable.
1. The CDF is \(F(x) = 1 - e^{-\lambda x}\).
2. Set \(u = 1 - e^{-\lambda x}\).
3. Solve for \(x\):
\(1 - u = e^{-\lambda x}\)
\(\ln(1 - u) = -\lambda x\)
\(x = -\frac{1}{\lambda} \ln(1 - u)\)
Since \(u\) is Uniform(0,1), \(1-u\) is also effectively Uniform(0,1), so we often simplify this to: \(x = -\frac{1}{\lambda} \ln(u)\).

Step-by-Step: Discrete Distributions

For discrete distributions (like the Binomial or Poisson), we can't always "solve" an equation because the CDF looks like a staircase. Instead, we use a lookup approach.

If we have a random number \(u\):
- If \(0 < u \le P(X = x_0)\), then \(X = x_0\).
- If \(P(X = x_0) < u \le P(X = x_0) + P(X = x_1)\), then \(X = x_1\).
- In general, we choose the value \(x_i\) such that \(F(x_{i-1}) < u \le F(x_i)\).

Analogy: Imagine a 1-meter ruler divided into sections. If the probability of "0" is 0.3 and the probability of "1" is 0.7, you mark the ruler at 30cm. You throw a dart; if it lands before 30cm, you picked "0". If it lands after, you picked "1".

Key Takeaway:

The Inverse Transform Method is powerful because it only requires one uniform random number to produce one sample from your target distribution. It is the "gold standard" for basic simulation in CS1.

3. Common Pitfalls to Avoid

1. Confusing \(f(x)\) and \(F(x)\): Always use the Cumulative Distribution Function (\(F(x)\)), not the Probability Density Function (\(f(x)\)), when performing the inverse transform.

2. Solving Algebra Incorrectly: When rearranging \(u = F(x)\), be very careful with natural logs (\(\ln\)) and signs. A common mistake is losing a minus sign when dealing with the Exponential distribution.

3. Misinterpreting the Poisson Rate: In a Poisson process, the rate \(\lambda\) is "per unit of time." If the exam asks for a 5-minute interval and \(\lambda\) is per minute, your parameter for the Poisson distribution is \(5\lambda\).

4. Quick Review Box

  • Poisson Process: Events occur at rate \(\lambda\); count of events \(N(t) \sim \text{Poisson}(\lambda t)\).
  • Inverse Transform (Continuous): Solve \(x = F^{-1}(u)\).
  • Inverse Transform (Discrete): Find the interval in the cumulative probabilities where \(u\) falls.
  • Standard Uniform: The starting point for all simulations (\(U \sim \text{Uniform}(0, 1)\)).

Did you know? The Poisson process is often used by actuaries to model the arrival of insurance claims. While real-world claims are more complex, the Poisson process provides the fundamental building block for understanding risk over time!

Next Steps: In Paper B, you will likely use R to perform these simulations. For Paper A, practice your algebra so you can quickly invert CDFs for the Exponential, Pareto, or Weibull distributions.