Statistical Distributions: AS 2 Section C (Statistics)
Welcome to your study notes for Statistical Distributions! Whether you love statistics or find it a bit daunting, this guide will walk you through each distribution step-by-step. In this unit, we explore how mathematics helps us model real-world uncertainty—from predicting how many customer emails arrive at a helpdesk to measuring the exact lifespan of an electronic component. Let's make these concepts clear, manageable, and easy to master!
1. Discrete Random Variables & Expectation Algebra
Before diving into named distributions, let's review the fundamental rules governing any discrete random variable \(X\).
Probability Distribution Properties
A discrete random variable takes distinct, separate values (like integers from rolling a die). For any valid discrete probability distribution:
• Every individual probability must be between 0 and 1: \(0 \le \text{P}(X = x) \le 1\)
• The sum of all probabilities must equal 1: \(\sum \text{P}(X = x) = 1\)
Expected Value and Variance
• Expected Value (Mean): \(\text{E}(X) = \mu = \sum x \, \text{P}(X = x)\)
This represents the long-term average value of \(X\) if we repeated the experiment thousands of times.
• Expected Value of a Function: \(\text{E}(g(X)) = \sum g(x) \, \text{P}(X = x)\)
Specifically, \(\text{E}(X^2) = \sum x^2 \, \text{P}(X = x)\).
• Variance: \(\text{Var}(X) = \sigma^2 = \text{E}(X^2) - [\text{E}(X)]^2\)
Variance measures how spread out the outcomes are from the mean.
Linear Coding Rules (Expectation Algebra)
When you transform a random variable by multiplying by a constant \(a\) and adding a constant \(b\):
• \(\text{E}(aX + b) = a\text{E}(X) + b\)
• \(\text{Var}(aX + b) = a^2\text{Var}(X)\)
Memory Trick: Adding a constant \(b\) shifts every data point along, so the centre (\(\text{E}(X)\)) shifts by \(b\), but the spread (\(\text{Var}(X)\)) remains totally unchanged! Multiplying by \(a\) scales the spread by \(a^2\) because variance is measured in squared units.
Key Takeaway: Always check that \(\sum \text{P}(X=x) = 1\). Remember that variance is always strictly non-negative: \(\text{Var}(X) \ge 0\).
2. The Poisson Distribution
The Poisson distribution models the number of times an independent event occurs within a fixed interval of time or space.
Conditions for a Poisson Model
A random variable \(X\) follows a Poisson distribution, written \(X \sim \text{Po}(\lambda)\), if events occur:
1. Independently: One event occurring does not change the likelihood of another.
2. Randomly: In time or space.
3. Singly: Events cannot occur simultaneously at the exact same instant.
4. At a constant average rate: \(\lambda\) events per specified interval.
Real-world examples: Number of cars passing a speed camera per minute, number of misprints per page of a book, or radioactive decays per second.
Probability Formula
If \(X \sim \text{Po}(\lambda)\), the probability of observing exactly \(x\) events is:
\(\text{P}(X = x) = \frac{\text{e}^{-\lambda}\lambda^x}{x!}\quad \text{for } x = 0, 1, 2, 3, \dots\)
Mean and Variance
A unique and famous feature of the Poisson distribution is that its mean and variance are identical:
• \(\text{E}(X) = \lambda\)
• \(\text{Var}(X) = \lambda\)
Did you know? If a question asks you to evaluate whether a data set is well-modelled by a Poisson distribution, compare the sample mean \(\bar{x}\) and sample variance \(s^2\). If \(\bar{x} \approx s^2\), a Poisson model is appropriate!
Scaling the Rate Parameter \(\lambda\)
The parameter \(\lambda\) is directly proportional to the size of the interval. If emails arrive at an average rate of 3 per minute (\(\lambda = 3\)), then over a 5-minute period, the rate becomes \(\lambda = 3 \times 5 = 15\).
Sum of Independent Poisson Variables
If \(X \sim \text{Po}(\lambda_1)\) and \(Y \sim \text{Po}(\lambda_2)\) are independent random variables, their sum is also a Poisson random variable:
\(X + Y \sim \text{Po}(\lambda_1 + \lambda_2)\)
Poisson Approximation to the Binomial Distribution
When the number of trials \(n\) in a Binomial distribution is very large and the probability of success \(p\) is very small, calculating \(\binom{n}{x}p^x(1-p)^{n-x}\) becomes cumbersome. We can approximate \(X \sim \text{B}(n, p)\) with a Poisson distribution:
• Conditions: \(n\) is large (\(n > 50\)) and \(p\) is small (\(p < 0.1\)), such that \(np \le 10\).
• Approximation: \(X \approx \text{Po}(\lambda)\), where \(\lambda = np\).
Key Takeaway: In Poisson problems, always adjust \(\lambda\) to match the exact time frame or area specified in the question before plugging numbers into the formula!
3. The Geometric Distribution
While the Binomial distribution counts the number of successes in a fixed number of trials, the Geometric distribution models the number of trials required to achieve the first success.
Conditions for a Geometric Model
A random variable \(X\) follows a Geometric distribution, written \(X \sim \text{Geo}(p)\), if:
• Trials are repeated until the first success occurs.
• Each trial has only two possible outcomes: success (probability \(p\)) and failure (probability \(q = 1 - p\)).
• Trials are independent of each other.
• The probability of success \(p\) remains constant for each trial.
Probability Formula
To get the first success on trial \(x\), we must have \(x-1\) failures followed by 1 success:
\(\text{P}(X = x) = (1-p)^{x-1}p\quad \text{for } x = 1, 2, 3, \dots\)
Tail and Cumulative Probabilities
You can find cumulative probabilities without summing endless terms using simple logic:
• More than \(x\) trials needed: This means the first \(x\) trials were all failures:
\(\text{P}(X > x) = (1-p)^x\)
• At most \(x\) trials needed: Using the complement rule:
\(\text{P}(X \le x) = 1 - \text{P}(X > x) = 1 - (1-p)^x\)
Mean and Variance
• \(\text{E}(X) = \frac{1}{p}\)
• \(\text{Var}(X) = \frac{1-p}{p^2}\)
Everyday Intuition: If you roll a fair six-sided die, the probability of rolling a six is \(p = \frac{1}{6}\). On average, how many rolls should it take to get your first six? \(\text{E}(X) = \frac{1}{1/6} = 6\) rolls!
Key Takeaway: Note that the smallest possible value for a Geometric distribution is \(x = 1\) (you cannot succeed in 0 trials). Be careful with inequality signs: \(\text{P}(X \ge x) = \text{P}(X > x - 1) = (1-p)^{x-1}\).
4. Continuous Random Variables (CRVs)
Unlike discrete variables that jump from one distinct value to the next, a continuous random variable can take any real value within a specified range (e.g., time, height, weight).
Probability Density Function (PDF), \(f(x)\)
A continuous random variable is defined by its Probability Density Function (PDF), \(f(x)\). For \(f(x)\) to be a valid PDF, it must satisfy two essential conditions:
1. \(f(x) \ge 0\) for all \(x\) (probabilities can never be negative).
2. The total area under the curve equals 1: \(\int_{-\infty}^{\infty} f(x) \, \text{d}x = 1\)
Vital Concept: For a continuous variable, the probability of taking any exact single value is zero: \(\text{P}(X = c) = 0\). Therefore, \(\text{P}(a \le X \le b) = \text{P}(a < X < b)\).
Calculating Probabilities
Probability corresponds to the area under the PDF curve between two boundaries:
\(\text{P}(a \le X \le b) = \int_{a}^{b} f(x) \, \text{d}x\)
Cumulative Distribution Function (CDF), \(F(x)\)
The Cumulative Distribution Function (CDF), denoted \(F(x)\), gives the probability that \(X\) is less than or equal to a value \(x\):
\(F(x) = \text{P}(X \le x) = \int_{-\infty}^{x} f(t) \, \text{d}t\)
• Converting from CDF to PDF: Differentiate the CDF: \(f(x) = \frac{\text{d}}{\text{d}x}[F(x)]\)
• Boundary Conditions: As \(x \to -\infty\), \(F(x) = 0\), and as \(x \to \infty\), \(F(x) = 1\).
Expectation and Variance of a CRV
To find the mean and spread for a continuous variable, replace discrete sums (\(\sum\)) with integrals (\(\int\)):
• Mean: \(\text{E}(X) = \mu = \int_{-\infty}^{\infty} x f(x) \, \text{d}x\)
• Expected value of \(X^2\): \(\text{E}(X^2) = \int_{-\infty}^{\infty} x^2 f(x) \, \text{d}x\)
• Variance: \(\text{Var}(X) = \text{E}(X^2) - [\text{E}(X)]^2\)
Mode, Median, and Quartiles
• Mode: The value of \(x\) at which \(f(x)\) attains its maximum value (find by sketching or using differentiation \(\frac{\text{d}f}{\text{d}x} = 0\)).
• Median (\(m\)): The value \(m\) where half the total probability lies below it: \(F(m) = 0.5\) or \(\int_{-\infty}^{m} f(x) \, \text{d}x = 0.5\).
• Lower Quartile (\(Q_1\)): The value where \(F(Q_1) = 0.25\).
• Upper Quartile (\(Q_3\)): The value where \(F(Q_3) = 0.75\).
• Interquartile Range (IQR): \(\text{IQR} = Q_3 - Q_1\)
Key Takeaway: Integration is your master tool for continuous variables. Integrate \(f(x)\) to find probabilities and CDFs; integrate \(x f(x)\) to find the mean.
5. The Continuous Uniform (Rectangular) Distribution
The simplest continuous distribution is the Continuous Uniform distribution, where the probability density is completely flat and constant across a specified interval \([a, b]\).
Definition and PDF
If \(X\) is uniformly distributed between \(a\) and \(b\), we write \(X \sim \text{U}(a, b)\). Since the total area must be a rectangle of area 1 with base length \((b - a)\), its height is:
\(f(x) = \begin{cases} \frac{1}{b - a} & \text{for } a \le x \le b \\ 0 & \text{otherwise} \end{cases}\)
Cumulative Distribution Function (CDF)
\(F(x) = \begin{cases} 0 & \text{for } x < a \\ \frac{x - a}{b - a} & \text{for } a \le x \le b \\ 1 & \text{for } x > b \end{cases}\)
Summary Statistics for \(X \sim \text{U}(a, b)\)
• Mean: \(\text{E}(X) = \frac{a + b}{2}\) (the exact midpoint of the interval)
• Variance: \(\text{Var}(X) = \frac{(b - a)^2}{12}\)
• Standard Deviation: \(\sigma = \frac{b - a}{\sqrt{12}} = \frac{b - a}{2\sqrt{3}}\)
• Median: \(\text{Median} = \frac{a + b}{2}\) (due to symmetry, \(\text{Mean} = \text{Median}\))
Quick Review Example: If a bus arrives randomly between 0 and 10 minutes from now, \(X \sim \text{U}(0, 10)\). The mean waiting time is \(\frac{0+10}{2} = 5\) minutes, and \(\text{Var}(X) = \frac{(10-0)^2}{12} = \frac{100}{12} = \frac{25}{3} \approx 8.33\).
Key Takeaway: Because the shape is a simple rectangle, you can often find probabilities using simple geometry: \(\text{P}(c \le X \le d) = \frac{\text{width}}{\text{total width}} = \frac{d - c}{b - a}\).
6. Common Mistakes to Avoid
• Confusing Discrete and Continuous Inequalities: In continuous distributions, \(\text{P}(X < 3) = \text{P}(X \le 3)\). In discrete distributions, \(\text{P}(X < 3) = \text{P}(X \le 2)\). Be extremely careful with your bounds!
• Forgetting to Adjust \(\lambda\): In Poisson problems, ensure your rate matches the given time/area interval before computing probabilities.
• Squaring the Mean in Variance: A classic algebra slip is forgetting to subtract \([\text{E}(X)]^2\) when finding \(\text{Var}(X) = \text{E}(X^2) - [\text{E}(X)]^2\).
• Limits of Integration: When calculating \(\text{E}(X)\) or probabilities, always integrate over the specific domain where \(f(x) > 0\), not necessarily from \(-\infty\) to \(\infty\).
• Coding Constants: Remember that \(\text{Var}(3X - 5) = 3^2 \text{Var}(X) = 9\text{Var}(X)\). The \(-5\) does not affect the variance at all!