Welcome to Linear Combinations of Random Variables!
In your previous studies, you’ve learned how to find the mean (expectation) and variance of a single random variable. But what happens when we start mixing them together? In the real world, things rarely happen in isolation. If you are calculating the total time for a journey, you might have the time spent walking and the time spent on a bus. If you are a business owner, your total profit is your revenue minus your costs.
In this chapter, we explore how to calculate the average outcome and the spread of risk when we combine different random variables. Don't worry if this seems tricky at first—once you see the patterns, it’s mostly just following a few golden rules!
1. Combining Any Random Variables
We start with the rules that apply to any random variables, regardless of their distribution (Binomial, Poisson, etc.).
The Expectation Rule (The "Friendly" Rule)
The expectation, denoted by \(E(X)\), is very well-behaved. It follows the linear combination exactly as you would expect. If you scale a variable or add variables together, the mean follows suit.
The general formula from your syllabus is:
\(E(aX + bY + c) = aE(X) + bE(Y) + c\)
Where \(a, b,\) and \(c\) are constants.
Real-World Example: Imagine you have a side hustle. You earn \(X\) dollars per hour and a fixed tip of \(c\) dollars. If you work 3 hours (\(a=3\)), your expected earnings are simply \(3 \times E(X) + c\).
The Variance Rule (The "Square" Rule)
Variance, denoted by \(Var(X)\), is a bit more sensitive. There is one vital condition you must check before combining variances: The variables must be independent.
If \(X\) and \(Y\) are independent, the formula is:
\(Var(aX + bY + c) = a^2Var(X) + b^2Var(Y)\)
Why is \(a\) squared? Remember that variance is a "squared" measure. If you double the size of something, its area (variance) increases by \(2^2 = 4\).
Why did \(c\) disappear? Adding a constant \(c\) shifts the entire distribution up or down, but it doesn't change how "spread out" the data is. Therefore, constants have zero effect on variance.
Quick Review Box:
1. Expectation uses \(a\).
2. Variance uses \(a^2\).
3. Constants \(c\) are added to Expectations but ignored by Variances.
Key Takeaway: Expectation is straightforward—just "plug and play." Variance requires independence and remember to square your coefficients!
2. Adding vs. Subtracting Variables
This is where many students trip up! Let's look at what happens when we subtract one variable from another, like \(X - Y\).
Expectation of a Difference
\(E(X - Y) = E(X) - E(Y)\)
Example: If you expect to earn £50 and expect to spend £30, you expect to have £20 left. Simple!
Variance of a Difference
\(Var(X - Y) = Var(X) + Var(Y)\) (if independent)
Wait, why is that a plus sign? This is the most common mistake in Further Maths! Think of it this way: Uncertainty always adds up. If you are unsure about how much you'll earn (\(X\)) and unsure about how much you'll spend (\(Y\)), you are even more unsure about your final profit. You never subtract variances to find a total spread.
Memory Aid: Think of variance like "mess." If you combine two messy rooms, you always end up with more mess, never less!
Key Takeaway: Whether you are calculating \(X+Y\) or \(X-Y\), you always add the variances: \(Var(X) + Var(Y)\).
3. Summing Identical Variables vs. Scaling
There is a big difference between \(2X\) (taking one observation and doubling it) and \(X_1 + X_2\) (taking two separate observations and adding them).
The "One Big Item" (Scaling: \(nX\))
If you take one random variable and multiply it by \(n\):
\(E(nX) = nE(X)\)
\(Var(nX) = n^2Var(X)\)
The "Multiple Small Items" (Summing: \(X_1 + X_2 + ... + X_n\))
If you take \(n\) independent observations of the same variable:
\(E(X_1 + ... + X_n) = nE(X)\)
\(Var(X_1 + ... + X_n) = nVar(X)\)
Analogy: Imagine you are buying 10 apples.
Scenario A (\(10X\)): You pick one apple and the cashier tells you it weighs 10 times that specific apple. If that apple is unusually heavy, the whole batch is heavy. This is very risky (high variance).
Scenario B (\(X_1 + ... + X_{10}\)): You pick 10 different apples. Some might be heavy, some light. They tend to "cancel each other out." This is less risky (lower variance).
Key Takeaway: Summing \(n\) independent variables results in a variance of \(nVar(X)\), while scaling a single variable by \(n\) results in a much larger variance of \(n^2Var(X)\).
4. Linear Combinations of Normal Variables
The Normal distribution has a very special "superpower": if you combine Normal variables, the result is always another Normal variable. This is known as the Reproductive Property.
According to your syllabus (5.04b):
1. If \(X \sim N(\mu, \sigma^2)\), then \(aX + b\) is also normally distributed.
2. If \(X\) and \(Y\) are independent normal distributions, then \(aX + bY\) is also normally distributed.
Step-by-Step: Finding the new Distribution
If you are told \(X \sim N(\mu_X, \sigma_X^2)\) and \(Y \sim N(\mu_Y, \sigma_Y^2)\) and you need to find the distribution of \(W = aX + bY\):
Step 1: Find the new mean.
\(E(W) = a\mu_X + b\mu_Y\)
Step 2: Find the new variance.
\(Var(W) = a^2\sigma_X^2 + b^2\sigma_Y^2\)
Step 3: State the distribution.
\(W \sim N(E(W), Var(W))\)
Did you know? This property is why the Normal distribution is so famous. In many science experiments, the "total error" is a sum of many small normal errors, meaning the total error is also normal!
Key Takeaway: "Normal in, Normal out." Just calculate the new mean and the new variance, and you can then use your standard normal distribution steps (or your calculator) to find probabilities.
Summary Checklist
Before you head off to practice questions, keep these points in your pocket:
• Check for Independence: You cannot add variances unless the variables are independent.
• Square the Coefficients: When moving a number out of a \(Var(...)\) bracket, square it (\(a \rightarrow a^2\)).
• Never Subtract Variance: Even if the variables are being subtracted (\(X - Y\)), the variances are added.
• Constants: Constants shift the mean but are ignored by the variance.
• Normal stays Normal: Linear combinations of Normal variables remain Normal.
Encouraging Note: You've got this! Start with the expectation (the easy part) to build your confidence, then carefully work through the variance step-by-step.