Linear combinations of independent variables

Introduction: Why Combine Random Variables?

Welcome to the chapter on Linear Combinations of Independent Variables! In real-world statistics, we rarely look at just a single measurement in isolation. Think about a courier van carrying several parcels, an elevator loaded with passengers, or a baker combining flour, sugar, and butter. Each individual item has its own mean and variation, and when we combine them, we need to know the overall expected value and total variability.

Don't worry if this seems tricky at first. The core rules covered here are completely logical, and once you master a few fundamental formulas and avoid a couple of classic exam traps, you will find these questions very predictable and high-scoring.

Key Concept to Remember: A linear combination simply means multiplying random variables by constants and adding or subtracting them, such as \(aX + bY + c\), where \(a\), \(b\), and \(c\) are fixed real numbers.

1. Expected Value (Mean) of Linear Combinations

The expected value operator, \(E(\dots)\), is strictly linear. It behaves naturally: multiplying a variable multiplies its mean, adding a constant shifts its mean, and adding two variables adds their means.

Fundamental Rules for Expectation:

For any random variables \(X\) and \(Y\), and constants \(a\), \(b\), and \(c\):
1. \(E(aX) = aE(X)\)
2. \(E(aX + b) = aE(X) + b\)
3. \(E(aX + bY) = aE(X) + bE(Y)\)
4. \(E(aX - bY) = aE(X) - bE(Y)\)
5. In general: \(E(aX + bY + c) = aE(X) + bE(Y) + c\)

Did you know? The expectation rules above hold true regardless of whether \(X\) and \(Y\) are independent or dependent! You can always add expectations directly.

Quick Summary: What you see is what you do. If you scale or shift the data, the mean scales and shifts by the exact same amount.

2. Variance of Linear Combinations

Variance measures spread or uncertainty. Unlike expectation, variance does not behave completely linearly, and we must be extra careful when signs and scaling factors are involved.

Fundamental Rules for Variance:

For random variables \(X\) and \(Y\), and constants \(a\), \(b\), and \(c\):
1. \(Var(c) = 0\) (a constant does not vary!)
2. \(Var(aX + b) = a^2Var(X)\) (shifting data by \(+b\) doesn't change its spread, but scaling by \(a\) scales variance by \(a^2\))
3. If \(X\) and \(Y\) are independent:
\(Var(aX + bY) = a^2Var(X) + b^2Var(Y)\)
4. If \(X\) and \(Y\) are independent:
\(Var(aX - bY) = a^2Var(X) + (-b)^2Var(Y) = a^2Var(X) + b^2Var(Y)\)

The Golden Rule: Variances ALWAYS Add!

One of the most common mistakes in Further Mathematics is writing \(Var(X - Y) = Var(X) - Var(Y)\). This is incorrect!

Why? Think about taking the difference between two uncertain measurements (like cutting a piece of wood using two rough measurements). The difference is more uncertain, not less uncertain! Because variance is always positive, subtracting random quantities increases total uncertainty.

\(Var(X - Y) = Var(1 \cdot X + (-1) \cdot Y) = 1^2Var(X) + (-1)^2Var(Y) = Var(X) + Var(Y)\)

Key Takeaway: Whenever you see a minus sign inside a variance calculation, remember that \((-1)^2 = +1\). Provided the variables are independent, variances always add.

3. Sum of Independent Observations vs. Multiple of a Single Variable

This is a major topic in CCEA A Level examinations. Make sure you understand the difference between these two scenarios:

Scenario A: Multiplying one random variable by \(n\)
Example: Picking one bag of sugar \(X\) and scaling its weight by 3: \(3X\).
- \(E(3X) = 3E(X) = 3\mu\)
- \(Var(3X) = 3^2Var(X) = 9\sigma^2\)

Scenario B: Summing \(n\) separate, independent observations of \(X\)
Example: Picking three separate bags of sugar: \(X_1 + X_2 + X_3\).
- \(E(X_1 + X_2 + X_3) = E(X_1) + E(X_2) + E(X_3) = \mu + \mu + \mu = 3\mu\)
- \(Var(X_1 + X_2 + X_3) = Var(X_1) + Var(X_2) + Var(X_3) = \sigma^2 + \sigma^2 + \sigma^2 = 3\sigma^2\)

Analogy: If you buy 3 individual apples, a lighter apple might balance out a heavier apple, so the overall variability is smaller (\(3\sigma^2\)). But if you clone 1 single apple 3 times, any fluke heaviness is magnified 9-fold (\(9\sigma^2\))!

In general:
- For \(nX\): Mean = \(n\mu\), Variance = \(n^2\sigma^2\)
- For \(X_1 + X_2 + \dots + X_n\): Mean = \(n\mu\), Variance = \(n\sigma^2\)

4. Distribution of the Sample Mean

When we take an independent random sample of size \(n\) from a population with mean \(\mu\) and variance \(\sigma^2\), the sample mean is defined as:

\(\bar{X} = \frac{X_1 + X_2 + \dots + X_n}{n} = \frac{1}{n}\sum_{i=1}^{n} X_i\)

Expected Value of \(\bar{X}\):

\(E(\bar{X}) = E\left(\frac{1}{n}\sum X_i\right) = \frac{1}{n}\sum E(X_i) = \frac{1}{n}(n\mu) = \mu\)

Variance of \(\bar{X}\):

\(Var(\bar{X}) = Var\left(\frac{1}{n}\sum X_i\right) = \left(\frac{1}{n}\right)^2 \sum Var(X_i) = \frac{1}{n^2}(n\sigma^2) = \frac{\sigma^2}{n}\)

Key Takeaway: The sample mean \(\bar{X}\) is an unbiased estimator of \(\mu\) with a variance of \(\frac{\sigma^2}{n}\). As the sample size \(n\) gets larger, the variance shrinks towards zero, meaning larger samples give much more precise estimates of the population mean.

5. Linear Combinations of Independent Normal Variables

A crucial and elegant property of the Normal Distribution is that any linear combination of independent Normal variables is also Normally distributed.

If \(X \sim N(\mu_X, \sigma_X^2)\) and \(Y \sim N(\mu_Y, \sigma_Y^2)\) are independent, and \(W = aX + bY + c\), then:
\(W \sim N(\mu_W, \sigma_W^2)\)
where:
- \(\mu_W = a\mu_X + b\mu_Y + c\)
- \(\sigma_W^2 = a^2\sigma_X^2 + b^2\sigma_Y^2\)

Step-by-Step Problem Solving Strategy:

1. Define the new variable: Clearly state the combination, e.g., let \(W = aX + bY\).
2. Calculate the new Mean: Find \(E(W)\).
3. Calculate the new Variance: Find \(Var(W)\) remembering to square all coefficients.
4. State the distribution: Write \(W \sim N(E(W), Var(W))\).
5. Standardise and calculate probability: Use \(Z = \frac{W - \mu_W}{\sigma_W}\) and statistical tables or calculator.

Worked Example 1: Lift Capacity Problem

Question: An elevator has a maximum safe load of \(450\text{ kg}\). The mass of a man is \(M \sim N(80, 25)\) and the mass of a woman is \(W \sim N(65, 16)\). If 3 men and 3 women enter the lift independently, find the probability that the safe limit is exceeded.

Step 1: Define the combined variable.
Let total mass \(T = M_1 + M_2 + M_3 + W_1 + W_2 + W_3\).
(Note: This is 6 independent people, NOT \(3M + 3W\)).

Step 2: Find the mean.
\(E(T) = 3E(M) + 3E(W) = 3(80) + 3(65) = 240 + 195 = 435\text{ kg}\)

Step 3: Find the variance.
\(Var(T) = 3Var(M) + 3Var(W) = 3(25) + 3(16) = 75 + 48 = 123\)

Step 4: State the distribution.
\(T \sim N(435, 123)\)

Step 5: Calculate the required probability.
We want \(P(T > 450)\).
Standardising:
\(Z = \frac{450 - 435}{\sqrt{123}} = \frac{15}{11.0905} \approx 1.352\)
\(P(T > 450) = P(Z > 1.352) = 1 - \Phi(1.352) \approx 1 - 0.9118 = 0.0882\) (or \(8.82\%\)).

Worked Example 2: Difference Between Two Random Variables

Question: Bolt lengths are distributed as \(B \sim N(50, 0.04)\text{ mm}\) and nut depths are \(D \sim N(49.6, 0.05)\text{ mm}\). Find the probability that a randomly chosen bolt is longer than a randomly chosen nut.

Step 1: Set up the inequality as a single variable.
We want \(P(B > D)\), which is equivalent to \(P(B - D > 0)\).
Let \(Y = B - D\).

Step 2: Calculate \(E(Y)\) and \(Var(Y)\).
\(E(Y) = E(B) - E(D) = 50 - 49.6 = 0.4\)
\(Var(Y) = 1^2Var(B) + (-1)^2Var(D) = 0.04 + 0.05 = 0.09\)
Therefore, the standard deviation is \(\sigma_Y = \sqrt{0.09} = 0.3\).

Step 3: State distribution and calculate probability.
\(Y \sim N(0.4, 0.09)\)
\(P(Y > 0) = P\left(Z > \frac{0 - 0.4}{0.3}\right) = P(Z > -1.333) = \Phi(1.333) \approx 0.9088\).

6. Summary of Common Mistakes to Avoid

Mistake 1: Subtracting variances.
Never write \(Var(X - Y) = Var(X) - Var(Y)\). Remember: \(Var(X - Y) = Var(X) + Var(Y)\).

Mistake 2: Forgetting to square coefficients in variance.
Remember that \(Var(3X) = 3^2 Var(X) = 9Var(X)\), not \(3Var(X)\).

Mistake 3: Confusing \(nX\) with \(\sum_{i=1}^n X_i\).
Read the question carefully. If it refers to "\(n\) separate items chosen at random", you are dealing with \(X_1 + X_2 + \dots + X_n\), where the variance is \(n\sigma^2\).

Mistake 4: Mixing up variance and standard deviation.
When writing down parameters for the Normal distribution \(N(\mu, \sigma^2)\), always verify whether you are given \(\sigma\) or \(\sigma^2\). If given standard deviation \(\sigma = 4\), the variance is \(4^2 = 16\).

7. Quick Review Checklist

Before sitting your exam, make sure you can:
- Apply \(E(aX + bY + c) = aE(X) + bE(Y) + c\) in any scenario.
- Apply \(Var(aX + bY + c) = a^2Var(X) + b^2Var(Y)\) for independent variables.
- State and use the distribution of the sample mean: \(\bar{X} \sim N\left(\mu, \frac{\sigma^2}{n}\right)\).
- Set up difference variables like \(D = X - Y\) to solve probability problems such as \(P(X > Y)\).