Welcome to the World of Multivariate Moments!

Hi there! You’ve already learned how to find the mean (expected value) and variance for a single variable. Now, we are stepping into the Multivariate world. Think of this as moving from looking at a single person’s height to looking at the relationship between their height and weight.

In this chapter, we explore "moments"—which is just a fancy statistical word for averages and spreads—when two or more variables are interacting. Don't worry if this seems a bit "busy" with all the double integrals or sums; once you see the patterns, it becomes much more manageable!

1. Joint Expected Values

If we have two random variables, \(X\) and \(Y\), we might want to find the average of a function that involves both of them, like \(E[X + Y]\) or \(E[XY]\). This is called a Joint Expected Value.

To find the expected value of any function \(g(X, Y)\), you simply multiply the function by the joint probability density (or mass) function and sum/integrate over everything.

For Discrete Variables:

\( E[g(X, Y)] = \sum_{x} \sum_{y} g(x, y) \cdot p(x, y) \)

For Continuous Variables:

\( E[g(X, Y)] = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(x, y) \cdot f(x, y) \,dy \,dx \)

Analogy: Imagine a grid of tiles on the floor. Each tile has a value written on it, and each tile also has a certain "weight" (probability). To find the overall average value of the room, you multiply each tile's value by its weight and add them all up.

Quick Review: Remember that the expected value is linear! This means \(E[aX + bY + c] = aE[X] + bE[Y] + c\). This works even if \(X\) and \(Y\) are dependent!

2. Covariance: Measuring the Relationship

Covariance measures how two variables move together. If \(X\) goes up when \(Y\) goes up, the covariance is positive. If \(X\) goes down when \(Y\) goes up, it’s negative.

The formula you will use most often on Exam P is:

\( \mathbf{Cov(X, Y) = E[XY] - E[X]E[Y]} \)

Important Note: If \(X\) and \(Y\) are independent, then \(E[XY] = E[X]E[Y]\), which means their Covariance is 0. However, be careful! Having a covariance of 0 does not always mean they are independent (though for Exam P purposes, it usually suggests no linear relationship).

Correlation (\(\rho\))

Covariance can be any number, which makes it hard to compare. Correlation scales the covariance to a range between -1 and 1.

\( \rho_{XY} = \frac{Cov(X, Y)}{\sigma_X \sigma_Y} \)

Where \(\sigma_X\) and \(\sigma_Y\) are the standard deviations of \(X\) and \(Y\).

Key Takeaway: Covariance tells you the direction of the relationship; Correlation tells you both the direction and the strength.

3. Conditional Moments

Sometimes we are given information about one variable and asked to find the mean or variance of the other. These are Conditional Moments.

Conditional Expectation

The conditional mean of \(X\) given that \(Y = y\) is:

\( E[X | Y = y] = \int x \cdot f_{X|Y}(x|y) \,dx \)

Essentially, you are treating \(Y\) as a constant and using the conditional density to find the average.

Conditional Variance

This measures the "spread" of \(X\) after we already know what \(Y\) is:

\( Var(X | Y = y) = E[X^2 | Y = y] - (E[X | Y = y])^2 \)

Did you know? \(E[X|Y]\) is actually a random variable itself because its value depends on what \(Y\) turns out to be! This leads us to the two most important laws in this chapter.

4. The Heavy Hitters: Double Expectation and Total Variance

These two formulas are "Exam P Gold." You will almost certainly need them.

The Law of Iterated Expectations (Adam's Law)

This law says that the "average of the averages" is the overall average.

\( \mathbf{E[X] = E[E[X|Y]]} \)

Example: If you want to find the average height of all students in a school, you can find the average height in each classroom (\(E[X|Y]\)) and then take the average of those results.

The Law of Total Variance (Eve's Law)

This breaks down the total variance into two parts: the variance within groups and the variance between groups.

\( \mathbf{Var(X) = E[Var(X|Y)] + Var(E[X|Y])} \)

Memory Aid: Many students remember this as "EV + VE" (Expected Value of Variance + Variance of Expected Value).

Don't worry if this seems tricky at first! Just remember that the first part (\(E[Var]\)) handles the "noise" inside the conditional groups, and the second part (\(Var(E)\)) handles how much the "averages" themselves change.

5. Common Mistakes to Avoid

1. Mixing up Marginal and Conditional: Always check if you should be using \(f(x)\) (marginal) or \(f(x|y)\) (conditional). If the problem says "Given that \(Y=2\)...", you must use the conditional density!

2. Forgetting to square the mean: In the variance formula \(Var(X) = E[X^2] - (E[X])^2\), students often forget to square the second term. This applies to conditional variance too!

3. Assuming \(E[XY] = E[X]E[Y]\): This is only true if the variables are independent. If the joint density \(f(x,y)\) doesn't factorize into \(f(x)f(y)\), you must integrate \(xy \cdot f(x,y)\).

6. Summary Checklist

Before you move to the practice problems, make sure you can:

  • Calculate \(E[XY]\) using a double integral or double sum.
  • Find \(Cov(X, Y)\) and understand that it is 0 for independent variables.
  • Calculate the Correlation coefficient and ensure it is between -1 and 1.
  • Apply the Law of Iterated Expectations to simplify complex problems.
  • Use Eve's Law (EV + VE) to find the total variance of a variable.

Final Tip: When you see a problem where one variable's behavior depends on another (e.g., "The number of claims \(N\) depends on the risk class \(\Theta\)"), that is a huge hint to use the Laws of Total Expectation or Total Variance!