Conditional and marginal probability functions (discrete)

Welcome to the World of Multivariate Variables!

In your probability journey so far, you’ve likely spent a lot of time looking at one random variable at a time—like the number of claims a car insurance policyholder makes in a year. But in the real world (and on Exam P), things rarely happen in isolation. Usually, we are looking at two or more variables at once. For example, an insurance company might look at the number of car insurance claims and the number of home insurance claims for the same person.

In this chapter, we are going to learn how to "isolate" one variable from a group (Marginal Probability) and how to update our knowledge of one variable when we already know something about another (Conditional Probability). Don't worry if this seems tricky at first; we’ll break it down step-by-step using simple tables and analogies!

1. The Big Picture: Discrete Joint PMFs

Before we dive into marginals and conditionals, remember that for discrete variables, we use a Joint Probability Mass Function (PMF), written as \( p(x, y) = P(X = x, Y = y) \). Think of this as a grid or a table where every cell represents the probability of \( X \) and \( Y \) happening at the same time.

2. Marginal Probability Functions: Finding the "Total"

Imagine you have a table showing the probability of someone owning \( X \) dogs and \( Y \) cats. If I ask you, "What is the probability that someone owns 2 dogs, regardless of how many cats they have?", you are looking for the Marginal Probability of \( X \).

To find the marginal probability of \( X \), you simply sum up all the probabilities in the row (or column) where \( X = x \). You are essentially "summing out" the variable you don't care about.

The Formula:
To find the marginal PMF of \( X \):
\( p_X(x) = \sum_{all y} p(x, y) \)

To find the marginal PMF of \( Y \):
\( p_Y(y) = \sum_{all x} p(x, y) \)

A Simple Analogy:
Think of a spreadsheet. The Joint probabilities are the numbers in the middle of the spreadsheet. The Marginal probabilities are the totals written in the margins (the far right or the very bottom) after you add up the rows and columns. That's actually why they are called "marginal" probabilities!

Quick Review: The Rules of Marginals

1. The marginal probability \( p_X(x) \) must always be between 0 and 1.
2. If you sum all the marginal probabilities for \( X \), they must equal 1. ( \( \sum p_X(x) = 1 \) )

3. Conditional Probability Functions: Narrowing Your Focus

Conditional probability is all about new information. If I tell you, "Given that this person owns exactly 1 cat, what is the probability they own 2 dogs?", I have changed the "universe" of possibilities. We no longer care about people with 0 cats or 2 cats. We only care about the "1 cat" row.

The Formula:
The conditional PMF of \( X \) given \( Y=y \) is:
\( p_{X|Y}(x|y) = P(X=x | Y=y) = \frac{p(x, y)}{p_Y(y)} \)

Note: This formula only works if \( p_Y(y) > 0 \). You can't condition on something that has a 0% chance of happening!

Step-by-Step Explanation:
1. Identify the "Given": Look at the value of the variable you are told is true (e.g., \( Y = y \)).
2. Find the "Slice": Look only at the row or column in the joint table corresponding to that value.
3. Re-scale: Divide the joint probability in that cell by the total probability of that entire row or column (the marginal). This ensures that your new "conditional" probabilities sum up to 1.

Common Mistake to Avoid

Students often forget to divide by the marginal probability. Remember: a conditional probability distribution is its own valid probability distribution. If you don't divide by the marginal, your probabilities won't add up to 1!

4. Independence in the Discrete Case

How do you know if two variables \( X \) and \( Y \) have nothing to do with each other? In probability terms, we say they are independent.

The Test:
\( X \) and \( Y \) are independent if and only if:
\( p(x, y) = p_X(x) \cdot p_Y(y) \)
for all pairs of \( (x, y) \).

Alternatively, they are independent if the conditional probability is the same as the marginal probability:
\( p_{X|Y}(x|y) = p_X(x) \)

Did you know? If you find even one cell in your joint table where \( p(x,y) \neq p_X(x)p_Y(y) \), the variables are dependent! You don't need to check the whole table if you find one mismatch early on.

5. Example Walkthrough

Let's look at a joint PMF table for \( X \) (Number of accidents) and \( Y \) (Number of vehicles):

\( X=0, Y=1: 0.40 \)
\( X=0, Y=2: 0.10 \)
\( X=1, Y=1: 0.20 \)
\( X=1, Y=2: 0.30 \)

Task 1: Find the marginal probability \( p_X(1) \).
Sum the probabilities where \( X=1 \):
\( p_X(1) = P(X=1, Y=1) + P(X=1, Y=2) = 0.20 + 0.30 = 0.50 \).

Task 2: Find the conditional probability \( P(Y=2 | X=0) \).
First, find the marginal \( p_X(0) \):
\( p_X(0) = 0.40 + 0.10 = 0.50 \).
Now, use the formula:
\( P(Y=2 | X=0) = \frac{P(X=0, Y=2)}{p_X(0)} = \frac{0.10}{0.50} = 0.20 \).

Summary and Key Takeaways

1. Marginals: Think "Totals." Sum across the variable you want to get rid of.
2. Conditionals: Think "Slices." Take the joint probability and divide it by the "given" marginal.
3. Independence: If the joint probability is just the two marginals multiplied together, the variables don't affect each other.
4. Check Your Work: Marginal probabilities for a single variable must sum to 1. Conditional probabilities for a given condition must also sum to 1.

Don't worry if the summation notation looks scary. Just remember: Marginals are the margins of the table, and Conditionals are just a "zoomed-in" look at one specific row or column!