M1 Statistics: Conditional Probability and Bayes' Theorem
Hello everyone! Welcome to the study notes for one of the most core topics in HKDSE M1 statistics: Conditional Probability and Bayes' Theorem. Don't worry if the names sound a bit intimidating. The ideas behind them are highly logical and are widely tested in both Section A and Section B conventional questions.
In this chapter, we will learn how to calculate conditional probabilities, test for independent events, apply the multiplication rule, and use the Law of Total Probability and Bayes' Theorem to reverse conditional events. Let's get started!
1. What is Conditional Probability?
At its heart, conditional probability is the probability of an event happening, given that another event has already occurred. We are updating our probability based on new information.
A Quick Review: Basic Probability
Remember that the basic probability of an event \(A\) is:
\(P(A) = \frac{\text{Number of favourable outcomes for } A}{\text{Total number of possible outcomes}}\raggedright\)
Example: The probability of rolling a '4' on a fair six-sided die is \(P(\text{rolling a 4}) = \frac{1}{6}\), because there is one '4' and six total possible outcomes.
The Core Idea: A Reduced Sample Space
Conditional probability shrinks our sample space. When we know that event \(B\) has already happened, we can ignore all outcomes outside of \(B\).
Analogy: Finding a specific student in a school.
Imagine your school has 1000 students. The probability of randomly picking a Form 6 girl is \(P(\text{Form 6 Girl})\).
Now, suppose you receive new information: "The chosen student is in Form 6."
Your sample space immediately shrinks from 1000 students to only the Form 6 students. The probability of picking a girl given that the student is from Form 6 is evaluated only among Form 6 students.
Notation and Formula
We write the conditional probability of "Event \(A\) happening given that Event \(B\) has occurred" as \(P(A|B)\).
The vertical bar \(|\) denotes "given". Thus, \(P(A|B)\) is read as "the probability of \(A\), given \(B\)".
The fundamental formula is:
\(P(A|B) = \frac{P(A \cap B)}{P(B)}\quad \text{where } P(B) > 0\)
Let's break down each component:
\(P(A|B)\): The probability of \(A\) occurring under the condition that \(B\) occurs.
\(P(A \cap B)\): The probability of both \(A\) and \(B\) occurring together (the intersection of \(A\) and \(B\)).
\(P(B)\): The probability of the given condition \(B\) (the reduced sample space).
Step-by-Step Example: Two-Way Contingency Table
A survey of 100 secondary students classifies them by the sports they play:
| Plays Football (\(F\)) | Does Not Play Football (\(F'\)) | Total | |
|---|---|---|---|
| Plays Basketball (\(B\)) | 20 | 25 | 45 |
| Does Not Play Basketball (\(B'\)) | 30 | 25 | 55 |
| Total | 50 | 50 | 100 |
Question: Find the probability that a randomly chosen student plays basketball, given that they play football.
Step 1: Identify the required probability.
We want to find \(P(B|F)\).
Step 2: Extract probabilities from the table.
\(P(B \cap F) = \frac{20}{100} = 0.2\)
\(P(F) = \frac{50}{100} = 0.5\)
Step 3: Apply the conditional probability formula.
\(P(B|F) = \frac{P(B \cap F)}{P(F)} = \frac{0.2}{0.5} = 0.4\)
Alternatively, looking directly at the row of football players: out of 50 football players, 20 play basketball, giving \(\frac{20}{50} = 0.4\).
Common Mistakes to Avoid:
• \(P(A|B) \neq P(B|A)\) in general. For example, \(P(B|F) = \frac{20}{50} = 0.4\), but \(P(F|B) = \frac{20}{45} \approx 0.444\).
• The denominator must always be the probability of the given conditioning event: for \(P(A|B)\), divide by \(P(B)\).
2. Independent Events
Two events \(A\) and \(B\) are independent if the occurrence of one event does not affect the probability of the occurrence of the other.
Formal Conditions for Independence
Two events \(A\) and \(B\) are independent if and only if any one of the following equivalent statements holds:
\(P(A|B) = P(A)\) (where \(P(B) > 0\))
\(P(B|A) = P(B)\) (where \(P(A) > 0\))
Multiplication Rule for Independent Events: \(P(A \cap B) = P(A) \times P(B)\)
Note for HKDSE M1: To prove whether two events are independent, compute \(P(A \cap B)\) and compare it with the product \(P(A) \times P(B)\). If \(P(A \cap B) = P(A)P(B)\), they are independent; otherwise, they are dependent.
3. The General Multiplication Rule & Tree Diagrams
The General Multiplication Rule
Rearranging the conditional probability formula gives the general multiplication rule, valid for any two events \(A\) and \(B\):
\(P(A \cap B) = P(A) \times P(B|A) = P(B) \times P(A|B)\)
Sequential Experiments (Without Replacement)
Example: A bag contains 5 red balls and 3 blue balls (total 8 balls). Two balls are drawn at random without replacement. What is the probability that both balls drawn are red?
Step 1: First draw.
Probability of drawing a red ball first: \(P(R_1) = \frac{5}{8}\).
Step 2: Second draw (conditional on the first).
After drawing 1 red ball, 4 red and 3 blue remain (total 7 balls).
\(P(R_2 | R_1) = \frac{4}{7}\).
Step 3: Apply the multiplication rule.
\(P(R_1 \cap R_2) = P(R_1) \times P(R_2|R_1) = \frac{5}{8} \times \frac{4}{7} = \frac{20}{56} = \frac{5}{14}\)
4. The Law of Total Probability & Bayes' Theorem
The Law of Total Probability
Let events \(A_1, A_2, \dots, A_k\) form a partition of the sample space (meaning they are mutually exclusive, \(A_i \cap A_j = \emptyset\) for \(i \neq j\), and exhaustive, \(A_1 \cup A_2 \cup \dots \cup A_k = S\)).
For any event \(B\):
\(P(B) = \sum_{i=1}^k P(B \cap A_i) = \sum_{i=1}^k P(A_i)P(B|A_i)\)
For two complementary events \(A\) and \(A'\), this simplifies to:
\(P(B) = P(A)P(B|A) + P(A')P(B|A')\)
Bayes' Theorem
Bayes' Theorem allows us to reverse conditional probabilities (finding \(P(A_j|B)\) when we know the conditional likelihoods \(P(B|A_i)\)):
\(P(A_j|B) = \frac{P(A_j \cap B)}{P(B)} = \frac{P(A_j)P(B|A_j)}{\sum_{i=1}^k P(A_i)P(B|A_i)}\)
For two partitions \(A\) and \(A'\):
\(P(A|B) = \frac{P(A)P(B|A)}{P(A)P(B|A) + P(A')P(B|A')}\)
Step-by-Step Worked Example
Two production lines, \(A\) and \(B\), produce light bulbs in a factory. Line \(A\) produces 60% of all bulbs, and Line \(B\) produces 40%. It is known that 2% of bulbs from Line \(A\) are defective, while 5% of bulbs from Line \(B\) are defective.
Question: If a randomly selected bulb is defective, find the probability that it came from Line \(A\).
Step 1: Define events and identify given values.
Let \(A\) = bulb comes from Line \(A\), \(B\) = bulb comes from Line \(B\), \(D\) = bulb is defective.
\(P(A) = 0.60\), \(P(B) = 0.40\)
\(P(D|A) = 0.02\), \(P(D|B) = 0.05\)
Step 2: Find total probability of defective bulb \(P(D)\).
Using the Law of Total Probability:
\(P(D) = P(A)P(D|A) + P(B)P(D|B)\)
\(P(D) = (0.60)(0.02) + (0.40)(0.05) = 0.012 + 0.020 = 0.032\)
Step 3: Apply Bayes' Theorem.
\(P(A|D) = \frac{P(A \cap D)}{P(D)} = \frac{P(A)P(D|A)}{P(D)} = \frac{0.012}{0.032} = 0.375 = \frac{3}{8}\)
Chapter Summary: Quick Review
- Conditional Probability: \(P(A|B) = \frac{P(A \cap B)}{P(B)}\), where \(P(B) > 0\).
- Multiplication Rule: \(P(A \cap B) = P(A)P(B|A) = P(B)P(A|B)\).
- Independent Events: \(A\) and \(B\) are independent if and only if \(P(A \cap B) = P(A)P(B)\) (or \(P(A|B) = P(A)\)).
- Law of Total Probability: \(P(B) = \sum_{i=1}^k P(A_i)P(B|A_i)\).
- Bayes' Theorem: \(P(A_j|B) = \frac{P(A_j)P(B|A_j)}{\sum_{i=1}^k P(A_i)P(B|A_i)}\).