Statistical hypothesis testing

Welcome to the World of Statistical Detective Work!

Have you ever wondered if a "lucky" coin is actually biased, or if a new medicine really works better than an old one? In Statistical Hypothesis Testing, we use mathematics to decide whether an outcome is a genuine discovery or just a bit of random luck. Don't worry if this seems a bit abstract at first—we’re basically just learning how to be "math detectives"!

1. The Language of Hypothesis Testing

Before we start calculating, we need to understand the "legal system" of statistics. Think of a hypothesis test like a court trial.

The Two Rival Hypotheses

Every test involves two competing statements about a population parameter (usually the probability \( p \) in a Binomial distribution):

1. The Null Hypothesis \( (H_0) \): This is the "status quo." It assumes that nothing has changed, or that a coin is fair. In our court analogy, the defendant is innocent until proven guilty. We always write this as \( H_0: p = \text{something} \).

2. The Alternative Hypothesis \( (H_1) \): This is the claim we are investigating. It's the "guilty" verdict. We write this as \( H_1: p < \dots \), \( H_1: p > \dots \), or \( H_1: p \neq \dots \).

Key Terms You Need to Know

• Test Statistic: This is the actual result we observe in our sample (e.g., the number of heads we got when flipping the coin).
• Significance Level \( (\alpha) \): This is the "threshold" for proof. Common levels are 5% (\( 0.05 \)) or 1% (\( 0.01 \)). If the chance of our result happening by pure luck is lower than this level, we "reject" the Null Hypothesis.
• Critical Region: The range of values for the test statistic that would cause us to reject \( H_0 \). If our result falls in this "rejection zone," we've found enough evidence for a change.
• Critical Value: The first value in the critical region. It's the "boundary line" of our detective work.

Memory Aid: Think of \( H_0 \) as "Ho-hum"—nothing interesting is happening here! Think of \( H_1 \) as the "Hypothesis of Hope"—the thing you're hoping to prove.

Key Takeaway: We always start by assuming \( H_0 \) is true and only change our minds if the evidence is very strong.

2. One-Tail vs. Two-Tail Tests

Depending on what we are looking for, we choose a specific "direction" for our test.

One-Tail Test

We use this when we are interested in a change in a specific direction. For example: "The new seeds have a higher germination rate" (\( p > \dots \)) or "The medicine reduces recovery time" (\( p < \dots \)).

Two-Tail Test

We use this when we want to know if the value has changed at all, but we don't know (or care) if it has gone up or down. For example: "The probability of a machine error has changed" (\( p \neq \dots \)).

Important Tip: In a two-tail test, you must split the significance level in half. If your total significance level is 5%, you look for a 2.5% tail at the bottom and a 2.5% tail at the top!

Did you know? Most scientific papers use a 5% significance level. This means they accept there is a 1 in 20 chance that their "discovery" was actually just random luck!

3. Testing with the Binomial Distribution

At AS Level, your hypothesis tests will be based on the Binomial Model \( X \sim B(n, p) \).

Step-by-Step Guide to Conducting a Test:

Step 1: Write down your hypotheses. State \( H_0 \) and \( H_1 \) clearly using the parameter \( p \).

Step 2: Define the distribution. State what the distribution would be if \( H_0 \) were true (e.g., \( X \sim B(10, 0.5) \)).

Step 3: Find the probability of the observed result. Use your calculator to find the probability of getting a result "as extreme or more extreme" than the one you observed.

• If testing \( p > \dots \), calculate \( P(X \geq \text{observed value}) \).
• If testing \( p < \dots \), calculate \( P(X \leq \text{observed value}) \).

Step 4: Compare with the significance level.

• If the probability is less than the significance level, Reject \( H_0 \).
• If the probability is greater than the significance level, Do not reject \( H_0 \).

Step 5: Write a conclusion in context. Always relate it back to the story (e.g., "There is sufficient evidence to suggest the coin is biased").

4. Understanding the "P-value"

The p-value is simply the probability of getting your observed result (or something more extreme) if the Null Hypothesis is true.

Analogy: Imagine you flip a coin 10 times and get 10 heads. The p-value is the probability of that happening with a fair coin (\( 0.5^{10} \approx 0.0009 \)). Because 0.0009 is much smaller than 0.05 (5%), you would conclude the coin is definitely not fair!

Quick Review Box:
- Small p-value (\( < \alpha \)) = Strong evidence = Reject \( H_0 \).
- Large p-value (\( > \alpha \)) = Weak evidence = Fail to reject \( H_0 \).

5. Common Mistakes to Avoid

• Wrong notation: Never write hypotheses using \( \bar{x} \) or \( X \). Always use the population parameter \( \mathbf{p} \).
• Forgetting context: Don't just stop at "Reject \( H_0 \)." You must say what that means for the gardener, the doctor, or the gambler in the question.
• Two-tail confusion: Forgetting to halve the significance level when the question says "change" or "different."
• Calculating the wrong tail: If the observed value is higher than expected, calculate \( P(X \geq x) \). If it's lower, calculate \( P(X \leq x) \).

Summary: The Big Picture

Hypothesis testing isn't about being 100% certain. It's about deciding if a result is statistically significant—meaning it is very unlikely to have happened by chance.

Key Takeaway: The significance level is the probability of incorrectly rejecting the null hypothesis. It represents the "risk" we are willing to take of being wrong when we claim to have found a discovery.

Don't worry if this seems tricky at first! The more examples you practice, the more the logic will start to feel like second nature. You've got this!