Hypothesis Testing

Welcome to the World of Hypothesis Testing!

In the FRM curriculum, Hypothesis Testing is like being a detective in the world of data. Up until now, you’ve learned how to calculate means and variances. Now, we are going to learn how to decide if those numbers actually mean something or if they are just the result of random luck.

Whether you are testing if a new investment strategy beats the market or checking if a risk model is accurate, hypothesis testing is your primary tool. Don't worry if this seems a bit "math-heavy" at first—we’ll break it down into simple, logical steps that anyone can follow!

1. The Starting Line: Null and Alternative Hypotheses

Every hypothesis test starts with two opposing statements. Think of this like a courtroom trial.

The Null Hypothesis (\(H_0\)): This is the "status quo" or the "innocent until proven guilty" statement. It usually states that there is no effect, no difference, or that a parameter equals a specific value. We always assume \(H_0\) is true until we have enough evidence to say otherwise.

The Alternative Hypothesis (\(H_a\) or \(H_1\)): This is what you are actually trying to prove. It’s the "guilty" verdict. If the data is strong enough, we reject \(H_0\) in favor of \(H_a\).

One-Tailed vs. Two-Tailed Tests

How do you know which one to use? It depends on your research question:

Two-Tailed Test: You are looking for any difference.
Example: Is the return of Fund A different from 5%? (\(H_a: \mu \neq 5\%\))

One-Tailed Test: You are looking for a specific direction (greater than or less than).
Example: Is the return of Fund A greater than 5%? (\(H_a: \mu > 5\%\))

Key Takeaway:

The Null Hypothesis (\(H_0\)) always contains an "equal to" sign (\(=\), \(\leq\), or \(\geq\)). The Alternative Hypothesis (\(H_a\)) never does.

2. The "Oops" Moments: Type I and Type II Errors

Statistical testing isn't perfect. Sometimes we make the wrong call. There are two specific ways we can mess up:

Type I Error (False Alarm): We reject the Null Hypothesis when it was actually true.
Analogy: An innocent person is sent to jail.
The probability of making a Type I error is called Significance Level (\(\alpha\)).

Type II Error (Missed Opportunity): We fail to reject the Null Hypothesis when it was actually false.
Analogy: A guilty person is set free.
The probability of making a Type II error is called \(\beta\).

Did you know? There is a trade-off! If you try too hard to avoid Type I errors (by being very strict), you increase the chance of making a Type II error. It’s a balancing act.

Power of the Test

The Power of a Test is the probability of correctly rejecting a false Null Hypothesis.
Formula: \(Power = 1 - \beta\).
We want our "test power" to be as high as possible!

3. The Decision Toolkit: Test Statistics and P-Values

To make a decision, we need to calculate a "Test Statistic." This is a value that tells us how far our sample result is from the Null Hypothesis.

The general formula for most test statistics is:
\(Test Statistic = \frac{Sample\ Statistic - Hypothesized\ Value}{Standard\ Error}\)

The Decision Rule

Once you have your test statistic, you compare it to a Critical Value (found in a stats table):

1. If your |Test Statistic| > |Critical Value|, you Reject \(H_0\). (Significant result!)
2. If your |Test Statistic| < |Critical Value|, you Fail to Reject \(H_0\). (Not enough evidence.)

The P-Value Approach

If you don't want to use critical values, use the P-value. This is the probability of getting a result as extreme as yours if the Null Hypothesis is true.
The Simple Rule: If P-value < \(\alpha\), Reject \(H_0\).
Mnemonic: "If the P is low, the Null must go!"

Quick Review:

Confidence Level: \(1 - \alpha\) (usually 95% or 99%).
Significance Level: \(\alpha\) (usually 5% or 1%).

4. Choosing the Right Test

In the FRM exam, you need to know which tool to grab from the shed. Here is a cheat sheet:

A. Testing the Mean (\(\mu\))

Use a Z-test if:
- You know the population variance (\(\sigma^2\)).
- Or if the sample size is large (\(n \geq 30\)).
\(Z = \frac{\bar{x} - \mu_0}{\sigma / \sqrt{n}}\)

Use a T-test if:
- The population variance is unknown (which is almost always the case in the real world).
- The sample size is small (\(n < 30\)).
\(t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}\) (where \(s\) is the sample standard deviation)

B. Testing the Variance (\(\sigma^2\))

Use a Chi-Square (\(\chi^2\)) Test if you are testing the variance of a single population.
Note: The Chi-Square distribution is not symmetrical and cannot be negative.

C. Comparing Two Variances

Use an F-test if you want to know if the variance of one group is equal to the variance of another.
Formula: \(F = \frac{s_1^2}{s_2^2}\) (Always put the larger variance on top!)

5. Step-by-Step Guide to Hypothesis Testing

Don't worry if this seems tricky at first. Just follow these 5 steps every time:

Step 1: State the Null (\(H_0\)) and Alternative (\(H_a\)) hypotheses.
Step 2: Select the significance level (\(\alpha\))—usually 0.05.
Step 3: Identify the appropriate test statistic (Z, T, Chi-square, or F).
Step 4: Calculate the Test Statistic using your sample data.
Step 5: Compare the Test Statistic to the Critical Value and make your decision.

6. Common Pitfalls to Avoid

1. "Accepting" the Null: We never "accept" the Null Hypothesis. We only "fail to reject" it. It sounds like a lawyer's trick, but it's important! It just means we don't have enough proof to dump the Null yet.

2. Confusing Tails: For a two-tailed test at a 5% significance level, you put 2.5% in each tail. For a one-tailed test, all 5% goes into one tail. This changes your critical value!

3. Ignoring the "Normal" Assumption: Most of these tests assume the underlying data follows a normal distribution. If the data is weirdly shaped, these tests might lie to you.

Summary Table:

Test for Mean (Known Variance): Z-test
Test for Mean (Unknown Variance): T-test
Test for Single Variance: Chi-Square test
Comparing Two Variances: F-test

You've got this! Hypothesis testing is just a logical way to ask, "Is this result special, or just a coincidence?" Master these basics, and the Quantitative Analysis section will be much easier to navigate.