Estimation and Hypothesis Testing

Q: What is survivorship bias in sampling?

Survivorship bias occurs when a sample only includes entities that have survived a period, excluding those that failed or dropped out.

Q: What is data-mining bias?

Data-mining bias occurs from repeatedly searching a dataset for patterns until a statistically significant result is found by chance.

CFA Level I · CFA · 17 min read · Updated: 22 Oct 2026

Welcome to Estimation and Inference!

In our previous studies, we looked at descriptive statistics—ways to summarize data we already have. But in the real world of finance, we rarely have all the data. We can't know the future returns of every single stock in existence, right? That’s where Estimation and Inference come in. This chapter is all about taking a small "sip" of data (a sample) and making a smart guess about the whole "pot" (the population). Don’t worry if this seems a bit abstract at first; we’ll break it down piece by piece!

1. The Basics: Populations and Samples

Before we dive into the math, we need to understand who we are studying.
Population: The entire group you want to know about (e.g., every public company in the world).
Sample: A smaller subset of that group that we actually measure (e.g., 100 companies we picked to study).
Parameter: A descriptive measure of a population (like the "true" average return).
Statistic: A descriptive measure of a sample (the average return of our 100 companies).

Quick Tip: Remember P for Population/Parameter and S for Sample/Statistic!

Types of Sampling

How we pick our sample matters a lot. If we pick badly, our results will be wrong.

Simple Random Sampling: Every member of the population has an equal chance of being selected. Think of drawing names out of a hat.
Stratified Random Sampling: We divide the population into sub-groups (called strata) based on a characteristic (like industry) and then take a proportional number from each. Analogy: If you’re making a fruit salad, you want to make sure you get a mix of apples, grapes, and melons in every spoonful, not just all grapes!
Cluster Sampling: We divide the population into "clusters" (like geographic regions) and then randomly select entire clusters to study.

Key Takeaway:

Sampling allows us to estimate population characteristics without having to look at every single data point, saving time and money.

2. The Magic of the Central Limit Theorem (CLT)

This is arguably the most important concept in all of statistics. The Central Limit Theorem tells us something amazing: If you take a large enough sample (usually n ≥ 30), the distribution of the sample means will look like a normal distribution (a bell curve), no matter what the original population looks like!

Why is this cool? Even if the underlying data is weird and skewed, the "averages of the samples" will behave predictably. This allows us to use the math of the normal distribution to make inferences.

Properties of the CLT:

The mean of all sample means will equal the population mean: \( \mu_{\bar{x}} = \mu \).
The variance of the sample means is the population variance divided by the sample size: \( \sigma^2_{\bar{x}} = \frac{\sigma^2}{n} \).
The standard deviation of the sample means is called the Standard Error.

The Standard Error

The Standard Error of the sample mean measures how much the sample mean is likely to vary from the true population mean.
Formula: \( \sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}} \)
If we don't know the population standard deviation (\( \sigma \)), we use the sample standard deviation (\( s \)):
\( s_{\bar{x}} = \frac{s}{\sqrt{n}} \)

Did you know? As your sample size (\( n \)) gets bigger, the Standard Error gets smaller. This makes sense: the more people you ask, the more confident you are in your average!

3. Point Estimates and Confidence Intervals

When we estimate something, we can do it in two ways:

1. Point Estimate: A single number (e.g., "I think the average return is 8%"). While simple, it's rarely exactly right.
2. Confidence Interval: A range of values (e.g., "I am 95% confident the return is between 6% and 10%").

Building a Confidence Interval

The general formula for any confidence interval is:
Point Estimate ± (Reliability Factor × Standard Error)

The Reliability Factor depends on how "confident" you want to be and which distribution you are using (z or t).

Common Z-Factors (When population variance is known):

90% Confidence: \( z = 1.645 \)
95% Confidence: \( z = 1.960 \)
99% Confidence: \( z = 2.575 \)

Example: If the sample mean is 10, the standard error is 2, and we want a 95% confidence interval:
\( 10 \pm (1.96 \times 2) = 10 \pm 3.92 \)
Interval: [6.08 to 13.92]

Key Takeaway:

A wider confidence interval (e.g., 99% vs 95%) means you are more certain the true value is inside, but the range is less precise.

4. Choosing Between Z and T

This is a common "gotcha" on the CFA exam! Which table do you use: the Z-table or the t-table?

Use the t-distribution when:

The population variance is unknown (this is almost always the case in the real world).
The sample size is small (\( n < 30 \)) BUT the population is normally distributed.

Use the z-distribution when:

The population variance is known.
The population variance is unknown but the sample size is large (\( n \ge 30 \)). However, many practitioners still prefer the t-distribution here as it's more conservative.

The t-distribution looks like the normal distribution but has "fatter tails." This means it allows for more outlier events because we are less certain about our estimate. As the degrees of freedom (\( df = n - 1 \)) increase, the t-distribution starts looking exactly like the z-distribution.

5. Sampling Biases (The Traps!)

Even with great math, your results can be "garbage in, garbage out" if your sampling is biased. Here are the ones to watch for:

1. Survivorship Bias: Only looking at companies that are still in business today. (Ignoring the ones that failed makes the average look better than it really was!)
2. Look-ahead Bias: Using information in your study that wouldn't have been available at the time of the investment.
3. Data-mining Bias: "Torturing" the data until it gives you the result you want by testing many different variables until one happens to work by chance.
4. Time-period Bias: The results only work during a specific time (e.g., a study of tech stocks only during the 90s boom).
5. Selection Bias: When the sample is not truly random (e.g., a "random" street survey that only talks to people wearing suits).

Quick Review Box:
- Standard Error: \( s / \sqrt{n} \)
- CLT: Makes sample means normal if \( n \ge 30 \).
- t-dist: Use when \( \sigma \) is unknown; has fatter tails.
- Degrees of Freedom: \( n - 1 \).

Closing Thoughts

Estimation and Inference are the bridge between raw data and actual decision-making. Don't worry if the formulas feel heavy—focus on the logic: we take a sample, calculate the mean, adjust for our uncertainty (standard error and reliability factor), and create a range where we think the truth lies. You're doing great—keep practicing those Z-table lookups!

Welcome to Hypothesis Testing: The "Fact-Checker" of Finance

Hello future Charterholders! Welcome to one of the most practical chapters in the Quantitative Methods section. Think of Hypothesis Testing as the "scientific method" for the investment world.

Imagine a fund manager claims their strategy beats the market. Is that true, or were they just lucky this year? Hypothesis testing gives us a mathematical way to decide whether a result is statistically significant or just a lucky coincidence. Don't worry if the math looks intimidating at first—we are going to break it down into simple, logical steps.

1. The Basics: Null vs. Alternative Hypotheses

Every test starts with two opposing statements.

The Null Hypothesis (\(H_0\)): This is the "status quo" or the statement we assume is true until we have enough evidence to prove otherwise. It almost always includes an "equal to" sign (=, ≤, or ≥).
Analogy: In a courtroom, the Null Hypothesis is "The defendant is innocent."

The Alternative Hypothesis (\(H_a\)): This is what the researcher actually wants to prove. It is the "challenge" to the status quo. It never includes an "equal to" sign (<, >, or ≠).
Analogy: The prosecutor's goal is to prove the Alternative Hypothesis: "The defendant is guilty."

Important Rule: We never "accept" the null hypothesis. We either "Reject \(H_0\)" (we found enough evidence for \(H_a\)) or "Fail to Reject \(H_0\)" (we didn't find enough evidence).

Key Takeaway:

Always look for the equals sign. If a statement says "is equal to" or "is at least," it belongs in \(H_0\). If it says "is different from" or "is greater than," it belongs in \(H_a\).

2. One-Tailed vs. Two-Tailed Tests

How do we know which way to look? It depends on the question.

Two-Tailed Test: Use this when you want to know if a value is different (either higher or lower) from the hypothesized value.
Example: \(H_0: \mu = 10\%\) vs. \(H_a: \mu \neq 10\%\).

One-Tailed Test: Use this when you are only interested in one direction (either "greater than" or "less than").
Example: \(H_0: \mu \le 0\) vs. \(H_a: \mu > 0\). (We only care if returns are positive).

Quick Trick: Look at the sign in the Alternative Hypothesis (\(H_a\)). If it is \(\neq\), it’s two-tailed. If it’s \(>\) or \(<\), the "arrow" points to the tail you are testing!

3. Errors in Testing: Type I and Type II

In a perfect world, we’d always be right. In statistics, we can make two types of mistakes:

Type I Error: Rejecting a true null hypothesis. This is like convicting an innocent person. The probability of making a Type I error is called Alpha (\(\alpha\)), also known as the Level of Significance.

Type II Error: Failing to reject a false null hypothesis. This is like letting a guilty person go free. The probability of this error is called Beta (\(\beta\)).

The Power of a Test: This is the probability of correctly rejecting a false null hypothesis. It is calculated as \(1 - \beta\). We want high power!

Memory Aid:

Type I is "False Positive" (You think you found something, but you didn't).
Type II is "False Negative" (There was something there, but you missed it).

4. The Test Statistic: Our Measuring Stick

To make a decision, we calculate a Test Statistic. The general formula for almost every test statistic in this chapter is:

\( \text{Test Statistic} = \frac{\text{Sample Statistic} - \text{Hypothesized Value}}{\text{Standard Error}} \)

Where Standard Error is usually the sample standard deviation divided by the square root of the sample size: \( \frac{s}{\sqrt{n}} \).

Did you know? The test statistic simply tells you how many "standard errors" your sample result is away from the null hypothesis. The further away it is, the more likely we are to reject the null!

5. Choosing the Right Test (z-test vs. t-test)

Students often struggle with which test to pick. Here is a simple guide:

Use the t-test if: You don't know the population variance (which is almost always the case in the real world) and you are using the sample standard deviation (\(s\)).
Note: For t-tests, you need to know the Degrees of Freedom (df), which is usually \(n - 1\).

Use the z-test if: You actually know the population variance (\(\sigma^2\)) or if the sample size is very large (though \(t\) is still technically preferred for unknown variance).

6. The Decision: Rejection Regions and P-values

Once you have your Test Statistic, how do you decide?

Method 1: Critical Values. You compare your calculated test statistic to a "critical value" from a table (based on your significance level, like 5%). If your statistic is more extreme than the critical value, you Reject \(H_0\).

Method 2: The P-value. This is the "probability" method. The p-value is the smallest level of significance at which the null hypothesis can be rejected.

The Golden Rule of P-values:
If p-value < \(\alpha\), then Reject \(H_0\).
If p-value ≥ \(\alpha\), then Fail to Reject \(H_0\).

Mnemonic: "If the p is low, the null must go!"

7. Testing for Variances

Sometimes we don't care about the average (mean); we care about the risk (variance).

Testing ONE variance: We use the Chi-square (\(\chi^2\)) test.
Formula: \( \chi^2 = \frac{(n-1)s^2}{\sigma_0^2} \)

Comparing TWO variances: We use the F-test. This is common when checking if two different stocks have the same level of risk.
Formula: \( F = \frac{s_1^2}{s_2^2} \) (Always put the larger variance on top so \(F > 1\)).

8. Parametric vs. Non-parametric Tests

Parametric Tests: These tests (like the t-test and z-test) assume the data follows a specific distribution (like the normal distribution) and rely on parameters like the mean and variance.

Non-parametric Tests: These are "distribution-free" tests. We use them when:
1. The data doesn't fit a normal distribution.
2. The data is "ranked" (like 1st place, 2nd place) rather than specific numbers.
3. The sample size is very small.

Common mistake to avoid: Don't assume non-parametric tests are "better." They are actually less powerful than parametric tests if the data is normally distributed.

9. Summary Checklist for the Exam

Before moving to the next chapter, make sure you can:
• Correctly identify \(H_0\) and \(H_a\) from a word problem.
• Distinguish between Type I and Type II errors.
• Calculate a t-statistic for a single mean.
• Decide whether to reject \(H_0\) using either critical values or p-values.
• Identify when to use Chi-square (one variance) vs. F-test (two variances).

Don't worry if this seems tricky at first! Hypothesis testing is like learning a new language. Once you get used to the "grammar" of nulls and alternatives, the logic will start to feel like second nature. Keep practicing those calculations!

Welcome to Parametric and Non-Parametric Tests of Independence!

In your CFA journey so far, you’ve learned how to describe data and test hypotheses about a single mean or variance. But in the real world of finance, we rarely look at one thing in isolation. We want to know: Is the performance of a stock related to its industry? Does a CEO’s education level affect company returns?

In this chapter, we explore how to test if two variables are independent (meaning they have no relationship) or dependent (meaning they do). We will look at two main tools: the Chi-Square Test and Spearman’s Rank Correlation. Don’t worry if these sound like a mouthful—we’ll break them down piece by piece!

Quick Review: What does "Independence" mean?
If two events are independent, the occurrence of one tells you nothing about the probability of the other. For example, flipping a coin and the weather in London are independent. However, interest rates and bond prices are dependent.

1. Parametric vs. Non-Parametric Tests: The Big Picture

Before we dive into the specific tests, we need to understand the two "families" of statistical tests.

Parametric Tests

These tests make strict assumptions about the underlying data. Usually, they assume the data follows a specific distribution (like the Normal Distribution) and use parameters like the mean and variance. Think of these like a strict baking recipe—if you don’t have the exact ingredients, the cake won't rise.

Non-Parametric Tests

These are more flexible. They don’t require the data to be normally distributed. They are often used when we have small sample sizes, outliers, or data that is ranked (like "1st place, 2nd place") rather than precise measurements. Think of these like a "taco bar"—you can swap ingredients and it still works!

Key Takeaway: We use non-parametric tests when our data is "messy" or doesn't meet the strict requirements of parametric tests.

2. The Chi-Square (\(\chi^2\)) Test of Independence

The Chi-Square test is used when we have categorical data. This means our data is sorted into groups or "buckets" rather than being continuous numbers. For example, sorting companies by Sector (Tech, Energy, Finance) and Dividend Policy (Pays Dividends, No Dividends).

Contingency Tables

To perform this test, we use a contingency table (also called a two-way table). It shows the counts (frequencies) for two categorical variables at the same time.

The Hypothesis

Null Hypothesis (\(H_0\)): The two variables are independent (no relationship).
Alternative Hypothesis (\(H_a\)): The two variables are dependent (there is a relationship).

Calculating the Test Statistic

The Chi-Square statistic measures the difference between what we actually observed (\(O\)) and what we would expect (\(E\)) to see if the variables were perfectly independent.

The formula is:
\( \chi^2 = \sum \frac{(O - E)^2}{E} \)

Step-by-Step Process:
1. Calculate Expected Values (\(E\)): For each cell in your table, use this trick:
\( E = \frac{(\text{Row Total} \times \text{Column Total})}{\text{Grand Total}} \)
2. Find the Difference: For each cell, subtract the expected value from the observed value (\(O - E\)).
3. Square and Divide: Square that difference and divide by the expected value.
4. Sum them up: Add those values together for all cells to get your \(\chi^2\) statistic.

Degrees of Freedom (\(df\))

For a Chi-Square test of independence, the degrees of freedom are calculated as:
\( df = (r - 1) \times (c - 1) \)
Where \(r\) is the number of rows and \(c\) is the number of columns.

Did you know? The Chi-Square distribution is always positive and skewed to the right. As the degrees of freedom increase, it starts to look more like a normal distribution!

Key Takeaway: If the calculated \(\chi^2\) is much larger than the "critical value" from the table, we reject the null hypothesis and conclude that the variables are related.

3. Spearman Rank Correlation Coefficient (\(r_s\))

Sometimes we want to know the strength of a relationship, but our data is ordinal (ranked) or has extreme outliers that would ruin a standard correlation calculation. This is where Spearman’s Rank Correlation comes in.

Analogy: Imagine two judges ranking 10 contestants in a talent show. Even if the judges don't give the exact same scores, if they both agree that Contestant A is better than Contestant B, their ranks will be correlated.

Why use it?

It is a non-parametric alternative to the Pearson correlation.
It is used when data is non-linear but still follows a trend (monotonic).
It is robust to outliers because it looks at the order of data, not the actual values.

How to calculate it:

1. Rank the data for both variables from smallest to largest (or largest to smallest).
2. Calculate the difference (\(d_i\)) between the ranks for each pair.
3. Use the formula:
\( r_s = 1 - \frac{6 \sum d_i^2}{n(n^2 - 1)} \)

Interpreting the Result:
The value of \(r_s\) will always be between -1 and +1.
- +1: Perfect positive rank correlation (as one rank goes up, the other always goes up).
- -1: Perfect negative rank correlation.
- 0: No relationship between the ranks.

Common Mistake: Don't forget the "6" in the numerator! It's a constant part of the formula, regardless of your sample size.

Key Takeaway: Spearman's Rank Correlation is the go-to tool when your data isn't normally distributed or when you are dealing with rankings rather than raw measurements.

4. When to use Non-Parametric Tests? (Quick Guide)

On the CFA exam, you might be asked to identify when a non-parametric test is more appropriate than a parametric one. Keep these four scenarios in mind:

Data is Categorical: Use Chi-Square when you are counting things in buckets.
Data is Ordinal (Ranked): Use Spearman Rank Correlation.
Small Sample Size: When your sample is too small to assume a normal distribution.
Extreme Outliers: When a few crazy data points would pull a mean or correlation too far in one direction.

Don't worry if this seems tricky at first! Just remember: Parametric = Precise/Strict, Non-Parametric = Ranks/Flexible.

Summary & Quick Review Box

Chi-Square Test: Tests for independence between two categorical variables using a contingency table. Uses \(df = (r-1)(c-1)\).

Spearman Rank Correlation: Measures the strength of the relationship between ranked data. Ranges from -1 to +1.

Independence: The null hypothesis usually states there is no relationship. If our test statistic is high, we "fire" (reject) the null hypothesis!

Check your understanding:
If you are comparing "Analyst Rating" (Buy/Hold/Sell) and "Industry" (Tech/Retail), which test would you use?
Answer: Chi-Square Test of Independence (because these are categories!).

Quick check

Can you answer these now?

Open each question to check the key ideas from this chapter.

What is the difference between a population and a sample?

A population is the entire group under study, while a sample is a smaller subset of that population actually measured to make inferences.

What is the distinction between a parameter and a statistic?

A parameter is a descriptive measure of a population, while a statistic is a descriptive measure of a sample.

How is stratified random sampling conducted?

The population is divided into sub-groups (strata) based on specific characteristics, and samples are drawn proportionally from each stratum.

What does the Central Limit Theorem (CLT) state regarding the distribution of sample means?

For a large enough sample size (\(n \ge 30\)), the distribution of sample means will be approximately normal, regardless of the population's underlying distribution.

What is the Standard Error of the sample mean, and what is its formula?

The Standard Error measures the variation of the sample mean from the population mean, calculated as \(\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}}\).

What is the difference between a point estimate and a confidence interval?

A point estimate is a single numerical value, while a confidence interval is a range of values within which the population parameter is expected to lie with a specified probability.

What is the general formula for a confidence interval?

\(\text{Point Estimate} \pm (\text{Reliability Factor} \times \text{Standard Error})\)

When should the t-distribution be used instead of the z-distribution for construction of confidence intervals?

Use the t-distribution when the population variance is unknown, or when the sample size is small (\(n < 30\)) and the population is normally distributed.

What is survivorship bias in sampling?

Survivorship bias occurs when a sample only includes entities that have survived a period, excluding those that failed or dropped out.

What is data-mining bias?

Data-mining bias occurs from repeatedly searching a dataset for patterns until a statistically significant result is found by chance.

What is the primary difference between the null hypothesis (\(H_0\)) and the alternative hypothesis (\(H_a\)) regarding the equality sign?

The null hypothesis (\(H_0\)) always contains an "equal to" sign (=, ≤, or ≥), while the alternative hypothesis (\(H_a\)) never contains an equality sign (<, >, or ≠).

Why do we use the phrase "fail to reject the null hypothesis" instead of "accepting" it?

We use "fail to reject" because hypothesis testing only determines if there is sufficient evidence to support the alternative; a lack of evidence does not prove the null is true.

How do you identify whether a test is one-tailed or two-tailed based on the sign in the Alternative Hypothesis (\(H_a\))?

If \(H_a\) contains the "not equal to" symbol (\(\neq\)), the test is two-tailed; if it contains "greater than" (\(>\)) or "less than" (\( < \)), the test is one-tailed.

What is the difference between a Type I Error and a Type II Error?

A Type I error is rejecting a true null hypothesis (false positive), while a Type II error is failing to reject a false null hypothesis (false negative).

How is the "Power of a Test" defined and calculated?

The Power of a Test is the probability of correctly rejecting a false null hypothesis, calculated as \(1 - \beta\).

What is the general formula for calculating a Test Statistic?

\(\text{Test Statistic} = \frac{\text{Sample Statistic} - \text{Hypothesized Value}}{\text{Standard Error}}\).

When is a t-test preferred over a z-test for testing a population mean?

A t-test is used when the population variance is unknown (which requires using the sample standard deviation, \(s\)) or when the sample size is small.

What is the decision rule for rejecting the null hypothesis using the p-value?

Reject \(H_0\) if the p-value is less than the chosen significance level (\(\alpha\)).

Which test is used for testing a single variance, and which is used for comparing two variances?

A Chi-square (\(\chi^2\)) test is used for a single variance, while an F-test is used for comparing two population variances.

What are the primary reasons to choose a non-parametric test over a parametric test?

Non-parametric tests are used when the data does not follow a normal distribution, the data is ranked (ordinal), or the sample size is very small.

What does it mean for two variables to be independent in statistics?

Two variables are independent if the occurrence or value of one variable provides no information about the probability of the other variable's occurrence.

What are the defining characteristics of parametric tests?

Parametric tests rely on specific assumptions about the distribution of the underlying data, typically assuming a normal distribution and focusing on parameters like the mean and variance.

When are non-parametric tests used instead of parametric tests?

Non-parametric tests are used when data does not meet strict distributional assumptions, such as when dealing with small samples, outliers, or ordinal (ranked) data.

What is the purpose of a Chi-Square (\(\chi^2\)) test of independence?

The Chi-Square test determines if there is a significant relationship between two categorical variables by comparing observed frequencies to expected frequencies in a contingency table.

What are the null and alternative hypotheses for the Chi-Square test of independence?

The null hypothesis (\(H_0\)) is that the two variables are independent (no relationship), while the alternative hypothesis (\(H_a\)) is that the variables are dependent.

What is the formula for the Chi-Square (\(\chi^2\)) test statistic?

\( \chi^2 = \sum \frac{(O - E)^2}{E} \), where \(O\) is the observed frequency and \(E\) is the expected frequency.

How do you calculate the expected frequency (\(E\)) for a cell in a contingency table?

\( E = \frac{(\text{Row Total} \times \text{Column Total})}{\text{Grand Total}} \)

How are degrees of freedom (\(df\)) calculated for a Chi-Square test of independence?

\( df = (r - 1) \times (c - 1) \), where \(r\) is the number of rows and \(c\) is the number of columns in the contingency table.

What does Spearman's rank correlation coefficient (\(r_s\)) measure?

Spearman's rank correlation measures the strength and direction of the monotonic relationship between two ranked variables.

What is the formula for the Spearman rank correlation coefficient (\(r_s\))?

\( r_s = 1 - \frac{6 \sum d_i^2}{n(n^2 - 1)} \), where \(d_i\) is the difference between ranks and \(n\) is the number of observations.

How is a Spearman rank correlation of +1, -1, or 0 interpreted?

+1 indicates a perfect positive rank correlation, -1 indicates a perfect negative rank correlation, and 0 indicates no relationship between the ranks.

What are the four main scenarios that call for non-parametric tests?

Non-parametric tests are preferred when data is categorical, when data is ordinal (ranked), when sample sizes are small, or when extreme outliers are present.

Ready to test yourself?

Turn these notes into exam-style practice. Get unlimited AI questions on this topic with instant marking and explanations.

Practise This Topic

More CFA Level I chapters

* The content provided by thinka is generated by AI and may not always be accurate or up-to-date. Please use it as a supplementary resource and verify with official materials.

Put These Notes into Practice

Reading the notes is a great start. Now practise with unlimited AI-generated questions and get instant feedback. 100,000+ students are already improving their grades.

Start Practising Now View Pricing

Done reading? Test yourself with AI practice questions

Practise This Topic Now