Welcome to Estimation and Inference!
In our previous studies, we looked at descriptive statistics—ways to summarize data we already have. But in the real world of finance, we rarely have all the data. We can't know the future returns of every single stock in existence, right? That’s where Estimation and Inference come in. This chapter is all about taking a small "sip" of data (a sample) and making a smart guess about the whole "pot" (the population). Don’t worry if this seems a bit abstract at first; we’ll break it down piece by piece!
1. The Basics: Populations and Samples
Before we dive into the math, we need to understand who we are studying.
Population: The entire group you want to know about (e.g., every public company in the world).
Sample: A smaller subset of that group that we actually measure (e.g., 100 companies we picked to study).
Parameter: A descriptive measure of a population (like the "true" average return).
Statistic: A descriptive measure of a sample (the average return of our 100 companies).
Quick Tip: Remember P for Population/Parameter and S for Sample/Statistic!
Types of Sampling
How we pick our sample matters a lot. If we pick badly, our results will be wrong.
- Simple Random Sampling: Every member of the population has an equal chance of being selected. Think of drawing names out of a hat.
- Stratified Random Sampling: We divide the population into sub-groups (called strata) based on a characteristic (like industry) and then take a proportional number from each. Analogy: If you’re making a fruit salad, you want to make sure you get a mix of apples, grapes, and melons in every spoonful, not just all grapes!
- Cluster Sampling: We divide the population into "clusters" (like geographic regions) and then randomly select entire clusters to study.
Key Takeaway:
Sampling allows us to estimate population characteristics without having to look at every single data point, saving time and money.
2. The Magic of the Central Limit Theorem (CLT)
This is arguably the most important concept in all of statistics. The Central Limit Theorem tells us something amazing: If you take a large enough sample (usually n ≥ 30), the distribution of the sample means will look like a normal distribution (a bell curve), no matter what the original population looks like!
Why is this cool? Even if the underlying data is weird and skewed, the "averages of the samples" will behave predictably. This allows us to use the math of the normal distribution to make inferences.
Properties of the CLT:
- The mean of all sample means will equal the population mean: \( \mu_{\bar{x}} = \mu \).
- The variance of the sample means is the population variance divided by the sample size: \( \sigma^2_{\bar{x}} = \frac{\sigma^2}{n} \).
- The standard deviation of the sample means is called the Standard Error.
The Standard Error
The Standard Error of the sample mean measures how much the sample mean is likely to vary from the true population mean.
Formula: \( \sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}} \)
If we don't know the population standard deviation (\( \sigma \)), we use the sample standard deviation (\( s \)):
\( s_{\bar{x}} = \frac{s}{\sqrt{n}} \)
Did you know? As your sample size (\( n \)) gets bigger, the Standard Error gets smaller. This makes sense: the more people you ask, the more confident you are in your average!
3. Point Estimates and Confidence Intervals
When we estimate something, we can do it in two ways:
1. Point Estimate: A single number (e.g., "I think the average return is 8%"). While simple, it's rarely exactly right.
2. Confidence Interval: A range of values (e.g., "I am 95% confident the return is between 6% and 10%").
Building a Confidence Interval
The general formula for any confidence interval is:
Point Estimate ± (Reliability Factor × Standard Error)
The Reliability Factor depends on how "confident" you want to be and which distribution you are using (z or t).
Common Z-Factors (When population variance is known):
- 90% Confidence: \( z = 1.645 \)
- 95% Confidence: \( z = 1.960 \)
- 99% Confidence: \( z = 2.575 \)
Example: If the sample mean is 10, the standard error is 2, and we want a 95% confidence interval:
\( 10 \pm (1.96 \times 2) = 10 \pm 3.92 \)
Interval: [6.08 to 13.92]
Key Takeaway:
A wider confidence interval (e.g., 99% vs 95%) means you are more certain the true value is inside, but the range is less precise.
4. Choosing Between Z and T
This is a common "gotcha" on the CFA exam! Which table do you use: the Z-table or the t-table?
Use the t-distribution when:
- The population variance is unknown (this is almost always the case in the real world).
- The sample size is small (\( n < 30 \)) BUT the population is normally distributed.
Use the z-distribution when:
- The population variance is known.
- The population variance is unknown but the sample size is large (\( n \ge 30 \)). However, many practitioners still prefer the t-distribution here as it's more conservative.
The t-distribution looks like the normal distribution but has "fatter tails." This means it allows for more outlier events because we are less certain about our estimate. As the degrees of freedom (\( df = n - 1 \)) increase, the t-distribution starts looking exactly like the z-distribution.
5. Sampling Biases (The Traps!)
Even with great math, your results can be "garbage in, garbage out" if your sampling is biased. Here are the ones to watch for:
1. Survivorship Bias: Only looking at companies that are still in business today. (Ignoring the ones that failed makes the average look better than it really was!)
2. Look-ahead Bias: Using information in your study that wouldn't have been available at the time of the investment.
3. Data-mining Bias: "Torturing" the data until it gives you the result you want by testing many different variables until one happens to work by chance.
4. Time-period Bias: The results only work during a specific time (e.g., a study of tech stocks only during the 90s boom).
5. Selection Bias: When the sample is not truly random (e.g., a "random" street survey that only talks to people wearing suits).
Quick Review Box:
- Standard Error: \( s / \sqrt{n} \)
- CLT: Makes sample means normal if \( n \ge 30 \).
- t-dist: Use when \( \sigma \) is unknown; has fatter tails.
- Degrees of Freedom: \( n - 1 \).
Closing Thoughts
Estimation and Inference are the bridge between raw data and actual decision-making. Don't worry if the formulas feel heavy—focus on the logic: we take a sample, calculate the mean, adjust for our uncertainty (standard error and reliability factor), and create a range where we think the truth lies. You're doing great—keep practicing those Z-table lookups!