Tests for a difference between two means

Introduction: Comparing Two Worlds

In our previous chapters, we looked at how to test a claim about a single group. But what if we want to compare two different groups? For example, do students who study with music perform differently than those who study in silence? Or is a new medication more effective than the current one? In this chapter, we learn how to use the Two-Sample t-test to determine if the observed difference between two sample means is statistically significant or if it likely happened by chance.

Step 1: Setting Up the Hypotheses

When we test the difference between two population means, \( \mu_1 \) and \( \mu_2 \), we are looking to see if the "gap" between them is real. We always start with the Null Hypothesis \( (H_0) \), which assumes there is "no difference."

The Null Hypothesis \( (H_0) \):
\( H_0: \mu_1 - \mu_2 = 0 \) (or \( \mu_1 = \mu_2 \))
Translation: There is no difference between the two population means.

The Alternative Hypothesis \( (H_a) \):
This is what we are trying to find evidence for. It can take three forms:
1. Two-sided: \( H_a: \mu_1 - \mu_2 \neq 0 \) (The means are just "different")
2. One-sided (Greater than): \( H_a: \mu_1 - \mu_2 > 0 \) (The first mean is larger)
3. One-sided (Less than): \( H_a: \mu_1 - \mu_2 < 0 \) (The first mean is smaller)

Quick Tip: Always define your parameters! For example: "Where \( \mu_1 \) is the true mean heart rate of runners and \( \mu_2 \) is the true mean heart rate of non-runners."

Step 2: Verifying Conditions

Before we calculate anything, we must justify that our inference method is valid. For a two-sample t-test, check these three conditions for both groups:

1. Randomization:
The data must come from independent random samples or a randomized experiment. If the problem says "randomly assigned," you've met this condition for an experiment!

2. Independence (10% Condition):
If we are sampling without replacement from a finite population, our sample sizes must be less than 10% of the population: \( n_1 < 0.10N_1 \) and \( n_2 < 0.10N_2 \).
Note: We do not check this for randomized experiments, only for observational studies/surveys.

3. Normal/Large Sample:
We need the sampling distribution of the difference in means to be approximately Normal. This is met if:
- The population distributions are stated to be Normal.
- OR the sample sizes are large enough (\( n_1 \ge 30 \) and \( n_2 \ge 30 \)) according to the Central Limit Theorem.
- OR if the samples are small, you must look at a graph of the sample data. If there is no extreme skewness or outliers, it is safe to proceed.

Step 3: Calculating the Test Statistic

We use a t-statistic because we almost never know the true population standard deviations \( (\sigma) \) and must use the sample standard deviations \( (s) \) instead. This introduces extra variability, which the t-distribution accounts for.

The formula for the standardized test statistic is:
\( t = \frac{(\text{statistic}) - (\text{parameter})}{\text{standard error}} \)

In our case:
\( t = \frac{(\bar{x}_1 - \bar{x}_2) - 0}{\text{SE}_{\bar{x}_1 - \bar{x}_2}} \)

Where the Standard Error is:
\( \text{SE}_{\bar{x}_1 - \bar{x}_2} = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}} \)

Degrees of Freedom (\( df \))

Calculating degrees of freedom for two samples by hand is very complex. On the AP Exam, the CED specifies that you should use technology (your graphing calculator) to find the exact degrees of freedom. If you are asked to show work, simply report the \( df \) provided by your calculator's "2-SampTTest" function.

Step 4: The P-value and the Decision

The P-value is the probability of getting a difference in sample means as extreme as (or more extreme than) the one observed, assuming the null hypothesis is true.

Making a Decision:
- If P-value \( \le \alpha \) (significance level, usually 0.05): Reject \( H_0 \). We have "convincing evidence" for \( H_a \).
- If P-value \( > \alpha \): Fail to reject \( H_0 \). We "do not have convincing evidence" for \( H_a \).

Did you know? A small P-value doesn't mean the difference is large; it just means the difference is unlikely to have happened by random chance alone.

Interpreting the Results in Context

A complete conclusion should always have three parts:
1. Compare the P-value to alpha.
2. State the decision (Reject or Fail to Reject \( H_0 \)).
3. State the conclusion in the context of the problem (use the language of the alternative hypothesis).

Example phrasing: "Because the P-value of 0.02 is less than \( \alpha = 0.05 \), we reject the null hypothesis. There is convincing evidence that the true mean score of students using the new software is higher than the true mean score of students using the old software."

Common Mistakes to Avoid

1. "Accepting" the Null: Never say we "accept" \( H_0 \) or that \( H_0 \) is "true." We only "fail to reject" it. Statistics is about evidence, not absolute proof!
2. Mixing up Means and Proportions: If the data is quantitative (numbers like height, time, or weight), use means (\( \bar{x} \) and \( \mu \)). If the data is categorical (yes/no, success/failure), use proportions (\( \hat{p} \) and \( p \)).
3. Forgetting Units: Always include context and units (e.g., "seconds," "grams," "points") in your interpretations.
4. Pooling: In AP Statistics, we generally do not pool variances for two-sample t-tests. Ensure your calculator's "Pooled" option is set to "No."

Quick Review Table

Method: Two-Sample t-test for \( \mu_1 - \mu_2 \)
Parameter: \( \mu_1 - \mu_2 \)
Statistic: \( \bar{x}_1 - \bar{x}_2 \)
Test Statistic: \( t = \frac{(\bar{x}_1 - \bar{x}_2) - 0}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}} \)
Degrees of Freedom: Provided by technology (calculator)

Key Takeaway

The Two-Sample t-test is our primary tool for comparing the averages of two independent groups. By checking our conditions and calculating the P-value, we can determine if the differences we see in our samples represent a real difference in the populations or just the "luck of the draw" from random sampling.