Introduction to Experimental Design

In the previous chapters, we looked at how to describe data and how to pick a random sample. But what if we want to know if one thing actually causes another? If we just watch people, we might see a pattern, but we can't prove why it's happening. That’s where Experimental Design comes in. This is the "gold standard" of statistics because it is the only way we can legally claim that one variable causes a change in another.

Don't worry if the terminology feels like a lot at first! We are going to break down the specific ingredients you need to build a perfect experiment.

1. Observational Study vs. Experiment

Before we design anything, we have to know what we are doing. There are two main ways to collect data about groups:

  • Observational Study: Researchers observe individuals and measure variables of interest but do not attempt to influence the responses. We are just "fly on the wall."
    Example: Following a group of students for a year to see if those who happen to eat breakfast have higher grades.
  • Experiment: Researchers deliberately impose a treatment on individuals to measure their responses. If you aren't changing something on purpose, it's not an experiment!
    Example: Randomly assigning 50 students to eat a specific breakfast and 50 students to fast, then comparing their test scores.

Key Takeaway: If the researchers assigned the "condition" (the treatment), it's an experiment. If the subjects chose it themselves or it just happened naturally, it's an observational study.

2. The Four Pillars of Experimental Design

To have a valid experiment that AP graders will love, you need to include these four principles:

I. Comparison

You must use a design that compares two or more treatments. If you only give a "new" medicine to one person and they get better, you don't know if it was the medicine or just luck. You need a control group (a group that receives no treatment or a placebo) to provide a baseline for comparison.

II. Random Assignment

This is the most important part! Random assignment means that experimental units (the people or objects) are assigned to treatments using a chance process.
Why do we do it? It helps "balance out" other variables that we might not be thinking about (like age, health, or sleep). It ensures that the treatment groups are as similar as possible before the treatments are applied.

III. Control

We need to keep other variables the same for all groups. If one group takes a pill with water and the other takes it with soda, we won't know if the results came from the pill or the drink. By keeping things the same, we prevent confounding variables (outside factors that confuse our results).

IV. Replication

In statistics, "replication" doesn't mean doing the whole experiment over again. It means using enough experimental units in each group so that any differences in the results can be distinguished from chance fluctuations.
Example: Testing a drug on \( n = 100 \) people is much more reliable than testing it on \( n = 2 \).

3. Advanced Design Techniques

Sometimes a simple "split the group in half" design isn't enough. Here are some more specific tools:

Blocking

A block is a group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments.
The Strategy: If you think men and women will react differently to a medication, you first separate them into "blocks" (Men and Women). Then, randomly assign treatments within each block. This reduces the variation in our results.

Blinding

To avoid psychological bias, we use "blinding":

  • Single-blind: The subjects don't know which treatment they are receiving, OR the people measuring the results don't know which treatment was given.
  • Double-blind: Neither the subjects nor the researchers interacting with them know who is getting which treatment. This is the best way to prevent the "placebo effect."

The Placebo Effect

A placebo is a "fake" treatment (like a sugar pill) that has no active ingredient. The placebo effect occurs when subjects provide a response simply because they think they are being treated. Using a placebo as a control allows us to see if the actual medicine works better than just the "power of suggestion."

4. Scope of Inference: What can we actually say?

This is a very common topic on the AP Exam! The way you collect your data limits what you can claim:

  • Can we claim cause-and-effect? ONLY if we used random assignment to treatments.
  • Can we generalize to the whole population? ONLY if we used random sampling to pick our participants (see chapter 1.11).

Common Mistake to Avoid: Many students say, "The experiment proves the medicine works." In AP Statistics, we use non-definitive language. Instead, say, "The evidence suggests a causal relationship between the medicine and the recovery rate."

Quick Review: The "Recipe" for a Free-Response Answer

When asked to describe an experimental design, always include these steps:
1. Label: Give every subject a number from \( 1 \) to \( n \).
2. Randomize: Use a random number generator to pick numbers for Group A and Group B.
3. Treatment: State clearly what treatment each group gets.
4. Measure/Compare: State what variable you will measure at the end and that you will compare the results between the groups.

Memory Aid: Remember CRAC — Control, Randomization, Assignment, Comparison!

Did you know? The concept of "blocking" is very similar to "stratified sampling" from the previous chapter. The difference is that we stratify to get a sample, but we block to conduct an experiment.