Welcome to "Assess the Representativeness of a Sample"

In your Thinking Skills course, you will often encounter arguments based on evidence. Frequently, that evidence comes from a sample—a small group chosen to represent a much larger population. But can we trust a conclusion about everyone based on only a few people? This chapter teaches you how to decide if a sample is a "fair" reflection of the whole group or if it’s leading us to a false conclusion.

Note: This chapter is part of the "Evaluating Evidence" section. To see if the person providing the evidence is trustworthy, check out the chapter on Credibility of Evidence.

What is "Representativeness"?

Imagine you are cooking a massive pot of vegetable soup. To check if it needs more salt, you take one spoonful and taste it. That spoonful is your sample. The whole pot is the target population.

If the spoonful (the sample) tastes exactly like the rest of the pot, it is representative. If you only scooped up a piece of potato and no broth, your sample is unrepresentative—it won't tell you the truth about the saltiness of the soup!

According to the syllabus, there are three key factors you must assess to see if a sample is representative: Number, Characteristic, and Selectivity.

1. Number (Sample Size)

The first thing to look at is the size of the sample. Is it "big enough"?

The Concept: Generally, a larger sample is more likely to be representative than a very small one. If a sample is too small, a single unusual "outlier" can skew the results.

Example: If you ask \( 2 \) students if they like the school cafeteria, and both say "No," can you claim that "all students hate the cafeteria"? Of course not! That is a rash generalization. However, if you ask \( 200 \) students and \( 180 \) say "No," your conclusion is much stronger.

Don't worry if this seems tricky... You don't need to know the exact mathematical formula for the perfect sample size. In Thinking Skills exams, just look for samples that seem suspiciously small (like asking only \( 3 \) or \( 4 \) people) to represent thousands.

Key Takeaway: A small number increases the risk that the evidence is just a coincidence rather than a fact about the whole group.

2. Characteristic (Diversity)

Even a large sample can be "bad" if it doesn't include the right types of people or things.

The Concept: A representative sample should have the same characteristics as the target population. This includes things like age, gender, location, occupation, or interests.

Example: Imagine a researcher wants to know the average height of people in a city. They measure \( 500 \) people (a good number), but they only measure members of the local professional basketball team.
Even though the number is large, the characteristic is wrong. Basketball players are much taller than the average person. The sample does not reflect the "diversity" of the city.

Quick Review: Ask yourself, "Does this small group look like a 'mini-version' of the big group?" If the big group is \( 50\% \) women, but the sample is \( 90\% \) men, it is unrepresentative.

3. Selectivity (How the sample was chosen)

This is often where the most "sneaky" errors happen. Selectivity refers to the method used to pick the participants.

The Concept: If a sample is "selected" in a way that favors a certain outcome, it is biased. We want samples to be chosen as randomly as possible to avoid this.

Common Problems with Selectivity:

  • Self-Selection: This happens when people choose to be in the sample (like an online poll or a radio call-in). People with very strong, angry opinions are more likely to volunteer than people who are satisfied.
  • Location Bias: If you survey people about "car usage" but only talk to people standing at a bus stop, your sample is unfairly selected.
  • Vested Interest: If a company only selects "happy customers" to provide testimonials for their website, they are being selective to make themselves look good.

Key Takeaway: If the process of picking the sample excludes certain groups or favors others, the evidence is weak.

How to Evaluate a Sample in an Exam (Step-by-Step)

When you are reading a source in Paper 2 or Paper 4 and you see a survey or a study, follow these steps:

Step 1: Identify the Target Population. Who is the argument trying to reach a conclusion about? (e.g., "All teenagers in the UK").

Step 2: Check the Number. Is the sample size mentioned? Is it large enough to represent that population? If it says "a few" or a very low number like \( 10 \), point this out as a weakness.

Step 3: Check the Characteristics. Does the sample include a mix of people that reflects the target population? Look for "missing" groups (e.g., "They only asked boys, but the conclusion is about all children").

Step 4: Look for Selectivity/Bias. How did the researchers find these people? Was it a "street survey" at \( 10:00 \) AM? (If so, they missed everyone who was at work!). Was it an online poll? (If so, they missed people without internet!).

Step 5: State the Impact. Don't just say the sample is small. Say: "The sample is unrepresentative because it only includes \( 5 \) people; therefore, the conclusion cannot be reliably generalized to the whole population."

Common Mistakes to Avoid

Mistake 1: Thinking a large sample is ALWAYS representative.
Correction: You could ask \( 1,000,000 \) people a question, but if they are all from the same political party, the sample is still unrepresentative of the whole country!

Mistake 2: Confusing "Representativeness" with "Credibility."
Correction: Credibility is about whether the source is lying or mistaken (see the RAVEN criteria). Representativeness is about whether the data allows for a broad conclusion. A very honest person can still use a bad sample.

Quick Review: The "Big Three"

To assess a sample, look for:

1. Number: Is the quantity sufficient? \( (n=?) \)
2. Characteristic: Does the group’s "flavor" match the whole pot?
3. Selectivity: Was the picking process fair and random, or biased?

Top Tip: In your exam, use the specific word "generalizability." If a sample is representative, you can generalize the findings to the whole group. If it isn't, the argument "fails to justify its generalization."