The Golden Rules of Good Data: Bias, Reliability, and Validity

Welcome to one of the most important parts of your Statistics course! Imagine you are building a house: if your bricks are cracked or your measurements are wrong, the whole house will eventually fall down. In Statistics, data are your bricks. If your data is "bad," your conclusions will be "bad" too.

In this chapter, we will learn how to make sure our data is high-quality, how to spot "unfair" data, and how to "clean" data before we use it to solve problems. This chapter focuses on the Collection of Data stage of the Statistical Enquiry Cycle.

1. Reliability and Validity

These two words sound similar, but they mean very different things in Statistics. Think of them as the "consistency" and "truth" of your data.

Reliability: The Consistency Test

Reliability is about whether your data-collection method produces consistent results. If you repeated the same test under the same conditions, would you get the same answer?

  • Example: If you weigh yourself on a scale and it says \(70kg\), then you step off and back on and it says \(75kg\), that scale is unreliable.
  • In Statistics: A survey is reliable if different researchers asking the same questions to the same people get the same answers.

Validity: The Truth Test

Validity is about whether you are actually measuring what you intended to measure. Is the data "fit for purpose"?

  • Example: Imagine you want to find out how good someone is at Math, but you give them a test written in a language they don't speak. Their low score doesn't mean they are bad at Math; it means the test wasn't valid because it was accidentally testing their language skills instead!

Quick Review: Imagine a target.
- If all your arrows hit the same spot but it's far from the bullseye, you are reliable but not valid.
- If your arrows hit the bullseye, you are both.

2. Bias: The Enemy of Fair Data

Bias happens when something in your investigation design causes the results to be "skewed" or unfair in one direction. Bias makes your findings inaccurate.

Common Causes of Bias:

  • Leading Questions: Asking a question in a way that suggests a particular answer.
    Bad: "Don't you agree that school lunches are delicious?"
    Better: "How would you rate the quality of school lunches?"
  • Biased Sources: Using a secondary source that has a reason to be one-sided (e.g., a company’s own website praising its products).
  • Sensitivity of Subject Matter: People often lie or "gloss over" the truth if a question is personal or embarrassing (e.g., "Have you ever cheated on a test?"). This leads to data distortion.
  • Time Factors: Collecting data at a specific time that doesn't represent the whole picture (e.g., surveying people about their favorite sport only during the Olympics).

How to Minimise Bias:

To keep things fair, you should use neutral language, ensure anonymity for sensitive topics, and use random sampling (covered in the Sampling chapter) to ensure everyone has an equal chance of being picked.

Higher Tier Only: The Random Response Technique
If you must ask a very sensitive "Yes/No" question (like "Have you ever stolen?"), you can use the Random Response technique. You ask the person to flip a coin in secret.
- If it's Heads, they must say "Yes."
- If it's Tails, they tell the truth.
Because the researcher doesn't know if a "Yes" came from the coin or the truth, the person feels safe being honest. You can then use probability to estimate the true percentage of "Yes" answers in the population.

3. Extraneous Variables

An extraneous variable is an "unwanted guest" in your investigation. It is a variable that you aren't interested in, but it might still affect your results.

Example: You are investigating if "Time spent revision" affects "Exam score."
- Independent Variable: Revision time.
- Dependent Variable: Exam score.
- Extraneous Variables: The temperature of the exam room, how much sleep the student had, or their natural ability.
If you don't control these (keep them the same), you won't know if the revision time caused the score or if it was just the extra sleep!

4. Cleaning Data

Raw data is often "messy." Before you start drawing charts or calculating means, you must clean it. This means looking for errors or problems that could ruin your calculations.

Common Problems to Look For:

  • Missing Responses: Someone left a question blank on a survey. Do you leave it as zero (risky!) or remove that person from the sample?
  • Errors in Recording: A person's age is listed as \(250\) years old. This is clearly a typo and should be corrected or removed.
  • Outliers: Values that are much, much higher or lower than the rest. These might be genuine "extreme values" or just mistakes. (Note: You'll learn the math for spotting outliers in the "Measures of Dispersion" chapter).
  • Formatting Issues: In a spreadsheet, some people might write dates as "1st May" while others write "01/05/2023." Cleaning makes them all the same so the computer can understand them.
  • Duplicate Data: The same person accidentally entering their data twice.

Key Takeaway: Cleaning data ensures that your final results are based on valid and accurate information.

5. Summary Checklist

Before moving on, make sure you can answer these questions:

  • Can I explain the difference between reliability (consistency) and validity (accuracy)?
  • Do I know how leading questions or sensitive topics can cause bias?
  • (Higher Tier) Do I understand that random response helps get honest answers to sensitive questions?
  • Can I identify extraneous variables that might "mess up" an experiment?
  • Do I know what it means to clean data (fixing typos, missing values, and spreadsheet errors)?

Don't worry if this seems like a lot of "common sense"—Statistics is all about applying that common sense in a logical, step-by-step way!