Introduction to Outliers

In Statistics, we often find that most data points like to "hang out" together in a cluster. But every now and then, you will come across a value that is far away from the rest of the group. We call these outliers.

Think of it like a group of friends where everyone is between 160cm and 180cm tall, but one person is 215cm tall. That person is an outlier! Understanding outliers is vital because they can "pull" your averages (like the mean) and give you a misleading picture of your data.

1. Identifying Outliers by Inspection (Foundation & Higher)

The simplest way to find an outlier is just by looking! This is called identification by inspection. You can do this by looking at a list of numbers or a statistical diagram.

What to look for:

  • In a list: A number that is significantly larger or smaller than all the other numbers.
  • On a Scatter Diagram: A point that sits far away from the "line of best fit" or the main cluster of dots.
  • On a Box Plot: A value that sits far to the left or right, often plotted as a small "x" or a dot beyond the whiskers.

Why do outliers happen?

When you spot an outlier, you need to comment on why it might be there. Usually, it falls into one of two categories:

  1. A Recording Error: This is a mistake. For example, if you are measuring the height of students in metres and you see \(175.0\) instead of \(1.75\), someone likely forgot the decimal point!
  2. A Genuine Extreme Value: This is a real, accurate measurement that just happens to be very unusual. For example, a professional athlete's salary in a survey of normal office workers.

Quick Tip: If the exam asks you to "comment on an outlier," always check if it looks like a typo first. If it looks like a realistic (but rare) value, call it a "genuine extreme."

2. Identifying Outliers by Calculation (Higher Tier Only)

Sometimes "looking at it" isn't enough. To be more scientific, we use specific formulas to decide if a value is officially an outlier. There are two main methods you need to know.

Method A: The Interquartile Range (IQR) Rule

This is the most common method. We calculate "boundaries." Anything outside these boundaries is an outlier.

  • Small Outlier: Any value \( < \text{Lower Quartile} - 1.5 \times \text{IQR} \)
  • Large Outlier: Any value \( > \text{Upper Quartile} + 1.5 \times \text{IQR} \)

Step-by-Step Example:
Imagine a dataset where the Lower Quartile (\(LQ\)) is \(20\), the Upper Quartile (\(UQ\)) is \(30\), and we want to check if the value \(50\) is an outlier.

  1. Find the \(IQR\): \(30 - 20 = 10\)
  2. Multiply the \(IQR\) by \(1.5\): \(10 \times 1.5 = 15\)
  3. Find the Upper Boundary: \(UQ + 15 = 30 + 15 = 45\)
  4. Conclusion: Since \(50\) is greater than \(45\), it is officially a large outlier.

Method B: The Standard Deviation Rule

If you know the mean and the standard deviation (\(\sigma\)), you can use this rule. In a typical distribution, almost all data sits within 3 standard deviations of the mean.

  • Outlier: Any value that lies outside the range: \(\text{mean} \pm 3 \times \text{standard deviation}\)

So, a value is an outlier if it is smaller than \(\text{mean} - 3\sigma\) or larger than \(\text{mean} + 3\sigma\).

Note: For more on how to calculate standard deviation, see the specific chapter in the Measures of Dispersion section.

3. "Cleaning" the Data

Once you have identified an outlier, you have to decide what to do with it. This is part of cleaning the data.

  • If it's an error: You should usually remove it or correct it if you can find the original data.
  • If it's genuine: You might keep it, but you should be aware that it will affect your mean and range significantly.

Did you know? The Median and the Interquartile Range are much better to use when you have outliers, because they aren't affected by extreme values as much as the Mean and the Range are!

Summary & Key Takeaways

For All Students:

  • An outlier is a value that doesn't fit the pattern of the rest of the data.
  • Identify them by inspection (looking) for gaps in lists or dots far away on diagrams.
  • Outliers can be errors (typos) or genuine extremes (real but rare).

For Higher Tier Students:

  • Use the \(1.5 \times \text{IQR}\) rule: \(< LQ - 1.5 \times \text{IQR}\) or \(> UQ + 1.5 \times \text{IQR}\).
  • Use the Standard Deviation rule: any value outside \(\text{mean} \pm 3\sigma\).
  • Common Mistake: Forgetting to multiply the \(IQR\) by \(1.5\) before adding/subtracting it. Always do the multiplication first!

Quick Review: If \(LQ = 10\) and \(UQ = 20\), is \(36\) an outlier? (Calculation: \(IQR = 10\). \(1.5 \times 10 = 15\). Boundary \( = 20 + 15 = 35\). Yes, \(36\) is just barely an outlier!)