Topic 4: Statistics and Probability — Sampling, Data Types and Bias
Welcome to the foundation of statistics! Before we can calculate averages, draw beautiful regression lines, or perform complex hypothesis tests, we need data. But where does data come from, and can we trust it? In this chapter, we explore how data is collected, how to classify different forms of data, how to choose an appropriate sample, and how to spot potential bias and errors that could ruin an investigation.
Cross-reference note: Once your data is collected and verified, you will summarise it using measures of location and spread in Central tendency and dispersion, and display it using tools covered in Presenting data: tables, histograms and box plots.
---1. Populations vs. Samples
Whenever a statistical inquiry takes place, we must define the group of individuals, objects, or measurements we are interested in studying.
Population: The entire group of individuals or items from which information is desired. For example, all students enrolled in an international school, or every lightbulb produced by a factory in one week.
Census: An investigation that collects data from every single member of the population.
Why not always do a census? A census provides complete certainty, but it is often expensive, time-consuming, practically impossible if the population is infinite, or destructive (for instance, if testing the lifetime of batteries until they run out!).
Sample: A subset or part of the population chosen to represent the entire population. We collect data from the sample and use it to draw conclusions about the whole population.
Random Sample: A sample chosen in such a way that every member of the population has a known and equal chance of being selected. This is essential for minimising selection bias.
Analogy to remember: Think of tasting soup. You don't need to drink the entire pot (a census) to know if it needs salt. You stir it well and taste a single spoonful (a representative sample)!
Key Takeaway: A sample saves time and money, but its usefulness depends entirely on whether it accurately represents the target population.
---2. Classification of Data Types
Data can be broadly divided into qualitative (categorical) and quantitative (numerical) data. Quantitative data is further divided into discrete and continuous data. Understanding this distinction is vital because the type of data dictates which charts, calculations, and tests are mathematically valid.
A. Qualitative (Categorical) Data
Data consisting of non-numerical qualities, labels, or categories.
Examples: Eye colour (blue, brown, green), favourite subject (Maths, History, Visual Arts), blood group (\(\text{A, B, AB, O}\)).
B. Quantitative (Numerical) Data
Data consisting of numbers that represent counts or measurements. This is split into two types:
1. Discrete Data: Data that can only take specific, distinct values (often integers). It usually arises from counting items. There are gaps between possible values where no intermediate values can exist.
Examples:
• The number of siblings a student has: \(x \in \{0, 1, 2, 3, \dots\}\) (you cannot have \(2.37\) siblings).
• The number of cars passing a toll booth in one hour.
• Shoe sizes (e.g., UK sizes \(6, 6.5, 7, 7.5\) — even though half-sizes exist, they are discrete because there is no size \(6.214\)).
2. Continuous Data: Data that can take any value within a given numerical interval or range. It arises from measuring a physical quantity. Continuous values are limited only by the accuracy of the measuring instrument.
Examples:
• The height of an athlete: \(h = 178.45\text{ cm}\).
• The time taken to run \(100\text{ metres}\): \(t = 11.82\text{ seconds}\).
• The mass of an apple: \(m = 142.6\text{ grams}\).
Quick Memory Aid: If you count it, it is usually discrete. If you measure it with a ruler, scale, or stopwatch, it is continuous.
Key Takeaway: Make sure you can instantly label a variable as qualitative, quantitative discrete, or quantitative continuous in an exam question.
---3. Sampling Techniques and Their Effectiveness
Selecting a sample properly ensures our findings are generalisable to the wider population. The IB syllabus requires familiarity with the main sampling methods, including their advantages and limitations.
A. Simple Random Sampling
Method: Every member of the population is assigned a unique number. A random number generator or a lottery system is used to pick the required sample size.
• Advantages: Completely unbiased; every individual has an equal chance of selection; simple to understand.
• Limitations: Requires an up-to-date, complete list of the entire population (a sampling frame); can be impractical or expensive if the population is geographically dispersed.
B. Systematic Sampling
Method: Members of the population are listed in order. A starting point is chosen at random from the first \(k\) items, and then every \(k\)-th item is chosen, where \(k = \frac{N}{n}\) (\(N\) is population size, \(n\) is desired sample size).
Example: To select \(50\) students from a school of \(1000\), calculate \(k = \frac{1000}{50} = 20\). Pick a random number between \(1\) and \(20\) (say \(7\)), then choose the \(7^{\text{th}}, 27^{\text{th}}, 47^{\text{th}}, 67^{\text{th}}, \dots\) student on the roll.
• Advantages: Quick, simple, and evenly spreads the sample across the sampling frame.
• Limitations: If the list has a hidden periodic pattern or cycle that coincides with \(k\), the sample can become heavily biased.
C. Stratified Sampling
Method: The population is divided into distinct, non-overlapping sub-groups called strata based on shared characteristics (such as age, grade level, gender). A simple random sample is then taken from each stratum in exact proportion to the size of that stratum in the population.
The number of items to select from stratum \(i\) is calculated by:
\(\text{Sample from stratum } i = \frac{\text{Size of stratum } i}{\text{Total population size } N} \times \text{Total desired sample size } n\)
Worked Example: A school has \(200\) Year 10 students, \(180\) Year 11 students, and \(120\) Year 12 students (Total \(N = 500\)). You want a stratified sample of \(n = 50\) students.
• Year 10: \(\frac{200}{500} \times 50 = 20\) students
• Year 11: \(\frac{180}{500} \times 50 = 18\) students
• Year 12: \(\frac{120}{500} \times 50 = 12\) students
Check: \(20 + 18 + 12 = 50\). Within each year group, the students are chosen using a random number generator.
• Advantages: Guarantees proportional representation of key groups; highly representative; reduces sampling error between strata.
• Limitations: Requires prior knowledge of the strata and population numbers; can be time-consuming to organise.
D. Quota Sampling
Method: Similar to stratified sampling, the population is divided into groups based on characteristics. However, instead of selecting people randomly within each group, the researcher chooses anyone available until the target quota for each category is filled.
Example: An interviewer in a shopping mall is told to survey \(30\) women aged under 25, \(30\) women aged 25–50, and \(30\) women aged over 50. The interviewer approaches people until each quota is reached.
• Advantages: Quick, cost-effective, no complete list of the population is needed.
• Limitations: Non-random; selection is left to the interviewer's discretion, introducing researcher bias; cannot estimate sampling error.
E. Convenience (Opportunity) Sampling
Method: Choosing individuals who are easiest to reach or readily available at the time of data collection.
Example: Asking your five closest friends in the school cafeteria what their favourite sports team is.
• Advantages: Very easy, fast, and cheap.
• Limitations: Highly prone to bias; unlikely to represent the broader population; results cannot be reliably generalised.
Key Takeaway: Random methods (Simple Random, Systematic, Stratified) eliminate researcher selection bias, whereas non-random methods (Quota, Convenience) are easier and cheaper but carry a high risk of bias.
---4. Reliability of Sources and Bias in Sampling
A sample is only as good as the process used to gather it. When a sampling method consistently over-represents or under-represents certain portions of a population, it suffers from bias.
A. Common Sources of Bias
1. Selection (Sampling) Bias: Occurs when the sampling frame excludes certain segments of the population. For instance, conducting an online survey about internet access naturally excludes individuals who do not have internet access.
2. Voluntary Response (Self-Selection) Bias: Occurs when participants choose whether to take part (e.g., an open poll on social media or an optional review form). People with strong, often negative, opinions are much more likely to respond than those with neutral feelings.
3. Non-Response Bias: Occurs when a significant number of selected participants do not return questionnaires or refuse to answer. If those who choose not to respond differ systematically from those who do, the sample becomes unrepresentative.
4. Response / Question Wording Bias: Occurs when the wording of a question encourages a particular answer, or when respondents feel pressured to answer untruthfully due to social stigma.
Biased question: "Given the terrible environmental damage caused by plastic bottles, do you agree we should ban them?"
Neutral question: "To what extent do you support or oppose a ban on single-use plastic bottles?"
B. Reliability of Data Sources
When working with secondary data (data collected by someone else), a critical statistician must evaluate:
• Authority and Credibility: Was the data gathered by a respected, neutral organisation (such as a national census bureau or an academic research institute), or by an interest group with an agenda?
• Recency: Is the data current, or has the population changed significantly since the data was gathered?
• Methodology: Did the original study disclose their sample size, sampling method, and response rate?
Key Takeaway: Bias cannot be fixed by simply increasing the sample size. Surveying \(100\,000\) biased respondents still produces a biased result!
---5. Outliers
What is an outlier? An outlier is an observed data point that lies an abnormal distance away from other values in the dataset.
A. Why do outliers occur?
• Measurement or Recording Error: Human error when entering data (e.g., typing a student's height as \(1850\text{ cm}\) instead of \(185\text{ cm}\)) or an equipment malfunction.
• Genuine Extreme Variation: Natural diversity within a population (e.g., a professional basketball player in a sample of high school students' heights).
B. How should an outlier be handled?
• If an outlier is proven to be a recording error or does not belong to the target population, it can be corrected or discarded.
• If an outlier represents a genuine, valid measurement, it must not be removed simply because it makes the data look messy. Instead, its presence should be noted, and non-parametric or robust statistics (like the median and interquartile range) should be considered.
Note: The formal mathematical rules for identifying outliers using quartiles (\(Q_1 - 1.5 \times \text{IQR}\) and \(Q_3 + 1.5 \times \text{IQR}\)) are covered in detail in Presenting data: tables, histograms and box plots.
Key Takeaway: Never delete an unusual data value without a justified reason. Investigate its origin first!
---6. Designing Valid Data Collections (AHL Focus)
For Higher Level candidates (and essential for writing your Internal Assessment exploration), you must be able to evaluate the design of surveys, questionnaires, and experiments.
A. Principles of Questionnaire Design
• Relevant Variables: Ensure every question collects data that directly answers the research question. Avoid gathering unnecessary personal information.
• Clear and Unambiguous Language: Avoid double-barrelled questions (e.g., "Do you enjoy maths and science?"). Split them into two separate questions.
• Neutrality: Ensure questions are free from emotional adjectives or leading phrasing.
• Appropriate Response Options: For multiple-choice questions, ensure categories are mutually exclusive (no overlap) and collectively exhaustive (all possibilities covered, often using an "Other" option).
B. Reliability vs. Validity
These two terms are fundamental when evaluating data collection instruments:
• Reliability: Relates to the consistency and repeatability of the measurement. If you repeat the survey or test under identical conditions, do you obtain the same results?
Methods to test reliability: Test-retest reliability (administering the same test to the same group at two different times) or internal consistency.
• Validity: Relates to accuracy and whether the tool actually measures what it claims to measure. For example, testing maths ability by setting a test with extremely difficult English vocabulary may measure reading comprehension rather than mathematical skill.
Methods to test validity: Face validity (does the test look suitable at first glance?) and content validity (does it cover the full curriculum being tested?).
The Dartboard Analogy:
• Shots clustered together far from the bullseye = Reliable, but not valid (consistent, but systematically wrong).
• Shots spread all over the board with no pattern = Neither reliable nor valid.
• Shots clustered tightly in the bullseye = Both reliable and valid.
Key Takeaway: A measurement tool can be highly reliable without being valid, but it cannot be valid unless it is first reliable.
---7. Common Pitfalls and Exam Hints
1. Confusing Stratified and Quota Sampling: Both divide the population into groups, but stratified sampling uses random selection within each group (requiring a full list of names), whereas quota sampling relies on the researcher picking whoever is available until the quota is filled.
2. Misclassifying Discrete Variables with Fractions: Shoe sizes or values like \(\$1.50, \$2.25\) are discrete, not continuous, because they jump in set increments without intermediate possibilities.
3. Vague Exam Answers on Bias: If asked to "Suggest a limitation of this sampling method", do not simply write "It might be biased." You must explain why: "Students surveyed at 08:00 outside the school library are more likely to be studious and well-rested, leading to over-representation of high-achieving students."
4. Command Terms Matter:
• State / Write down: Give the answer directly with no calculation or working needed.
• Explain / Justify: Give a detailed reason supported by context or mathematical definitions.