Supervised and unsupervised learning

Welcome to the World of Learning!

In our previous look at machine learning, we learned that it is all about computers getting better at tasks through experience. But how exactly do they "experience" data? In the IB syllabus, we categorize this into two main styles: Supervised Learning and Unsupervised Learning. Think of these like two different ways of studying for an exam—one with a tutor and one by exploring a library on your own.

Don't worry if these terms sound high-tech; by the end of these notes, you'll see they are actually very logical ways to process information.

1. Supervised Learning: Learning with a Guide

Supervised learning is the most common type of machine learning. In this method, the algorithm is trained on a labeled dataset. This means the computer is given the "input" and the "correct answer" at the same time.

The Analogy: Imagine you are learning to identify fruit. A teacher shows you a picture of an apple and says, "This is an apple." Then they show you an orange and say, "This is an orange." After seeing hundreds of these labeled examples, you can look at a new, unseen fruit and predict what it is based on what you learned.

Types of Supervised Learning

Supervised learning generally falls into two categories based on what we want to predict:

A. Classification (Category Prediction)
This is used when the output is a discrete label or a "category."
Example: A spam filter. The computer looks at an email and classifies it as either "Spam" or "Not Spam." There is no middle ground.
Example: Identifying a handwritten digit (is it a \(0, 1, 2...\) or \(9\)?).

B. Regression (Value Prediction)
This is used when the output is a continuous numerical value.
Example: Predicting the price of a house. The answer isn't "Yes" or "No," but a specific number like \(\$450,000\).
Example: Predicting tomorrow’s temperature in Celsius based on historical weather data.

Key Takeaway: If you have historical data where you already know the "right answer" (the label), you are using Supervised Learning. Its goal is to find a mapping function \(y = f(x)\) to predict the output \(y\) for new inputs \(x\).

2. Unsupervised Learning: Finding Hidden Patterns

In Unsupervised Learning, the algorithm is given unlabeled data. There is no "teacher" telling the computer what is right or wrong. Instead, the computer looks at the data and tries to find its own structure or patterns.

The Analogy: Imagine you are given a giant box of random LEGO bricks. Nobody tells you what to build or how to sort them. You might naturally start putting all the red bricks in one pile, the blue bricks in another, and the long bricks in a third. You aren't "predicting" anything; you are just organizing based on what looks similar.

Types of Unsupervised Learning

A. Clustering
This is the process of grouping data points together because they share similar characteristics.
Example: Customer Segmentation. A clothing company gives their database of 10,000 customers to an algorithm. The algorithm groups them into "High Spenders," "Bargain Hunters," and "Occasional Shoppers" based on their buying habits, even though the company didn't have those labels prepared beforehand.

B. Association
This is about finding "rules" that describe your data, specifically which things tend to happen together.
Example: Market Basket Analysis. A grocery store notices that people who buy diapers also tend to buy beer at the same time. This "association" helps them decide where to place items on the shelves.

Key Takeaway: Unsupervised learning is about discovery. It doesn't predict a specific "answer"; it describes the structure of the data.

3. Comparing the Two: A Quick Guide

To keep these straight for your exams, remember these three main differences:

1. Data Type:
Supervised = Labeled (Input + Answer).
Unsupervised = Unlabeled (Input only).

2. Goal:
Supervised = To predict an output for new data.
Unsupervised = To find hidden patterns or groupings in existing data.

3. Feedback:
Supervised = The model is "corrected" during training if it gets an answer wrong (see the chapter on Data, training and evaluation).
Unsupervised = There is no "wrong" answer; the computer just reports what it found.

Mnemonic Hint:
Supervised = Showing the answer.
Unsupervised = Unknown labels.

4. Common Mistakes to Avoid

Mistake 1: Thinking Regression is Unsupervised because it involves numbers.
Even though regression (like predicting a stock price) involves complex math, it is Supervised because we use past "correct" prices to train the model.

Mistake 2: Thinking Clustering and Classification are the same.
They both involve groups, but in Classification, you know the names of the groups beforehand (e.g., "Dog" or "Cat"). In Clustering, the computer creates the groups itself based on similarity, and you decide what to call them later.

Did you know?
Most modern AI, like recommendation engines on streaming services, uses a mix of both! They might use Unsupervised Learning to cluster you with similar users, and then Supervised Learning to predict exactly which movie you will click on next.

Quick Review Box

Supervised Learning: Labeled data. Includes Classification (discrete categories) and Regression (continuous values). Used for prediction.
Unsupervised Learning: Unlabeled data. Includes Clustering (grouping) and Association (finding rules). Used for pattern discovery.

Note: For more on how we measure if these models are actually working, check out the chapter "Data, training and evaluation."