Structured and unstructured data types

Welcome to Data Exploration: The "Shape" of Your Data

Hello there! Before we dive into complex models and fancy algorithms, we have to start with the most important ingredient: the data. Think of data like the building materials for a house. You wouldn't use the same tools to work with wood as you would for bricks, right? Similarly, in Exam PA, you need to recognize the different "types" of data so you know which "tools" (statistical methods) to use.

In this chapter, we are looking at Structured and Unstructured data. Don't worry if these terms sound a bit technical—by the end of this page, you’ll be spotting the difference in your sleep!

1. Structured Data: The Neat and Tidy Table

Structured data is the bread and butter of actuarial work. If you can imagine it sitting perfectly inside an Excel spreadsheet with clearly labeled rows and columns, it’s likely structured data.

What makes it structured?
1. It has a predefined format (a "schema").
2. Each column represents a specific variable (like Age, Premium, or Gender).
3. Each row represents a single observation (like a specific policyholder or a single insurance claim).

Real-World Example:
Imagine a list of car insurance policies. Every entry has a "Policy Number," a "Start Date," and a "Vehicle Make." It’s predictable and easy for a computer to search through.

The Analogy:
Think of structured data like a muffin tin. You know exactly where each muffin goes, and every muffin is roughly the same size and shape because the holes are already there waiting for them.

Quick Review: Key Features of Structured Data

• Easily stored in Relational Databases (SQL).
• Quantitative (usually numbers or clearly defined categories).
• Requires less "cleaning" than unstructured data before it can be used in a model.

2. Unstructured Data: The "Wild West" of Information

Unstructured data is everything else. It doesn’t have a predefined model or a tidy table to sit in. It is often "heavy" on text, images, or audio.

In the context of Exam PA, you might encounter unstructured data in the form of claim notes written by an adjuster or customer emails. Because there are no columns or rows, a computer can't "read" it as easily as a spreadsheet. We often have to use special techniques (like Text Mining) to turn this into numbers before we can model it.

Real-World Example:
A doctor's handwritten notes on a medical file. One doctor might write "Patient has a cough," while another might write "Respiratory distress noted." There is no fixed "column" for this—it’s just raw text.

The Analogy:
Think of unstructured data like a laundry basket full of different clothes. You have shirts, socks, and hats all tangled together. To make sense of it, you have to sort them out first!

Did You Know?

It is estimated that over 80% of all data generated today is unstructured! While actuaries traditionally use structured data, the "Big Data" revolution is making unstructured data much more important for predicting things like fraud or customer sentiment.

3. The Middle Ground: Semi-Structured Data

Sometimes, data is a bit of both. We call this Semi-Structured data. It doesn't reside in a neat table, but it has tags or markers that separate the elements. Common formats include JSON or XML files (which you might see when data is pulled from a website).

Memory Aid:
• Structured = A Brick Wall (Perfectly aligned).
• Semi-Structured = A LEGO Set (Pieces are different, but they have "studs" to help them fit together).
• Unstructured = A Pile of Clay (No shape until you mold it).

4. Diving Deeper: Types of Variables

Once we have our Structured Data, we need to categorize the variables (the columns). This is a very common topic on Exam PA!

A. Numerical (Quantitative) Variables

These represent measurable quantities. You can do math with these! We break these down into two types:

1. Continuous: Can take any value in a range (e.g., \( X = 152.45 \)). Examples: Height, Policy Premium, Claim Amount.
2. Discrete: Countable values, usually integers. Examples: Number of cars in a household, Number of claims in a year.

B. Categorical (Qualitative) Variables

These represent groups or labels. You can't calculate the "average" of these in a traditional way.

1. Nominal: Categories with no natural order. Example: Car Color (Red, Blue, Green). Is Red "higher" than Blue? No. They are just different.
2. Ordinal: Categories with a clear logical order. Example: Education Level (High School, Bachelors, Masters, PhD). We know a Masters is "higher" than High School, but we can't say exactly "how much" higher using a simple subtraction.

Common Mistake to Avoid:

The "Number" Trap: Just because a variable uses numbers doesn't mean it's Numerical!
Example: A Zip Code (90210) is a Nominal variable. Adding two zip codes together makes no sense—the number is just a label for a location!

Step-by-Step: How to Identify Data Types for Exam PA

When you are presented with a dataset in the exam, follow these steps:

Step 1: Look at the format. Is it in a table? (If yes, it's Structured).
Step 2: Look at a specific column. Are there numbers or words?
Step 3: If numbers, ask: "Does the average of this make sense?" (If yes, it's Numerical; if no, it might be Categorical/Nominal).
Step 4: If words/categories, ask: "Is there a specific rank or order?" (If yes, it's Ordinal; if no, it's Nominal).

Summary & Key Takeaways

• Structured Data: Organized in tables (rows/columns). Ready for analysis.
• Unstructured Data: No fixed format (text, images). Needs processing.
• Numerical Variables: Continuous (any value) or Discrete (counts).
• Categorical Variables: Nominal (no order) or Ordinal (ranked).
• Crucial Point: Identifying the data type is the first step in Data Exploration because it dictates which graph you will use (e.g., you use a Histogram for Numerical data but a Bar Chart for Categorical data).

Don't worry if this seems a bit basic—mastering these definitions ensures you won't lose easy points on the exam when the questions get more complex! Keep up the great work!