Exploratory plots and correlation output in R

CS1 – Actuarial Statistics · IFoA (Institute and Faculty of Actuaries) · 5 min read · Updated: 7 Sept 2026

Introduction: Seeing the Story in the Data

Welcome! In the world of actuarial science, we are often given massive datasets that look like nothing more than a wall of numbers. Before we dive into complex modeling, we need to perform Exploratory Data Analysis (EDA). This is the stage where we use "exploratory plots" to visualize our data and "correlation coefficients" to see how variables relate to one another.

In your Paper B (CS1B) exam, you won't just be asked to generate these plots; you'll be asked to interpret them. Think of yourself as a detective looking for clues—is the data skewed? Are there weird outliers? Do two variables move together? Let’s learn how to find these answers using R.

1. Visualizing Univariate Data: Looking at One Variable

Before looking at relationships, we need to understand the individual variables. The syllabus highlights the use of exploratory visualisations to understand the distribution of your data.

The Histogram: `hist()`

A histogram helps us see the "shape" of our data. It groups data into "bins" and shows how many observations fall into each bin. It is perfect for checking if data is symmetrical or skewed.

R Syntax:
hist(data_vector, main = "Title", xlab = "Label", col = "blue")

Actuarial Analogy: Imagine you are sorting claim amounts. If most claims are small but a few are massive, your histogram will have a "long tail" to the right. This is called positive skew, and it’s very common in insurance!

The Boxplot: `boxplot()`

A boxplot (or box-and-whisker plot) provides a visual summary of the five-number summary: Minimum, First Quartile (\(Q1\)), Median, Third Quartile (\(Q3\)), and Maximum.

Key features to spot:
- The "Box": Represents the Interquartile Range (IQR). It contains the middle 50% of the data.
- The Thick Line: This is the Median.
- The "Whiskers": Extend to the most extreme data points that are not considered outliers.
- Dots/Circles: Points outside the whiskers are outliers. These are critical for actuaries as they might represent "catastrophic" claims.

R Syntax:
boxplot(data_vector, horizontal = TRUE)

2. Visualizing Bivariate Data: Scatter Plots

When we want to see if there is a relationship between two variables (like "Age of Policyholder" and "Number of Claims"), we use a scatter plot.

The Plot Function: `plot()`

In R, plot(x, y) creates a scatter plot where \(x\) is the explanatory variable (usually on the horizontal axis) and \(y\) is the response variable (usually on the vertical axis).

What to look for:
- Direction: Do the points go up from left to right (positive relationship) or down (negative)?
- Form: Do the points look like a straight line (linear) or a curve?
- Strength: How closely are the points packed together? Tight packing means a strong relationship.

Quick Tip: In Paper B, always label your axes! Use xlab = "..." and ylab = "..." to ensure you don't lose easy marks.

3. Correlation: Measuring the Connection

While plots give us a visual "feel," correlation coefficients give us a numerical value for the strength and direction of a relationship. The CS1 syllabus requires you to understand three specific measures.

The Three Musketeers of Correlation

1. Pearson’s Product-Moment Correlation (\(r\))
This measures the strength of the linear relationship between two variables. It assumes the data is roughly normally distributed.
- Value of \(+1\): Perfect positive linear relationship.
- Value of \(-1\): Perfect negative linear relationship.
- Value of \(0\): No linear relationship.

2. Spearman’s Rank Correlation (\(\rho\))
This is a "non-parametric" measure. Instead of using the raw values, it uses the ranks of the data. It is great for detecting monotonic relationships (where variables move in the same direction, but not necessarily in a straight line). It is much less sensitive to outliers than Pearson’s.

3. Kendall’s Tau (\(\tau\))
Like Spearman's, this is rank-based. It looks at "concordant" and "discordant" pairs. It is often used for smaller datasets or when there are many tied ranks.

R Code for Correlation

The cor() function is your best friend here. You just need to specify the "method":

cor(x, y, method = "pearson")
cor(x, y, method = "spearman")
cor(x, y, method = "kendall")

Quick Review: Which one to use?
- If the relationship looks like a straight line: Pearson.
- If the relationship is curved but always increasing/decreasing: Spearman.
- If you have extreme outliers that might "pull" a straight line: Spearman or Kendall.

4. Common Pitfalls and Interpretation Tips

Don't worry if this seems tricky at first; interpreting output is a skill that grows with practice. Here are some things to keep in mind for your exam answers:

Correlation \(\neq\) Causation
Just because two variables have a high correlation coefficient (e.g., \(0.95\)) doesn't mean one causes the other. In your exam comments, use phrases like "There is a strong positive linear association between..." rather than saying "Variable X causes Variable Y."

The Effect of Outliers
A single outlier can significantly decrease (or increase!) a Pearson correlation coefficient. If you see an outlier in a scatter plot but the Spearman correlation is high, it suggests the underlying relationship is strong but the Pearson value is being distorted.

Checking for Non-Linearity
If cor(x, y) is near \(0\), it doesn't mean there is no relationship. There could be a quadratic (U-shaped) relationship. This is why we always plot the data before calculating the numbers!

Key Takeaways for Paper B

hist(): To see the distribution and skewness of a single variable.
boxplot(): To identify the median, quartiles, and outliers.
plot(x, y): To visualize the relationship between two variables.
cor(..., method = "..."): To quantify the relationship.
- Pearson: Linear relationship.
- Spearman/Kendall: Monotonic relationship (rank-based).
Interpretation: Always look at the plot before trusting the correlation coefficient!

Note: For further details on how to use these results in formal tests, see the chapter on "Tests and confidence intervals using software output". For using these variables in models, see "Fitting and interpreting linear regression output".

Quick check

Can you answer these now?

Open each question to check the key ideas from this chapter.

What is the primary purpose of Exploratory Data Analysis (EDA) in actuarial work?

EDA is used to visualize data through exploratory plots and calculate correlation coefficients to understand distributions and relationships before performing complex modeling.

What does a histogram help an actuary identify about a dataset's distribution?

A histogram reveals the 'shape' of the data, specifically whether the distribution is symmetrical or skewed (e.g., positive skew in claim amounts).

Which five values make up the 'five-number summary' visualized in a boxplot?

The five-number summary consists of the Minimum, First Quartile (\(Q1\)), Median, Third Quartile (\(Q3\)), and Maximum.

How are outliers represented and why are they significant in a boxplot?

Outliers are represented as dots or circles outside the whiskers; for actuaries, they often represent critical 'catastrophic' events or extreme claims.

What three characteristics should you look for when interpreting a scatter plot?

You should look for the direction (positive or negative), the form (linear or curved), and the strength (how closely points are packed) of the relationship.

When is Pearson’s Product-Moment Correlation (\(r\)) the most appropriate measure of association?

Pearson's correlation is best for measuring the strength and direction of a linear relationship between variables that are roughly normally distributed.

What is the advantage of using Spearman’s Rank Correlation (\(\rho\)) over Pearson’s?

Spearman's correlation is non-parametric, uses ranks instead of raw values, and is effective for detecting non-linear monotonic relationships and resisting the influence of outliers.

Why is it dangerous to assume causation from a high correlation coefficient?

Correlation only measures association; it does not prove that one variable causes the other, as a third 'lurking' variable might be influencing both.

Why should you always plot your data before calculating a correlation coefficient?

Plotting data prevents misinterpreting a low correlation coefficient (near 0) as 'no relationship' when a strong non-linear (e.g., quadratic) relationship might actually exist.

Ready to test yourself?

Turn these notes into exam-style practice. Get unlimited AI questions on this topic with instant marking and explanations.

Practise This Topic

More CS1 – Actuarial Statistics chapters

* The content provided by thinka is generated by AI and may not always be accurate or up-to-date. Please use it as a supplementary resource and verify with official materials.

Put These Notes into Practice

Reading the notes is a great start. Now practise with unlimited AI-generated questions and get instant feedback. 100,000+ students are already improving their grades.

Start Practising Now View Pricing

Done reading? Test yourself with AI practice questions

Practise This Topic Now