Introduction to Tests and Confidence Intervals in R
Welcome to one of the most practical parts of the CS1 curriculum! In Paper A, you spend a lot of time calculating test statistics by hand and looking up values in the Goldstein Tables. In Paper B (the computer-based exam), the software does the heavy lifting for you. Your job shifts from "calculating" to interpreting.
In this chapter, we focus on how to use R to perform hypothesis tests and construct confidence intervals. This is a vital skill for any actuary, as real-world data is rarely "textbook-neat" and almost always requires software for analysis. We will look at how to read R output, make decisions about null hypotheses, and understand the range of values where a true parameter might lie.
Note: For a refresher on how to calculate basic summary statistics like the mean or variance before running these tests, see the chapter "Probabilities, quantiles and summary statistics in R".
The Core Logic: The \(p\)-value
In Paper B, the \(p\)-value is your best friend. While Paper A often requires you to find a critical value, Paper B output almost always provides a \(p\)-value.
What is a \(p\)-value? It is the probability of seeing a result as extreme as (or more extreme than) the one observed, assuming the null hypothesis (\(H_0\)) is true.
The Golden Rule of Testing:
If \(p\)-value \( < \alpha \) (the significance level, usually 0.05), we reject \(H_0\).
If \(p\)-value \( \ge \alpha \), we fail to reject \(H_0\).
Analogy: Think of the \(p\)-value as the "strength of evidence" against the null hypothesis. The smaller the number, the more "surprising" the data is if \(H_0\) were true, making us more likely to ditch \(H_0\).
Testing Means with t.test()
The t.test() function is the "Swiss Army Knife" of R's statistical functions. It is used for tests involving the Normal distribution when the variance is unknown.
1. One-Sample t-test
Used to test if the mean of a single group (\(\mu\)) is equal to a specific value (\(\mu_0\)).
R Syntax: t.test(data_vector, mu = mu_0)
2. Two-Sample t-test
Used to compare the means of two independent groups (\(\mu_1\) and \(\mu_2\)).
R Syntax: t.test(group1, group2)
By default, R performs Welch's t-test, which does not assume the variances of the two groups are equal. If the exam asks you to assume equal variances, you must add the argument var.equal = TRUE.
3. Paired t-test
Used when you have two measurements for the same subjects (e.g., weight before and after a diet). This is the test for paired data mentioned in the syllabus.
R Syntax: t.test(before_vector, after_vector, paired = TRUE)
Reading the Output
When you run t.test(), R gives you a block of text. Focus on these three areas:
1. t: The calculated test statistic.
2. p-value: Compare this to your significance level (e.g., 0.05).
3. 95 percent confidence interval: The range of values for the mean difference. If this interval includes 0, it usually means the difference is not significant!
Proportions and Rates: Binomial and Poisson
The syllabus requires you to handle tests for Binomial probabilities and Poisson means, including the normal approximation.
Binomial Tests
To test a proportion (e.g., the probability of a claim occurring), we use binom.test() for exact results or prop.test() for the normal approximation.
Example: If you have 30 successes out of 100 trials and want to test if \(p = 0.25\):
binom.test(x = 30, n = 100, p = 0.25)
Poisson Tests
To test the rate of a Poisson process (e.g., number of accidents per year), use poisson.test().
R Syntax: poisson.test(x = count_of_events, T = time_period, r = hypothesized_rate)
Key Takeaway: For both functions, R will provide a \(p\)-value and a confidence interval for the parameter (\(p\) or \(\lambda\)). Always check if the "hypothesized value" from your \(H_0\) falls inside the confidence interval provided.
Non-Parametric Approaches: Permutation and Bootstrap
Sometimes we don't want to assume our data follows a perfect Normal or Poisson distribution. The syllabus highlights two modern techniques: Permutation tests and Bootstrapping.
1. The Permutation Approach
This is used for non-parametric hypothesis testing. Instead of using a formula for a distribution, we "shuffle" the data labels many times to see how likely our result is by random chance.
In Paper B, you might be asked to write a simple for loop to shuffle data and calculate a test statistic repeatedly to build your own distribution.
2. The Bootstrap Method
This is used to obtain confidence intervals for an estimator when the standard formula is unknown or the distribution is messy.
How it works:
1. Resample your data with replacement many times (e.g., 1,000 times).
2. Calculate the statistic (like the mean or variance) for each sample.
3. The 2.5th and 97.5th percentiles of these 1,000 results form your 95% Bootstrap Confidence Interval.
Did you know? The term "bootstrapping" comes from the phrase "to pull oneself up by one's bootstraps"—it refers to the idea of the data "finding its own distribution" without external help!
Interpreting Regression Output
While regression has its own dedicated chapters, you must be able to identify tests and confidence intervals within a regression summary (summary(lm_model)).
1. Standard Error: Used to construct CIs for the slope (\(\beta\)).
2. t value: The test statistic for \(H_0: \beta = 0\).
3. Pr(>|t|): The \(p\)-value for that specific variable. If this is small, the variable is a significant predictor.
For Generalised Linear Models (GLMs), you will look at deviance and use Pearson's chi-square test or the likelihood-ratio test to determine if the model fits the data well.
Common Mistakes to Avoid
1. Wrong Alternative: R defaults to a "two-sided" test. If your hypothesis is "greater than" (\(H_1: \mu > \mu_0\)), you must specify alternative = "greater" in your R code, or your \(p\)-value will be wrong!
2. Misinterpreting the CI: Remember, a 95% Confidence Interval means that if we repeated the experiment many times, 95% of those calculated intervals would contain the true parameter. It does not mean there is a 95% probability the parameter is in this specific interval (though this is a very common student mistake!).
3. Confusing paired vs independent: Always ask: "Are these two different groups (Independent) or the same group measured twice (Paired)?" Using the wrong one in t.test() will lead to incorrect marks.
Quick Review Box
- p-value < 0.05: Significant result, reject \(H_0\).
- t.test(): Means (Normal data). Use paired = TRUE for paired data.
- binom.test() / prop.test(): Proportions (Binomial data).
- poisson.test(): Rates (Poisson data).
- Bootstrap: Resampling with replacement to get CIs.
- Permutation: Shuffling labels to get \(p\)-values for non-parametric tests.
Don't worry if the R code feels overwhelming at first. In Paper B, you usually have access to R's help files (e.g., by typing ?t.test), which can remind you of the arguments needed!