T-Test Primer

A primer of the T-test without examples of how to calculate.

Introduction

One of the simplest inferential tests is ‘Students T-test’ which is used to compare the means of two groups. The T-test is used when you want to determine if there is a significant difference between the means of two groups, and the sample sizes are small, typically less than 30.

It was developed by William Sealy Gosset while working as a brewer at ‘Guinness’ where publication was allowed if papers did not mention “1) beer, 2) Guinness, or 3) their own surname”[ref] on fear of disclosing trade secrets. Gosset hence published under the pseudonym “Student” in 1908.

Assumptions

The T-test makes several key assumptions:

Each observation is independent of the others. E.g. no person has been measured twice.
The outcome is continuous and measured on an interval or ratio scale.
Two, and only two, distinct groups are being compared.
The data is normally distributed.
The variances of the two groups are equal. I.e. the spread of the data is similar in both groups.

Types of T-tests

While the assumptions are strict, there are options for analysing data when they are not met. For example:

If the data is not normally distributed, a non-parametric test such as the Mann-Whitney U test can be used.
If the variances are not equal, a Welch’s T-test can be used instead of the standard T-test.
If there are multiple groups, an ANOVA can be used to compare the means of more than two groups.
If the data is paired, a paired T-test can be used to compare the means of two related groups.

Interpreting the results of a T-test

When you perform a T-test, you will get a T-statistic and a p-value. The T-statistic is a measure of how different the means of the two groups are (relative to the variability in the data). The larger the absolute value of the T-statistic, the better separated the means are. Remember that the T-statistic is measured relative to the standard deviation of the data, so a T-statistic of 3 does not mean the data is separated by 3 units on the original scale. For the practical relevant difference, the T-statistic should be multiplied by the estimated standard deviation of the data to give the difference in the original units.

The p-value tells you how likely the ‘null-hypothesis’ is to be true. For the T-test, the null hypothesis is:

\(H_0: \text{there is no difference between the means of the two groups}\)

A small p-value (typically less than 0.05) indicates that the null hypothesis can be rejected, suggesting that there is a significant difference between the means of the two groups, and hence accept the ‘alternative hypothesis’:

\(H_1: \text{there is a difference between the means of the two groups}\)

The T-test can provide a ‘confidence interval’ for the difference between the means of the two groups i.e. a range between which the true difference is likely to lie. This is useful for understanding the magnitude of the difference, not just whether it is statistically significant. As we often assume 5% as a threshold for ‘significance’, the confidence interval is often given as a 95% confidence interval. This means that if the experiment were repeated many times, 95% of the time the true difference would lie within this range (not that the true difference is 95% likely to lie within this range).

Conclusion

The T-test is a powerful and widely used statistical test for comparing the means of two groups. It is important to understand the assumptions of the T-test and the alternatives available when those assumptions are not met. The T-test is a fundamental tool in statistics and is used in many fields, including psychology, medicine, and social sciences.