Descriptive Statistics

Updated 1st November 2026Started 31st December 2022

Centre

Mean

\bar x = \frac{1}{n} \sum_{i=1}^n x_i

Median: the middle value of the sorted data. With an even number of values it is the average of the middle two.

Mode: the most common value (or values).

Spread

Range: the largest value minus the smallest. Simple, but set entirely by the two most extreme points.

Variance: roughly the average squared deviation from the mean. For a sample we divide by $n - 1$ rather than $n$ :

s^2 = \frac{\sum_{i=1}^n (x_i - \bar{x})^2}{n - 1}

Why $n - 1$ ? We measure deviations from the sample mean $\bar{x}$ , not the true mean $\mu$ , and $\bar{x}$ is by construction the point that minimises the sum of squared deviations for this sample. So the sum is systematically too small. Dividing by $n - 1$ (Bessel's correction) makes $s^2$ an unbiased estimate of the population variance. The 1 is there because one quantity, $\bar{x}$ , was estimated from the same data: once $\bar{x}$ is fixed, only $n - 1$ of the deviations are free to vary (they must sum to zero).

If you really do have the whole population, you know $\mu$ , and dividing by $n$ is correct. NumPy's np.var divides by $n$ by default (ddof=0); pandas' .var() divides by $n - 1$ (ddof=1).

Standard deviation: the square root of the variance, which puts it back in the units of the data.

s = \sqrt{\frac{\sum_{i=1}^n (x_i - \bar{x})^2}{n - 1}}

Interquartile range: the 75th percentile minus the 25th. Unlike the range and the standard deviation, it is barely affected by outliers.

Covariance

Variance measures how one variable deviates from its mean; covariance measures how two variables deviate from their means together:

\text{cov}(x, y) = \frac{\sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})}{n - 1}

A large positive covariance means $x$ tends to be large when $y$ is large and small when $y$ is small. A large negative covariance means the opposite. A covariance near zero means no such linear relationship.

Covariance is hard to interpret on its own:

Its units are the product of the two variables' units.
Doubling every $x$ doubles the covariance, though the two variables are just as related as before. So it is hard to say what counts as "large".

Correlation

Pearson's correlation coefficient divides the covariance by both standard deviations, which removes the units and bounds it between −1 and 1:

\rho(X, Y) = \frac{\text{cov}(X, Y)}{\sigma_X \sigma_Y}

Things to remember about it:

It only picks up linear relationships, where knowing how $x_i$ compares with the mean of $x$ tells you how $y_i$ compares with the mean of $y$ . A perfect $y = x^2$ relationship on symmetric data has a correlation of zero.
It says nothing about how large the effect is. A correlation of 0.9 is compatible with $y$ rising by a tiny amount per unit of $x$ ; the slope comes from linear regression.
A correlation in the whole population can reverse inside every subgroup when a confounding variable is ignored (Simpson's paradox).

For a rank-based alternative that handles any monotonic relationship, see Spearman's correlation in Feature Importance.

Adapted in part from Joel Grus, Data Science from Scratch (2nd edition), chapter 5.

Revision log

1st November 2026Migrated from data-science-notes

Tags: Statistics