Why √n — Reading the Central Limit Theorem

Roll a single die and the outcomes are flat: 1 through 6, each equally likely, no peak anywhere. Roll two and sum them, and a triangle appears with its apex at 7. Roll ten, or fifty, and the histogram of the average sharpens into the unmistakable bell — symmetric, concentrated, falling off fast at the edges. Nothing about a die is bell-shaped. The bell is manufactured entirely by averaging, and the same bell appears whether you average dice, coin flips, exam scores, or photon counts. This universality, and the precise rate at which the bell sharpens, is the content of Plate IX — the Central Limit Theorem.

The headline result is easy to state and worth holding in mind before we justify it. Take random variables $X_1, X_2, \dots, X_n$ that are independent, identically distributed, with a finite mean $\mu$ and a finite variance $\sigma^2$ . Form the sample mean

$\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i.$

Then $\bar{X}_n$ clusters around $\mu$ , its typical deviation from $\mu$ shrinks like $\sigma/\sqrt{n}$ , and after rescaling by exactly that amount the fluctuations look Gaussian. Two numbers do all the work here, the expectation $\mu$ and the variance $\sigma^2$ , and the theorem says the shape of the original distribution is forgotten. Only its first two moments survive.

Where the center comes from: linearity

The easy half is the center. Expectation is linear without any independence assumption at all: for any random variables and any constants, $\mathbb{E}[aX + bY] = a\mathbb{E}[X] + b\mathbb{E}[Y]$ . Applying this to the sum,

$\mathbb{E}[\bar{X}_n] = \frac{1}{n}\sum_{i=1}^n \mathbb{E}[X_i] = \frac{1}{n}\cdot n\mu = \mu.$

So the sample mean is correctly centered for every $n$ — it is an unbiased estimate of $\mu$ , no large-sample limit required.Linearity is why the centre never moves; it holds even when the $X_i$ are dependent. This is the same accounting that makes the sample mean the maximum-likelihood estimate of $\mu$ for Gaussian data. Knowing the center, though, says nothing about how tightly the mean concentrates. For that we need the second moment, and there independence finally earns its keep.

Where √n comes from: variance adds

Variance is not linear, but for independent variables it is additive. If $X$ and $Y$ are independent, the cross term $\mathrm{Cov}(X,Y)$ vanishes and

$\mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y).$

Variances of independent contributions add; standard deviations do not. Summing $n$ independent copies, each with variance $\sigma^2$ ,

$\mathrm{Var}\!\left(\sum_{i=1}^n X_i\right) = n\sigma^2.$

Now divide by $n$ to get the mean, remembering that variance scales by the square of any constant factor, $\mathrm{Var}(cZ) = c^2\,\mathrm{Var}(Z)$ :

$\mathrm{Var}(\bar{X}_n) = \frac{1}{n^2}\cdot n\sigma^2 = \frac{\sigma^2}{n}.$

This single line is the origin of the $\sqrt{n}$ . The variance of the mean falls like $1/n$ , so the standard deviation — the typical spread, in the original units — falls like the square root, $\sigma/\sqrt{n}$ . The asymmetry between summing (variance grows like $n$ ) and averaging (you divided by $n^2$ ) is the whole story. It also dispels a common wrong guess: the spread does not shrink like $1/n$ . Quadrupling your sample only halves your error bar. To gain one more decimal digit of precision you need a hundredfold more data — the brutal arithmetic behind why large studies are expensive.

The standardized sum and “in distribution”

To see a shape rather than just a width, strip out the center and the scale. Define the standardized sum

$Z_n = \frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} = \frac{1}{\sigma\sqrt{n}}\sum_{i=1}^n (X_i - \mu).$

By construction $Z_n$ has mean $0$ and variance $1$ for every $n$ — we have factored out exactly the two effects computed above. What remains is pure shape, and the central limit theorem states that this shape converges to the standard normal:

$Z_n \xrightarrow{\ d\ } \mathcal{N}(0, 1).$

The arrow means convergence in distribution, and the precise meaning matters. It does not say that the random number $Z_n$ settles down to some fixed value — it keeps fluctuating forever. It says the distribution function converges: for every threshold $z$ ,

$\mathbb{P}(Z_n \le z) \longrightarrow \Phi(z),$

where $\Phi$ is the standard normal CDF. The probabilities line up, not the realizations. This is a weaker and subtler statement than the law of large numbers, which says $\bar{X}_n \to \mu$ for the values themselves. The CLT is a claim about the odds, not about where any particular average lands.In distribution: the histograms agree in the limit, even though no single draw of $Z_n$ converges to anything.

Why a Gaussian and not some other limit shape? Because the standardization above is a fixed point. The sum of two independent normals is again normal, and the normal is the unique mean-zero, finite-variance distribution stable under this add-and-rescale operation. Any finite-variance starting distribution, run through the averaging machine, is pulled toward that fixed point. The original shape is washed out; only $\mu$ and $\sigma$ are remembered.

What the theorem does not promise

The CLT is generous but it is not magic, and reading it carelessly causes real errors.

It is a statement about the center, not the tails. The convergence is best near the mean and weakest far out. For modest $n$ , the true probability of a rare, many-sigma event can differ from the Gaussian prediction by orders of magnitude. Skewed or heavy-tailed data converge slowly, and using a normal approximation to price a five-sigma loss is exactly where the approximation is least trustworthy.

It requires independence. Variance addition relied on the covariances vanishing. If the $X_i$ are correlated — consecutive days of a market, neighboring pixels, repeated measurements of a drifting instrument — the cross terms do not cancel, the effective variance is not $n\sigma^2$ , and the $\sqrt{n}$ rate is wrong, usually optimistic. Treating dependent data as independent silently shrinks your error bars below their true size.

It requires finite variance. Drop the assumption $\sigma^2 < \infty$ and the result collapses. The standard counterexample is the Cauchy distribution, with density $f(x) = 1/[\pi(1+x^2)]$ . Its tails decay so slowly that the variance integral diverges — indeed the mean does not even exist. Average $n$ independent Cauchy draws and you get a remarkable failure: the sample mean has the same Cauchy distribution as a single draw.Averaging Cauchy variables buys nothing — one extreme draw can dominate the whole sum. It does not concentrate at all. There is no $\sqrt{n}$ shrinkage, no Gaussian limit, no benefit whatsoever from collecting more data. The bell never forms, because the hypothesis that built it was never met.

The lesson behind Plate IX is therefore two-sided. When its three conditions hold — independent draws, identical law, finite variance — an extraordinary universality emerges: the underlying shape is irrelevant, the limit is always the same bell, and the error shrinks at the knowable rate $\sigma/\sqrt{n}$ . When any condition fails, the theorem withdraws its guarantee precisely and without apology. The $\sqrt{n}$ is not a law of nature. It is the visible signature of variances that add, which is to say, of contributions that do not conspire.

Where the center comes from: linearity

Where √n comes from: variance adds

The standardized sum and “in distribution”

What the theorem does not promise

Further reading