Cauchy Distribution: Where Limit Theorems Fail

The Cauchy Distribution

The Cauchy distribution is a continuous probability distribution that serves as a cautionary example: neither the WLLN nor CLT apply.

PDF

$f(x) = \frac{1}{\pi(1 + x^2)}, \quad x \in \mathbb{R}$

Standard Cauchy has parameters $x_0 = 0$ (location) and $\gamma = 1$ (scale).

CDF

$F(x) = \frac{1}{2} + \frac{1}{\pi}\arctan(x)$

Heavy Tails

The Cauchy distribution has very heavy tails. Compare to standard normal:

$\lim_{x \to \infty} \frac{P(\text{Cauchy} > x)}{P(\text{Normal} > x)} = \infty$

The Cauchy tails decay like $x^{-2}$, much slower than the normal’s $e^{-x^2/2}$.

No Finite Mean

The mean of Cauchy does not exist.

Attempt to Compute Mean

$\int_{-\infty}^{\infty} x \cdot \frac{1}{\pi(1+x^2)} dx = \frac{1}{\pi} \int_{-\infty}^{\infty} \frac{x}{1+x^2} dx$

This integral is principally undefined:

$\int_{-\infty}^{0} \frac{x}{1+x^2} dx = -\infty$

$\int_{0}^{\infty} \frac{x}{1+x^2} dx = +\infty$

We get $\infty - \infty$, which is undefined. Even the Cauchy principal value is 0, but this is not a proper expectation.

No Finite Variance

Without a finite mean, variance is also undefined. In fact, $E[X^2] = \infty$ for Cauchy.

Why WLLN and CLT Fail

Both require finite mean and/or variance:

• WLLN: Requires $E[X] < \infty$
• SLLN: Requires $E[X] < \infty$
• CLT: Requires $E[X^2] < \infty$ (finite variance)

Cauchy has neither finite mean nor finite variance.

Average of Cauchy Variables

If $X_1, X_2, \ldots$ are i.i.d. Cauchy, then $\bar{X}_n$ is also Cauchy. It does not concentrate around any value as $n \to \infty$.

Intuition: Outliers are so extreme that they dominate the average, even as $n$ grows.

Physical Interpretation

The Cauchy distribution arises as the distribution of $X = \tan(\theta)$ where $\theta$ is uniform on $(-\pi/2, \pi/2)$.

This models:

• Light refracted at random angle
• Ratio of two independent normals with mean 0
• Resonance lines in spectroscopy

Example: Averages Don’t Converge

Generate Cauchy samples and compute running averages. The average will keep jumping around rather than settling to a value.

Unlike normal or exponential samples where $\bar{X}_n$ converges, Cauchy averages are “sticky” to extreme values.

Visually Comparing Averages

• Normal: sample mean stays near 0 after moderate $n$
• Cauchy: sample mean can suddenly jump even for large $n$

Moral

Limit theorems are powerful, but they have assumptions:

• i.i.d. samples
• Finite mean (WLLN, SLLN)
• Finite variance (CLT, Chebyshev-based WLLN)

The Cauchy distribution shows what happens when these assumptions fail.

Summary

Property	Cauchy	Normal
PDF tails	$O(x^{-2})$	$O(e^{-x^2/2})$
Mean	Undefined	0
Variance	Undefined	1
CLT applies	No	Yes
Sample mean	Also Cauchy	Concentrates