Strong Law of Large Numbers

Statement

Let $X_1, X_2, \ldots$ be i.i.d. random variables with finite mean $\mu$. The Strong Law of Large Numbers states:

$P\left(\lim_{n \to \infty} \bar{X}_n = \mu\right) = 1$

where $\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i$.

Interpretation

With probability 1, the sequence $\bar{X}_n$ converges to $\mu$ and stays close forever.

This is stronger than the Weak Law:

• Weak Law: For any $\epsilon$, $P(|\bar{X}_n - \mu| > \epsilon) \to 0$
• Strong Law: The probability that $\bar{X}_n \to \mu$ is 1

Almost Sure Convergence

The SLLN says $\bar{X}_n \xrightarrow{a.s.} \mu$ (converges almost surely).

This means: the set of outcomes where $\lim_{n \to \infty} \bar{X}_n = \mu$ has probability 1.

Almost Surely vs In Probability

Type	Statement
In probability	For each $\epsilon$: $P(
Almost surely	$P(\lim_{n \to \infty} \bar{X}_n = \mu) = 1$

The difference:

• In probability: for large $n$, $\bar{X}_n$ is probably close to $\mu$ (but may occasionally jump away)
• Almost surely: with probability 1, $\bar{X}_n$ eventually stays close to $\mu$ forever

Analogy

Think of $\bar{X}_n$ as a particle:

• WLLN: At time $n$, particle is probably near $\mu$
• SLLN: Particle eventually stays near $\mu$ forever (with probability 1)

Why Stronger?

Almost sure convergence implies convergence in probability, but not vice versa.

In probability allows infinitely many “large” deviations, as long as their frequency goes to 0. Almost surely forces convergence with only finitely many large deviations.

Example: Repeated Experiments

Suppose we repeatedly estimate a probability by sampling. SLLN says:

• With probability 1, our estimate sequence converges to the true probability
• Eventually, our estimate “locks in” to the truth

This is why long-run frequencies converge to probabilities.

Intuition: Borel-Cantelli

Under appropriate conditions, the event $\{|\bar{X}_n - \mu| > \epsilon\}$ happens only finitely many times almost surely.

Since for any $\epsilon > 0$:

$\sum_{n=1}^{\infty} P(|\bar{X}_n - \mu| > \epsilon) < \infty$

By Borel-Cantelli: $P(|\bar{X}_n - \mu| > \epsilon \text{ infinitely often}) = 0$.

What Conditions Are Needed?

SLLN holds if either:

1. $E[|X_i|] < \infty$ (finite mean suffices)
2. Or: $E[X_i^2] < \infty$ (finite variance, easier proof)

The variance condition makes the proof simpler but isn’t necessary.

No Guarantee of Rate

Like WLLN, the SLLN doesn’t tell us how fast convergence occurs. For rate, use the CLT.

Summary Table

Law	Convergence	Statement
WLLN	Probability	$P(
SLLN	Almost sure	$P(\lim \bar{X}_n = \mu) = 1$

Key Takeaway

Both laws justify using sample means as estimators. The Strong Law is conceptually stronger: it says our estimator will ultimately stabilise at the truth with certainty.