Comparing WLLN, SLLN, and CLT

Overview

These three theorems describe different aspects of how sample means behave as sample size grows.

Theorem	What It Says	Type of Convergence
WLLN	$\bar{X}_n \to \mu$	In probability
SLLN	$\bar{X}_n \to \mu$	Almost surely
CLT	$\sqrt{n}(\bar{X}_n - \mu) \to N(0, \sigma^2)$	In distribution

Weak Law of Large Numbers (WLLN)

Statement: $P(|\bar{X}_n - \mu| > \epsilon) \to 0$ as $n \to \infty$.

Meaning: For large $n$, $\bar{X}_n$ is probably close to $\mu$.

Key insight: Each fixed $\epsilon$ bound gives a probability going to zero.

Analogy: Like a particle that is probably near $\mu$ at any large time $n$, but might occasionally wander away.

Strong Law of Large Numbers (SLLN)

Statement: $P(\bar{X}_n \to \mu \text{ as } n \to \infty) = 1$.

Meaning: With probability 1, $\bar{X}_n$ converges to $\mu$ and stays there.

Key insight: Almost all sequences converge and stay converged.

Analogy: Like a particle that eventually settles at $\mu$ and stays there forever.

Central Limit Theorem (CLT)

Statement: $\sqrt{n}(\bar{X}_n - \mu) \xrightarrow{d} N(0, \sigma^2)$.

Meaning: The scaled deviation from $\mu$ has a normal distribution.

Key insight: Gives the distribution and rate, not just convergence.

Analogy: Like knowing how the particle’s distance from $\mu$ is distributed at each $n$.

Visual Comparison

When to Use Each

Goal	Use
Show estimator is consistent	WLLN or SLLN
показать “probably close”	WLLN
Show “eventually stays close”	SLLN
Compute approximate probabilities	CLT
Find confidence intervals	CLT
Determine rate of convergence	CLT

Mathematical Hierarchy

$\text{Almost sure convergence} \Rightarrow \text{Convergence in probability} \Rightarrow \text{Convergence in distribution}$

SLLN $\Rightarrow$ WLLN is immediate.

CLT is about distribution, not about $\bar{X}_n$ directly converging.

Rate Information

Theorem	Rate Information
WLLN	None (just says probability → 0)
SLLN	None (just says stays converged)
CLT	$O(1/\sqrt{n})$ (explicit distribution)

Chebyshev gives WLLN rate: $P(|\bar{X}_n - \mu| > \epsilon) \leq \sigma^2/(n\epsilon^2)$.

CLT gives exact asymptotic distribution: $\sqrt{n}(\bar{X}_n - \mu) \sim N(0, \sigma^2)$.

Example: Estimating a Probability

Estimate $p = P(\text{success})$ by sample proportion $\hat{p}_n$.

• WLLN: For large $n$, $\hat{p}_n$ is probably near $p$
• SLLN: With probability 1, $\hat{p}_n \to p$ eventually
• CLT: $\sqrt{n}(\hat{p}_n - p) \approx N(0, p(1-p))$

For confidence interval (CLT): $\hat{p}_n \pm 1.96\sqrt{\frac{\hat{p}_n(1-\hat{p}_n)}{n}}$

Example: Average Height

Measure heights of random people.

• WLLN: Sample average converges to population mean in probability
• SLLN: Sample average eventually stabilises at population mean
• CLT: For $n = 100$, $\bar{X}_{100} \approx N(\mu, \sigma^2/100)$

Why All Three?

They answer different questions:

1. Does it converge? WLLN/SLLN
2. Does it stay converged? SLLN
3. What’s the distribution? CLT
4. How fast? CLT

Practical Takeaway

• WLLN: Justifies using sample mean as point estimator
• SLLN: Guarantees long-run stability of estimators
• CLT: Enables probabilistic statements, confidence intervals, hypothesis tests

Summary

Aspect	WLLN	SLLN	CLT
Convergence type	Probability	Almost sure	Distribution
Rate	Implicit	None	Explicit $O(1/\sqrt{n})$
Distribution info	No	No	Yes (normal)
Use case	Basic justification	Long-run guarantees	Probability computation