Unbiased Estimators for Mean and Variance

Sample Mean

$\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$ is unbiased for $\mu$.

$E[\bar{X}] = \mu$

Variance: $\text{Var}(\bar{X}) = \frac{\sigma^2}{n}$

Sample Variance (Biased)

$S_n^2 = \frac{1}{n}\sum_{i=1}^n(X_i - \bar{X})^2$

This is biased: $E[S_n^2] = \frac{n-1}{n}\sigma^2$.

Proof

$E[S_n^2] = E\left[\frac{1}{n}\sum X_i^2 - \bar{X}^2\right]$

$= E[X^2] - (E[X])^2 - \text{Var}(\bar{X})$

$= \sigma^2 - \frac{\sigma^2}{n} = \frac{n-1}{n}\sigma^2$

Sample Variance (Unbiased)

$S^2 = \frac{1}{n-1}\sum_{i=1}^n(X_i - \bar{X})^2$

This is unbiased: $E[S^2] = \sigma^2$.

Why n-1?

Degrees of freedom: Estimating $\mu$ with $\bar{X}$ uses one degree of freedom before estimating variance.

$n - 1$ corrects for this.

Standard Error

$SE = \frac{S}{\sqrt{n}}$

Estimated standard deviation of $\bar{X}$.

Example: Heights

$n = 100$ heights: $\bar{X} = 170$ cm, $S = 10$ cm.

Unbiased estimate of population variance: $S^2 = 100$ cm$^2$.

Standard error: $SE = \frac{10}{10} = 1$ cm.

Summary

Estimator	Formula	Unbiased?
Sample mean	$\bar{X} = \frac{1}{n}\sum X_i$	Yes
Sample var (n)	$S_n^2 = \frac{1}{n}\sum(X_i - \bar{X})^2$	No
Sample var (n-1)	$S^2 = \frac{1}{n-1}\sum(X_i - \bar{X})^2$	Yes