LOTUS and Moments

LOTUS: Law of the Unconscious Statistician

When computing $E[g(X)]$, you might think we need the distribution of $Y = g(X)$ first. LOTUS says no: we can compute directly using the distribution of $X$.

Statement: For a discrete random variable $X$:

$E[g(X)] = \sum_{x} g(x) \cdot p(x)$

For a continuous random variable $X$:

$E[g(X)] = \int_{-\infty}^{\infty} g(x) f(x) \, dx$

Why “unconscious”? Many people apply this “unconsciously” without realising it’s a theorem requiring proof.

Proof (Discrete Case)

Let $Y = g(X)$. By definition:

$E[Y] = \sum_y y \cdot P(Y = y) = \sum_y y \cdot P(g(X) = y)$

But $P(g(X) = y) = \sum_{x: g(x) = y} p(x)$. So:

$E[Y] = \sum_y y \sum_{x: g(x) = y} p(x) = \sum_y \sum_{x: g(x) = y} y \cdot p(x) = \sum_y \sum_{x: g(x) = y} g(x) \cdot p(x)$

Each $x$ appears exactly once (grouped by the value $g(x)$), giving:

$E[Y] = \sum_x g(x) \cdot p(x)$

Why LOTUS Matters

Instead of finding the PMF/PDF of $Y = g(X)$, we compute directly. This is often much easier.

Example: Squared Die

$X$ = fair die roll. Find $E[X^2]$.

Without LOTUS: Find distribution of $X^2$ (values $\{1, 4, 9, 16, 25, 36\}$), then compute expectation.

With LOTUS:

$E[X^2] = \sum_{i=1}^{6} i^2 \cdot \frac{1}{6} = \frac{1 + 4 + 9 + 16 + 25 + 36}{6} = \frac{91}{6}$

Moments

nth Moment

The $n$th moment of $X$ is:

$E[X^n] = \sum_x x^n \cdot p(x)$

The first moment is the expectation: $E[X]$.

Central Moments

The $n$th central moment is:

$E[(X - E[X])^n]$

The second central moment is the variance.

Common Functions

Linear Function

$E[aX + b] = aE[X] + b$

This follows from linearity, not specifically LOTUS.

Positive Powers

$E[X^n] = \sum_x x^n p(x)$

Used for computing variance and higher moments.

Absolute Value

$E[|X|] = \sum_x |x| p(x)$

The expectation of $|X|$ is called the mean absolute deviation from zero.

Example: $E[2^X]$

If $X$ ~ Binomial($n$, $p$), find $E[2^X]$.

By LOTUS:

$E[2^X] = \sum_{k=0}^{n} 2^k \binom{n}{k} p^k (1-p)^{n-k} = \sum_{k=0}^{n} \binom{n}{k} (2p)^k (1-p)^{n-k}$

This is the binomial expansion of $(2p + (1-p))^n = (1 + p)^n$.

$E[2^X] = (1 + p)^n$

Probability Generating Functions

For a non-negative integer-valued random variable $X$, the probability generating function is:

$G_X(t) = E[t^X] = \sum_{k=0}^{\infty} t^k P(X = k)$

This is LOTUS with $g(x) = t^x$.

Key property: $P(X = k) = \frac{G_X^{(k)}(0)}{k!}$ (the $k$th derivative at 0).

Variance via LOTUS

$\text{Var}(X) = E[X^2] - E[X]^2$

LOTUS computes $E[X^2]$ directly.

For a fair die:

$\text{Var}(X) = E[X^2] - E[X]^2 = \frac{91}{6} - \left(\frac{7}{2}\right)^2 = \frac{91}{6} - \frac{49}{4} = \frac{182 - 147}{12} = \frac{35}{12}$

Summary

Concept	Formula
LOTUS (discrete)	$E[g(X)] = \sum_x g(x) p(x)$
LOTUS (continuous)	$E[g(X)] = \int g(x) f(x) dx$
$n$th moment	$E[X^n]$
$n$th central moment	$E[(X - E[X])^n]$
PGF	$G(t) = E[t^X]$