Linearity of Expectation

The Linearilty Property

Expectation preserves linearity. For any random variables $X$ and $Y$, and constants $a$ and $b$:

$E[aX + b] = aE[X] + b$

$E[X + Y] = E[X] + E[Y]$

$E[aX + bY] = aE[X] + bE[Y]$

Crucially: $X$ and $Y$ need NOT be independent. Linearity holds regardless of dependence.

Proof for Constants

$E[aX + b] = \sum_x (ax + b) p(x) = a\sum_x x p(x) + b\sum_x p(x) = aE[X] + b \cdot 1 = aE[X] + b$

Proof for Sum of Two Variables

For discrete $X$ and $Y$:

$E[X + Y] = \sum_x \sum_y (x + y) p(x, y) = \sum_x \sum_y x p(x, y) + \sum_x \sum_y y p(x, y)$

$= \sum_x x \sum_y p(x, y) + \sum_y y \sum_x p(x, y) = \sum_x x p_X(x) + \sum_y y p_Y(y) = E[X] + E[Y]$

Why This Matters

Linearity is perhaps the most useful property of expectation. Many problems that would otherwise be intractable become straightforward.

Example: Transformed Die

Roll a fair die. Let $Y = 3X + 1$.

We could compute $E[Y]$ directly: values are $\{4, 7, 10, 13, 16, 19\}$ with equal probability, giving $E[Y] = \frac{4+7+10+13+16+19}{6} = \frac{69}{6} = 11.5$.

But using linearity:

$E[Y] = E[3X + 1] = 3E[X] + 1 = 3(3.5) + 1 = 11.5$

Indicator Trick

Many problems become easy by expressing the quantity of interest as a sum of indicators.

Example: Matching Problem

$n$ people throw hats into a pile, then each picks one randomly. Expected number who get their own hat?

Let $I_i$ = indicator that person $i$ gets their own hat. Then:

$X = \sum_{i=1}^{n} I_i$

is the number of matches.

$E[I_i] = P(\text{person } i \text{ gets own hat}) = \frac{1}{n}$

By linearity:

$E[X] = \sum_{i=1}^{n} E[I_i] = n \cdot \frac{1}{n} = 1$

Regardless of $n$, the expected number of matches is 1.

Example: Expected Number of Aces

Draw 5 cards from a standard deck. Expected number of aces?

Let $I_i$ = indicator that card $i$ is an ace.

$E[I_i] = P(\text{card } i \text{ is ace}) = \frac{4}{52} = \frac{1}{13}$

$E[\text{number of aces}] = \sum_{i=1}^{5} E[I_i] = 5 \cdot \frac{1}{13} = \frac{5}{13}$

Example: Email Spam

100 emails, each independently has probability 0.4 of being spam. Expected number of spam emails?

Let $I_i$ indicate email $i$ is spam.

$E[I_i] = 0.4$

$E[\text{spam count}] = 100 \times 0.4 = 40$

Linearity Extends to $n$ Variables

By induction:

$E\left[\sum_{i=1}^{n} X_i\right] = \sum_{i=1}^{n} E[X_i]$

Regardless of dependence among the $X_i$.

Caution: Products and Functions

Linearity does NOT extend to products:

$E[XY] \neq E[X] E[Y] \text{ in general}$

Equality holds if $X$ and $Y$ are independent.

Linearity does NOT extend to arbitrary functions:

$E[g(X)] \neq g(E[X]) \text{ in general}$

Counterexample: $E[X^2] \neq E[X]^2$ for a die roll.

Application: Sums of Identical Distributions

If $X_1, X_2, \ldots, X_n$ are i.i.d. with mean $\mu$:

$E[X_1 + X_2 + \cdots + X_n] = n\mu$

This is the foundation of the Law of Large Numbers.

Probabilistic Method

Linearity enables the probabilistic method: to prove existence of an object with property $P$, show $E[X] > 0$ for some non-negative random variable $X$, where $X > 0$ implies property $P$. Then there must exist some outcome with $X > 0$.

Summary

Property	Formula
Scaling and shift	$E[aX + b] = aE[X] + b$
Additivity	$E[X + Y] = E[X] + E[Y]$
General	$E[\sum_i X_i] = \sum_i E[X_i]$
No independence needed	Holds for dependent variables