Expectation

Definition

The expected value (or expectation, mean) of a discrete random variable $X$ is:

$E[X] = \sum_{x} x \cdot p(x)$

where the sum is over all values in the support of $X$.

Interpretation: The long-run average if we repeatedly sample from $X$.

Intuition

The expectation is a weighted average of possible values, weighted by their probabilities. Values that occur more often contribute more to the average.

Example: Fair Die

$E[X] = \sum_{i=1}^{6} i \cdot \frac{1}{6} = \frac{1+2+3+4+5+6}{6} = \frac{21}{6} = 3.5$

Note: 3.5 is not a possible outcome. The expectation need not be a value $X$ can take.

Example: Biased Coin

Let $X$ = number of heads in one flip, with $P(X=1) = p$.

$E[X] = 0 \cdot (1-p) + 1 \cdot p = p$

The expected number of heads equals the success probability.

Example: Lottery

A lottery ticket costs £2. With probability $10^{-6}$, it wins £1{,}000{,}000. Otherwise, it wins £0.

$E[\text{winnings}] = 10^{-6} \times 10^6 + (1 - 10^{-6}) \times 0 = 1$

Expected winnings: £1. Expected profit: £1 - £2 = -£1 (a loss).

Does Expectation Always Exist?

Not always. The expectation requires:

$\sum_{x} |x| \cdot p(x) < \infty$

If $X$ can take values with $|x|$ so large that the weighted sum diverges, expectation does not exist.

Example: The Cauchy distribution has no finite expectation: its tails are too heavy.

Properties

1. Constant

$E[c] = c$

for any constant $c$.

2. Monotonicity

If $X \leq Y$ (with probability 1), then $E[X] \leq E[Y]$.

3. Non-negativity Preservation

If $X \geq 0$ (with probability 1), then $E[X] \geq 0$.

Interpretations

Physical: Centre of Mass

Place masses $p(x)$ at positions $x$ on a line. The expectation is the centre of mass: the balance point.

Financial: Fair Price

If you pay $E[X]$ to play a game with payout $X$, you break even on average. This is the “fair price.”

Statistical: Sample Mean

By the Law of Large Numbers, the sample mean $\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i$ converges to $E[X]$ as $n \to \infty$.

Expectation of Functions

For a function $g$:

$E[g(X)] = \sum_{x} g(x) \cdot p(x)$

Note: In general, $E[g(X)] \neq g(E[X])$. The expectation of a function is not the function of the expectation.

Example: Squared Die

$E[X^2] = \sum_{i=1}^{6} i^2 \cdot \frac{1}{6} = \frac{1+4+9+16+25+36}{6} = \frac{91}{6} \approx 15.17$

Compare: $E[X]^2 = 3.5^2 = 12.25$. Not equal.

Jensen’s Inequality

For a convex function $g$:

$E[g(X)] \geq g(E[X])$

For a concave function $g$:

$E[g(X)] \leq g(E[X])$

Example: $g(x) = x^2$ is convex, so $E[X^2] \geq E[X]^2$.

Summary

Concept	Formula
Definition	$E[X] = \sum_x x \cdot p(x)$
Constant	$E[c] = c$
Function	$E[g(X)] = \sum_x g(x) p(x)$
Centre of mass	Physical interpretation