Sums of Random Variables

Distribution of Sums

If $X$ and $Y$ are random variables, what is the distribution of $X + Y$?

Discrete Case

The PMF of $S = X + Y$:

$P(S = s) = \sum_x P(X = x, Y = s - x)$

If independent:

$P(S = s) = \sum_x P(X = x) P(Y = s - x)$

This is a convolution of the PMFs.

Continuous Case

The PDF of $S = X + Y$:

$f_S(s) = \int_{-\infty}^{\infty} f(x, s - x) \, dx$

If independent:

$f_S(s) = \int_{-\infty}^{\infty} f_X(x) f_Y(s - x) \, dx$

Known Sums

Sum of Independent Binomials (Same $p$)

$X \sim \text{Bin}(n_1, p), \quad Y \sim \text{Bin}(n_2, p)$

$X + Y \sim \text{Bin}(n_1 + n_2, p)$

Intuition: Adding two independent counts from Bernoulli trials with same $p$.

Sum of Independent Poissons

$X \sim \text{Pois}(\lambda_1), \quad Y \sim \text{Pois}(\lambda_2)$

$X + Y \sim \text{Pois}(\lambda_1 + \lambda_2)$

Intuition: Merging two independent event streams.

Sum of Independent Normals

$X \sim N(\mu_1, \sigma_1^2), \quad Y \sim N(\mu_2, \sigma_2^2)$

$X + Y \sim N(\mu_1 + \mu_2, \sigma_1^2 + \sigma_2^2)$

Intuition: Adding Gaussian noise gives Gaussian noise.

Sum of Independent Exponentials (Same Rate)

$X_1, \ldots, X_n \sim \text{Exp}(\lambda) \text{ independent}$

$\sum_{i=1}^n X_i \sim \text{Gamma}(n, \lambda)$

Moment Computations

Regardless of distribution:

Expectation

$E[X + Y] = E[X] + E[Y]$

Always holds (no independence needed).

Variance

$\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y) + 2\text{Cov}(X, Y)$

If independent:

$\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$

Example: Sum of Dice

Roll two fair dice. $X, Y \in \{1, \ldots, 6\}$ uniform, independent.

$S = X + Y$ has range $\{2, 3, \ldots, 12\}$.

$P(S = 7) = \sum_{x=1}^{6} P(X = x) P(Y = 7 - x) = \sum_{x=1}^{6} \frac{1}{6} \cdot \frac{1}{6} = \frac{6}{36} = \frac{1}{6}$

$P(S = 2) = P(X = 1) P(Y = 1) = \frac{1}{36}$

Convolution Visualisation

The sum distribution is obtained by “sliding” one distribution past the other and multiplying.

Linear Combinations

For constants $a$ and $b$:

$E[aX + bY] = aE[X] + bE[Y]$

$\text{Var}(aX + bY) = a^2\text{Var}(X) + b^2\text{Var}(Y) + 2ab\text{Cov}(X, Y)$

If independent:

$\text{Var}(aX + bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)$

Sum of I.I.D. Variables

Let $X_1, \ldots, X_n$ be i.i.d. with mean $\mu$ and variance $\sigma^2$.

Define $S_n = \sum_{i=1}^n X_i$ and $\bar{X} = S_n / n$.

$E[S_n] = n\mu, \quad \text{Var}(S_n) = n\sigma^2$

$E[\bar{X}] = \mu, \quad \text{Var}(\bar{X}) = \frac{\sigma^2}{n}$

Averaging reduces variance proportionally to $n$.

Summary

Variables	Sum Distribution	Sum Mean	Sum Variance
Binomial (same $p$)	Binomial	$(n_1+n_2)p$	$(n_1+n_2)p(1-p)$
Poisson	Poisson	$\lambda_1 + \lambda_2$	$\lambda_1 + \lambda_2$
Normal	Normal	$\mu_1 + \mu_2$	$\sigma_1^2 + \sigma_2^2$
General	Convolution	$E[X] + E[Y]$	Always additive if independent