Conditional Distributions

Motivation

Conditional distributions describe the distribution of one random variable given that another has a specific value. This is fundamental to regression, prediction, and Bayesian inference.

Discrete Case

Conditional PMF

$p_{Y|X}(y \mid x) = P(Y = y \mid X = x) = \frac{p(x, y)}{p_X(x)}$

provided $p_X(x) > 0$.

Properties

For fixed $x$ with $p_X(x) > 0$, $p_{Y|X}(y \mid x)$ is a valid PMF:

• $p_{Y|X}(y \mid x) \geq 0$
• $\sum_y p_{Y|X}(y \mid x) = \frac{\sum_y p(x,y)}{p_X(x)} = \frac{p_X(x)}{p_X(x)} = 1$

Example: Dice Game

Roll two dice: $X$ = first die, $Y$ = second die. Joint: $p(x,y) = \frac{1}{36}$.

Given $X + Y = 7$, what is the distribution of $X$?

Let $S = X + Y$. We want $p_{X|S}(x \mid 7)$.

Possible pairs with sum 7: $(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)$.

$P(S = 7) = \frac{6}{36} = \frac{1}{6}$

$p_{X|S}(x \mid 7) = P(X = x \mid S = 7) = \frac{1}{6} \text{ for } x = 1, \ldots, 6$

Given the sum is 7, the first die is equally likely to be any value 1 through 6.

Continuous Case

Conditional PDF

$f_{Y|X}(y \mid x) = \frac{f(x, y)}{f_X(x)}$

provided $f_X(x) > 0$.

Interpretation

$f_{Y|X}(y \mid x)$ gives the density of $Y$ given that $X$ takes value $x$.

Example

$f(x, y) = 2e^{-x} e^{-2y}$ for $x, y > 0$.

We found $f_X(x) = e^{-x}$.

$f_{Y|X}(y \mid x) = \frac{2e^{-x} e^{-2y}}{e^{-x}} = 2e^{-2y}$

The conditional distribution of $Y$ given $X = x$ is $\text{Exp}(2)$, independent of $x$. This confirms $X$ and $Y$ are independent.

Conditional Expectation

The conditional expectation of $Y$ given $X = x$:

$E[Y \mid X = x] = \sum_y y \cdot p_{Y|X}(y \mid x) \quad \text{(discrete)}$

$E[Y \mid X = x] = \int y \cdot f_{Y|X}(y \mid x) \, dy \quad \text{(continuous)}$

Example

From the example above: $E[Y \mid X = x] = E[\text{Exp}(2)] = \frac{1}{2}$.

The conditional expectation is constant (doesn’t depend on $x$) when $X$ and $Y$ are independent.

Law of Total Expectation

$E[Y] = E[E[Y \mid X]]$

More explicitly:

$E[Y] = \sum_x E[Y \mid X = x] \cdot p_X(x) \quad \text{(discrete)}$

$E[Y] = \int E[Y \mid X = x] \cdot f_X(x) \, dx \quad \text{(continuous)}$

Application

This law lets us compute $E[Y]$ by conditioning on $X$, often simplifying complex problems.

Example

$X \sim \text{Pois}(\lambda)$ and given $X = n$, $Y \sim \text{Bin}(n, p)$.

Find $E[Y]$.

By law of total expectation:

$E[Y] = E[E[Y \mid X]] = E[X \cdot p] = p E[X] = p \lambda$

Conditional Variance

$\text{Var}(Y \mid X = x) = E[Y^2 \mid X = x] - (E[Y \mid X = x])^2$

Summary

Concept	Discrete	Continuous
Conditional PMF/PDF	$\frac{p(x,y)}{p_X(x)}$	$\frac{f(x,y)}{f_X(x)}$
Conditional expectation	$\sum_y y \cdot p_{Y	X}(y
Law of total expectation	$E[E[Y	X]]$