Joint Distributions for Discrete Random Variables

Motivation

Many experiments involve multiple random variables simultaneously. We need to describe how they behave together, not just individually.

Examples:

• Height and weight of a person
• Roll of two dice
• Price and quantity sold

Joint PMF

For discrete random variables $X$ and $Y$, the joint PMF is:

$p(x, y) = P(X = x, Y = y)$

Properties

1. $p(x, y) \geq 0$ for all $(x, y)$
2. $\sum_x \sum_y p(x, y) = 1$
3. $P((X, Y) \in A) = \sum_{(x,y) \in A} p(x, y)$

Example: Two Dice

Roll two dice. Let $X$ = first die, $Y$ = second die.

Since independent: $p(x, y) = P(X = x) P(Y = y) = \frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36}$ for all $x, y \in \{1, \ldots, 6\}$.

All 36 outcomes $(x, y)$ have probability $\frac{1}{36}$.

Example: Dependent Variables

Draw two cards without replacement. Let $X$ = number of hearts in first draw, $Y$ = number of hearts in second draw.

$X, Y \in \{0, 1\}$. Total hearts in deck: 13.

• $P(X = 0) = \frac{39}{52}$
• $P(X = 1) = \frac{13}{52}$
• $P(Y = 0, Y = 1 \mid X = 0) = \frac{39}{51}, \frac{12}{51}$

Joint PMF:

$X \backslash Y$	$Y = 0$	$Y = 1$
$X = 0$	$\frac{39}{52} \cdot \frac{38}{51}$	$\frac{39}{52} \cdot \frac{13}{51}$
$X = 1$	$\frac{13}{52} \cdot \frac{39}{51}$	$\frac{13}{52} \cdot \frac{12}{51}$

Computing Joint Probabilities

Product Rule

$p(x, y) = P(Y = y \mid X = x) \cdot P(X = x)$

If independent:

$p(x, y) = P(X = x) \cdot P(Y = y) = p_X(x) \cdot p_Y(y)$

Example

Two items from a batch. Each is defective with probability $p = 0.1$, independent. Let $X$ = number defective in item 1 (Bernoulli), $Y$ = number defective in item 2.

$X \backslash Y$	$Y = 0$	$Y = 1$
$X = 0$	$0.81$	$0.09$
$X = 1$	$0.09$	$0.01$

The entry for $(X=0, Y=0)$: $P(X=0) P(Y=0) = 0.9 \times 0.9 = 0.81$.

Marginal PMFs

From the joint PMF, we can recover the marginal PMF of each variable:

$p_X(x) = \sum_y p(x, y)$

$p_Y(y) = \sum_x p(x, y)$

Intuition: To find the distribution of $X$ alone, “marginalise out” $Y$ by summing over all possible $y$ values.

Example

From the joint table for two defective items:

$P(X = 0) = 0.81 + 0.09 = 0.9$

$P(X = 1) = 0.09 + 0.01 = 0.1$

These are the original marginal probabilities.

Visualisation

Joint PMFs can be visualised as a 3D bar chart or as a 2D heatmap with values in each cell.

Independence from Joint PMF

$X$ and $Y$ are independent if and only if:

$p(x, y) = p_X(x) \cdot p_Y(y) \quad \text{for all } x, y$

Checking Independence

Compute the product of marginals. If it equals the joint PMF everywhere, they are independent.

Counterexample: In cards without replacement:

$P(X = 0, Y = 0) = \frac{39}{52} \cdot \frac{38}{51} = \frac{1482}{2652}$

$P(X = 0) P(Y = 0) = \frac{39}{52} \cdot \frac{39}{52} = \frac{1521}{2704}$

Not equal $\Rightarrow$ not independent. (Drawing a non-heart first affects probability of non-heart second.)

Summary

Concept	Formula
Joint PMF	$p(x,y) = P(X=x, Y=y)$
Marginal of $X$	$p_X(x) = \sum_y p(x,y)$
Independence	$p(x,y) = p_X(x) p_Y(y)$