Conditional Probability

The Intuition

New information changes probabilities. If we roll a die and learn the result is even, the probability of rolling 6 changes from $\frac{1}{6}$ to $\frac{1}{3}$.

Conditional probability formalises this: given that some event occurred, what is the probability of another?

Definition

The conditional probability of $E$ given $F$ (with $P(F) > 0$) is:

$P(E \mid F) = \frac{P(E \cap F)}{P(F)}$

Interpretation: We restrict our attention to outcomes where $F$ occurs. The probability of $E$ is then the relative proportion of these outcomes that also lie in $E$.

Equivalent Forms

Rearranging:

$P(E \cap F) = P(E \mid F) \cdot P(F)$

This is useful for computing joint probabilities: decompose into conditional times marginal.

Similarly:

$P(E \cap F) = P(F \mid E) \cdot P(E)$

Properties

Conditional probability satisfies all probability axioms:

• $P(E \mid F) \geq 0$ for all $E$
• $P(\Omega \mid F) = 1$
• $P(\bigcup_{i} E_i \mid F) = \sum_{i} P(E_i \mid F)$ for disjoint $E_i$

Conditional on $F$, we have a valid probability measure on the restricted sample space $F$.

Examples

Example 1: Dice

Roll two dice. Given that the sum is 7, what is the probability the first die shows 4?

Let $E$ = “first die is 4”, $F$ = “sum is 7”. The outcomes in $F$ are: $\{(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)\}$.

Out of these 6 equally likely outcomes, only $(4,3)$ has first die 4.

$P(E \mid F) = \frac{1}{6}$

Compare: $P(E) = \frac{1}{6}$ (unconditionally). In this case, knowing the sum is 7 doesn’t change the probability.

Example 2: Cards

Draw two cards without replacement from a standard deck. Given the first card is a heart, what is the probability the second card is a heart?

Let $E$ = “second card is a heart”, $F$ = “first card is a heart”.

After drawing a heart first, 12 hearts remain out of 51 cards.

$P(E \mid F) = \frac{12}{51} = \frac{4}{17}$

Compare: $P(E) = \frac{1}{2}$ (unconditionally, by symmetry).

Example 3: Medical Tests

A test for a disease has 99% sensitivity (true positive rate) and 95% specificity (true negative rate). Disease prevalence is 1%. What is the probability of having the disease given a positive test?

Let $D$ = “has disease”, $T$ = “tests positive”.

We want $P(D \mid T)$.

$P(D \mid T) = \frac{P(T \mid D) \cdot P(D)}{P(T)}$

where $P(T) = P(T \mid D) P(D) + P(T \mid D^c) P(D^c) = 0.99 \times 0.01 + 0.05 \times 0.99 = 0.0099 + 0.0495 = 0.0594$

$P(D \mid T) = \frac{0.99 \times 0.01}{0.0594} \approx \frac{0.0099}{0.0594} \approx 16.7\%$

Despite a positive test, there’s only a 16.7% chance of having the disease, due to low prevalence.

The Chain Rule

For multiple events:

$P(E_1 \cap E_2 \cap \cdots \cap E_n) = P(E_1) \cdot P(E_2 \mid E_1) \cdot P(E_3 \mid E_1 \cap E_2) \cdots P(E_n \mid E_1 \cap \cdots \cap E_{n-1})$

This decomposes complex joint probabilities into simpler conditional probabilities.

Example: Drawing Without Replacement

Draw 3 cards from 52 without replacement. Probability all are spades:

$P(\text{all spades}) = \frac{13}{52} \cdot \frac{12}{51} \cdot \frac{11}{50} = \frac{13 \cdot 12 \cdot 11}{52 \cdot 51 \cdot 50}$

Common Misconceptions

Confusion with Inverse

$P(E \mid F) \neq P(F \mid E)$ in general.

In the medical test example:

• $P(T \mid D) = 0.99$ (sensitivity)
• $P(D \mid T) \approx 0.167$ (much smaller)

The Prosecutor’s Fallacy

Confusing $P(\text{evidence} \mid \text{innocent})$ with $P(\text{innocent} \mid \text{evidence})$. A small false positive rate doesn’t guarantee a small probability of innocence.

Conditioning as Restriction

Conditional probability can be understood as:

1. Restrict the sample space to $F$
2. Renormalise probabilities so they sum to 1

For equally likely outcomes:

$P(E \mid F) = \frac{|E \cap F|}{|F|}$

Count outcomes in $E \cap F$, divide by outcomes in $F$.

Summary

Concept	Formula
Definition	$P(E \mid F) = \frac{P(E \cap F)}{P(F)}$
Product rule	$P(E \cap F) = P(E \mid F) P(F)$
Chain rule	$P(E_1 \cap \cdots \cap E_n) = \prod_{i=1}^{n} P(E_i \mid E_1 \cap \cdots \cap E_{i-1})$