Probability Axioms

The Need for Axioms

Intuitively, probability measures “how likely” an event is. But intuition can mislead. The axioms provide a rigorous foundation, ensuring all probability calculations are consistent.

Kolmogorov’s Axioms

Let $\Omega$ be a sample space and $\mathcal{F}$ an event space. A probability measure $P: \mathcal{F} \to [0,1]$ satisfies:

Axiom 1: Non-negativity

$P(E) \geq 0 \text{ for all events } E$

Probabilities are never negative.

Axiom 2: Normalisation

$P(\Omega) = 1$

The certain event has probability 1. It always occurs.

Axiom 3: Additivity

For mutually exclusive events $E_1, E_2, \ldots$:

$P\left(\bigcup_{i=1}^{\infty} E_i\right) = \sum_{i=1}^{\infty} P(E_i)$

The probability of the union of disjoint events equals the sum of their probabilities.

For finite collections of mutually exclusive events:

$P(E_1 \cup E_2 \cup \cdots \cup E_n) = P(E_1) + P(E_2) + \cdots + P(E_n)$

when all $E_i \cap E_j = \emptyset$ for $i \neq j$.

Consequences of the Axioms

Probability of the Empty Set

$P(\emptyset) = 0$

Proof: $\Omega$ and $\emptyset$ are mutually exclusive with $\Omega \cup \emptyset = \Omega$. By Axiom 3: $P(\Omega) = P(\Omega) + P(\emptyset)$, so $P(\emptyset) = 0$.

Probability of Complement

$P(E^c) = 1 - P(E)$

Proof: $E$ and $E^c$ partition $\Omega$: $E \cup E^c = \Omega$ and $E \cap E^c = \emptyset$. By additivity: $P(\Omega) = P(E) + P(E^c) = 1$, hence $P(E^c) = 1 - P(E)$.

Probability Bounds

$0 \leq P(E) \leq 1$

The lower bound is Axiom 1. The upper bound follows from $P(E) = 1 - P(E^c)$ and $P(E^c) \geq 0$.

Monotonicity

If $E \subseteq F$, then $P(E) \leq P(F)$.

Proof: Write $F = E \cup (F \setminus E)$ where these are disjoint. Then $P(F) = P(E) + P(F \setminus E) \geq P(E)$.

Inclusion-Exclusion for Two Events

$P(E \cup F) = P(E) + P(F) - P(E \cap F)$

Why subtract $P(E \cap F)$? The sum $P(E) + P(F)$ counts outcomes in $E \cap F$ twice. We correct by subtracting once.

Inclusion-Exclusion for Three Events

$P(E \cup F \cup G) = P(E) + P(F) + P(G) - P(E \cap F) - P(E \cap G) - P(F \cap G) + P(E \cap F \cap G)$

The pattern: add singletons, subtract pairwise intersections, add triple intersections.

General Inclusion-Exclusion

For $n$ events $E_1, E_2, \ldots, E_n$:

$P\left(\bigcup_{i=1}^{n} E_i\right) = \sum_{i} P(E_i) - \sum_{i<j} P(E_i \cap E_j) + \sum_{i<j<k} P(E_i \cap E_j \cap E_k) - \cdots$

Alternating signs: add, subtract, add, subtract, and so on.

Example: Card Drawing

Draw a card from a standard deck. Let:

• $E$ = card is a heart
• $F$ = card is an ace

$P(E \cup F) = P(E) + P(F) - P(E \cap F) = \frac{13}{52} + \frac{4}{52} - \frac{1}{52} = \frac{16}{52} = \frac{4}{13}$

Countable vs Uncountable

The axioms extend to countable unions, but not uncountable ones. For uncountable sample spaces (like $\mathbb{R}$), we cannot assign positive probability to every singleton while maintaining total probability 1.

This motivates using probability density functions for continuous random variables rather than assigning probability to individual points.

Constructing Probability Measures

For finite sample spaces with equally likely outcomes:

$P(E) = \frac{|E|}{|\Omega|} = \frac{\text{number of outcomes in } E}{\text{total number of outcomes}}$

This construction satisfies all axioms.

Example (fair die): $\Omega = \{1,2,3,4,5,6\}$ with each outcome having probability $1/6$. Event “even” has probability $|\{2,4,6\}|/6 = 3/6 = 1/2$.

Summary

Property	Formula
Certain event	$P(\Omega) = 1$
Impossible event	$P(\emptyset) = 0$
Complement	$P(E^c) = 1 - P(E)$
Union (2 events)	$P(E \cup F) = P(E) + P(F) - P(E \cap F)$
Monotonicity	$E \subseteq F \Rightarrow P(E) \leq P(F)$
Bounds	$0 \leq P(E) \leq 1$