Probability Density Functions

From Discrete to Continuous

Discrete random variables take countably many values. But many quantities (time, weight, temperature) can take any value in an interval.

Problem: For continuous $X$, $P(X = x) = 0$ for any specific $x$. What does “exactly 3.214159… seconds” mean?”

We need a different approach.

Probability Density Function

A continuous random variable $X$ has a probability density function (PDF) $f(x)$ such that:

$P(a \leq X \leq b) = \int_a^b f(x) \, dx$

Properties:

1. $f(x) \geq 0$ for all $x$
2. $\int_{-\infty}^{\infty} f(x) \, dx = 1$

Key difference from PMF: $f(x)$ is NOT a probability. It can exceed 1. Only the integral over an interval gives a probability.

Visual Interpretation

The probability $P(a \leq X \leq b)$ is the area under the curve $f(x)$ from $a$ to $b$.

Point Probabilities are Zero

For any specific value $c$:

$P(X = c) = \int_c^c f(x) \, dx = 0$

A continuous random variable never equals an exact value “with probability 1.” This is why we work with intervals.

The CDF

For continuous $X$, the CDF is:

$F(x) = P(X \leq x) = \int_{-\infty}^x f(t) \, dt$

The PDF is the derivative of the CDF:

$f(x) = F'(x)$

Relationship

$P(a \leq X \leq b) = F(b) - F(a)$

This works because:

$\int_a^b f(x) \, dx = F(b) - F(a)$

by the Fundamental Theorem of Calculus.

Example: Uniform on [0, 2]

$f(x) = \begin{cases} \frac{1}{2} & 0 \leq x \leq 2 \\ 0 & \text{otherwise} \end{cases}$

• Total area: $2 \times \frac{1}{2} = 1$
• $P(X \leq 1) = \int_0^1 \frac{1}{2} dx = \frac{1}{2}$
• $f(1) = \frac{1}{2}$, but $P(X = 1) = 0$

Example: Exponential Density

$f(x) = \lambda e^{-\lambda x}, \quad x \geq 0$

$F(x) = 1 - e^{-\lambda x}, \quad x \geq 0$

$P(X > t) = e^{-\lambda t}$

Finding the Normalisation Constant

For $f(x) = c x^2$ on $[0, 1]$, find $c$ such that $f$ is a valid PDF.

$\int_0^1 c x^2 \, dx = 1$

$c \cdot \frac{1}{3} = 1 \Rightarrow c = 3$

The PDF is $f(x) = 3x^2$ for $x \in [0, 1]$.

Properties of PDFs

1. Non-negative: $f(x) \geq 0$
2. Integrates to 1: $\int_{-\infty}^{\infty} f(x) dx = 1$
3. Not probabilities: $f(x)$ may exceed 1
4. Point values: $P(X = x) = 0$
5. Intervals: Use integration for $P(a \leq X \leq b)$

Support

The support of a continuous RV is where $f(x) > 0$:

$\text{supp}(X) = \{x : f(x) > 0\}$

• Uniform on $[0,1]$: support is $[0, 1]$
• Exponential: support is $[0, \infty)$
• Normal: support is $\mathbb{R}$

Summary

Concept	Formula
PDF definition	$P(a \leq X \leq b) = \int_a^b f(x) dx$
CDF	$F(x) = \int_{-\infty}^x f(t) dt$
Derivative relation	$f(x) = F'(x)$
Normalisation	$\int_{-\infty}^{\infty} f(x) dx = 1$
Point probability	$P(X = c) = 0$