Bernoulli and Binomial Distributions

Bernoulli Distribution

Definition

A random variable $X$ follows the Bernoulli distribution with parameter $p$ if:

$P(X = 1) = p, \quad P(X = 0) = 1 - p$

Write: $X \sim \text{Bern}(p)$.

Context: Models a single trial with two outcomes: success (1) with probability $p$, or failure (0) with probability $1-p$.

PMF

$p(x) = \begin{cases} 1 - p & x = 0 \\ p & x = 1 \\ 0 & \text{otherwise} \end{cases}$

Or compactly: $p(x) = p^x(1-p)^{1-x}$ for $x \in \{0,1\}$.

Expectation

$E[X] = 0 \cdot (1-p) + 1 \cdot p = p$

Variance

$E[X^2] = 0^2 \cdot (1-p) + 1^2 \cdot p = p$

$\text{Var}(X) = E[X^2] - E[X]^2 = p - p^2 = p(1-p)$

Examples

• Coin flip: $X \sim \text{Bern}(0.5)$ for a fair coin
• Defective item: $X \sim \text{Bern}(0.02)$ if 2% are defective
• Pass/fail: $X \sim \text{Bern}(p)$ where $p$ is pass rate

Binomial Distribution

Definition

$X \sim \text{Bin}(n, p)$ if $X$ counts the number of successes in $n$ independent Bernoulli trials, each with success probability $p$.

PMF

$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}, \quad k = 0, 1, \ldots, n$

Why this formula?

• Choose which $k$ of $n$ trials are successes: $\binom{n}{k}$ ways
• Each such arrangement has probability $p^k (1-p)^{n-k}$

Range

$X \in \{0, 1, 2, \ldots, n\}$

Expectation

Let $X = X_1 + X_2 + \cdots + X_n$ where $X_i \sim \text{Bern}(p)$ are independent.

$E[X] = E[X_1] + \cdots + E[X_n] = np$

Intuition: If each trial succeeds with probability $p$, expect $np$ successes in $n$ trials.

Variance

Since the $X_i$ are independent:

$\text{Var}(X) = \text{Var}(X_1) + \cdots + \text{Var}(X_n) = n \cdot p(1-p)$

Shape

• For small $p$: distribution is skewed right (concentration near 0)
• For $p = 0.5$: symmetric around $n/2$
• For large $p$: distribution is skewed left (concentration near $n$)

Example: Coin Flips

Flip a fair coin 100 times. Expected number of heads?

$X \sim \text{Bin}(100, 0.5)$

$E[X] = 100 \times 0.5 = 50$ $\text{Var}(X) = 100 \times 0.5 \times 0.5 = 25$ $\sigma_X = 5$

Example: Quality Control

A factory produces items with 2% defect rate. In a sample of 200, what is the probability of exactly 3 defects?

$X \sim \text{Bin}(200, 0.02)$

$P(X = 3) = \binom{200}{3} (0.02)^3 (0.98)^{197}$

Compute: $\binom{200}{3} = \frac{200 \cdot 199 \cdot 198}{6} = 1{,}313{,}400$

$P(X = 3) \approx 1{,}313{,}400 \times 0.000008 \times 0.018 \approx 0.19$

Relation Between Bernoulli and Binomial

The sum of $n$ independent $\text{Bern}(p)$ variables is $\text{Bin}(n, p)$:

$\sum_{i=1}^{n} \text{Bern}(p) = \text{Bin}(n, p)$

This gives us the expectation and variance immediately by linearity and independence.

Binomial as Sampling

The binomial arises from sampling with replacement or from a large population where the success probability stays constant.

If sampling is without replacement from a small population, use the hypergeometric distribution instead.

Normal Approximation

For large $n$, $\text{Bin}(n, p)$ is approximately normal:

$X \approx N(np, np(1-p))$

Rule of thumb: use when $np \geq 5$ and $n(1-p) \geq 5$.

Summary Table

Distribution	PMF	Mean	Variance
Bernoulli($p$)	$p^1(1-p)^{1-x}$	$p$	$p(1-p)$
Binomial($n,p$)	$\binom{n}{k}p^k(1-p)^{n-k}$	$np$	$np(1-p)$