Examinable Formulas and Results

Foundational Formulas

Probability Axioms

• $P(\Omega) = 1$, $P(\emptyset) = 0$
• $P(A^c) = 1 - P(A)$
• $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Conditional Probability

• $P(A \mid B) = \frac{P(A \cap B)}{P(B)}$ for $P(B) > 0$
• Bayes’ theorem: $P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$
• Law of total probability: $P(A) = \sum_i P(A \mid F_i) P(F_i)$ for partition $\{F_i\}$

Independence

• Events: $P(A \cap B) = P(A) P(B)$
• Random variables: $F(x, y) = F_X(x) F_Y(y)$

Discrete Distributions

Distribution	PMF	$E[X]$	$\text{Var}(X)$
Bernoulli($p$)	$p^x(1-p)^{1-x}$	$p$	$p(1-p)$
Binomial($n,p$)	$\binom{n}{k}p^k(1-p)^{n-k}$	$np$	$np(1-p)$
Poisson($\lambda$)	$\frac{\lambda^k e^{-\lambda}}{k!}$	$\lambda$	$\lambda$
Geo($p$)	$(1-p)^{k-1}p$	$\frac{1}{p}$	$\frac{1-p}{p^2}$
NegBin($r,p$)	$\binom{n-1}{r-1}(1-p)^{n-r}p^r$	$\frac{r}{p}$	$\frac{r(1-p)}{p^2}$
Hyp($N,n,m$)	$\frac{\binom{m}{i}\binom{N-m}{n-i}}{\binom{N}{n}}$	$\frac{nm}{N}$	$\frac{nm}{N}(1-\frac{m}{N})\frac{N-n}{N-1}$
Unif$\{1,\ldots,N\}$	$\frac{1}{N}$	$\frac{N+1}{2}$	$\frac{N^2-1}{12}$

Continuous Distributions

Distribution	PDF	$E[X]$	$\text{Var}(X)$
Unif$[a,b]$	$\frac{1}{b-a}$	$\frac{a+b}{2}$	$\frac{(b-a)^2}{12}$
Exp($\lambda$)	$\lambda e^{-\lambda x}$	$\frac{1}{\lambda}$	$\frac{1}{\lambda^2}$
$N(\mu, \sigma^2)$	$\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$	$\mu$	$\sigma^2$

Distribution Properties

Memoryless Distributions

• Discrete: Geometric
• Continuous: Exponential

Sum of Independent Variables

• Bin($n_1, p$) + Bin($n_2, p$) = Bin($n_1 + n_2, p$)
• Pois($\lambda_1$) + Pois($\lambda_2$) = Pois($\lambda_1 + \lambda_2$)
• $N(\mu_1, \sigma_1^2)$ + $N(\mu_2, \sigma_2^2)$ = $N(\mu_1 + \mu_2, \sigma_1^2 + \sigma_2^2)$
• $n \times$ Exp($\lambda$) = Gamma($n, \lambda$)

Poisson Limit

Bin($n, p$) $\to$ Pois($\lambda$) as $n \to \infty$, $np \to \lambda$

Inequalities

Markov’s Inequality

For $X \geq 0$: $P(X \geq a) \leq \frac{E[X]}{a}$

Chebyshev’s Inequality

$P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}$

Jensen’s Inequality

For convex $g$: $E[g(X)] \geq g(E[X])$

For concave $g$: $E[g(X)] \leq g(E[X])$

Limit Theorems

Weak Law of Large Numbers

$\bar{X}_n \xrightarrow{P} \mu$

Strong Law of Large Numbers

$P(\bar{X}_n \to \mu) = 1$

Central Limit Theorem

$\sqrt{n}(\bar{X}_n - \mu) \xrightarrow{d} N(0, \sigma^2)$

Equivalently: $\frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \xrightarrow{d} N(0, 1)$

Joint Distributions

Covariance

$\text{Cov}(X, Y) = E[XY] - E[X]E[Y]$

Correlation

$\rho_{X,Y} = \frac{\text{Cov}(X,Y)}{\sigma_X \sigma_Y}$

Variance of Sum

$\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y) + 2\text{Cov}(X, Y)$

If independent: $\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y)$

Estimation

Bias

$\text{Bias}(\hat{\theta}) = E[\hat{\theta}] - \theta$

MSE

$\text{MSE}(\hat{\theta}) = E[(\hat{\theta} - \theta)^2] = \text{Var}(\hat{\theta}) + \text{Bias}^2$

Sample Mean

$\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$, unbiased for $\mu$

Sample Variance

$S^2 = \frac{1}{n-1}\sum_{i=1}^n(X_i - \bar{X})^2$, unbiased for $\sigma^2$

Confidence Interval (mean, known $\sigma$)

$\bar{X} \pm z_{\alpha/2} \frac{\sigma}{\sqrt{n}}$

Confidence Interval (mean, unknown $\sigma$)

$\bar{X} \pm t_{n-1, \alpha/2} \frac{S}{\sqrt{n}}$

Standard Normal Values

$z$	$\Phi(z)$
1.645	0.95
1.96	0.975
2.576	0.995

Empirical Rule

• $\mu \pm \sigma$: ~68%
• $\mu \pm 2\sigma$: ~95%
• $\mu \pm 3\sigma$: ~99.7%