Eigenvalues and Eigenvectors

Definition

For a square matrix $A$, an eigenvector $\mathbf{v} \neq \mathbf{0}$ satisfies:

$A\mathbf{v} = \lambda\mathbf{v}$

where $\lambda$ is the eigenvalue.

Key: Applying $A$ to its eigenvector just scales it by $\lambda$.

---

Characteristic Polynomial

Rearrange: $(A - \lambda I)\mathbf{v} = \mathbf{0}$

Non-trivial solutions exist when $\det(A - \lambda I) = 0$.

This gives the characteristic polynomial:

$P_A(\lambda) = \det(A - \lambda I) = 0$

An $n \times n$ matrix has exactly $n$ eigenvalues (counting multiplicities).

---

Finding Eigenvalues and Eigenvectors

Step 1: Solve $P_A(\lambda) = 0$ for eigenvalues.

Step 2: For each $\lambda$, solve $(A - \lambda I)\mathbf{v} = \mathbf{0}$ for $\mathbf{v}$.

---

Example

Find eigenvalues and eigenvectors of $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$.

Characteristic polynomial:

$\det(A - \lambda I) = \det\begin{pmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{pmatrix}$

$= (2-\lambda)^2 - 1 = \lambda^2 - 4\lambda + 3 = (\lambda - 1)(\lambda - 3)$

Eigenvalues: $\lambda_1 = 1$, $\lambda_2 = 3$.

Eigenvector for $\lambda = 1$:

$(A - I)\mathbf{v} = \begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}\mathbf{v} = \mathbf{0}$

$\Rightarrow v_1 + v_2 = 0$

$\mathbf{v}_1 = \begin{pmatrix} 1 \\ -1 \end{pmatrix}$

Eigenvector for $\lambda = 3$:

$(A - 3I)\mathbf{v} = \begin{pmatrix} -1 & 1 \\ 1 & -1 \end{pmatrix}\mathbf{v} = \mathbf{0}$

$\Rightarrow v_1 = v_2$

$\mathbf{v}_2 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}$

---

Properties

Property	Formula
Sum of eigenvalues	$\lambda_1 + \cdots + \lambda_n = \text{Tr}(A)$
Product of eigenvalues	$\lambda_1 \cdots \lambda_n = \det(A)$
Inverse eigenvalues	If $A\mathbf{v} = \lambda\mathbf{v}$, then $A^{-1}\mathbf{v} = \lambda^{-1}\mathbf{v}$
Polynomial	$A^k\mathbf{v} = \lambda^k\mathbf{v}$

---

Real Symmetric Matrices

For real symmetric $A$ ($A = A^T$):

1. All eigenvalues are real
2. Eigenvectors for different eigenvalues are orthogonal
3. Can always find $n$ orthonormal eigenvectors

---

Diagonalisation

If $A$ has $n$ linearly independent eigenvectors $\mathbf{v}_1, \ldots, \mathbf{v}_n$:

$A = S\Lambda S^{-1}$

where $S$ has eigenvectors as columns, and $\Lambda = \text{diag}(\lambda_1, \ldots, \lambda_n)$.

For symmetric matrices: $S$ can be orthogonal, so $S^{-1} = S^T$.

---

Diagonalisation Example

For $A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}$:

$S = \begin{pmatrix} 1 & 1 \\ -1 & 1 \end{pmatrix}$, $\Lambda = \begin{pmatrix} 1 & 0 \\ 0 & 3 \end{pmatrix}$

Verify: $S\Lambda S^{-1} = A$.

---

Application: Hessian Matrix

For stationary point analysis, diagonalise the Hessian $H$:

• All $\lambda_i > 0$ → local minimum
• All $\lambda_i < 0$ → local maximum
• Mixed signs → saddle point

Eigenvalues give the curvatures in principal directions.

---

Similarity Transformation

Matrices $A$ and $B$ are similar if $B = P^{-1}AP$ for some invertible $P$.

Similar matrices represent the same linear map in different bases. They share:

• Eigenvalues
• Determinant
• Trace

---

Why Eigenvalues Matter

• Dynamics: $A^k\mathbf{x} = S\Lambda^k S^{-1}\mathbf{x}$ (long-term behaviour)
• Stability: Eigenvalues of Jacobian determine stability of fixed points
• Vibrations: Natural frequencies are eigenvalues
• Quantum mechanics: Observable values are eigenvalues
• Principal component analysis: Covariance matrix eigenvectors