Eigendecomposition of a matrix

In linear algebra, eigendecomposition is the factorization of a matrix into a canonical form, whereby the matrix is represented in terms of its eigenvalues and eigenvectors. Only diagonalizable matrices can be factorized in this way. When the matrix being factorized is a normal or real symmetric matrix, the decomposition is called "spectral decomposition", derived from the spectral theorem.

A (nonzero) vector v of dimension N is an eigenvector of a square N × N matrix A if it satisfies a linear equation of the form $${\displaystyle \mathbf {A} \mathbf {v} =\lambda \mathbf {v} }$$ for some scalar λ. Then λ is called the eigenvalue corresponding to v. Geometrically speaking, the eigenvectors of A are the vectors that A merely elongates or shrinks, and the amount that they elongate/shrink by is the eigenvalue. The above equation is called the eigenvalue equation or the eigenvalue problem.

This yields an equation for the eigenvalues $${\displaystyle p\left(\lambda \right)=\det \left(\mathbf {A} -\lambda \mathbf {I} \right)=0.}$$ We call p(λ) the characteristic polynomial, and the equation, called the characteristic equation, is an Nth-order polynomial equation in the unknown λ. This equation will have N_λ distinct solutions, where 1 ≤ N_λ ≤ N. The set of solutions, that is, the eigenvalues, is called the spectrum of A.

If the field of scalars is algebraically closed, then we can factor p as $${\displaystyle p(\lambda )=\left(\lambda -\lambda _{1}\right)^{n_{1}}\left(\lambda -\lambda _{2}\right)^{n_{2}}\cdots \left(\lambda -\lambda _{N_{\lambda }}\right)^{n_{N_{\lambda }}}=0.}$$ The integer n_i is termed the algebraic multiplicity of eigenvalue λ_i. The algebraic multiplicities sum to N: ${\textstyle \sum _{i=1}^{N_{\lambda }}{n_{i}}=N.}$

For each eigenvalue λ_i, we have a specific eigenvalue equation $${\displaystyle \left(\mathbf {A} -\lambda _{i}\mathbf {I} \right)\mathbf {v} =0.}$$ There will be 1 ≤ m_i ≤ n_i linearly independent solutions to each eigenvalue equation. The linear combinations of the m_i solutions (except the one which gives the zero vector) are the eigenvectors associated with the eigenvalue λ_i. The integer m_i is termed the geometric multiplicity of λ_i. It is important to keep in mind that the algebraic multiplicity n_i and geometric multiplicity m_i may or may not be equal, but we always have m_i ≤ n_i. The simplest case is of course when m_i = n_i = 1. The total number of linearly independent eigenvectors, N_v, can be calculated by summing the geometric multiplicities $${\displaystyle \sum _{i=1}^{N_{\lambda }}{m_{i}}=N_{\mathbf {v} }.}$$

The eigenvectors can be indexed by eigenvalues, using a double index, with v_ij being the jth eigenvector for the ith eigenvalue. The eigenvectors can also be indexed using the simpler notation of a single index v_k, with k = 1, 2, ..., N_v.

Let A be a square n × n matrix with n linearly independent eigenvectors q_i (where i = 1, ..., n). Then A can be factored as $${\displaystyle \mathbf {A} =\mathbf {Q} \mathbf {\Lambda } \mathbf {Q} ^{-1}}$$ where Q is the square n × n matrix whose ith column is the eigenvector q_i of A, and Λ is the diagonal matrix whose diagonal elements are the corresponding eigenvalues, Λ_ii = λ_i. Note that only diagonalizable matrices can be factorized in this way. For example, the defective matrix ${\displaystyle \left[{\begin{smallmatrix}1&1\\0&1\end{smallmatrix}}\right]}$ (which is a shear matrix) cannot be diagonalized.

The n eigenvectors q_i are usually normalized, but they don't have to be. A non-normalized set of n eigenvectors, v_i can also be used as the columns of Q. That can be understood by noting that the magnitude of the eigenvectors in Q gets canceled in the decomposition by the presence of Q⁻¹. If one of the eigenvalues λ_i has multiple linearly independent eigenvectors (that is, the geometric multiplicity of λ_i is greater than 1), then these eigenvectors for this eigenvalue λ_i can be chosen to be mutually orthogonal; however, if two eigenvectors belong to two different eigenvalues, it may be impossible for them to be orthogonal to each other (see Example below). One special case is that if A is a normal matrix, then by the spectral theorem, it's always possible to diagonalize A in an orthonormal basis {q_i}.