In mathematics, an invariant subspace of a linear mapping T : V → V i.e. from some vector space V to itself, is a subspace W of V that is preserved by T. More generally, an invariant subspace for a collection of linear mappings is a subspace preserved by each mapping individually.

Consider a vector space ${\displaystyle V}$ and a linear map ${\displaystyle T:V\to V.}$ A subspace ${\displaystyle W\subseteq V}$ is called an invariant subspace for ${\displaystyle T}$, or equivalently, T-invariant, if T transforms any vector ${\displaystyle \mathbf {v} \in W}$ back into W. In formulas, this can be written$${\displaystyle \mathbf {v} \in W\implies T(\mathbf {v} )\in W}$$or $${\displaystyle TW\subseteq W{\text{.}}}$$

In this case, T restricts to an endomorphism of W:$${\displaystyle T|_{W}:W\to W{\text{;}}\quad T|_{W}(\mathbf {w} )=T(\mathbf {w} ){\text{.}}}$$

The existence of an invariant subspace also has a matrix formulation. Pick a basis C for W and complete it to a basis B of V. With respect to B, the operator T has form $${\displaystyle T={\begin{bmatrix}T|_{W}&T_{12}\\0&T_{22}\end{bmatrix}}}$$ for some T₁₂ and T₂₂, where ${\displaystyle T|_{W}}$ here denotes the matrix of ${\displaystyle T|_{W}}$ with respect to the basis C.

Any linear map ${\displaystyle T:V\to V}$ admits the following invariant subspaces:

These are the improper and trivial invariant subspaces, respectively. Certain linear operators have no proper non-trivial invariant subspace: for instance, rotation of a two-dimensional real vector space. However, the axis of a rotation in three dimensions is always an invariant subspace.

If U is a 1-dimensional invariant subspace for operator T with vector v ∈ U, then the vectors v and Tv must be linearly dependent. Thus $${\displaystyle \forall \mathbf {v} \in U\;\exists \alpha \in \mathbb {R} :T\mathbf {v} =\alpha \mathbf {v} {\text{.}}}$$In fact, the scalar α does not depend on v.

The equation above formulates an eigenvalue problem. Any eigenvector for T spans a 1-dimensional invariant subspace, and vice-versa. In particular, a nonzero invariant vector (i.e. a fixed point of T) spans an invariant subspace of dimension 1.