Block matrix

The matrix

${\displaystyle \mathbf {P} ={\begin{bmatrix}1&2&2&7\\1&5&6&2\\3&3&4&5\\3&3&6&7\end{bmatrix}}}$

can be visualized as divided into four blocks, as

${\displaystyle \mathbf {P} =\left[{\begin{array}{cc|cc}1&2&2&7\\1&5&6&2\\\hline 3&3&4&5\\3&3&6&7\end{array}}\right]}$.

The horizontal and vertical lines have no special mathematical meaning,^[6][7] but are a common way to visualize a partition.^[6][7] By this partition, ${\displaystyle P}$ is partitioned into four 2×2 blocks, as

${\displaystyle \mathbf {P} _{11}={\begin{bmatrix}1&2\\1&5\end{bmatrix}},\quad \mathbf {P} _{12}={\begin{bmatrix}2&7\\6&2\end{bmatrix}},\quad \mathbf {P} _{21}={\begin{bmatrix}3&3\\3&3\end{bmatrix}},\quad \mathbf {P} _{22}={\begin{bmatrix}4&5\\6&7\end{bmatrix}}.}$

The partitioned matrix can then be written as

${\displaystyle \mathbf {P} ={\begin{bmatrix}\mathbf {P} _{11}&\mathbf {P} _{12}\\\mathbf {P} _{21}&\mathbf {P} _{22}\end{bmatrix}}.}$^[8]

Let ${\displaystyle A\in \mathbb {C} ^{m\times n}}$. A partitioning of ${\displaystyle A}$ is a representation of ${\displaystyle A}$ in the form

${\displaystyle A={\begin{bmatrix}A_{11}&A_{12}&\cdots &A_{1q}\\A_{21}&A_{22}&\cdots &A_{2q}\\\vdots &\vdots &\ddots &\vdots \\A_{p1}&A_{p2}&\cdots &A_{pq}\end{bmatrix}}}$,

where ${\displaystyle A_{ij}\in \mathbb {C} ^{m_{i}\times n_{j}}}$ are contiguous submatrices, ${\displaystyle \sum _{i=1}^{p}m_{i}=m}$, and ${\displaystyle \sum _{j=1}^{q}n_{j}=n}$.^[9] The elements ${\displaystyle A_{ij}}$ of the partition are called blocks.^[9]

By this definition, the blocks in any one column must all have the same number of columns.^[9] Similarly, the blocks in any one row must have the same number of rows.^[9]

A matrix can be partitioned in many ways.^[9] For example, a matrix ${\displaystyle A}$ is said to be partitioned by columns if it is written as

${\displaystyle A=(a_{1}\ a_{2}\ \cdots \ a_{n})}$,

where ${\displaystyle a_{j}}$ is the ${\displaystyle j}$th column of ${\displaystyle A}$.^[9] A matrix can also be partitioned by rows:

${\displaystyle A={\begin{bmatrix}a_{1}^{T}\\a_{2}^{T}\\\vdots \\a_{m}^{T}\end{bmatrix}}}$,

where ${\displaystyle a_{i}^{T}}$ is the ${\displaystyle i}$th row of ${\displaystyle A}$.^[9]

Often,^[9] we encounter the 2x2 partition

${\displaystyle A={\begin{bmatrix}A_{11}&A_{12}\\A_{21}&A_{22}\end{bmatrix}}}$,^[9]

particularly in the form where ${\displaystyle A_{11}}$ is a scalar:

${\displaystyle A={\begin{bmatrix}a_{11}&a_{12}^{T}\\a_{21}&A_{22}\end{bmatrix}}}$.^[9]

Let

${\displaystyle A={\begin{bmatrix}A_{11}&A_{12}&\cdots &A_{1q}\\A_{21}&A_{22}&\cdots &A_{2q}\\\vdots &\vdots &\ddots &\vdots \\A_{p1}&A_{p2}&\cdots &A_{pq}\end{bmatrix}}}$

where ${\displaystyle A_{ij}\in \mathbb {C} ^{k_{i}\times \ell _{j}}}$. (This matrix ${\displaystyle A}$ will be reused in § Addition and § Multiplication.) Then its transpose is

${\displaystyle A^{T}={\begin{bmatrix}A_{11}^{T}&A_{21}^{T}&\cdots &A_{p1}^{T}\\A_{12}^{T}&A_{22}^{T}&\cdots &A_{p2}^{T}\\\vdots &\vdots &\ddots &\vdots \\A_{1q}^{T}&A_{2q}^{T}&\cdots &A_{pq}^{T}\end{bmatrix}}}$,^[9][10]

and the same equation holds with the transpose replaced by the conjugate transpose.^[9]

A special form of matrix transpose can also be defined for block matrices, where individual blocks are reordered but not transposed. Let ${\displaystyle A=(B_{ij})}$ be a ${\displaystyle k\times l}$ block matrix with ${\displaystyle m\times n}$ blocks ${\displaystyle B_{ij}}$, the block transpose of ${\displaystyle A}$ is the ${\displaystyle l\times k}$ block matrix ${\displaystyle A^{\mathcal {B}}}$ with ${\displaystyle m\times n}$ blocks ${\displaystyle \left(A^{\mathcal {B}}\right)_{ij}=B_{ji}}$.^[11] As with the conventional trace operator, the block transpose is a linear mapping such that ${\displaystyle (A+C)^{\mathcal {B}}=A^{\mathcal {B}}+C^{\mathcal {B}}}$.^[10] However, in general the property ${\displaystyle (AC)^{\mathcal {B}}=C^{\mathcal {B}}A^{\mathcal {B}}}$ does not hold unless the blocks of ${\displaystyle A}$ and ${\displaystyle C}$ commute.

Let

${\displaystyle B={\begin{bmatrix}B_{11}&B_{12}&\cdots &B_{1s}\\B_{21}&B_{22}&\cdots &B_{2s}\\\vdots &\vdots &\ddots &\vdots \\B_{r1}&B_{r2}&\cdots &B_{rs}\end{bmatrix}}}$,

where ${\displaystyle B_{ij}\in \mathbb {C} ^{m_{i}\times n_{j}}}$, and let ${\displaystyle A}$ be the matrix defined in § Transpose. (This matrix ${\displaystyle B}$ will be reused in § Multiplication.) Then if ${\displaystyle p=r}$, ${\displaystyle q=s}$, ${\displaystyle k_{i}=m_{i}}$, and ${\displaystyle \ell _{j}=n_{j}}$, then

${\displaystyle A+B={\begin{bmatrix}A_{11}+B_{11}&A_{12}+B_{12}&\cdots &A_{1q}+B_{1q}\\A_{21}+B_{21}&A_{22}+B_{22}&\cdots &A_{2q}+B_{2q}\\\vdots &\vdots &\ddots &\vdots \\A_{p1}+B_{p1}&A_{p2}+B_{p2}&\cdots &A_{pq}+B_{pq}\end{bmatrix}}}$.^[9]

It is possible to use a block partitioned matrix product that involves only algebra on submatrices of the factors. The partitioning of the factors is not arbitrary, however, and requires "conformable partitions"^[12] between two matrices ${\displaystyle A}$ and ${\displaystyle B}$ such that all submatrix products that will be used are defined.^[13]

Two matrices ${\displaystyle A}$ and ${\displaystyle B}$ are said to be partitioned conformally for the product ${\displaystyle AB}$, when ${\displaystyle A}$ and ${\displaystyle B}$ are partitioned into submatrices and if the multiplication ${\displaystyle AB}$ is carried out treating the submatrices as if they are scalars, but keeping the order, and when all products and sums of submatrices involved are defined.

Let ${\displaystyle A}$ be the matrix defined in § Transpose, and let ${\displaystyle B}$ be the matrix defined in § Addition. Then the matrix product

${\displaystyle C=AB}$

can be performed blockwise, yielding ${\displaystyle C}$ as an ${\displaystyle (p\times s)}$ matrix. The matrices in the resulting matrix ${\displaystyle C}$ are calculated by multiplying:

${\displaystyle C_{ij}=\sum _{k=1}^{q}A_{ik}B_{kj}.}$^[6]

Or, using the Einstein notation that implicitly sums over repeated indices:

${\displaystyle C_{ij}=A_{ik}B_{kj}.}$

Depicting ${\displaystyle C}$ as a matrix, we have

${\displaystyle C=AB={\begin{bmatrix}\sum _{i=1}^{q}A_{1i}B_{i1}&\sum _{i=1}^{q}A_{1i}B_{i2}&\cdots &\sum _{i=1}^{q}A_{1i}B_{is}\\\sum _{i=1}^{q}A_{2i}B_{i1}&\sum _{i=1}^{q}A_{2i}B_{i2}&\cdots &\sum _{i=1}^{q}A_{2i}B_{is}\\\vdots &\vdots &\ddots &\vdots \\\sum _{i=1}^{q}A_{pi}B_{i1}&\sum _{i=1}^{q}A_{pi}B_{i2}&\cdots &\sum _{i=1}^{q}A_{pi}B_{is}\end{bmatrix}}}$.^[9]

If a matrix is partitioned into four blocks, it can be inverted blockwise as follows:

${\displaystyle {P}={\begin{bmatrix}{A}&{B}\\{C}&{D}\end{bmatrix}}^{-1}={\begin{bmatrix}{A}^{-1}+{A}^{-1}{B}\left({D}-{CA}^{-1}{B}\right)^{-1}{CA}^{-1}&-{A}^{-1}{B}\left({D}-{CA}^{-1}{B}\right)^{-1}\\-\left({D}-{CA}^{-1}{B}\right)^{-1}{CA}^{-1}&\left({D}-{CA}^{-1}{B}\right)^{-1}\end{bmatrix}},}$

where A and D are square blocks of arbitrary size, and B and C are conformable with them for partitioning. Furthermore, A and the Schur complement of A in P: P/A = D − CA⁻¹B must be invertible.^[15]

Equivalently, by permuting the blocks:

${\displaystyle {P}={\begin{bmatrix}{A}&{B}\\{C}&{D}\end{bmatrix}}^{-1}={\begin{bmatrix}\left({A}-{BD}^{-1}{C}\right)^{-1}&-\left({A}-{BD}^{-1}{C}\right)^{-1}{BD}^{-1}\\-{D}^{-1}{C}\left({A}-{BD}^{-1}{C}\right)^{-1}&\quad {D}^{-1}+{D}^{-1}{C}\left({A}-{BD}^{-1}{C}\right)^{-1}{BD}^{-1}\end{bmatrix}}.}$^[16]

Here, D and the Schur complement of D in P: P/D = A − BD⁻¹C must be invertible.

If A and D are both invertible, then:

${\displaystyle {\begin{bmatrix}{A}&{B}\\{C}&{D}\end{bmatrix}}^{-1}={\begin{bmatrix}\left({A}-{B}{D}^{-1}{C}\right)^{-1}&{0}\\{0}&\left({D}-{C}{A}^{-1}{B}\right)^{-1}\end{bmatrix}}{\begin{bmatrix}{I}&-{B}{D}^{-1}\\-{C}{A}^{-1}&{I}\end{bmatrix}}.}$

By the Weinstein–Aronszajn identity, one of the two matrices in the block-diagonal matrix is invertible exactly when the other is.

By the symmetry between a matrix and its inverse in the block inversion formula, if a matrix P and its inverse P⁻¹ are partitioned conformally:

${\displaystyle P={\begin{bmatrix}{A}&{B}\\{C}&{D}\end{bmatrix}},\quad P^{-1}={\begin{bmatrix}{E}&{F}\\{G}&{H}\end{bmatrix}}}$

then the inverse of any principal submatrix can be computed from the corresponding blocks of P⁻¹:

${\displaystyle {A}^{-1}={E}-{FH}^{-1}{G}}$

${\displaystyle {D}^{-1}={H}-{GE}^{-1}{F}}$

This relationship follows from recognizing that E⁻¹ = A − BD⁻¹C (the Schur complement), and applying the same block inversion formula with the roles of P and P⁻¹ reversed.^{[17] [18]}

The formula for the determinant of a ${\displaystyle 2\times 2}$-matrix above continues to hold, under appropriate further assumptions, for a matrix composed of four submatrices ${\displaystyle A,B,C,D}$ with ${\displaystyle A}$ and ${\displaystyle D}$ square. The easiest such formula, which can be proven using either the Leibniz formula or a factorization involving the Schur complement, is

${\displaystyle \det {\begin{bmatrix}A&0\\C&D\end{bmatrix}}=\det(A)\det(D)=\det {\begin{bmatrix}A&B\\0&D\end{bmatrix}}.}$^[16]

Using this formula, we can derive that characteristic polynomials of ${\displaystyle {\begin{bmatrix}A&0\\C&D\end{bmatrix}}}$ and ${\displaystyle {\begin{bmatrix}A&B\\0&D\end{bmatrix}}}$ are same and equal to the product of characteristic polynomials of ${\displaystyle A}$ and ${\displaystyle D}$. Furthermore, If ${\displaystyle {\begin{bmatrix}A&0\\C&D\end{bmatrix}}}$ or ${\displaystyle {\begin{bmatrix}A&B\\0&D\end{bmatrix}}}$ is diagonalizable, then ${\displaystyle A}$ and ${\displaystyle D}$ are diagonalizable too. The converse is false; simply check ${\displaystyle {\begin{bmatrix}1&1\\0&1\end{bmatrix}}}$.

If ${\displaystyle A}$ is invertible, one has

${\displaystyle \det {\begin{bmatrix}A&B\\C&D\end{bmatrix}}=\det(A)\det \left(D-CA^{-1}B\right),}$^[16]

and if ${\displaystyle D}$ is invertible, one has

${\displaystyle \det {\begin{bmatrix}A&B\\C&D\end{bmatrix}}=\det(D)\det \left(A-BD^{-1}C\right).}$^[19][16]

If the blocks are square matrices of the same size further formulas hold. For example, if ${\displaystyle C}$ and ${\displaystyle D}$ commute (i.e., ${\displaystyle CD=DC}$), then

${\displaystyle \det {\begin{bmatrix}A&B\\C&D\end{bmatrix}}=\det(AD-BC).}$^[20]

Similar statements hold when ${\displaystyle AB=BA}$, ${\displaystyle AC=CA}$, or ⁠${\displaystyle BD=DB}$⁠. Namely, if ${\displaystyle AC=CA}$, then

${\displaystyle \det {\begin{bmatrix}A&B\\C&D\end{bmatrix}}=\det(AD-CB).}$

Note the change in order of ${\displaystyle C}$ and ${\displaystyle B}$ (we have ${\displaystyle CB}$ instead of ${\displaystyle BC}$). Similarly, if ${\displaystyle BD=DB}$, then ${\displaystyle AD}$ should be replaced with ${\displaystyle DA}$ (i.e. we get ${\displaystyle \det(DA-BC)}$) and if ${\displaystyle AB=BA}$, then we should have ${\displaystyle \det(DA-CB)}$. Note for the last two results, you have to use commutativity of the underlying ring, but not for the first two.

This formula has been generalized to matrices composed of more than ${\displaystyle 2\times 2}$ blocks, again under appropriate commutativity conditions among the individual blocks.^[21]

For ${\displaystyle A=D}$ and ${\displaystyle B=C}$, the following formula holds (even if ${\displaystyle A}$ and ${\displaystyle B}$ do not commute)

${\displaystyle \det {\begin{bmatrix}A&B\\B&A\end{bmatrix}}=\det(A-B)\det(A+B).}$^[16]

For any arbitrary matrices A (of size m × n) and B (of size p × q), we have the direct sum of A and B, denoted by A ${\displaystyle \oplus }$ B and defined as

${\displaystyle {A}\oplus {B}={\begin{bmatrix}a_{11}&\cdots &a_{1n}&0&\cdots &0\\\vdots &\ddots &\vdots &\vdots &\ddots &\vdots \\a_{m1}&\cdots &a_{mn}&0&\cdots &0\\0&\cdots &0&b_{11}&\cdots &b_{1q}\\\vdots &\ddots &\vdots &\vdots &\ddots &\vdots \\0&\cdots &0&b_{p1}&\cdots &b_{pq}\end{bmatrix}}.}$^[10]

For instance,

${\displaystyle {\begin{bmatrix}1&3&2\\2&3&1\end{bmatrix}}\oplus {\begin{bmatrix}1&6\\0&1\end{bmatrix}}={\begin{bmatrix}1&3&2&0&0\\2&3&1&0&0\\0&0&0&1&6\\0&0&0&0&1\end{bmatrix}}.}$

This operation generalizes naturally to arbitrary dimensioned arrays (provided that A and B have the same number of dimensions).

Note that any element in the direct sum of two vector spaces of matrices could be represented as a direct sum of two matrices.

A block diagonal matrix is a block matrix that is a square matrix such that the main-diagonal blocks are square matrices and all off-diagonal blocks are zero matrices.^[16] That is, a block diagonal matrix A has the form

${\displaystyle {A}={\begin{bmatrix}{A}_{1}&{0}&\cdots &{0}\\{0}&{A}_{2}&\cdots &{0}\\\vdots &\vdots &\ddots &\vdots \\{0}&{0}&\cdots &{A}_{n}\end{bmatrix}}}$

where A_k is a square matrix for all k = 1, ..., n. In other words, matrix A is the direct sum of A₁, ..., A_n.^[16] It can also be indicated as A₁ ⊕ A₂ ⊕ ... ⊕ A_n^[10] or diag(A₁, A₂, ..., A_n)^[10] (the latter being the same formalism used for a diagonal matrix). Any square matrix can trivially be considered a block diagonal matrix with only one block.

For the determinant and trace, the following properties hold:

${\displaystyle {\begin{aligned}\det {A}&=\det {A}_{1}\times \cdots \times \det {A}_{n},\end{aligned}}}$^[22][23] and

${\displaystyle {\begin{aligned}\operatorname {tr} {A}&=\operatorname {tr} {A}_{1}+\cdots +\operatorname {tr} {A}_{n}.\end{aligned}}}$^[16][23]

A block diagonal matrix is invertible if and only if each of its main-diagonal blocks are invertible, and in this case its inverse is another block diagonal matrix given by

${\displaystyle {\begin{bmatrix}{A}_{1}&{0}&\cdots &{0}\\{0}&{A}_{2}&\cdots &{0}\\\vdots &\vdots &\ddots &\vdots \\{0}&{0}&\cdots &{A}_{n}\end{bmatrix}}^{-1}={\begin{bmatrix}{A}_{1}^{-1}&{0}&\cdots &{0}\\{0}&{A}_{2}^{-1}&\cdots &{0}\\\vdots &\vdots &\ddots &\vdots \\{0}&{0}&\cdots &{A}_{n}^{-1}\end{bmatrix}}.}$^[24]

The eigenvalues^[25] and eigenvectors of ${\displaystyle {A}}$ are simply those of the ${\displaystyle {A}_{k}}$s combined.^[23]

A block tridiagonal matrix is another special block matrix, which is just like the block diagonal matrix a square matrix, having square matrices (blocks) in the lower diagonal, main diagonal and upper diagonal, with all other blocks being zero matrices. It is essentially a tridiagonal matrix but has submatrices in places of scalars. A block tridiagonal matrix ${\displaystyle A}$ has the form

${\displaystyle {A}={\begin{bmatrix}{B}_{1}&{C}_{1}&&&\cdots &&{0}\\{A}_{2}&{B}_{2}&{C}_{2}&&&&\\&\ddots &\ddots &\ddots &&&\vdots \\&&{A}_{k}&{B}_{k}&{C}_{k}&&\\\vdots &&&\ddots &\ddots &\ddots &\\&&&&{A}_{n-1}&{B}_{n-1}&{C}_{n-1}\\{0}&&\cdots &&&{A}_{n}&{B}_{n}\end{bmatrix}}}$

where ${\displaystyle {A}_{k}}$, ${\displaystyle {B}_{k}}$ and ${\displaystyle {C}_{k}}$ are square sub-matrices of the lower, main and upper diagonal respectively.^[26][27]

Block tridiagonal matrices are often encountered in numerical solutions of engineering problems (e.g., computational fluid dynamics). Optimized numerical methods for LU factorization are available^[28] and hence efficient solution algorithms for equation systems with a block tridiagonal matrix as coefficient matrix. The Thomas algorithm, used for efficient solution of equation systems involving a tridiagonal matrix can also be applied using matrix operations to block tridiagonal matrices (see also Block LU decomposition).

An ${\displaystyle n\times n}$ matrix ${\displaystyle A}$ is upper block triangular (or block upper triangular^[29]) if there are positive integers ${\displaystyle n_{1},\ldots ,n_{k}}$ such that ${\displaystyle n=n_{1}+n_{2}+\ldots +n_{k}}$ and $${\displaystyle A={\begin{bmatrix}A_{11}&A_{12}&\cdots &A_{1k}\\0&A_{22}&\cdots &A_{2k}\\\vdots &\vdots &\ddots &\vdots \\0&0&\cdots &A_{kk}\end{bmatrix}},}$$ where the matrix ${\displaystyle A_{ij}}$ is ${\displaystyle n_{i}\times n_{j}}$ for all ${\displaystyle i,j=1,\ldots ,k}$.^[25][29] Similarly, ${\displaystyle A}$ is lower block triangular if $${\displaystyle A={\begin{bmatrix}A_{11}&0&\cdots &0\\A_{21}&A_{22}&\cdots &0\\\vdots &\vdots &\ddots &\vdots \\A_{k1}&A_{k2}&\cdots &A_{kk}\end{bmatrix}},}$$ where ${\displaystyle A_{ij}}$ is ${\displaystyle n_{i}\times n_{j}}$ for all ${\displaystyle i,j=1,\ldots ,k}$.^[25]

A block Toeplitz matrix is another special block matrix, which contains blocks that are repeated down the diagonals of the matrix, as a Toeplitz matrix has elements repeated down the diagonal.

A matrix ${\displaystyle A}$ is block Toeplitz if ${\displaystyle A_{(i,j)}=A_{(k,l)}}$ for all ${\displaystyle k-i=l-j}$, that is,

${\displaystyle A={\begin{bmatrix}A_{1}&A_{2}&A_{3}&\cdots \\A_{4}&A_{1}&A_{2}&\cdots \\A_{5}&A_{4}&A_{1}&\cdots \\\vdots &\vdots &\vdots &\ddots \end{bmatrix}}}$,

where ${\displaystyle A_{i}\in \mathbb {F} ^{n_{i}\times m_{i}}}$.^[25]

A matrix ${\displaystyle A}$ is block Hankel if ${\displaystyle A_{(i,j)}=A_{(k,l)}}$ for all ${\displaystyle i+j=k+l}$, that is,

${\displaystyle A={\begin{bmatrix}A_{1}&A_{2}&A_{3}&\cdots \\A_{2}&A_{3}&A_{4}&\cdots \\A_{3}&A_{4}&A_{5}&\cdots \\\vdots &\vdots &\vdots &\ddots \end{bmatrix}}}$,

where ${\displaystyle A_{i}\in \mathbb {F} ^{n_{i}\times m_{i}}}$.^[25]

• Kronecker product (matrix direct product resulting in a block matrix)
• Jordan normal form (canonical form of a linear operator on a finite-dimensional complex vector space)
• Strassen algorithm (algorithm for matrix multiplication that is faster than the conventional matrix multiplication algorithm)