Vectorization (mathematics)

The vectorization is frequently used together with the Kronecker product to express matrix multiplication as a linear transformation on matrices. In particular, $${\displaystyle \operatorname {vec} (ABC)=(C^{\mathrm {T} }\otimes A)\operatorname {vec} (B)}$$ for matrices A, B, and C of dimensions k×l, l×m, and m×n.^{[note 1]} For example, if ${\displaystyle \operatorname {ad} _{A}(X)=AX-XA}$ (the adjoint endomorphism of the Lie algebra gl(n, C) of all n×n matrices with complex entries), then ${\displaystyle \operatorname {vec} (\operatorname {ad} _{A}(X))=(A\otimes I_{n}-I_{n}\otimes A^{\mathrm {T} }){\text{vec}}(X)}$, where ${\displaystyle I_{n}}$ is the n×n identity matrix.

There are two other useful formulations: $${\displaystyle {\begin{aligned}\operatorname {vec} (ABC)&=(I_{n}\otimes AB)\operatorname {vec} (C)=(C^{\mathrm {T} }B^{\mathrm {T} }\otimes I_{k})\operatorname {vec} (A)\\\operatorname {vec} (AB)&=(I_{m}\otimes A)\operatorname {vec} (B)=(B^{\mathrm {T} }\otimes I_{k})\operatorname {vec} (A)\end{aligned}}}$$

If B is a diagonal matrix (i.e., ${\textstyle B=\operatorname {diag} (b_{1},\dots ,b_{n})}$), the vectorization can be written using the column-wise Kronecker product ${\textstyle \ast }$ (see Khatri-Rao product) and the main diagonal ${\textstyle b={\begin{bmatrix}b_{1},\dots ,b_{n}\end{bmatrix}}^{\mathrm {T} }}$ of B: $${\displaystyle \operatorname {vec} (ABC)=(C^{\mathrm {T} }\ast A)b}$$

More generally, it has been shown that vectorization is a self-adjunction in the monoidal closed structure of any category of matrices.^[1]

Vectorization is an algebra homomorphism from the space of n × n matrices with the Hadamard (entrywise) product to Cⁿ² with its Hadamard product: $${\displaystyle \operatorname {vec} (A\circ B)=\operatorname {vec} (A)\circ \operatorname {vec} (B).}$$

Vectorization is a unitary transformation from the space of n×n matrices with the Frobenius (or Hilbert–Schmidt) inner product to Cⁿ²: $${\displaystyle \operatorname {tr} (A^{\dagger }B)=\operatorname {vec} (A)^{\dagger }\operatorname {vec} (B),}$$ where the superscript ^† denotes the conjugate transpose.

The matrix vectorization operation can be written in terms of a linear sum. Let X be an m × n matrix that we want to vectorize, and let e_i be the i-th canonical basis vector for the n-dimensional space, that is ${\textstyle \mathbf {e} _{i}=\left[0,\dots ,0,1,0,\dots ,0\right]^{\mathrm {T} }}$. Let B_i be a (mn) × m block matrix defined as follows: $${\displaystyle \mathbf {B} _{i}={\begin{bmatrix}\mathbf {0} \\\vdots \\\mathbf {0} \\\mathbf {I} _{m}\\\mathbf {0} \\\vdots \\\mathbf {0} \end{bmatrix}}=\mathbf {e} _{i}\otimes \mathbf {I} _{m}}$$

B_i consists of n block matrices of size m × m, stacked column-wise, and all these matrices are all-zero except for the i-th one, which is a m × m identity matrix I_m.

Then the vectorized version of X can be expressed as follows: $${\displaystyle \operatorname {vec} (\mathbf {X} )=\sum _{i=1}^{n}\mathbf {B} _{i}\mathbf {X} \mathbf {e} _{i}}$$

Multiplication of X by e_i extracts the i-th column, while multiplication by B_i puts it into the desired position in the final vector.

Alternatively, the linear sum can be expressed using the Kronecker product: $${\displaystyle \operatorname {vec} (\mathbf {X} )=\sum _{i=1}^{n}\mathbf {e} _{i}\otimes \mathbf {X} \mathbf {e} _{i}}$$

For a symmetric matrix A, the vector vec(A) contains more information than is strictly necessary, since the matrix is completely determined by the symmetry together with the lower triangular portion, that is, the n(n + 1)/2 entries on and below the main diagonal. For such matrices, the half-vectorization is sometimes more useful than the vectorization. The half-vectorization, vech(A), of a symmetric n × n matrix A is the n(n + 1)/2 × 1 column vector obtained by vectorizing only the lower triangular part of A: $${\displaystyle \operatorname {vech} (A)=[A_{1,1},\ldots ,A_{n,1},A_{2,2},\ldots ,A_{n,2},\ldots ,A_{n-1,n-1},A_{n,n-1},A_{n,n}]^{\mathrm {T} }.}$$

For example, for the 2×2 matrix ${\displaystyle A={\begin{bmatrix}a&b\\b&d\end{bmatrix}}}$, the half-vectorization is ${\displaystyle \operatorname {vech} (A)={\begin{bmatrix}a\\b\\d\end{bmatrix}}}$.

There exist unique matrices transforming the half-vectorization of a matrix to its vectorization and vice versa called, respectively, the duplication matrix and the elimination matrix.

Programming languages that implement matrices may have easy means for vectorization. In Matlab/GNU Octave a matrix A can be vectorized by A(:). GNU Octave also allows vectorization and half-vectorization with vec(A) and vech(A) respectively. Julia has the vec(A) function as well. In Python NumPy arrays implement the flatten method,^{[note 1]} while in R the desired effect can be achieved via the c() or as.vector() functions or, more efficiently, by removing the dimensions attribute of a matrix A with dim(A) <- NULL. In R, function vec() of package 'ks' allows vectorization and function vech() implemented in both packages 'ks' and 'sn' allows half-vectorization.^[2][3][4]

Vectorization is used in matrix calculus and its applications in establishing e.g., moments of random vectors and matrices, asymptotics, as well as Jacobian and Hessian matrices.^[5] It is also used in local sensitivity and statistical diagnostics.^[6]

• Duplication and elimination matrices
• Voigt notation
• Packed storage matrix
• Column-major order
• Matricization