Centering matrix

Centering matrixMain

Community hub

Centering matrix

7 pages, 0 posts

0 subscribers

Recent from talks

Be the first to start a discussion here.

Recent from talks

Be the first to start a discussion here.

Contribute something

About hubMembersContent overviewUpdatesRules

Main reference articles

Centering matrix

View on Wikipedia

from Wikipedia

This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.
Find sources: "Centering matrix" – news · newspapers · books · scholar · JSTOR (August 2024) (Learn how and when to remove this message)

In mathematics and multivariate statistics, the centering matrix^[1] is a symmetric and idempotent matrix, which when multiplied with a vector has the same effect as subtracting the mean of the components of the vector from every component of that vector.

Definition

[edit]

The centering matrix of size n is defined as the n-by-n matrix

C_{n}=I_{n}-{\tfrac {1}{n}}J_{n}

where $I_{n}\,$ is the identity matrix of size n and $J_{n}$ is an n-by-n matrix of all 1's.

For example

C_{1}={\begin{bmatrix}0\end{bmatrix}}

C_{2}=\left[{\begin{array}{rrr}1&0\\0&1\end{array}}\right]-{\frac {1}{2}}\left[{\begin{array}{rrr}1&1\\1&1\end{array}}\right]=\left[{\begin{array}{rrr}{\frac {1}{2}}&-{\frac {1}{2}}\\-{\frac {1}{2}}&{\frac {1}{2}}\end{array}}\right]

C_{3}=\left[{\begin{array}{rrr}1&0&0\\0&1&0\\0&0&1\end{array}}\right]-{\frac {1}{3}}\left[{\begin{array}{rrr}1&1&1\\1&1&1\\1&1&1\end{array}}\right]=\left[{\begin{array}{rrr}{\frac {2}{3}}&-{\frac {1}{3}}&-{\frac {1}{3}}\\-{\frac {1}{3}}&{\frac {2}{3}}&-{\frac {1}{3}}\\-{\frac {1}{3}}&-{\frac {1}{3}}&{\frac {2}{3}}\end{array}}\right]

Properties

[edit]

Given a column-vector, $\mathbf {v} \,$ of size n, the centering property of $C_{n}\,$ can be expressed as

C_{n}\,\mathbf {v} =\mathbf {v} -({\tfrac {1}{n}}J_{n,1}^{\textrm {T}}\mathbf {v} )J_{n,1}

where $J_{n,1}$ is a column vector of ones and ${\tfrac {1}{n}}J_{n,1}^{\textrm {T}}\mathbf {v}$ is the mean of the components of $\mathbf {v} \,$ .

$C_{n}\,$ is symmetric positive semi-definite.

$C_{n}\,$ is idempotent, so that $C_{n}^{k}=C_{n}$ , for $k=1,2,\ldots$ . Once the mean has been removed, it is zero and removing it again has no effect.

$C_{n}\,$ is singular. The effects of applying the transformation $C_{n}\,\mathbf {v}$ cannot be reversed.

$C_{n}\,$ has the eigenvalue 1 of multiplicity n − 1 and eigenvalue 0 of multiplicity 1.

$C_{n}\,$ has a nullspace of dimension 1, along the vector $J_{n,1}$ .

$C_{n}\,$ is an orthogonal projection matrix. That is, $C_{n}\mathbf {v}$ is a projection of $\mathbf {v} \,$ onto the (n − 1)-dimensional subspace that is orthogonal to the nullspace $J_{n,1}$ . (This is the subspace of all n-vectors whose components sum to zero.)

The trace of $C_{n}$ is $n(n-1)/n=n-1$ .

Application

[edit]

Although multiplication by the centering matrix is not a computationally efficient way of removing the mean from a vector, it is a convenient analytical tool. It can be used not only to remove the mean of a single vector, but also of multiple vectors stored in the rows or columns of an m-by-n matrix $X$ .

The left multiplication by $C_{m}$ subtracts a corresponding mean value from each of the n columns, so that each column of the product $C_{m}\,X$ has a zero mean. Similarly, the multiplication by $C_{n}$ on the right subtracts a corresponding mean value from each of the m rows, and each row of the product $X\,C_{n}$ has a zero mean. The multiplication on both sides creates a doubly centred matrix $C_{m}\,X\,C_{n}$ , whose row and column means are equal to zero.

The centering matrix provides in particular a succinct way to express the scatter matrix, $S=(X-\mu J_{n,1}^{\mathrm {T} })(X-\mu J_{n,1}^{\mathrm {T} })^{\mathrm {T} }$ of a data sample $X\,$ , where $\mu ={\tfrac {1}{n}}XJ_{n,1}$ is the sample mean. The centering matrix allows us to express the scatter matrix more compactly as

S=X\,C_{n}(X\,C_{n})^{\mathrm {T} }=X\,C_{n}\,C_{n}\,X\,^{\mathrm {T} }=X\,C_{n}\,X\,^{\mathrm {T} }.

$C_{n}$ is the covariance matrix of the multinomial distribution, in the special case where the parameters of that distribution are $k=n$ , and $p_{1}=p_{2}=\cdots =p_{n}={\frac {1}{n}}$ .

References

[edit]

^ John I. Marden, Analyzing and Modeling Rank Data, Chapman & Hall, 1995, ISBN 0-412-99521-2, page 59.

v t e Matrix classes
Explicitly constrained entries	Alternant Anti-diagonal Anti-Hermitian Anti-symmetric Arrowhead Band Bidiagonal Bisymmetric Block-diagonal Block Block tridiagonal Boolean Cauchy Centrosymmetric Conference Complex Hadamard Copositive Diagonally dominant Diagonal Discrete Fourier Transform Elementary Equivalent Frobenius Generalized permutation Hadamard Hankel Hermitian Hessenberg Hollow Integer Logical Matrix unit Metzler Moore Nonnegative Pentadiagonal Permutation Persymmetric Polynomial Quaternionic Signature Skew-Hermitian Skew-symmetric Skyline Sparse Sylvester Symmetric Toeplitz Triangular Tridiagonal Vandermonde Walsh Z
Constant	Exchange Hilbert Identity Lehmer Of ones Pascal Pauli Redheffer Shift Zero
Conditions on eigenvalues or eigenvectors	Companion Convergent Defective Definite Diagonalizable Hurwitz-stable Positive-definite Stieltjes
Satisfying conditions on products or inverses	Congruent Idempotent or Projection Invertible Involutory Nilpotent Normal Orthogonal Unimodular Unipotent Unitary Totally unimodular Weighing
With specific applications	Adjugate Alternating sign Augmented Bézout Carleman Cartan Circulant Cofactor Commutation Confusion Coxeter Distance Duplication and elimination Euclidean distance Fundamental (linear differential equation) Generator Gram Hessian Householder Jacobian Moment Payoff Pick Random Rotation Routh-Hurwitz Seifert Shear Similarity Symplectic Totally positive Transformation
Used in statistics	Centering Correlation Covariance Design Doubly stochastic Fisher information Hat Precision Stochastic Transition
Used in graph theory	Adjacency Biadjacency Degree Edmonds Incidence Laplacian Seidel adjacency Tutte
Used in science and engineering	Cabibbo–Kobayashi–Maskawa Density Fundamental (computer vision) Fuzzy associative Gamma Gell-Mann Hamiltonian Irregular Overlap S State transition Substitution Z (chemistry)
Related terms	Jordan normal form Linear independence Matrix exponential Matrix representation of conic sections Perfect matrix Pseudoinverse Row echelon form Wronskian
Mathematics portal List of matrices Category:Matrices (mathematics)

Revisions and contributors Edit on Wikipedia Read on Wikipedia

View on Grokipedia

from Grokipedia

The centering matrix, also known as the data centering matrix, is an

n \times n

symmetric and idempotent matrix in linear algebra and statistics, defined as

\mathbf{H} = \mathbf{I}_n - \frac{1}{n} \mathbf{1}_n \mathbf{1}_n^\top

, where

\mathbf{I}_n

is the

n \times n

identity matrix and

\mathbf{1}_n

is the

n \times 1

column vector of ones.^[1]^[2] This matrix serves as an orthogonal projection onto the hyperplane orthogonal to

\mathbf{1}_n

, effectively centering a vector

\mathbf{x}

by subtracting its mean

\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i

from each component, yielding

\mathbf{Hx} = \mathbf{x} - \bar{x} \mathbf{1}_n

.^[3]^[1] In multivariate statistics, the centering matrix plays a crucial role in data preprocessing by transforming an

n \times p

data matrix

\mathbf{X}

(with

n

observations and

p

variables) into

\mathbf{HX}

, which subtracts the mean of each column from its entries, resulting in a centered dataset where each variable has zero mean.^[3] This operation is essential for computing unbiased sample covariance matrices, as the sample covariance

\mathbf{S} = \frac{1}{n-1} \mathbf{X}^\top \mathbf{H} \mathbf{X}

ensures that the means are removed, preventing bias from non-zero centroids.^[3]^[2] The matrix's idempotence (

\mathbf{H}^2 = \mathbf{H}

) and symmetry (

\mathbf{H}^\top = \mathbf{H}

) guarantee that repeated applications do not alter the centered data, and its eigenvalues consist of 1 with multiplicity

n-1

and 0 with multiplicity 1, reflecting its rank of

n-1

and the one-dimensional kernel spanned by

\mathbf{1}_n

.^[1]^[2] Beyond centering, the matrix facilitates principal component analysis (PCA) and other dimensionality reduction techniques by ensuring that variance is computed around the origin after mean subtraction, and it appears in regression models to orthogonalize data against constant terms.^[3] Its rows and columns sum to zero, underscoring its role in removing the overall mean effect across observations.^[1]

Definition and Construction

Matrix Representation

The centering matrix

H

n

-dimensional space is defined as the symmetric matrix given by

H = I_n - \frac{1}{n} \mathbf{1}_n \mathbf{1}_n^T,

where

I_n

denotes the

n \times n

identity matrix and

\mathbf{1}_n

is the

n \times 1

column vector of all ones.^[1]^[3] This formulation represents the orthogonal projection onto the subspace perpendicular to

\mathbf{1}_n

.^[3] A common notational convention denotes the all-ones matrix as

J_n = \mathbf{1}_n \mathbf{1}_n^T

, which is the

n \times n

matrix with every entry equal to 1; thus, the centering matrix simplifies to

H = I_n - \frac{1}{n} J_n

.^[1] This matrix arises from the general formula for the orthogonal projection onto the column space of a matrix

A

, which is

P = A (A^T A)^{-1} A^T

, assuming

A

has full column rank./06%3A_Orthogonality/6.03%3A_Orthogonal_Projection) Applying this to

A = \mathbf{1}_n

, we obtain

A^T A = n

, so

(A^T A)^{-1} = \frac{1}{n}

and

P = \mathbf{1}_n \left( \frac{1}{n} \right) \mathbf{1}_n^T = \frac{1}{n} J_n

, the projection onto the span of

\mathbf{1}_n

. The centering matrix is then the complementary projection

H = I_n - P

.^[3]^[1] For explicit construction with small

n

, consider

n=3

. Substituting into the formula yields

H = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} - \frac{1}{3} \begin{pmatrix} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{pmatrix} = \begin{pmatrix} \frac{2}{3} & -\frac{1}{3} & -\frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} & -\frac{1}{3} \\ -\frac{1}{3} & -\frac{1}{3} & \frac{2}{3} \end{pmatrix}.

100010001

−31

111111111

32−31−31−3132−31−31−3132

. This matrix has equal off-diagonal entries of $-\frac{1}{3}$ and diagonal entries of $\frac{2}{3}$ , illustrating the pattern for general $n$ where diagonals are $1 - \frac{1}{n} = \frac{n-1}{n}$ and off-diagonals are $-\frac{1}{n}$ .^[1]

Geometric Interpretation

The centering matrix serves as an orthogonal projection operator onto the hyperplane defined by

\{ \mathbf{x} \in \mathbb{R}^n \mid \mathbf{1}_n^T \mathbf{x} = 0 \}

, which is the subspace consisting of all vectors whose components sum to zero, equivalent to having a zero mean.^[4]^[5] This projection removes the component of a vector along the all-ones vector

\mathbf{1}_n

, effectively centering the data by subtracting the mean without altering the relative positions within the subspace orthogonal to

\mathbf{1}_n

.^[4] When applied to a data matrix

X \in \mathbb{R}^{n \times p}

whose columns represent data points, the operation

HX

centers each column by projecting it onto this hyperplane, thereby eliminating the mean vector and shifting the dataset's centroid to the origin.^[5] Geometrically, this corresponds to

HX = X - \left( \frac{\mathbf{1}_n \mathbf{1}_n^T}{n} \right) X

, where the subtracted term replicates the mean vector across all observations, isolating the deviations from the mean in the resulting centered matrix.^[4] In low-dimensional settings, this projection is intuitive to visualize. For

n=2

, the hyperplane is the line orthogonal to

\mathbf{1}_2 = [1, 1]^T

, specifically the line

x_1 + x_2 = 0

, onto which points are projected perpendicularly to remove their average.^[4] In three dimensions (

n=3

), it projects onto the plane perpendicular to

\mathbf{1}_3 = [1, 1, 1]^T

, defined by

x_1 + x_2 + x_3 = 0

, centering a set of points by shifting them such that their centroid lies at the origin while preserving distances within that plane.^[4] These visualizations highlight how the centering matrix reveals the intrinsic geometry of the data by stripping away the uniform offset along the mean direction.^[5]

Properties

Algebraic Properties

The centering matrix

H = I_n - \frac{1}{n} J_n

, where

I_n

is the

n \times n

identity matrix and

J_n

is the

n \times n

matrix of all ones, exhibits several key algebraic properties arising from its explicit form.^[6] Symmetry. The matrix

H

is symmetric, satisfying

H = H^T

. This follows directly from the transposition of its defining expression:

H^T = \left( I_n - \frac{1}{n} J_n \right)^T = I_n^T - \frac{1}{n} J_n^T = I_n - \frac{1}{n} J_n = H,

since both

I_n

and

J_n

are symmetric.^[6]^[7] Idempotence. The matrix

H

is idempotent, meaning

H^2 = H

. To verify this, compute

H^2 = \left( I_n - \frac{1}{n} J_n \right)^2 = I_n - \frac{2}{n} J_n + \frac{1}{n^2} J_n^2.

The key relation is

J_n^2 = n J_n

, so

\frac{1}{n^2} J_n^2 = \frac{1}{n^2} (n J_n) = \frac{1}{n} J_n.

Substituting yields

H^2 = I_n - \frac{2}{n} J_n + \frac{1}{n} J_n = I_n - \frac{1}{n} J_n = H.

This confirms idempotence.^[6]^[7] Orthogonality to the all-ones matrix. The centering matrix

H

is orthogonal to

J_n

, in the sense that

H J_n = 0

and

J_n H = 0

. For the first,

H J_n = \left( I_n - \frac{1}{n} J_n \right) J_n = J_n - \frac{1}{n} J_n^2 = J_n - \frac{1}{n} (n J_n) = J_n - J_n = 0.

Symmetry of

H

and

J_n

implies

J_n H = (H J_n)^T = 0^T = 0

. This property demonstrates that

H

annihilates constant vectors, as

H \mathbf{1}_n = \mathbf{0}_n

, where

\mathbf{1}_n

is the all-ones vector.^[6] Trace and determinant. The trace of

H

n-1

, derived as

\operatorname{tr}(H) = \operatorname{tr}(I_n) - \frac{1}{n} \operatorname{tr}(J_n) = n - \frac{1}{n} \cdot n = n - 1,

since the diagonal entries of

J_n

are all 1. The determinant is

\det(H) = 0

, as

H

is singular with rank

n-1

(its null space is the one-dimensional span of

\mathbf{1}_n

).^[6]^[7] Uniqueness. The centering matrix

H

is the unique symmetric idempotent matrix satisfying

H \mathbf{1}_n = \mathbf{0}_n

, as it represents the orthogonal projection onto the subspace orthogonal to

\operatorname{span}\{\mathbf{1}_n\}

, and such projections are unique among symmetric idempotents.^[8]

Spectral Properties

The centering matrix

H

, defined for an

n \times n

dimension, possesses a simple eigenvalue spectrum consisting of 1 with algebraic multiplicity

n-1

and 0 with algebraic multiplicity 1.^[9] This structure arises because

H

projects onto the subspace orthogonal to the all-ones vector

\mathbf{1}_n

, preserving vectors in that subspace while annihilating

\mathbf{1}_n

.^[10] The eigenspace associated with the eigenvalue 1 is the set of all vectors

v \in \mathbb{R}^n

satisfying

\mathbf{1}_n^T v = 0

, forming a hyperplane of dimension

n-1

that serves as the column space of

H

.^[9] Conversely, the null space of

H

, corresponding to the eigenvalue 0, is one-dimensional and spanned by

\mathbf{1}_n

.^[10] These eigenspaces are orthogonal complements, reflecting the matrix's role as a projection operator.^[9] The rank of

H

equals

n-1

, directly following from the multiplicity of the nonzero eigenvalue or the dimension of its image.^[10] As a real symmetric matrix,

H

is orthogonally diagonalizable via the spectral theorem:

H = Q D Q^T,

where

Q

is an orthogonal matrix with columns as orthonormal eigenvectors, and

D

is the diagonal matrix with

n-1

entries of 1 followed by a single 0.^[9] The eigenvectors for eigenvalue 1 are not unique, as any orthonormal basis of the

(n-1)

-dimensional eigenspace suffices, leading to non-uniqueness in the choice of

Q

beyond the fixed eigenvector for 0 (up to scaling).^[9] In the context of projections, the Moore-Penrose pseudoinverse of

H

coincides with

H

itself, since

H

is an orthogonal projection matrix satisfying the required properties for self-inverse under the pseudoinverse operation.^[11]

Applications

In Statistics

In statistics, the centering matrix facilitates data preprocessing by subtracting the sample means from each variable, yielding a dataset where each variable has a zero mean. For an

n \times p

data matrix

\mathbf{X}

, with

n

observations and

p

variables, the centered version is

\mathbf{HX}

, where

\mathbf{H} = \mathbf{I}_n - \frac{1}{n} \mathbf{1}_n \mathbf{1}_n^T

and

\mathbf{1}_n

is the

n \times 1

vector of ones. This transformation ensures the sample means of the columns are zero while leaving the sample variances unchanged, as

\mathbf{H}

projects onto the space orthogonal to

\mathbf{1}_n

.^[12] The sample covariance matrix is obtained as

\frac{1}{n-1} (\mathbf{HX})^T (\mathbf{HX}) = \frac{1}{n-1} \mathbf{X}^T \mathbf{H} \mathbf{X}

, serving as an unbiased estimator of the population covariance matrix

\boldsymbol{\Sigma}

under standard assumptions, such as finite second moments. The properties of

\mathbf{H}

, including its symmetry and idempotence, ensure that this expression correctly captures the covariances among deviations from the means without bias from the location of the data.^[12] In multivariate analysis, centering precedes computations like principal component analysis or correlation matrices to emphasize deviations from the mean, thereby isolating patterns of variability rather than absolute levels. For principal components, this step aligns the data origin with the sample mean, allowing eigenvectors of the covariance matrix to represent directions of maximal variance in the centered space, which aids in dimensionality reduction and interpretation of data structure. The practice of centering data for least squares adjustments and variance decomposition originated in early 20th-century statistics, as seen in Ronald Fisher's development of analysis of variance for experimental designs, where means were subtracted to partition total variance into components, though the matrix formulation was not explicitly named at the time.^[13] From a computational perspective,

\mathbf{H}

is efficient for large

n

due to its structure as an identity minus a rank-one matrix, enabling centered data to be computed via

\mathbf{HX} = \mathbf{X} - \mathbf{1}_n \mathbf{1}_n^T \mathbf{X} / n

in a single matrix-vector operation, avoiding iterative mean subtractions and scaling well for high-dimensional datasets.^[3]

In Linear Algebra and Regression

In ordinary least squares (OLS) regression, the centering matrix

H = I_n - \frac{1}{n} \mathbf{1}_n \mathbf{1}_n^T

, where

I_n

is the

n \times n

identity matrix and

\mathbf{1}_n

is an

n \times 1

vector of ones, is applied to the predictor matrix

X

to obtain centered predictors

HX

. This transformation subtracts the column means from each predictor, effectively removing multicollinearity between the predictors and the intercept term, which simplifies coefficient estimation and interpretation without altering the overall model fit.^[14] The resulting model

y = \mathbf{1}_n \alpha + HX \boldsymbol{\beta} + \varepsilon

allows the intercept

\alpha

to directly represent the mean response, enhancing numerical stability in computations.^[14] The centering matrix relates closely to the hat matrix in regression analysis. For the linear model

Y = X \boldsymbol{\beta} + \varepsilon

, the hat matrix is

H_X = X (X^T X)^{-1} X^T

, which projects the response vector

Y

onto the column space of

X

to yield fitted values

\hat{Y} = H_X Y

. In the intercept-only model, where

X = \mathbf{1}_n

, the hat matrix simplifies to

H_X = \frac{1}{n} \mathbf{1}_n \mathbf{1}_n^T

, and the centering matrix emerges as the orthogonal complement

H = I_n - H_X

, projecting data orthogonal to the constant subspace.^[11] This connection facilitates residual computation as

r = H Y - X_c (X_c^T X_c)^{-1} X_c^T Y

, where

X_c = H X

denotes centered predictors, underscoring the centering matrix's role in decomposing the projection.^[11] In analysis of variance (ANOVA) and balanced experimental designs, the centering matrix projects residuals orthogonal to the constant vector, enabling the partitioning of total variance into components attributable to model effects and error. For instance, in a balanced one-way ANOVA, applying

H

to the response vector isolates deviations from the grand mean, with the sum of squares for error given by

Y^T H Y

, distinct from the total sum of squares

Y^T Y

.^[15] This orthogonal projection ensures that variance components are additively decomposed, supporting F-tests for treatment effects in balanced layouts.^[15] Centering via the matrix

H

also ensures parameter identifiability in general linear hypothesis testing under constraints, such as those imposed by linear restrictions on coefficients. By centering predictors, the design matrix's column space is orthogonalized relative to the intercept, preventing singular configurations that could otherwise lead to non-unique solutions in constrained optimization.^[14] This preprocessing step maintains the model's degrees of freedom while allowing testable hypotheses like

C \boldsymbol{\beta} = \mathbf{0}

to be evaluated without aliasing due to constants.^[16] Extensions to generalized linear models (GLMs) incorporate the centering matrix for preprocessing to enforce mean-zero assumptions on predictors, aiding convergence and interpretability in non-normal response settings. In multilevel GLMs, grand-mean centering

x_{ij} = X_{ij} - \bar{X}_{..}

adjusts level-1 predictors to zero overall mean, clarifying fixed effects as deviations from the population average and reducing bias in variance estimates. This approach, while not altering slope estimates, refines intercept interpretations and supports hypothesis testing in hierarchical structures.

Examples

Numerical Illustration

To illustrate the construction and effect of the centering matrix, consider a univariate dataset given by the column vector

\mathbf{x} = \begin{pmatrix} 1 \\ 3 \\ 5 \end{pmatrix}

135

, which has a sample mean $\mu = \frac{1+3+5}{3} = 3$ .^[17] The centering matrix $H$ for $n=3$ is constructed as $H = I_3 - \frac{1}{3} J_3$ , where $I_3$ is the 3×3 identity matrix and $J_3$ is the 3×3 matrix of all ones.^[17] This yields $H = \begin{pmatrix} \frac{2}{3} & -\frac{1}{3} & -\frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} & -\frac{1}{3} \\ -\frac{1}{3} & -\frac{1}{3} & \frac{2}{3} \end{pmatrix}.$

32−31−31−3132−31−31−3132

. Applying $H$ to $\mathbf{x}$ gives the centered vector $H\mathbf{x} = \begin{pmatrix} -2 \\ 0 \\ 2 \end{pmatrix},$

−202

, which subtracts the mean from each element: $1-3 = -2$ , $3-3 = 0$ , $5-3 = 2$ .^[17] The elements of $H\mathbf{x}$ sum to zero ( $-2 + 0 + 2 = 0$ ), confirming the mean of the centered data is zero.^[17] Additionally, the squared Euclidean norm $\|H\mathbf{x}\|^2 = (-2)^2 + 0^2 + 2^2 = 8$ equals the sum of squared deviations from the mean, $\sum (x_i - \mu)^2 = (1-3)^2 + (3-3)^2 + (5-3)^2 = 8$ .^[3] To verify the idempotence property in this example, compute $H^2$ : $H^2 = H \cdot H = \begin{pmatrix} \frac{2}{3} & -\frac{1}{3} & -\frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} & -\frac{1}{3} \\ -\frac{1}{3} & -\frac{1}{3} & \frac{2}{3} \end{pmatrix} \begin{pmatrix} \frac{2}{3} & -\frac{1}{3} & -\frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} & -\frac{1}{3} \\ -\frac{1}{3} & -\frac{1}{3} & \frac{2}{3} \end{pmatrix} = \begin{pmatrix} \frac{2}{3} & -\frac{1}{3} & -\frac{1}{3} \\ -\frac{1}{3} & \frac{2}{3} & -\frac{1}{3} \\ -\frac{1}{3} & -\frac{1}{3} & \frac{2}{3} \end{pmatrix} = H.$

32−31−31−3132−31−31−3132

=H. Thus, $H\mathbf{x}$ remains unchanged under further application of $H$ , as expected for centered data.^[17] This centering operation produces data with zero mean, which is essential for subsequent computations such as sample variance, defined as $s^2 = \frac{1}{n-1} \|H\mathbf{x}\|^2 = \frac{8}{2} = 4$ here.^[3]

Multivariate Case

In the multivariate case, consider a bivariate dataset with

n = 4

observations and

p = 2

variables, represented by the data matrix

\mathbf{X} = \begin{pmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ 7 & 8 \end{pmatrix},

13572468

, where each row corresponds to an observation $[x_{i1}, x_{i2}]$ . The column means are $\bar{\mathbf{x}}_1 = 4$ and $\bar{\mathbf{x}}_2 = 5$ , computed as the average of each variable across observations.^[9] The centering matrix for $n = 4$ is $\mathbf{H} = \mathbf{I}_4 - \frac{1}{4} \mathbf{1}_4 \mathbf{1}_4^\top = \begin{pmatrix} 0.75 & -0.25 & -0.25 & -0.25 \\ -0.25 & 0.75 & -0.25 & -0.25 \\ -0.25 & -0.25 & 0.75 & -0.25 \\ -0.25 & -0.25 & -0.25 & 0.75 \end{pmatrix},$

0.75−0.25−0.25−0.25−0.250.75−0.25−0.25−0.25−0.250.75−0.25−0.25−0.25−0.250.75

, which is symmetric and idempotent. Applying $\mathbf{H}$ to $\mathbf{X}$ yields the centered matrix $\mathbf{HX} = \begin{pmatrix} -3 & -3 \\ -1 & -1 \\ 1 & 1 \\ 3 & 3 \end{pmatrix},$

−3−113−3−113

, where each column now has a mean of zero, confirming that centering subtracts the respective column means from each entry while preserving the deviations between observations.^[9]^[18] This operation maintains the cross-covariance structure of the data, as the relative relationships between variables (such as perfect positive correlation in this linear example) remain unchanged after centering, since a constant is subtracted from each variable independently.^[18] In a scatter plot of the two variables, the original points form a line segment centered at $(4, 5)$ ; post-centering, the plot shifts the point cloud so the origin coincides with this mean, without altering the orientation, spread, or relative positions of the points.^[3] Euclidean distances between the centered points are identical to those in the original data, as $\mathbf{H}$ is an orthogonal projection onto the subspace orthogonal to the all-ones vector, but distances from the points to the original origin are no longer preserved due to the mean shift.^[9]

Info Pages

Talk Pages

Special Pages

Centering matrix

Recent from talks

Recent from talks

Contribute something

Contribute something

Media Pages

Timelines

Articles

Notes collections

Notes

Notes

Days in Chronicle

Centering matrix

Definition

Properties

Application

References