Hessian matrix

In mathematics, the Hessian matrix, Hessian or (less commonly) Hesse matrix is a square matrix of second-order partial derivatives of a scalar-valued function, or scalar field. It describes the local curvature of a function of many variables. The Hessian matrix was developed in the 19th century by the German mathematician Ludwig Otto Hesse and later named after him. Hesse originally used the term "functional determinants". The Hessian is sometimes denoted by H or ${\displaystyle \nabla \nabla }$ or ${\displaystyle \nabla ^{2}}$ or ${\displaystyle \nabla \otimes \nabla }$ or ${\displaystyle D^{2}}$.

Suppose ${\displaystyle f:\mathbb {R} ^{n}\to \mathbb {R} }$ is a function taking as input a vector ${\displaystyle \mathbf {x} \in \mathbb {R} ^{n}}$ and outputting a scalar ${\displaystyle f(\mathbf {x} )\in \mathbb {R} .}$ If all second-order partial derivatives of ${\displaystyle f}$ exist, then the Hessian matrix ${\displaystyle \mathbf {H} }$ of ${\displaystyle f}$ is a square ${\displaystyle n\times n}$ matrix, usually defined and arranged as $${\displaystyle \mathbf {H} _{f}={\begin{bmatrix}{\dfrac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\dfrac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\dfrac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\[2.2ex]{\dfrac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\dfrac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\dfrac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\[2.2ex]\vdots &\vdots &\ddots &\vdots \\[2.2ex]{\dfrac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\dfrac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\dfrac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}.}$$ That is, the entry of the ith row and the jth column is $${\displaystyle (\mathbf {H} _{f})_{i,j}={\frac {\partial ^{2}f}{\partial x_{i}\,\partial x_{j}}}.}$$

If furthermore the second partial derivatives are all continuous, the Hessian matrix is a symmetric matrix by the symmetry of second derivatives.

The determinant of the Hessian matrix is called the Hessian determinant.

The Hessian matrix of a function ${\displaystyle f}$ is the transpose of the Jacobian matrix of the gradient of the function ${\displaystyle f}$; that is: ${\displaystyle \mathbf {H} (f(\mathbf {x} ))=\mathbf {J} (\nabla f(\mathbf {x} ))^{\mathsf {T}}.}$

If ${\displaystyle f}$ is a homogeneous polynomial in three variables, the equation ${\displaystyle f=0}$ is the implicit equation of a plane projective curve. The inflection points of the curve are exactly the non-singular points where the Hessian determinant is zero. It follows by Bézout's theorem that a cubic plane curve has at most 9 inflection points, since the Hessian determinant is a polynomial of degree 3.

The Hessian matrix of a convex function is positive semi-definite. Refining this property allows us to test whether a critical point ${\displaystyle x}$ is a local maximum, local minimum, or a saddle point, as follows:

If the Hessian is positive-definite at ${\displaystyle x,}$ then ${\displaystyle f}$ attains an isolated local minimum at ${\displaystyle x.}$ If the Hessian is negative-definite at ${\displaystyle x,}$ then ${\displaystyle f}$ attains an isolated local maximum at ${\displaystyle x.}$ If the Hessian has both positive and negative eigenvalues, then ${\displaystyle x}$ is a saddle point for ${\displaystyle f.}$ Otherwise the test is inconclusive. This implies that at a local minimum the Hessian is positive-semidefinite, and at a local maximum the Hessian is negative-semidefinite.