Gaussian process

In probability theory and statistics, a Gaussian process is a stochastic process (a collection of random variables indexed by time or space), such that every finite collection of those random variables has a multivariate normal distribution. The distribution of a Gaussian process is the joint distribution of all those (infinitely many) random variables, and as such, it is a distribution over functions with a continuous domain, e.g. time or space.

The concept of Gaussian processes is named after Carl Friedrich Gauss because it is based on the notion of the Gaussian distribution (normal distribution). Gaussian processes can be seen as an infinite-dimensional generalization of multivariate normal distributions.

Gaussian processes are useful in statistical modelling, benefiting from properties inherited from the normal distribution. For example, if a random process is modelled as a Gaussian process, the distributions of various derived quantities can be obtained explicitly. Such quantities include the average value of the process over a range of times and the error in estimating the average using sample values at a small set of times. While exact models often scale poorly as the amount of data increases, multiple approximation methods have been developed which often retain good accuracy while drastically reducing computation time.

A time continuous stochastic process ${\displaystyle \left\{X_{t};t\in T\right\}}$ is Gaussian if and only if for every finite set of indices ${\displaystyle t_{1},\ldots ,t_{k}}$ in the index set ${\displaystyle T}$

$${\displaystyle \mathbf {X} _{t_{1},\ldots ,t_{k}}=(X_{t_{1}},\ldots ,X_{t_{k}})}$$

is a multivariate Gaussian random variable. As the sum of independent and Gaussian distributed random variables is again Gaussian distributed, that is the same as saying every linear combination of ${\displaystyle (X_{t_{1}},\ldots ,X_{t_{k}})}$ has a univariate Gaussian (or normal) distribution.

Using characteristic functions of random variables with ${\displaystyle i}$ denoting the imaginary unit such that ${\displaystyle i^{2}=-1}$, the Gaussian property can be formulated as follows: ${\displaystyle \left\{X_{t};t\in T\right\}}$ is Gaussian if and only if, for every finite set of indices ${\displaystyle t_{1},\ldots ,t_{k}}$, there are real-valued ${\displaystyle \sigma _{\ell j}}$, ${\displaystyle \mu _{\ell }}$ with ${\displaystyle \sigma _{jj}>0}$ such that the following equality holds for all ${\displaystyle s_{1},s_{2},\ldots ,s_{k}\in \mathbb {R} }$,

$${\displaystyle {\mathbb {E} }\left[\exp \left(i\sum _{\ell =1}^{k}s_{\ell }\,\mathbf {X} _{t_{\ell }}\right)\right]=\exp \left(-{\tfrac {1}{2}}\sum _{\ell ,j}\sigma _{\ell j}s_{\ell }s_{j}+i\sum _{\ell }\mu _{\ell }s_{\ell }\right),}$$