Minimum-distance estimation

Minimum-distance estimation (MDE) is a conceptual method for fitting a statistical model to data, usually the empirical distribution. Often-used estimators such as ordinary least squares can be thought of as special cases of minimum-distance estimation.

While consistent and asymptotically normal, minimum-distance estimators are generally not statistically efficient when compared to maximum likelihood estimators, because they omit the Jacobian usually present in the likelihood function. This, however, substantially reduces the computational complexity of the optimization problem.

Let ${\displaystyle \displaystyle X_{1},\ldots ,X_{n}}$ be an independent and identically distributed (iid) random sample from a population with distribution ${\displaystyle F(x;\theta )\colon \theta \in \Theta }$ and ${\displaystyle \Theta \subseteq \mathbb {R} ^{k}(k\geq 1)}$.

Let ${\displaystyle \displaystyle F_{n}(x)}$ be the empirical distribution function based on the sample.

Let ${\displaystyle {\hat {\theta }}}$ be an estimator for ${\displaystyle \displaystyle \theta }$. Then ${\displaystyle F(x;{\hat {\theta }})}$ is an estimator for ${\displaystyle \displaystyle F(x;\theta )}$.

Let ${\displaystyle d[\cdot ,\cdot ]}$ be a functional returning some measure of "distance" between its two arguments. The functional ${\displaystyle \displaystyle d}$ is also called the criterion function.

If there exists a ${\displaystyle {\hat {\theta }}\in \Theta }$ such that ${\displaystyle d[F(x;{\hat {\theta }}),F_{n}(x)]=\inf\{d[F(x;\theta ),F_{n}(x)];\theta \in \Theta \}}$, then ${\displaystyle {\hat {\theta }}}$ is called the minimum-distance estimate of ${\displaystyle \displaystyle \theta }$.

(Drossos & Philippou 1980, p. 121)