Exponential dispersion model

In probability and statistics, the class of exponential dispersion models (EDM), also called exponential dispersion family (EDF), is a set of probability distributions that represents a generalisation of the natural exponential family.^[1][2][3] Exponential dispersion models play an important role in statistical theory, in particular in generalized linear models because they have a special structure which enables deductions to be made about appropriate statistical inference.

There are two versions to formulate an exponential dispersion model.

In the univariate case, a real-valued random variable ${\displaystyle X}$ belongs to the additive exponential dispersion model with canonical parameter ${\displaystyle \theta }$ and index parameter ${\displaystyle \lambda }$, ${\displaystyle X\sim \mathrm {ED} ^{*}(\theta ,\lambda )}$, if its probability density function can be written as

${\displaystyle f_{X}(x\mid \theta ,\lambda )=h^{*}(\lambda ,x)\exp \left(\theta x-\lambda A(\theta )\right)\,\!.}$

The distribution of the transformed random variable ${\displaystyle Y={\frac {X}{\lambda }}}$ is called reproductive exponential dispersion model, ${\displaystyle Y\sim \mathrm {ED} (\mu ,\sigma ^{2})}$, and is given by

${\displaystyle f_{Y}(y\mid \mu ,\sigma ^{2})=h(\sigma ^{2},y)\exp \left({\frac {\theta y-A(\theta )}{\sigma ^{2}}}\right)\,\!,}$

with ${\displaystyle \sigma ^{2}={\frac {1}{\lambda }}}$ and ${\displaystyle \mu =A'(\theta )}$, implying ${\displaystyle \theta =(A')^{-1}(\mu )}$. The terminology dispersion model stems from interpreting ${\displaystyle \sigma ^{2}}$ as dispersion parameter. For fixed parameter ${\displaystyle \sigma ^{2}}$, the ${\displaystyle \mathrm {ED} (\mu ,\sigma ^{2})}$ is a natural exponential family.

In the multivariate case, the n-dimensional random variable ${\displaystyle \mathbf {X} }$ has a probability density function of the following form^[1]

${\displaystyle f_{\mathbf {X} }(\mathbf {x} |{\boldsymbol {\theta }},\lambda )=h(\lambda ,\mathbf {x} )\exp \left(\lambda ({\boldsymbol {\theta }}^{\top }\mathbf {x} -A({\boldsymbol {\theta }}))\right)\,\!,}$

where the parameter ${\displaystyle {\boldsymbol {\theta }}}$ has the same dimension as ${\displaystyle \mathbf {X} }$.

The cumulant-generating function of ${\displaystyle Y\sim \mathrm {ED} (\mu ,\sigma ^{2})}$ is given by

${\displaystyle K(t;\mu ,\sigma ^{2})=\log \operatorname {E} [e^{tY}]={\frac {A(\theta +\sigma ^{2}t)-A(\theta )}{\sigma ^{2}}}\,\!,}$

with ${\displaystyle \theta =(A')^{-1}(\mu )}$

Mean and variance of ${\displaystyle Y\sim \mathrm {ED} (\mu ,\sigma ^{2})}$ are given by

${\displaystyle \operatorname {E} [Y]=\mu =A'(\theta )\,,\quad \operatorname {Var} [Y]=\sigma ^{2}A''(\theta )=\sigma ^{2}V(\mu )\,\!,}$

with unit variance function ${\displaystyle V(\mu )=A''((A')^{-1}(\mu ))}$.

If ${\displaystyle Y_{1},\ldots ,Y_{n}}$ are i.i.d. with ${\displaystyle Y_{i}\sim \mathrm {ED} \left(\mu ,{\frac {\sigma ^{2}}{w_{i}}}\right)}$, i.e. same mean ${\displaystyle \mu }$ and different weights ${\displaystyle w_{i}}$, the weighted mean is again an ${\displaystyle \mathrm {ED} }$ with

${\displaystyle \sum _{i=1}^{n}{\frac {w_{i}Y_{i}}{w_{\bullet }}}\sim \mathrm {ED} \left(\mu ,{\frac {\sigma ^{2}}{w_{\bullet }}}\right)\,\!,}$

with ${\displaystyle w_{\bullet }=\sum _{i=1}^{n}w_{i}}$. Therefore ${\displaystyle Y_{i}}$ are called reproductive.

The probability density function of an ${\displaystyle \mathrm {ED} (\mu ,\sigma ^{2})}$ can also be expressed in terms of the unit deviance ${\displaystyle d(y,\mu )}$ as

${\displaystyle f_{Y}(y\mid \mu ,\sigma ^{2})={\tilde {h}}(\sigma ^{2},y)\exp \left(-{\frac {d(y,\mu )}{2\sigma ^{2}}}\right)\,\!,}$

where the unit deviance takes the special form ${\displaystyle d(y,\mu )=yf(\mu )+g(\mu )+h(y)}$ or in terms of the unit variance function as ${\displaystyle d(y,\mu )=2\int _{\mu }^{y}\!{\frac {y-t}{V(t)}}\,dt}$.

Many very common probability distributions belong to the class of EDMs, among them are: normal distribution, binomial distribution, Poisson distribution, negative binomial distribution, gamma distribution, inverse Gaussian distribution, and Tweedie distribution.