Efficiency (statistics)

In statistics, efficiency is a measure of quality of an estimator, of an experimental design, or of a hypothesis testing procedure. Essentially, a more efficient estimator needs fewer input data or observations than a less efficient one to achieve the Cramér–Rao bound. An efficient estimator is characterized by having the smallest possible variance, indicating that there is a small deviance between the estimated value and the "true" value in the L2 norm sense.

The relative efficiency of two procedures is the ratio of their efficiencies, although often this concept is used where the comparison is made between a given procedure and a notional "best possible" procedure. The efficiencies and the relative efficiency of two procedures theoretically depend on the sample size available for the given procedure, but it is often possible to use the asymptotic relative efficiency (defined as the limit of the relative efficiencies as the sample size grows) as the principal comparison measure.

The efficiency of an unbiased estimator, T, of a parameter θ is defined as

where ${\displaystyle {\mathcal {I}}(\theta )}$ is the Fisher information of the sample. Thus e(T) is the minimum possible variance for an unbiased estimator divided by its actual variance. The Cramér–Rao bound can be used to prove that e(T) ≤ 1.

An efficient estimator is an estimator that estimates the quantity of interest in some “best possible” manner. The notion of “best possible” relies upon the choice of a particular loss function — the function which quantifies the relative degree of undesirability of estimation errors of different magnitudes. The most common choice of the loss function is quadratic, resulting in the mean squared error criterion of optimality.

In general, the spread of an estimator around the parameter θ is a measure of estimator efficiency and performance. This performance can be calculated by finding the mean squared error. More formally, let T be an estimator for the parameter θ. The mean squared error of T is the value ${\displaystyle \operatorname {MSE} (T)=E[(T-\theta )^{2}]}$, which can be decomposed as a sum of its variance and bias:

An estimator T₁ performs better than an estimator T₂ if ${\displaystyle \operatorname {MSE} (T_{1})<\operatorname {MSE} (T_{2})}$. For a more specific case, if T₁ and T₂ are two unbiased estimators for the same parameter θ, then the variance can be compared to determine performance. In this case, T₂ is more efficient than T₁ if the variance of T₂ is smaller than the variance of T₁, i.e. ${\displaystyle \operatorname {var} (T_{1})>\operatorname {var} (T_{2})}$ for all values of θ. This relationship can be determined by simplifying the more general case above for mean squared error; since the expected value of an unbiased estimator is equal to the parameter value, ${\displaystyle \operatorname {E} [T]=\theta }$. Therefore, for an unbiased estimator, ${\displaystyle \operatorname {MSE} (T)=\operatorname {var} (T)}$, as the ${\displaystyle (\operatorname {E} [T]-\theta )^{2}}$ term drops out for being equal to 0.

If an unbiased estimator of a parameter θ attains ${\displaystyle e(T)=1}$ for all values of the parameter, then the estimator is called efficient.