Poisson limit theorem - Сommunity Hub

Poisson limit theoremMain

Poisson limit theorem

Community hub

Poisson limit theorem

8 pages, 0 posts

0 subscribers

Recent from talks

Be the first to start a discussion here.

Be the first to start a discussion here.

Recent from talks

Be the first to start a discussion here.

Be the first to start a discussion here.

Contribute something

About hubMembersContent overviewUpdatesRules

Main reference articles

Poisson limit theorem

Poisson limit theorem

Poisson limit theorem

View on Wikipedia

from Wikipedia

Comparison of the Poisson distribution (black lines) and the binomial distribution with $n$ = 10 (red circles), $n$ = 20 (blue circles), $n$ = 1000 (green circles). All distributions have a mean of 5. The horizontal axis shows the number of events $k$ . As $n$ gets larger, the Poisson distribution becomes an increasingly better approximation for the binomial distribution with the same mean.

In probability theory, the law of rare events or Poisson limit theorem states that the Poisson distribution may be used as an approximation to the binomial distribution, under certain conditions.^[1] The theorem was named after Siméon Denis Poisson (1781–1840). A generalization of this theorem is Le Cam's theorem.

Theorem

Let $p_{n}$ be a sequence of real numbers in $[0,1]$ such that the sequence $np_{n}$ converges to a finite limit $\lambda$ . Then:

\lim _{n\to \infty }{n \choose k}p_{n}^{k}(1-p_{n})^{n-k}=e^{-\lambda }{\frac {\lambda ^{k}}{k!}}

First proof

Assume $\lambda >0$ (the case $\lambda =0$ is easier). Then

{\begin{aligned}\lim \limits _{n\rightarrow \infty }{n \choose k}p_{n}^{k}(1-p_{n})^{n-k}&=\lim _{n\to \infty }{\frac {n(n-1)(n-2)\dots (n-k+1)}{k!}}\left({\frac {\lambda }{n}}(1+o(1))\right)^{k}\left(1-{\frac {\lambda }{n}}(1+o(1))\right)^{n-k}\\&=\lim _{n\to \infty }{\frac {n^{k}+O\left(n^{k-1}\right)}{k!}}{\frac {\lambda ^{k}}{n^{k}}}\left(1-{\frac {\lambda }{n}}(1+o(1))\right)^{n}\left(1-{\frac {\lambda }{n}}(1+o(1))\right)^{-k}\\&=\lim _{n\to \infty }{\frac {\lambda ^{k}}{k!}}\left(1-{\frac {\lambda }{n}}(1+o(1))\right)^{n}.\end{aligned}}

Since

\lim _{n\to \infty }\left(1-{\frac {\lambda }{n}}(1+o(1))\right)^{n}=e^{-\lambda }

this leaves

{n \choose k}p^{k}(1-p)^{n-k}\simeq {\frac {\lambda ^{k}e^{-\lambda }}{k!}}.

Alternative proof

Using Stirling's approximation, it can be written:

{\begin{aligned}{n \choose k}p^{k}(1-p)^{n-k}&={\frac {n!}{(n-k)!k!}}p^{k}(1-p)^{n-k}\\&\simeq {\frac {{\sqrt {2\pi n}}\left({\frac {n}{e}}\right)^{n}}{{\sqrt {2\pi \left(n-k\right)}}\left({\frac {n-k}{e}}\right)^{n-k}k!}}p^{k}(1-p)^{n-k}\\&={\sqrt {\frac {n}{n-k}}}{\frac {n^{n}e^{-k}}{\left(n-k\right)^{n-k}k!}}p^{k}(1-p)^{n-k}.\end{aligned}}

Letting $n\to \infty$ and $np=\lambda$ :

{\begin{aligned}{n \choose k}p^{k}(1-p)^{n-k}&\simeq {\frac {n^{n}\,p^{k}(1-p)^{n-k}e^{-k}}{\left(n-k\right)^{n-k}k!}}\\&={\frac {n^{n}\left({\frac {\lambda }{n}}\right)^{k}\left(1-{\frac {\lambda }{n}}\right)^{n-k}e^{-k}}{n^{n-k}\left(1-{\frac {k}{n}}\right)^{n-k}k!}}\\&={\frac {\lambda ^{k}\left(1-{\frac {\lambda }{n}}\right)^{n-k}e^{-k}}{\left(1-{\frac {k}{n}}\right)^{n-k}k!}}\\&\simeq {\frac {\lambda ^{k}\left(1-{\frac {\lambda }{n}}\right)^{n}e^{-k}}{\left(1-{\frac {k}{n}}\right)^{n}k!}}.\end{aligned}}

As $n\to \infty$ , $\left(1-{\frac {x}{n}}\right)^{n}\to e^{-x}$ so:

{\begin{aligned}{n \choose k}p^{k}(1-p)^{n-k}&\simeq {\frac {\lambda ^{k}e^{-\lambda }e^{-k}}{e^{-k}k!}}\\&={\frac {\lambda ^{k}e^{-\lambda }}{k!}}\end{aligned}}

Ordinary generating functions

It is also possible to demonstrate the theorem through the use of ordinary generating functions of the binomial distribution:

G_{\operatorname {bin} }(x;p,N)\equiv \sum _{k=0}^{N}\left[{\binom {N}{k}}p^{k}(1-p)^{N-k}\right]x^{k}={\Big [}1+(x-1)p{\Big ]}^{N}

by virtue of the binomial theorem. Taking the limit $N\rightarrow \infty$ while keeping the product $pN\equiv \lambda$ constant, it can be seen:

\lim _{N\rightarrow \infty }G_{\operatorname {bin} }(x;p,N)=\lim _{N\rightarrow \infty }\left[1+{\frac {\lambda (x-1)}{N}}\right]^{N}=\mathrm {e} ^{\lambda (x-1)}=\sum _{k=0}^{\infty }\left[{\frac {\mathrm {e} ^{-\lambda }\lambda ^{k}}{k!}}\right]x^{k}

which is the OGF for the Poisson distribution. (The second equality holds due to the definition of the exponential function.)

See also

References

^ Papoulis, Athanasios; Pillai, S. Unnikrishna. Probability, Random Variables, and Stochastic Processes (4th ed.).

Revisions and contributors Edit on Wikipedia Read on Wikipedia

Poisson limit theorem

View on Grokipedia

from Grokipedia

The Poisson limit theorem, also known as the law of rare events, states that under certain conditions, the binomial distribution converges in distribution to the Poisson distribution. Specifically, consider a sequence of independent Bernoulli random variables

X_{n,1}, \dots, X_{n,n}

each with success probability

p_n = \lambda / n

, where

\lambda > 0

is fixed; let

S_n = \sum_{i=1}^n X_{n,i}

. As

n \to \infty

, the distribution of

S_n

converges to a Poisson distribution with parameter

\lambda

, meaning

P(S_n = k) \to e^{-\lambda} \lambda^k / k!

for each integer

k \geq 0

.^[1]^[2] This theorem provides a foundational approximation in probability theory, particularly for modeling the number of occurrences of rare, independent events within a fixed interval, where the expected number of events

\lambda

remains constant even as the opportunities for events proliferate but each becomes increasingly unlikely.^[1] It complements the central limit theorem by addressing scenarios where the variance

\lambda

is finite and small, rather than growing with

n

, and is especially useful when

n

is large and

p

is small, such as in queueing theory, reliability analysis, and ecology for counting infrequent incidents like defects or arrivals.^[2]^[3] The result was first derived by Siméon Denis Poisson in his 1837 work Recherches sur la probabilité des jugements en matière criminelle et en matière civile, where it emerged in the context of error analysis and rare judicial errors, though the modern interpretation as a limit theorem developed later.^[2] In 1898, Ladislaus von Bortkiewicz popularized its application to real-world rare events, such as horse-kick fatalities in Prussian cavalry, dubbing it the "law of small numbers" to emphasize the stability of small counts under Poisson-like behavior.^[4] Proofs typically rely on the method of generating functions, where the probability generating function of the binomial

(1 - p + p s)^n

approaches

e^{\lambda (s-1)}

as

n \to \infty

with

np = \lambda

, or alternatively via Stirling's approximation for factorials in the direct probability computation.^[2] Extensions of the theorem appear in more general settings, such as for dependent events or compound processes, but the core version remains central to introductory probability.^[5]

Preliminaries

Binomial distribution

The binomial distribution models the number of successes in a fixed number

n

of independent Bernoulli trials, where each trial has the same probability

p

of success and

1-p

of failure.^[6] This discrete probability distribution is fundamental in scenarios involving repeated independent experiments with binary outcomes, providing a framework for calculating probabilities of specific success counts.^[7] The probability mass function of a binomial random variable

K

is given by

P(K = k) = \binom{n}{k} p^k (1-p)^{n-k},

where

k = 0, 1, \dots, n

and

\binom{n}{k}

denotes the binomial coefficient, representing the number of ways to choose

k

successes out of

n

trials.^[8] The expected value (mean) is

E[K] = np

, and the variance is

\operatorname{Var}(K) = np(1-p)

, both of which scale linearly with

n

for fixed

p

.^[9] Named after Jacob Bernoulli, the distribution was formally introduced in his posthumously published book Ars Conjectandi in 1713, marking a foundational contribution to probability theory.^[10] Common examples include modeling the number of heads in

n

flips of a fair coin (where

p = 0.5

) or the count of defective items in a quality control sample of size

n

from a production process with defect probability

p

.^[11]^[12] In the Poisson limit theorem, the binomial distribution serves as the starting point, approximating the Poisson distribution under certain conditions on

n

and

p

.^[6]

Poisson distribution

The Poisson distribution is a discrete probability distribution that models the number of events occurring within a fixed interval of time or space, under the assumptions of a constant average rate

\lambda > 0

and independent occurrences of rare events.^[13] It arises naturally in scenarios where events are sporadic and the probability of more than one event in a very small interval is negligible.^[14] The probability mass function of a Poisson random variable

X

is given by

P(X = k) = \frac{[e](/page/E!)^{-\lambda} \lambda^[k](/page/K')}{[k](/page/K)!}, \quad k = 0, 1, 2, \dots

where

[e](/page/E!)

is the base of the natural logarithm and

[k](/page/K')!

denotes the factorial of

[k](/page/K')

.^[15] The expected value (mean) is

E[X] = \lambda

, and the variance is

\operatorname{Var}(X) = \lambda

, making the distribution equidispersed with mean equal to variance.^[15] This distribution has infinite support over the non-negative integers and is commonly applied to count data, such as the number of defects in a manufactured item or the number of arrivals at a service facility.^[13] For instance, it describes the number of radioactive decays in a sample over a fixed period or the number of customer arrivals in a queue during a given hour, assuming the events occur independently at a constant rate.^[13] The Poisson distribution serves as a limiting case for the binomial distribution when the number of trials is large and the success probability is small, with

\lambda = np

.^[16]

Theorem

Statement

The Poisson limit theorem, also known as the law of rare events, asserts that under suitable conditions, the binomial distribution converges to the Poisson distribution as the number of trials increases indefinitely while the expected number of successes remains fixed.^[17] Formally, let

X_n

follow a binomial distribution with parameters

n

and

p_n

, where

n p_n = \lambda

for a fixed

\lambda > 0

,

n \to \infty

, and thus

p_n \to 0

. Then

X_n

converges in distribution to a Poisson random variable with parameter

\lambda

, denoted

X_n \xrightarrow{d} \mathrm{Poisson}(\lambda)

Xnd

Poisson(λ).^[17] This convergence is equivalent to pointwise convergence of the probability mass functions: for each fixed nonnegative integer $k$ , $\lim_{n \to \infty} P(X_n = k) = e^{-\lambda} \frac{\lambda^k}{k!}.$ The conditions $p_n \to 0$ and $n p_n = \lambda$ (constant) ensure the approximation is valid in the regime of rare but numerous independent events.^[17]

Interpretation

The Poisson limit theorem, also known as the law of rare events, intuitively captures how the binomial distribution simplifies to the Poisson distribution under conditions of rarity and abundance. Consider a scenario with n independent Bernoulli trials, each having a small success probability p, such that the expected number of successes λ = np remains fixed as n grows large. In this regime, successes become rare events across the many trials, with the probability of exactly k successes approximating the Poisson form due to the limiting behavior of the binomial probabilities, where higher-order terms diminish and the distribution is dominated by isolated occurrences.^[18] The law of rare events formalizes this intuition: when events are sufficiently improbable yet numerous trials are conducted, the probability of exactly k occurrences approximates the Poisson form e^{-λ} λ^k / k!, assuming independence. This arises because the binomial PMF

\binom{n}{k} p^k (1-p)^{n-k}

limits to

\frac{\lambda^k e^{-\lambda}}{k!}

as n → ∞ with np = λ fixed. Originally derived by Siméon Denis Poisson in his analysis of judicial error probabilities, this principle underscores the theorem's role in modeling phenomena where successes are sparse but collectively informative.^[19]^[20]

Proofs

Direct probabilistic proof

The direct probabilistic proof establishes the Poisson limit theorem by computing the pointwise limit of the probability mass function of a binomial random variable

X_n \sim \operatorname{Bin}(n, p)

as

n \to \infty

with

p = \lambda / n

fixed, for each fixed nonnegative integer

k

. This approach relies on the known limiting behavior of the binomial probabilities under the rare events regime where the expected number of successes

\lambda = np

is constant.^[17] The probability mass function of

X_n

is given by

P(X_n = k) = \binom{n}{k} p^k (1-p)^{n-k} = \binom{n}{k} \left( \frac{\lambda}{n} \right)^k \left(1 - \frac{\lambda}{n}\right)^{n-k},

for

k = 0, 1, \dots, n

. Substituting the binomial coefficient

\binom{n}{k} = \frac{n(n-1) \cdots (n-k+1)}{k!}

yields

P(X_n = k) = \frac{n(n-1) \cdots (n-k+1)}{k!} \cdot \frac{\lambda^k}{n^k} \cdot \left(1 - \frac{\lambda}{n}\right)^{n-k}.

This can be rewritten as

P(X_n = k) = \frac{\lambda^k}{k!} \cdot \prod_{j=0}^{k-1} \left(1 - \frac{j}{n}\right) \cdot \left(1 - \frac{\lambda}{n}\right)^{n-k}.[](https://pages.cs.wisc.edu/~matthewb/pages/notes/pdf/distributions/Poisson.pdf)

To evaluate the limit as

n \to \infty

for fixed

k

, observe that the product

\prod_{j=0}^{k-1} (1 - j/n) \to 1

since each term

1 - j/n \to 1

and there are finitely many factors. Next, factor the remaining exponential term as

\left(1 - \frac{\lambda}{n}\right)^{n-k} = \left(1 - \frac{\lambda}{n}\right)^n \cdot \left(1 - \frac{\lambda}{n}\right)^{-k}.

It is well-known that

\left(1 - \frac{\lambda}{n}\right)^n \to e^{-\lambda}

and

\left(1 - \frac{\lambda}{n}\right)^{-k} \to 1

as

n \to \infty

. Therefore,

\lim_{n \to \infty} P(X_n = k) = \frac{\lambda^k}{k!} \cdot 1 \cdot e^{-\lambda} \cdot 1 = \frac{e^{-\lambda} \lambda^k}{k!},

which is the probability mass function of the

\operatorname{Poisson}(\lambda)

distribution. This convergence holds for every fixed

k \geq 0

, completing the proof.^[21]

Generating functions proof

The proof of the Poisson limit theorem using probability generating functions (PGFs) provides an elegant transform-based approach to establishing convergence in distribution from the binomial to the Poisson distribution.^[22] The PGF of a non-negative integer-valued random variable

X

is defined as

G(s) = \mathbb{E}[s^X]

for

|s| \leq 1

, and it uniquely determines the probability distribution of

X

.^[23] Consider a binomial random variable

X_n

with parameters

n

and success probability

p_n = \lambda / n

, where

\lambda > 0

is fixed. The PGF of

X_n

is

G_n(s) = \left(1 - \frac{\lambda}{n} + \frac{\lambda}{n} s \right)^n, \quad |s| \leq 1.

^[20] As

n \to \infty

, this expression converges pointwise to

G(s) = e^{\lambda (s - 1)}, \quad |s| \leq 1,

^[22] which is the PGF of a Poisson random variable with parameter

\lambda

.^[21] The convergence follows from the standard limit

\lim_{n \to \infty} \left(1 + \frac{x}{n}\right)^n = e^x

applied after rewriting the binomial PGF and taking logarithms.^[20] Since the PGF uniquely determines the distribution and PGFs converge pointwise on the compact interval

[-1, 1]

, the continuity theorem for PGFs implies that

X_n

converges in distribution to a Poisson random variable with parameter

\lambda

.^[22] This establishes the Poisson limit theorem, where the binomial distribution approximates the Poisson under the specified conditions.^[21] Although ordinary generating functions (without the probabilistic expectation) bear a close resemblance, PGFs are the standard tool for discrete distributions in this context due to their direct connection to probability mass functions via series expansion.^[23]

Characteristic functions proof

The characteristic function provides an analytic tool for establishing the Poisson limit theorem, leveraging Fourier transforms to demonstrate convergence in distribution. Consider a sequence of independent Bernoulli trials where

X_n \sim \operatorname{Bin}(n, p_n)

with

n p_n \to \lambda > 0

as

n \to \infty

. The characteristic function of

X_n

is

\phi_{X_n}(t) = \mathbb{E}[e^{i t X_n}] = (1 - p_n + p_n e^{i t})^n.

Substituting

p_n = \lambda / n

, this simplifies to

\phi_{X_n}(t) = \left(1 + \frac{\lambda (e^{i t} - 1)}{n}\right)^n.

As

n \to \infty

, the expression converges pointwise to

e^{\lambda (e^{i t} - 1)}

, which is the characteristic function of the Poisson distribution with parameter

\lambda

.^[24] To verify the limit, consider the natural logarithm of the characteristic function:

\log \phi_{X_n}(t) = n \log \left(1 + \frac{\lambda (e^{i t} - 1)}{n}\right).

For large

n

, the argument of the logarithm is small, so the Taylor expansion

\log(1 + x) = x + O(x^2)

as

x \to 0

yields

\log \phi_{X_n}(t) = n \left[ \frac{\lambda (e^{i t} - 1)}{n} + O\left(\frac{1}{n^2}\right) \right] = \lambda (e^{i t} - 1) + O\left(\frac{1}{n}\right),

which approaches

\lambda (e^{i t} - 1)

as

n \to \infty

. Exponentiating gives the desired Poisson characteristic function.^[24] Lévy's continuity theorem then implies that pointwise convergence of the characteristic functions ensures convergence in distribution:

X_n \Rightarrow \operatorname{Poisson}(\lambda)

. This theorem, which links characteristic function convergence to distributional limits under mild continuity conditions on the limiting function, underpins the result.^[24] The characteristic function approach offers broader applicability than direct probabilistic methods, readily extending to sums of independent but non-identical Bernoulli random variables (where

\sum p_{n,m} \to \lambda

and

\max p_{n,m} \to 0

) and to general infinitely divisible distributions, facilitating proofs in more abstract settings beyond simple discrete counts.^[24]

Applications and extensions

Statistical applications

The Poisson limit theorem finds significant application in statistical hypothesis testing, particularly for approximating binomial distributions in scenarios involving rare events. When testing hypotheses about the success probability in a binomial model with large sample size

n

and small probability

p

, the Poisson approximation with parameter

\lambda = np

simplifies the computation of test statistics and p-values, avoiding the need for exact binomial calculations that become computationally intensive for large

n

.^[25] For instance, in chi-squared goodness-of-fit tests for binned data where expected frequencies are low (e.g., rare outcomes in contingency tables), the Poisson model provides a more accurate alternative to the standard chi-squared approximation, which assumes adequate cell counts and can lead to inflated Type I error rates otherwise.^[26] In constructing confidence intervals for rare event probabilities, the theorem enables the use of Poisson-based methods to approximate binomial intervals efficiently. Specifically, for estimating the proportion

p

in a binomial setting, confidence bounds for

p

can be derived from those of the Poisson mean

\lambda = np

by scaling, such as

\hat{p}_L = \hat{\lambda}_L / n

and

\hat{p}_U = \hat{\lambda}_U / n

, where

\hat{\lambda}_L

and

\hat{\lambda}_U

are Poisson lower and upper bounds; this approach is particularly reliable when

np \leq 10

.^[27] Such approximations are valuable in fields like epidemiology or reliability engineering, where events are infrequent, ensuring intervals remain conservative without requiring extensive numerical integration. A practical example arises in quality control for estimating defect rates in manufacturing processes. When inspecting a large number

n

of items with a low defect probability

p

, the number of defects follows a binomial distribution that can be well-approximated by a Poisson with

\lambda = np

, facilitating quick assessments of process stability and setting control limits for defect counts per batch.^[28] The Poisson approximation also enhances computational efficiency in software implementations for statistical analyses involving large-scale binomial data. Unlike the binomial probability mass function, which requires summing up to

n+1

terms and can suffer from numerical underflow for large

n

, the Poisson formula

P(X = k) = e^{-\lambda} \lambda^k / k!

involves fewer operations and is easier to implement stably, especially in simulations or iterative algorithms for rare event modeling.^[29] Historically, the theorem's implications were applied in early 20th-century biology to model mutation rates as rare events. J.B.S. Haldane utilized the Poisson approximation to analyze the distribution of mutations in populations, treating them as infrequent occurrences in large genetic samples, which informed estimates of mutation rates and their role in evolution.^[30]

Generalizations to other limits

The Poisson limit theorem extends beyond the independent and identically distributed (i.i.d.) Bernoulli case to sums of independent but non-identical indicators, as captured by Le Cam's theorem. This result provides a bound on the total variation distance between the distribution of the sum

S = \sum_{i=1}^n X_i

, where the

X_i

are independent Bernoulli random variables with possibly different success probabilities

p_i

, and a Poisson distribution with mean

\lambda = \sum p_i

. Specifically, the distance is at most

2 \sum p_i^2

, allowing for approximation even when probabilities vary, provided they are small and their sum remains moderate.^[31] In the multivariate setting, the theorem generalizes to the approximation of a multinomial distribution by a product of independent Poisson distributions. For a multinomial vector

(N_1, \dots, N_k)

with parameters

n

trials and probabilities

(p_1, \dots, p_k)

where

\sum p_j = 1

,

k \to \infty

,

\max p_j \to 0

, and

np_j \to \lambda_j < \infty

as

n \to \infty

(with

\sum \lambda_j = n \to \infty

), the multinomial approximates the conditional distribution of independent Poissons with means

\lambda_j

given that their sum equals

n

. This extension relies on semigroup methods and applies to superpositions of Bernoulli point processes approximating Poisson processes, with error bounds in total variation distance derived from univariate Poisson binomial approximations.^[32] Poissonization provides another generalization, particularly useful in random allocation problems such as balls and bins. Here, the fixed number of balls

m

is replaced by a Poisson random variable with mean

m

, transforming the joint distribution of bin occupancies from dependent binomials to independent Poissons with means

m p_i

. Conditioning on the total number of balls recovers the original model, facilitating analysis of limits like maximum load, where the Poisson paradigm yields asymptotic results equivalent to the binomial case as

m \to \infty

. This technique simplifies proofs for rare event approximations in combinatorial settings.^[33] For dependent rare events, compound Poisson limits arise, generalizing the theorem to sums where indicators may cluster due to dependence, such as in Markov chains. In multi-state Markov chains with rare transitions, the sum of indicators for state visits converges in distribution to a compound Poisson process, where the compounding distribution reflects the cluster sizes induced by dependence. This extends earlier results for two-state chains and applies to higher-order dependencies, bounding errors via generating functions under mixing conditions.^[34] Stein's method further extends Poisson approximation to dependent settings via the Chen-Stein approach, providing explicit error bounds for sums of locally dependent indicators. The total variation distance between the law of

W = \sum X_i

and a Poisson with mean

\lambda = E[W]

is at most

(1 - e^{-\lambda})/\lambda \cdot (b_1 + b_2 + b_3)

, where the

b

-terms quantify neighborhood dependencies:

b_1

sums squared probabilities,

b_2

expected neighbor influences, and

b_3

conditional expectations outside neighborhoods. This method applies to complex dependencies in random graphs, sequences, and point processes, improving on Le Cam's bounds for non-independent cases.^[3]

References

Add your contribution

Related Hubs

No comments yet.