Inverse transform sampling
View on WikipediaInverse transform sampling (also known as inversion sampling, the inverse probability integral transform, the inverse transformation method, or the Smirnov transform) is a basic method for pseudo-random number sampling, i.e., for generating sample numbers at random from any probability distribution given its cumulative distribution function.
Inverse transformation sampling takes uniform samples of a number between 0 and 1, interpreted as a probability, and then returns the smallest number such that for the cumulative distribution function of a random variable. For example, imagine that is the standard normal distribution with mean zero and standard deviation one. The table below shows samples taken from the uniform distribution and their representation on the standard normal distribution.
| .5 | 0 |
| .975 | 1.95996 |
| .995 | 2.5758 |
| .999999 | 4.75342 |
| 1-2−52 | 8.12589 |

We are randomly choosing a proportion of the area under the curve and returning the number in the domain such that exactly this proportion of the area occurs to the left of that number. Intuitively, we are unlikely to choose a number in the far end of tails because there is very little area in them which would require choosing a number very close to zero or one.
Computationally, this method involves computing the quantile function of the distribution — in other words, computing the cumulative distribution function (CDF) of the distribution (which maps a number in the domain to a probability between 0 and 1) and then inverting that function. This is the source of the term "inverse" or "inversion" in most of the names for this method. Note that for a discrete distribution, computing the CDF is not in general too difficult: we simply add up the individual probabilities for the various points of the distribution. For a continuous distribution, however, we need to integrate the probability density function (PDF) of the distribution, which is impossible to do analytically for most distributions (including the normal distribution). As a result, this method may be computationally inefficient for many distributions and other methods are preferred; however, it is a useful method for building more generally applicable samplers such as those based on rejection sampling.
For the normal distribution, the lack of an analytical expression for the corresponding quantile function means that other methods (e.g. the Box–Muller transform) may be preferred computationally. It is often the case that, even for simple distributions, the inverse transform sampling method can be improved on:[1] see, for example, the ziggurat algorithm and rejection sampling. On the other hand, it is possible to approximate the quantile function of the normal distribution extremely accurately using moderate-degree polynomials, and in fact the method of doing this is fast enough that inversion sampling is now the default method for sampling from a normal distribution in the statistical package R.[2]
Formal statement
[edit]For any random variable , the random variable has the same distribution as , where is the generalized inverse of the cumulative distribution function of and is uniform on .[3]
For continuous random variables, the inverse probability integral transform is indeed the inverse of the probability integral transform, which states that for a continuous random variable with cumulative distribution function , the random variable is uniform on .

Intuition
[edit]From , we want to generate with CDF We assume to be a continuous, strictly increasing function, which provides good intuition.
We want to see if we can find some strictly monotone transformation , such that . We will have
where the last step used that when is uniform on .
So we got to be the inverse function of , or, equivalently
Therefore, we can generate from
The method
[edit]

The problem that the inverse transform sampling method solves is as follows:
- Let be a random variable whose distribution can be described by the cumulative distribution function .
- We want to generate values of which are distributed according to this distribution.
The inverse transform sampling method works as follows:
- Generate a random number from the standard uniform distribution in the interval , i.e. from
- Find the generalized inverse of the desired CDF, i.e. .
- Compute . The computed random variable has distribution and thereby the same law as .
Expressed differently, given a cumulative distribution function and a uniform variable , the random variable has the distribution .[3]
In the continuous case, a treatment of such inverse functions as objects satisfying differential equations can be given.[4] Some such differential equations admit explicit power series solutions, despite their non-linearity.[5]
Examples
[edit]- As an example, suppose we have a random variable and a cumulative distribution function
- In order to perform an inversion we want to solve for
- From here we would perform steps one, two and three.
- As another example, we use the exponential distribution with for x ≥ 0 (and 0 otherwise). By solving y=F(x) we obtain the inverse function
- It means that if we draw some from a and compute This has exponential distribution.
- The idea is illustrated in the following graph:

Random numbers yi are generated from a uniform distribution between 0 and 1, i.e. Y ~ U(0, 1). They are sketched as colored points on the y-axis. Each of the points is mapped according to x=F−1(y), which is shown with gray arrows for two example points. In this example, we have used an exponential distribution. Hence, for x ≥ 0, the probability density is and the cumulative distribution function is . Therefore, . We can see that using this method, many points end up close to 0 and only few points end up having high x-values - just as it is expected for an exponential distribution. - Note that the distribution does not change if we start with 1-y instead of y. For computational purposes, it therefore suffices to generate random numbers y in [0, 1] and then simply calculate
Proof of correctness
[edit]Let be a cumulative distribution function, and let be its generalized inverse function (using the infimum because CDFs are weakly monotonic and right-continuous):[6]
Claim: If is a uniform random variable on then has as its CDF.
Proof:
Truncated distribution
[edit]Inverse transform sampling can be simply extended to cases of truncated distributions on the interval without the cost of rejection sampling: the same algorithm can be followed, but instead of generating a random number uniformly distributed between 0 and 1, generate uniformly distributed between and , and then again take .
Reduction of the number of inversions
[edit]In order to obtain a large number of samples, one needs to perform the same number of inversions of the distribution. One possible way to reduce the number of inversions while obtaining a large number of samples is the application of the so-called Stochastic Collocation Monte Carlo sampler (SCMC sampler) within a polynomial chaos expansion framework. This allows us to generate any number of Monte Carlo samples with only a few inversions of the original distribution with independent samples of a variable for which the inversions are analytically available, for example the standard normal variable.[7]
Software implementations
[edit]There are software implementations available for applying the inverse sampling method by using numerical approximations of the inverse in the case that it is not available in closed form. For example, an approximation of the inverse can be computed if the user provides some information about the distributions such as the PDF [8] or the CDF.
- C library UNU.RAN [9]
- R library Runuran [10]
- Python subpackage sampling in scipy.stats[11][12]
See also
[edit]- Probability integral transform
- Copula, defined by means of probability integral transform.
- Quantile function, for the explicit construction of inverse CDFs.
- Inverse distribution function for a precise mathematical definition for distributions with discrete components.
- Rejection sampling is another common technique to generate random variates that does not rely on inversion of the CDF.
References
[edit]- ^ Luc Devroye (1986). Non-Uniform Random Variate Generation (PDF). New York: Springer-Verlag. Archived from the original (PDF) on 2014-08-18. Retrieved 2012-04-12.
- ^ "R: Random Number Generation".
- ^ a b McNeil, Alexander J.; Frey, Rüdiger; Embrechts, Paul (2005). Quantitative risk management. Princeton Series in Finance. Princeton University Press, Princeton, NJ. p. 186. ISBN 0-691-12255-5.
- ^ Steinbrecher, György; Shaw, William T. (19 March 2008). "Quantile mechanics". European Journal of Applied Mathematics. 19 (2): 87–112. doi:10.1017/S0956792508007341. S2CID 6899308.
- ^ Arridge, Simon; Maass, Peter; Öktem, Ozan; Schönlieb, Carola-Bibiane (2019). "Solving inverse problems using data-driven models". Acta Numerica. 28: 1–174. doi:10.1017/S0962492919000059. ISSN 0962-4929. S2CID 197480023.
- ^ Luc Devroye (1986). "Section 2.2. Inversion by numerical solution of F(X) = U" (PDF). Non-Uniform Random Variate Generation. New York: Springer-Verlag.
- ^ L.A. Grzelak, J.A.S. Witteveen, M. Suarez, and C.W. Oosterlee. The stochastic collocation Monte Carlo sampler: Highly efficient sampling from “expensive” distributions. https://ssrn.com/abstract=2529691
- ^ Derflinger, Gerhard; Hörmann, Wolfgang; Leydold, Josef (2010). "Random variate generation by numerical inversion when only the density is known" (PDF). ACM Transactions on Modeling and Computer Simulation. 20 (4). doi:10.1145/945511.945517.
- ^ "UNU.RAN - Universal Non-Uniform RANdom number generators".
- ^ "Runuran: R Interface to the 'UNU.RAN' Random Variate Generators". 17 January 2023.
- ^ "Random Number Generators (Scipy.stats.sampling) — SciPy v1.12.0 Manual".
- ^ Baumgarten, Christoph; Patel, Tirth (2022). "Automatic random variate generation in Python". Proceedings of the 21st Python in Science Conference. pp. 46–51. doi:10.25080/majora-212e5952-007.
Inverse transform sampling
View on GrokipediaConceptual Foundations
Intuition
Inverse transform sampling provides a fundamental technique for generating random variables from a desired probability distribution using only uniform random numbers, typically drawn from a standard uniform distribution over the interval (0,1). The core idea relies on the cumulative distribution function (CDF) of the target distribution, which accumulates probability mass up to any given value. By applying the inverse of this CDF to a uniform random variable, the method effectively maps the flat, evenly distributed probabilities of the uniform input to the uneven shape of the target distribution, producing samples that mimic the original distribution's characteristics.[1][3] This mapping can be intuitively understood through the lens of percentiles or quantiles. A uniform random draw U between 0 and 1 represents a random probability level, akin to selecting a random percentile. The inverse CDF then translates this percentile into the corresponding value in the target distribution—for instance, if U is 0.75, it identifies the value below which 75% of the distribution's probability lies. This process ensures that the generated samples respect the relative frequencies and densities of the target distribution, transforming the simplicity of uniform randomness into targeted variability.[1][4] The method's appeal lies in its directness, particularly for distributions where the inverse CDF can be computed explicitly or approximated efficiently. When the inverse is readily available, such as for exponential or uniform distributions themselves, the transformation requires minimal computational overhead, making it a reliable starting point for simulation tasks without needing complex rejection or approximation schemes. This straightforwardness stems from the probabilistic guarantee that the inverse operation reverses the CDF's probability accumulation, yielding exact samples from the target.[3][4]Formal Statement
Inverse transform sampling relies on the cumulative distribution function (CDF) of a target continuous random variable , which is defined as , where is the probability density function (PDF) of .[5] The inverse CDF, or quantile function, is formally defined as for .[6] For simplicity, the following theorem assumes that is continuous and strictly increasing (hence invertible), though the method generalizes to discrete distributions where may have jumps by using the same generalized inverse definition.[7] Theorem: Let . Then the random variable has CDF , meaning follows the target distribution of the original random variable.[8]Algorithm and Proof
The Method
Inverse transform sampling generates random variates from a target probability distribution by leveraging its cumulative distribution function (CDF) and a uniform random number generator, as established by the formal theorem that maps uniform variates to the target distribution.[1] For continuous distributions, the method proceeds in three main steps to produce a single sample : first, generate a uniform random variate ; second, compute the inverse CDF applied to , denoted , where is the CDF of the target distribution; third, repeat the process independently for each desired sample.[1] This approach requires access to either the explicit form of the inverse CDF or the ability to evaluate the CDF itself.[1] When a closed-form expression for is unavailable, numerical methods such as the bisection algorithm can approximate the inversion by solving for , though exact closed-form inverses are preferred for efficiency and precision.[9] The adaptation for discrete distributions modifies the inversion to account for the step-like nature of the CDF, using the generalized inverse .[1] The algorithm follows similar steps: generate ; then set to the smallest support point such that the cumulative probability up to , , is at least , or equivalently, find where , with denoting the probability mass function; repeat for multiple samples.[10] This requires precomputing or incrementally calculating the cumulative probabilities over the discrete support, in addition to the uniform random number generator.[1] Unlike the continuous case, no numerical root-finding is typically needed, as the discrete search can be performed via direct comparison or binary search on the cumulative sums for larger supports.[10]Proof of Correctness
To prove the correctness of the inverse transform sampling method, consider a target random variable with cumulative distribution function (CDF) , where is continuous and strictly increasing, and let be an independent uniform random variable. The method generates , where is the generalized inverse CDF. The goal is to show that has the same distribution as , i.e., for all . For the strictly increasing case, the events and are equivalent. To see this, suppose ; then by the monotonicity of . Conversely, if , then , again using monotonicity. Thus,Variants and Optimizations
Truncated Distributions
A truncated distribution arises when a random variable is restricted to a finite interval , where , and the original probability density function (PDF) and cumulative distribution function (CDF) are renormalized to ensure the total probability over the interval sums to 1. The PDF of the truncated distribution is given byReduction of Inversions
Inverting the cumulative distribution function (CDF) $ F $ for complex distributions often lacks a closed-form solution, leading to high computational costs when repeated sampling is required, such as in Monte Carlo simulations. To address this, techniques focus on approximations and precomputation to minimize the need for full numerical inversions each time, enabling efficient generation of large numbers of variates while preserving the correctness of the inverse transform sampling method.[4] One common strategy employs lookup tables based on piecewise linear approximations of the inverse CDF $ F^{-1} $. The support of the distribution is partitioned into a finite number of intervals, and within each interval, $ F^{-1} $ is approximated by a linear function fitted to precomputed points from the exact CDF. For a uniform random variate $ U $, sampling involves locating the appropriate interval via binary search or direct indexing into the table, followed by linear interpolation to obtain the approximate $ X = F^{-1}(U) $. This approach significantly reduces per-sample computation to constant-time operations after an initial preprocessing step, with error controlled by the number of pieces; for instance, approximations with 16 segments can achieve root mean square errors around 10^{-4} for distributions like the non-central chi-squared using piecewise cubic interpolation. Such methods are particularly effective for distributions where $ F $ can be evaluated accurately but inversion is cumbersome.[12] Another key method is numerical inversion using the Newton-Raphson iteration, which solves $ F(x) = u $ (where $ u $ is the uniform input) through successive approximations:Applications and Examples
Uniform Distribution
The inverse transform sampling method applied to a uniform distribution on the interval is particularly straightforward, as the cumulative distribution function (CDF) is for , yielding the inverse where .[1] This directly maps the uniform input to the desired uniform output, illustrating the method's identity transformation in this trivial case.[1] For instance, with , , and , the sample is .[13]Exponential Distribution
For the exponential distribution with rate parameter , the CDF is for , and the inverse is .[1] To generate a sample, first draw ; then compute .[1] Consider and : , , so .[1] This step-by-step process confirms the sample follows the exponential distribution, as the transformation ensures the CDF of matches the target.[1]Normal Distribution
The standard normal distribution lacks a closed-form inverse CDF, denoted , where .[14] Numerical methods, such as the bisection algorithm, solve by iteratively narrowing an interval containing the root until convergence.[15] For example, to find , start with bounds and bisect: evaluate at the midpoint, retain the subinterval where brackets 0.3, repeating for desired precision; this yields .[16] Such inversion produces standard normal samples, scalable to general normals via $ \mu + \sigma x $.[14]Discrete Example: Bernoulli Distribution
For a Bernoulli distribution with success probability , the CDF is for , for , and for .[1] The inverse transform sets if and otherwise, where .[1] For and , since , ; if , .[1] This binary decision exemplifies the method for discrete cases.[13]Software Implementations
Inverse transform sampling is commonly implemented in Python using the SciPy library, which provides the percent point function (ppf) as the inverse cumulative distribution function for various distributions. To generate samples, one first imports the necessary modules and creates a random variable instance, then applies the ppf to uniform random variates generated by NumPy. For example, for an exponential distribution with rate parameter λ=1:import numpy as np
from scipy import stats
rv = stats.expon(scale=1) # Exponential with mean 1
n = 1000
u = np.random.uniform(size=n)
samples = rv.ppf(u)
This produces an array of samples following the target distribution, with the ppf handling the inversion efficiently for built-in distributions.
In R, the method is implemented through quantile functions (prefixed with 'q' for distributions), applied to uniform random variates from runif(). For the same exponential distribution:
set.seed(42)
n <- 1000
u <- runif(n)
samples <- qexp(u, rate=1)
The qexp() function serves as the inverse CDF, enabling straightforward sampling when the quantile is available, as it is for standard distributions in the stats package.
For C++, implementations typically use the #include <random>
#include <vector>
#include <cmath>
#include <iostream>
int main() {
std::random_device rd;
std::mt19937 gen(rd());
std::uniform_real_distribution<double> dis(0.0, 1.0);
int n = 1000;
std::vector<double> samples(n);
double lambda = 1.0;
for (int i = 0; i < n; ++i) {
double u = dis(gen);
samples[i] = -std::log(1.0 - u) / lambda;
}
// Output or process samples
return 0;
}
This approach is suitable for distributions with closed-form inverses, ensuring high performance in numerical simulations.
Major libraries facilitate general implementations: NumPy and SciPy in Python support vectorized ppf calls for efficiency across array sizes, while Boost.Math in C++ provides quantile functions (e.g., boost::math::quantile) for over 30 distributions, including approximations for non-invertible cases like the normal distribution via the error function inverse. These libraries often include numerical safeguards, such as handling edge cases where the inverse CDF is undefined.
Best practices for implementation emphasize vectorization to leverage optimized linear algebra routines, as in NumPy's array operations, which can generate millions of samples in seconds on modern hardware. Additionally, error handling is crucial for uniform variates exactly at 0 or 1, which may cause logarithmic singularities or unbounded outputs; libraries like SciPy clip or approximate these to avoid NaN or infinity, ensuring robust sampling in production code.

