Hubbry Logo
Nyquist–Shannon sampling theoremNyquist–Shannon sampling theoremMain
Open search
Nyquist–Shannon sampling theorem
Community hub
Nyquist–Shannon sampling theorem
logo
8 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
Contribute something
Nyquist–Shannon sampling theorem
Nyquist–Shannon sampling theorem
from Wikipedia

The Nyquist–Shannon sampling theorem is an essential principle for digital signal processing linking the frequency range of a signal and the sample rate required to avoid a type of distortion called aliasing. The theorem states that the sample rate must be at least twice the bandwidth of the signal to avoid aliasing. In practice, it is used to select band-limiting filters to keep aliasing below an acceptable amount when an analog signal is sampled or when sample rates are changed within a digital signal processing function.

Example of magnitude of the Fourier transform of a bandlimited function

The Nyquist–Shannon sampling theorem is a theorem in the field of signal processing which serves as a fundamental bridge between continuous-time signals and discrete-time signals. It establishes a sufficient condition for a sample rate that permits a discrete sequence of samples to capture all the information from a continuous-time signal of finite bandwidth.

Strictly speaking, the theorem only applies to a class of mathematical functions having a Fourier transform that is zero outside of a finite region of frequencies. Intuitively we expect that when one reduces a continuous function to a discrete sequence and interpolates back to a continuous function, the fidelity of the result depends on the density (or sample rate) of the original samples. The sampling theorem introduces the concept of a sample rate that is sufficient for perfect fidelity for the class of functions that are band-limited to a given bandwidth, such that no actual information is lost in the sampling process. It expresses the sufficient sample rate in terms of the bandwidth for the class of functions. The theorem also leads to a formula for perfectly reconstructing the original continuous-time function from the samples.

Perfect reconstruction may still be possible when the sample-rate criterion is not satisfied, provided other constraints on the signal are known (see § Sampling of non-baseband signals below and compressed sensing). In some cases (when the sample-rate criterion is not satisfied), utilizing additional constraints allows for approximate reconstructions. The fidelity of these reconstructions can be verified and quantified utilizing Bochner's theorem.[1]

The name Nyquist–Shannon sampling theorem honours Harry Nyquist and Claude Shannon, but the theorem was also previously discovered by E. T. Whittaker (published in 1915), and Shannon cited Whittaker's paper in his work. The theorem is thus also known by the names Whittaker–Shannon sampling theorem, Whittaker–Shannon, and Whittaker–Nyquist–Shannon, and may also be referred to as the cardinal theorem of interpolation.

Introduction

[edit]

Sampling is a process of converting a signal (for example, a function of continuous time or space) into a sequence of values (a function of discrete time or space). Shannon's version of the theorem states:[2]

TheoremIf a function contains no frequencies higher than B hertz, then it can be completely determined from its ordinates at a sequence of points spaced less than seconds apart.

A sufficient sample-rate is therefore anything larger than samples per second. Equivalently, for a given sample rate , perfect reconstruction is guaranteed possible for a bandlimit .

When the bandlimit is too high (or there is no bandlimit), the reconstruction exhibits imperfections known as aliasing. Modern statements of the theorem are sometimes careful to explicitly state that must contain no sinusoidal component at exactly frequency or that must be strictly less than one half the sample rate. The threshold is called the Nyquist rate and is an attribute of the continuous-time input to be sampled. The sample rate must exceed the Nyquist rate for the samples to suffice to represent The threshold is called the Nyquist frequency and is an attribute of the sampling equipment. All meaningful frequency components of the properly sampled exist below the Nyquist frequency. The condition described by these inequalities is called the Nyquist criterion, or sometimes the Raabe condition. The theorem is also applicable to functions of other domains, such as space, in the case of a digitized image. The only change, in the case of other domains, is the units of measure attributed to and

The normalized sinc function: sin(πx) / (πx) ... showing the central peak at x = 0, and zero-crossings at the other integer values of x.

The symbol is customarily used to represent the interval between adjacent samples and is called the sample period or sampling interval. The samples of function are commonly denoted by [3] (alternatively in older signal processing literature), for all integer values of The multiplier is a result of the transition from continuous time to discrete time (see Discrete-time Fourier transform#Relation to Fourier Transform), and it is needed to preserve the energy of the signal as varies.

A mathematically ideal way to interpolate the sequence involves the use of sinc functions. Each sample in the sequence is replaced by a sinc function, centered on the time axis at the original location of the sample with the amplitude of the sinc function scaled to the sample value, Subsequently, the sinc functions are summed into a continuous function. A mathematically equivalent method uses the Dirac comb and proceeds by convolving one sinc function with a series of Dirac delta pulses, weighted by the sample values. Neither method is numerically practical. Instead, some type of approximation of the sinc functions, finite in length, is used. The imperfections attributable to the approximation are known as interpolation error.

Practical digital-to-analog converters produce neither scaled and delayed sinc functions, nor ideal Dirac pulses. Instead they produce a piecewise-constant sequence of scaled and delayed rectangular pulses (the zero-order hold), usually followed by a lowpass filter (called an "anti-imaging filter") to remove spurious high-frequency replicas (images) of the original baseband signal.

Aliasing

[edit]
The samples of two sine waves can be identical when at least one of them is at a frequency above half the sample rate.

When is a function with a Fourier transform :

Then the samples of are sufficient to create a periodic summation of (see Discrete-time Fourier transform#Relation to Fourier Transform):

(top blue) and (bottom blue) are continuous Fourier transforms of two different functions, and (not shown). When the functions are sampled at rate , the images (green) are added to the original transforms (blue) when one examines the discrete-time Fourier transforms (DTFT) of the sequences. In this hypothetical example, the DTFTs are identical, which means the sampled sequences are identical, even though the original continuous pre-sampled functions are not. If these were audio signals, and might not sound the same. But their samples (taken at rate ) are identical and would lead to identical reproduced sounds; thus is an alias of at this sample rate.

which is a periodic function and its equivalent representation as a Fourier series, whose coefficients are . This function is also known as the discrete-time Fourier transform (DTFT) of the sample sequence.

As depicted, copies of are shifted by multiples of the sampling rate and combined by addition. For a band-limited function and sufficiently large it is possible for the copies to remain distinct from each other. But if the Nyquist criterion is not satisfied, adjacent copies overlap, and it is not possible in general to discern an unambiguous Any frequency component above is indistinguishable from a lower-frequency component, called an alias, associated with one of the copies. In such cases, the customary interpolation techniques produce the alias, rather than the original component. When the sample-rate is pre-determined by other considerations (such as an industry standard), is usually filtered to reduce its high frequencies to acceptable levels before it is sampled. The type of filter required is a lowpass filter, and in this application it is called an anti-aliasing filter.

Spectrum, , of a properly sampled bandlimited signal (blue) and the adjacent DTFT images (green) that do not overlap. A brick-wall low-pass filter, , removes the images, leaves the original spectrum, , and recovers the original signal from its samples.
The figure on the left shows a function (in gray/black) being sampled and reconstructed (in gold) at steadily increasing sample-densities, while the figure on the right shows the frequency spectrum of the gray/black function, which does not change. The highest frequency in the spectrum is half the width of the entire spectrum. The width of the steadily-increasing pink shading is equal to the sample-rate. When it encompasses the entire frequency spectrum it is twice as large as the highest frequency, and that is when the reconstructed waveform matches the sampled one.

Derivation as a special case of Poisson summation

[edit]

When there is no overlap of the copies (also known as "images") of , the term of Eq.1 can be recovered by the product:

where:

The sampling theorem is proved since uniquely determines .

All that remains is to derive the formula for reconstruction. need not be precisely defined in the region because is zero in that region. However, the worst case is when the Nyquist frequency. A function that is sufficient for that and all less severe cases is:

where is the rectangular function. Therefore:

      (from  Eq.1, above).
     [A]

The inverse transform of both sides produces the Whittaker–Shannon interpolation formula:

which shows how the samples, , can be combined to reconstruct .

  • Larger-than-necessary values of (smaller values of ), called oversampling, have no effect on the outcome of the reconstruction and have the benefit of leaving room for a transition band in which is free to take intermediate values. Undersampling, which causes aliasing, is not in general a reversible operation.
  • Theoretically, the interpolation formula can be implemented as a low-pass filter, whose impulse response is and whose input is which is a Dirac comb function modulated by the signal samples. Practical digital-to-analog converters (DAC) implement an approximation like the zero-order hold. In that case, oversampling can reduce the approximation error.

Shannon's original proof

[edit]

Poisson shows that the Fourier series in Eq.1 produces the periodic summation of , regardless of and . Shannon, however, only derives the series coefficients for the case . Virtually quoting Shannon's original paper:

Let be the spectrum of   Then
because is assumed to be zero outside the band   If we let where is any positive or negative integer, we obtain:
On the left are values of at the sampling points. The integral on the right will be recognized as essentially[a] the coefficient in a Fourier-series expansion of the function taking the interval to as a fundamental period. This means that the values of the samples determine the Fourier coefficients in the series expansion of   Thus they determine since is zero for frequencies greater than and for lower frequencies is determined if its Fourier coefficients are determined. But determines the original function completely, since a function is determined if its spectrum is known. Therefore the original samples determine the function completely.

Shannon's proof of the theorem is complete at that point, but he goes on to discuss reconstruction via sinc functions, what we now call the Whittaker–Shannon interpolation formula as discussed above. He does not derive or prove the properties of the sinc function, as the Fourier pair relationship between the rect (the rectangular function) and sinc functions was well known by that time.[4]

Let be the sample. Then the function is represented by:

As in the other proof, the existence of the Fourier transform of the original signal is assumed, so the proof does not say whether the sampling theorem extends to bandlimited stationary random processes.

Notes

[edit]
  1. ^ Multiplying both sides of Eq.2 by produces, on the left, the scaled sample values in Poisson's formula (Eq.1), and, on the right, the actual formula for Fourier expansion coefficients.

Application to multivariable signals and images

[edit]
Subsampled image showing a Moiré pattern
Properly sampled image

The sampling theorem is usually formulated for functions of a single variable. Consequently, the theorem is directly applicable to time-dependent signals and is normally formulated in that context. However, the sampling theorem can be extended in a straightforward way to functions of arbitrarily many variables. Grayscale images, for example, are often represented as two-dimensional arrays (or matrices) of real numbers representing the relative intensities of pixels (picture elements) located at the intersections of row and column sample locations. As a result, images require two independent variables, or indices, to specify each pixel uniquely—one for the row, and one for the column.

Color images typically consist of a composite of three separate grayscale images, one to represent each of the three primary colors—red, green, and blue, or RGB for short. Other colorspaces using 3-vectors for colors include HSV, CIELAB, XYZ, etc. Some colorspaces such as cyan, magenta, yellow, and black (CMYK) may represent color by four dimensions. All of these are treated as vector-valued functions over a two-dimensional sampled domain.

Similar to one-dimensional discrete-time signals, images can also suffer from aliasing if the sampling resolution, or pixel density, is inadequate. For example, a digital photograph of a striped shirt with high frequencies (in other words, the distance between the stripes is small), can cause aliasing of the shirt when it is sampled by the camera's image sensor. The aliasing appears as a moiré pattern. The "solution" to higher sampling in the spatial domain for this case would be to move closer to the shirt, use a higher resolution sensor, or to optically blur the image before acquiring it with the sensor using an optical low-pass filter.

Another example is shown here in the brick patterns. The top image shows the effects when the sampling theorem's condition is not satisfied. When software rescales an image (the same process that creates the thumbnail shown in the lower image) it, in effect, runs the image through a low-pass filter first and then downsamples the image to result in a smaller image that does not exhibit the moiré pattern. The top image is what happens when the image is downsampled without low-pass filtering: aliasing results.

The sampling theorem applies to camera systems, where the scene and lens constitute an analog spatial signal source, and the image sensor is a spatial sampling device. Each of these components is characterized by a modulation transfer function (MTF), representing the precise resolution (spatial bandwidth) available in that component. Effects of aliasing or blurring can occur when the lens MTF and sensor MTF are mismatched. When the optical image which is sampled by the sensor device contains higher spatial frequencies than the sensor, the under sampling acts as a low-pass filter to reduce or eliminate aliasing. When the area of the sampling spot (the size of the pixel sensor) is not large enough to provide sufficient spatial anti-aliasing, a separate anti-aliasing filter (optical low-pass filter) may be included in a camera system to reduce the MTF of the optical image. Instead of requiring an optical filter, the graphics processing unit of smartphone cameras performs digital signal processing to remove aliasing with a digital filter. Digital filters also apply sharpening to amplify the contrast from the lens at high spatial frequencies, which otherwise falls off rapidly at diffraction limits.

The sampling theorem also applies to post-processing digital images, such as to up or down sampling. Effects of aliasing, blurring, and sharpening may be adjusted with digital filtering implemented in software, which necessarily follows the theoretical principles.

A family of sinusoids at the critical frequency, all having the same sample sequences of alternating +1 and –1. That is, they all are aliases of each other, even though their frequency is not above half the sample rate.

Critical frequency

[edit]

To illustrate the necessity of consider the family of sinusoids generated by different values of in this formula:

With or equivalently the samples are given by:

regardless of the value of That sort of ambiguity is the reason for the strict inequality of the sampling theorem's condition.

Sampling of non-baseband signals

[edit]

As discussed by Shannon:[2]

A similar result is true if the band does not start at zero frequency but at some higher value, and can be proved by a linear translation (corresponding physically to single-sideband modulation) of the zero-frequency case. In this case the elementary pulse is obtained from by single-side-band modulation.

That is, a sufficient no-loss condition for sampling signals that do not have baseband components exists that involves the width of the non-zero frequency interval as opposed to its highest frequency component. See sampling for more details and examples.

For example, in order to sample FM radio signals in the frequency range of 100–102 MHz, it is not necessary to sample at 204 MHz (twice the upper frequency), but rather it is sufficient to sample at 4 MHz (twice the width of the frequency interval). (Reconstruction is not usually the goal with sampled IF or RF signals. Rather, the sample sequence can be treated as ordinary samples of the signal frequency-shifted to near baseband, and digital demodulation can proceed on that basis.)

Using the bandpass condition, where for all outside the open band of frequencies

for some nonnegative integer and some sampling frequency , it is possible to find an interpolation that reproduces the signal. Note that there may be several combinations of and that work, including the normal baseband condition as the case The corresponding interpolation filter to be convolved with the sample is the impulse response of an ideal "brick-wall" bandpass filter (as opposed to the ideal brick-wall lowpass filter used above) with cutoffs at the upper and lower edges of the specified band, which is the difference between a pair of lowpass impulse responses:

This function is 1 at and zero at any other multiple of (as well as at other times if ).

Other generalizations, for example to signals occupying multiple non-contiguous bands, are possible as well. Even the most generalized form of the sampling theorem does not have a provably true converse. That is, one cannot conclude that information is necessarily lost just because the conditions of the sampling theorem are not satisfied; from an engineering perspective, however, it is generally safe to assume that if the sampling theorem is not satisfied then information will most likely be lost.

Nonuniform sampling

[edit]

The sampling theory of Shannon can be generalized for the case of nonuniform sampling, that is, samples not taken equally spaced in time. The Shannon sampling theory for non-uniform sampling states that a band-limited signal can be perfectly reconstructed from its samples if the average sampling rate satisfies the Nyquist condition.[5] Therefore, although uniformly spaced samples may result in easier reconstruction algorithms, it is not a necessary condition for perfect reconstruction.

The general theory for non-baseband and nonuniform samples was developed in 1967 by Henry Landau.[6] He proved that the average sampling rate (uniform or otherwise) must be twice the occupied bandwidth of the signal, assuming it is a priori known what portion of the spectrum was occupied.

In the late 1990s, this work was partially extended to cover signals for which the amount of occupied bandwidth is known but the actual occupied portion of the spectrum is unknown.[7] In the 2000s, a complete theory was developed (see the section Sampling below the Nyquist rate under additional restrictions below) using compressed sensing. In particular, the theory, using signal processing language, is described in a 2009 paper by Mishali and Eldar.[8] They show, among other things, that if the frequency locations are unknown, then it is necessary to sample at least at twice the Nyquist criteria; in other words, you must pay at least a factor of 2 for not knowing the location of the spectrum. Note that minimum sampling requirements do not necessarily guarantee stability.

Sampling below the Nyquist rate under additional restrictions

[edit]

The Nyquist–Shannon sampling theorem provides a sufficient condition for the sampling and reconstruction of a band-limited signal. When reconstruction is done via the Whittaker–Shannon interpolation formula, the Nyquist criterion is also a necessary condition to avoid aliasing, in the sense that if samples are taken at a slower rate than twice the band limit, then there are some signals that will not be correctly reconstructed. However, if further restrictions are imposed on the signal, then the Nyquist criterion may no longer be a necessary condition.

A non-trivial example of exploiting extra assumptions about the signal is given by the recent field of compressed sensing, which allows for full reconstruction with a sub-Nyquist sampling rate. Specifically, this applies to signals that are sparse (or compressible) in some domain. As an example, compressed sensing deals with signals that may have a low overall bandwidth (say, the effective bandwidth ) but the frequency locations are unknown, rather than all together in a single band, so that the passband technique does not apply. In other words, the frequency spectrum is sparse. Traditionally, the necessary sampling rate is thus Using compressed sensing techniques, the signal could be perfectly reconstructed if it is sampled at a rate slightly lower than With this approach, reconstruction is no longer given by a formula, but instead by the solution to a linear optimization program.

Another example where sub-Nyquist sampling is optimal arises under the additional constraint that the samples are quantized in an optimal manner, as in a combined system of sampling and optimal lossy compression.[9] This setting is relevant in cases where the joint effect of sampling and quantization is to be considered, and can provide a lower bound for the minimal reconstruction error that can be attained in sampling and quantizing a random signal. For stationary Gaussian random signals, this lower bound is usually attained at a sub-Nyquist sampling rate, indicating that sub-Nyquist sampling is optimal for this signal model under optimal quantization.[10]

Historical background

[edit]

The sampling theorem was implied by the work of Harry Nyquist in 1928,[11] in which he showed that up to independent pulse samples could be sent through a system of bandwidth ; but he did not explicitly consider the problem of sampling and reconstruction of continuous signals. About the same time, Karl Küpfmüller showed a similar result[12] and discussed the sinc-function impulse response of a band-limiting filter, via its integral, the step-response sine integral; this bandlimiting and reconstruction filter that is so central to the sampling theorem is sometimes referred to as a Küpfmüller filter (but seldom so in English).

The sampling theorem, essentially a dual of Nyquist's result, was proved by Claude E. Shannon.[2] Edmund Taylor Whittaker published similar results in 1915,[13] as did his son John Macnaghten Whittaker in 1935,[14] and Dennis Gabor in 1946 ("Theory of communication").

In 1948 and 1949, Claude E. Shannon published the two revolutionary articles in which he founded information theory.[15][16][2] In Shannon's "A Mathematical Theory of Communication", the sampling theorem is formulated as "Theorem 13": Let contain no frequencies over W. Then

where

It was not until these articles were published that the theorem known as "Shannon's sampling theorem" became common property among communication engineers, although Shannon himself writes that this is a fact which is common knowledge in the communication art.[B] A few lines further on, however, he adds: "but in spite of its evident importance, [it] seems not to have appeared explicitly in the literature of communication theory". Despite his sampling theorem being published at the end of the 1940s, Shannon had derived his sampling theorem as early as 1940.[17]

Other discoverers

[edit]

Others who have independently discovered or played roles in the development of the sampling theorem have been discussed in several historical articles, for example, by Jerri[18] and by Lüke.[19] For example, Lüke points out that Herbert Raabe, an assistant to Küpfmüller, proved the theorem in his 1939 Ph.D. dissertation; the term Raabe condition came to be associated with the criterion for unambiguous representation (sampling rate greater than twice the bandwidth). Meijering[20] mentions several other discoverers and names in a paragraph and pair of footnotes:

As pointed out by Higgins, the sampling theorem should really be considered in two parts, as done above: the first stating the fact that a bandlimited function is completely determined by its samples, the second describing how to reconstruct the function using its samples. Both parts of the sampling theorem were given in a somewhat different form by J. M. Whittaker and before him also by Ogura. They were probably not aware of the fact that the first part of the theorem had been stated as early as 1897 by Borel.[Meijering 1] As we have seen, Borel also used around that time what became known as the cardinal series. However, he appears not to have made the link. In later years it became known that the sampling theorem had been presented before Shannon to the Russian communication community by Kotel'nikov. In more implicit, verbal form, it had also been described in the German literature by Raabe. Several authors have mentioned that Someya introduced the theorem in the Japanese literature parallel to Shannon. In the English literature, Weston introduced it independently of Shannon around the same time.[Meijering 2]

  1. ^ Several authors, following Black, have claimed that this first part of the sampling theorem was stated even earlier by Cauchy, in a paper published in 1841. However, the paper of Cauchy does not contain such a statement, as has been pointed out by Higgins.
  2. ^ As a consequence of the discovery of the several independent introductions of the sampling theorem, people started to refer to the theorem by including the names of the aforementioned authors, resulting in such catchphrases as "the Whittaker–Kotel'nikov–Shannon (WKS) sampling theorem" or even "the Whittaker–Kotel'nikov–Raabe–Shannon–Someya sampling theorem". To avoid confusion, perhaps the best thing to do is to refer to it as the sampling theorem, "rather than trying to find a title that does justice to all claimants".

— Eric Meijering, "A Chronology of Interpolation From Ancient Astronomy to Modern Signal and Image Processing" (citations omitted)

In Russian literature it is known as the Kotelnikov's theorem, named after Vladimir Kotelnikov, who discovered it in 1933.[21]

Why Nyquist?

[edit]

Exactly how, when, or why Harry Nyquist had his name attached to the sampling theorem remains obscure. The term Nyquist Sampling Theorem (capitalized thus) appeared as early as 1959 in a book from his former employer, Bell Labs,[22] and appeared again in 1963,[23] and not capitalized in 1965.[24] It had been called the Shannon Sampling Theorem as early as 1954,[25] but also just the sampling theorem by several other books in the early 1950s.

In 1958, Blackman and Tukey cited Nyquist's 1928 article as a reference for the sampling theorem of information theory,[26] even though that article does not treat sampling and reconstruction of continuous signals as others did. Their glossary of terms includes these entries:

Sampling theorem (of information theory)
Nyquist's result that equi-spaced data, with two or more points per cycle of highest frequency, allows reconstruction of band-limited functions. (See Cardinal theorem.)
Cardinal theorem (of interpolation theory)
A precise statement of the conditions under which values given at a doubly infinite set of equally spaced points can be interpolated to yield a continuous band-limited function with the aid of the function

Exactly what "Nyquist's result" they are referring to remains mysterious.

When Shannon stated and proved the sampling theorem in his 1949 article, according to Meijering,[20] "he referred to the critical sampling interval as the Nyquist interval corresponding to the band in recognition of Nyquist's discovery of the fundamental importance of this interval in connection with telegraphy". This explains Nyquist's name on the critical interval, but not on the theorem.

Similarly, Nyquist's name was attached to Nyquist rate in 1953 by Harold S. Black:

If the essential frequency range is limited to cycles per second, was given by Nyquist as the maximum number of code elements per second that could be unambiguously resolved, assuming the peak interference is less than half a quantum step. This rate is generally referred to as signaling at the Nyquist rate and has been termed a Nyquist interval.

— Harold Black, Modulation Theory[27] (bold added for emphasis; italics as in the original)

According to the Oxford English Dictionary, this may be the origin of the term Nyquist rate. In Black's usage, it is not a sampling rate, but a signaling rate.

See also

[edit]

Notes

[edit]

References

[edit]

Further reading

[edit]
[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia
from Grokipedia
The Nyquist–Shannon sampling theorem, also known as the cardinal theorem of sampling or the Shannon sampling theorem, is a foundational result in that establishes the conditions under which a continuous-time bandlimited signal can be perfectly reconstructed from a sequence of its discrete samples. Specifically, it states that if a signal contains no frequencies higher than a maximum value WW cycles per second, it is completely determined by its values at points spaced 1/(2W)1/(2W) seconds apart, provided the sampling rate exceeds twice the bandwidth to prevent . The theorem originated with Harry Nyquist's 1928 analysis of telegraph transmission, where he determined that the necessary frequency range for distortionless signaling does not exceed the signaling speed, implying that a bandwidth equal to half the sampling rate suffices for complete signal representation through independent sinusoidal components. Nyquist's work, published as "Certain Topics in Telegraph Transmission Theory" in the Transactions of the , laid the groundwork by linking the number of signal elements to the required frequency components, approximately half the signaling rate. This insight was later formalized and proven rigorously by in his 1949 paper "Communication in the Presence of Noise," published in the Proceedings of the IRE, which integrated it into the broader framework of and demonstrated exact reconstruction via sinc for bandlimited functions. Central to the theorem is the concept of the , defined as twice the highest frequency in the signal, which serves as the critical threshold for faithful . Sampling below this rate introduces , where higher frequencies masquerade as lower ones, distorting the signal irreversibly without prior . Reconstruction from samples is achieved through the , which sums scaled sinc functions centered at each sample point, ensuring no information loss for signals strictly bandlimited to below half the sampling frequency. The theorem's implications extend across , enabling efficient analog-to-digital and digital-to-analog conversions in applications such as audio recording (e.g., compact discs sampling at 44.1 kHz to capture up to 22.05 kHz), , medical imaging, and systems. It underpins modern technologies by guaranteeing that discrete representations preserve continuous signal fidelity, provided filters limit the input bandwidth appropriately. Extensions of the theorem address non-uniform sampling, multidimensional signals, and noisy environments, but the core principle remains essential for avoiding information loss in sampled systems.

Fundamentals

Statement of the Theorem

The Nyquist–Shannon sampling theorem, also known as the cardinal theorem of sampling or Whittaker–Shannon–Kotelnikov–Nyquist theorem, addresses the conditions under which a continuous-time signal can be perfectly reconstructed from its discrete samples. A bandlimited signal is defined as one whose Fourier transform X(ω)X(\omega) is zero for all frequencies ω>2πB|\omega| > 2\pi B, where BB is the bandwidth in hertz, meaning the signal contains no energy outside the frequency range [B,B][-B, B]. The theorem states that if a continuous-time signal x(t)x(t) is bandlimited to frequency BB, then it can be completely reconstructed from its uniformly spaced samples x(nT)x(nT), taken at a sampling rate fs=1/T2Bf_s = 1/T \geq 2B samples per second, without loss of information. The minimum such sampling rate of 2B samples per second is known as the , while the is B Hz (half the sampling rate). The theorem was first implied in the context of telegraph transmission by in 1928, who showed that a channel of bandwidth WW can transmit up to 2W2W independent pulses per second without , and rigorously proved by in 1949 for arbitrary bandlimited functions. Perfect reconstruction under the theorem requires several conditions: uniform sampling at intervals T1/(2B)T \leq 1/(2B), an infinite number of samples extending over all time, absence of or other distortions in the sampling , and, in some formulations, a strict inequality fs>2Bf_s > 2B to ensure no overlap at the boundary frequencies. Violation of the sampling rate condition leads to , where higher frequencies masquerade as lower ones. The reconstruction is achieved via ideal sinc interpolation, expressed by the formula: x(t)=n=x(nfs)sinc(fs(tnfs)),x(t) = \sum_{n=-\infty}^{\infty} x\left(\frac{n}{f_s}\right) \operatorname{sinc}\left(f_s \left(t - \frac{n}{f_s}\right)\right), where sinc(u)=sin(πu)/(πu)\operatorname{sinc}(u) = \sin(\pi u)/(\pi u) is the normalized . This formula weights each sample by a shifted , whose frequency response acts as an ideal to recover the original bandlimited signal. The factor of two in the arises because sampling at fsf_s produces periodic replicas of the signal's spaced by fsf_s; a rate of at least 2B2B ensures these replicas do not overlap, allowing the to be isolated via .

Aliasing Phenomenon

Aliasing refers to the distortion in a sampled signal where higher-frequency components are incorrectly represented as lower frequencies, resulting from the periodic replication of the signal's during the sampling process. This phenomenon arises because sampling a continuous-time signal at a rate fsf_s creates multiple copies of the original centered at multiples of fsf_s, and if fsf_s is less than twice the signal's bandwidth BB, these replicas overlap, causing frequencies above the to "fold" into the . Such overlap prevents the unique identification and perfect reconstruction of the original signal, directly contravening the Nyquist–Shannon sampling theorem's requirement for distortion-free recovery. In the frequency domain, the spectrum of the sampled signal Xs(f)X_s(f) is obtained by multiplying the original spectrum X(f)X(f) by a Dirac comb, which generates replicas at intervals of the sampling frequency fs=1/Tf_s = 1/T, where TT is the sampling period. This results in the key relation: Xs(f)=1Tk=X(fkT)X_s(f) = \frac{1}{T} \sum_{k=-\infty}^{\infty} X\left(f - \frac{k}{T}\right) If the replicas do not overlap—ensured when fs>2Bf_s > 2B—the baseband spectrum remains undistorted and can be isolated for reconstruction; otherwise, the superposition corrupts the low-frequency content with contributions from higher frequencies. A classic example of aliasing is the observed in motion pictures or video, where the spokes of a rotating appear to rotate backwards or stand still due to the sampling being insufficient relative to the 's , causing the perceived motion to alias into the opposite direction. In audio processing, aliasing can manifest as unintended low-frequency tones when high-frequency sounds are sampled inadequately, similar to moiré patterns in where fine details create false interference fringes. To detect aliasing, one can analyze the for unexpected low-frequency artifacts or use diagnostic signals exceeding half the sampling rate; prevention typically involves applying low-pass filters before sampling to attenuate frequencies above fs/2f_s/2, ensuring no overlap in the replicated spectra.

Nyquist Frequency

The , denoted as fnf_n, is defined as half the sampling frequency fsf_s, such that fn=fs2f_n = \frac{f_s}{2}. This represents the highest frequency component in a continuous-time signal that can be accurately captured and reconstructed from its discrete samples without distortion, assuming the signal is bandlimited. In the context of the Nyquist–Shannon sampling theorem, the sampling frequency must satisfy fs>2Bf_s > 2B, where BB is the bandwidth of the signal (its maximum content), implying fnBf_n \geq B. Theoretical reconstruction is possible when equality holds (fn=Bf_n = B) for strictly bandlimited signals, but this requires an ideal , which is unattainable in practice and leads to instability due to filter imperfections and . Consequently, real-world systems typically employ fsf_s slightly greater than 2B2B to provide a for practical filter . In digital signal processing systems, the Nyquist frequency establishes the upper limit of the for the sampled signal; any frequency components exceeding fnf_n will fold back into the (0 to fnf_n) as , distorting the representation. filters are thus essential prior to sampling to attenuate frequencies above fnf_n, ensuring the input signal adheres to the bandlimit. A prominent example is the (CD) audio format, which uses a sampling rate of 44.1 kHz, yielding a Nyquist frequency of 22.05 kHz. This exceeds the typical human up to 20 kHz, providing a margin to accommodate filter transition bands while preventing of ultrasonic components. The Nyquist frequency differs from the signal's bandwidth BB, which characterizes the inherent frequency extent of the original continuous signal, whereas fnf_n is a property of the sampling process that must be at least as large as BB to enable faithful digitization.

Mathematical Derivations

Proof via Poisson Summation

The Poisson summation formula serves as a powerful tool for deriving the Nyquist–Shannon sampling theorem by linking the discrete samples of a signal in the time domain to periodic replicas of its Fourier transform in the frequency domain. For a square-integrable function x(t)x(t) with Fourier transform X(f)=x(t)ei2πftdtX(f) = \int_{-\infty}^{\infty} x(t) e^{-i 2\pi f t} \, dt, the Poisson summation formula states that n=x(nT)=1Tk=X(kT)ei2πkt/T,\sum_{n=-\infty}^{\infty} x(nT) = \frac{1}{T} \sum_{k=-\infty}^{\infty} X\left(\frac{k}{T}\right) e^{i 2\pi k t / T}, evaluated at t=0t = 0 for the sampling context, yielding n=x(nT)=1Tk=X(k/T)\sum_{n=-\infty}^{\infty} x(nT) = \frac{1}{T} \sum_{k=-\infty}^{\infty} X(k/T). This relation, originally due to Poisson and extended in Fourier analysis, reveals the inherent periodicity introduced by sampling at interval TT. To derive the sampling theorem, assume x(t)x(t) is bandlimited such that X(f)=0X(f) = 0 for f>B|f| > B. The impulse-sampled signal is then xs(t)=n=x(nT)δ(tnT)x_s(t) = \sum_{n=-\infty}^{\infty} x(nT) \delta(t - nT), where δ\delta is the . The Fourier transform of xs(t)x_s(t) is Xs(f)=n=x(nT)ei2πfnTX_s(f) = \sum_{n=-\infty}^{\infty} x(nT) e^{-i 2\pi f n T}. Applying the in the distributional sense, Xs(f)X_s(f) simplifies to a periodic repetition of the original spectrum: Xs(f)=1Tk=X(fkT).X_s(f) = \frac{1}{T} \sum_{k=-\infty}^{\infty} X\left(f - \frac{k}{T}\right). This expression shows that sampling replicates X(f)X(f) at intervals of 1/T1/T in the , scaled by 1/T1/T. If the sampling period satisfies T1/(2B)T \leq 1/(2B), or equivalently, the sampling frequency fs=1/T2Bf_s = 1/T \geq 2B, the spectral replicas do not overlap within the f<B|f| < B. Under this condition, the original X(f)X(f) can be recovered perfectly from Xs(f)X_s(f) by applying an ideal low-pass filter with cutoff BB and gain TT, isolating the central replica without distortion. The reconstructed signal is then obtained via the inverse Fourier transform: x(t)=TBBXs(f)ei2πftdf,x(t) = T \int_{-B}^{B} X_s(f) \, e^{i 2\pi f t} \, df, which, given the non-overlapping replicas, equals BBX(f)ei2πftdf\int_{-B}^{B} X(f) \, e^{i 2\pi f t} \, df, yielding x(t)x(t). This step confirms that the continuous-time signal is uniquely recoverable from its samples. This Fourier-analytic approach highlights the theorem's connection to broader principles in harmonic analysis, demonstrating why uniform sampling suffices for bandlimited signals by exploiting the duality between summation in time and periodicity in frequency. It underscores the role of the sampling rate in preventing spectral folding, though it assumes ideal bandlimiting and infinite-duration signals. In practice, this relates to the discrete-time Fourier transform, where finite samples approximate the infinite sum, but deviations from ideality introduce errors.

Shannon's Information Theory Approach

Claude Shannon provided a foundational proof of the sampling theorem in his 1949 paper "Communication in the Presence of Noise," where he integrated it into the broader framework of to analyze communication systems affected by noise. In this work, Shannon linked the sampling process to the concept of , demonstrating how bandlimited signals can be represented efficiently for transmission over noisy channels. By viewing signals geometrically in a multidimensional space, he showed that sampling at the ensures lossless encoding of the signal's information content without excess redundancy. The core argument posits that a bandlimited signal with bandwidth WW (in cycles per second) observed over a time interval TT possesses exactly 2WT2WT degrees of freedom, meaning it requires no more than 2W2W independent samples per second to fully specify its information. This finite dimensionality arises because the signal's frequency content is confined, limiting the number of independent parameters needed for its description. Sampling at a rate fs=2Wf_s = 2W captures these degrees of freedom precisely, allowing exact reconstruction in the absence of noise, while lower rates lead to information loss and higher rates introduce unnecessary samples. Shannon's derivation begins by representing the bandlimited signal in a space of dimension 2WT2WT, where the signal can be expanded using an orthogonal basis of approximately 2WT2WT functions. He proposed sinc functions, specifically sin2πW(tn/(2W))π(tn/(2W))\frac{\sin 2\pi W (t - n/(2W))}{\pi (t - n/(2W))} for n=0,1,,2WT1n = 0, 1, \dots, 2WT - 1, which form an orthogonal set for large TT and span the space of bandlimited signals. The coordinates of the signal in this basis correspond to the values at the sampling points spaced 1/(2W)1/(2W) apart, proving that these 2WT2WT samples uniquely determine the signal, as any bandlimited function is a linear combination of this basis with coefficients given by the samples. In the context of noisy channels, Shannon derived the maximum information rate as CWlog2(1+S/N)C \leq W \log_2 (1 + S/N) bits per second, where SS is signal power and NN is noise power in bandwidth WW. For the noiseless case (N=0N = 0), this capacity becomes infinite, but the sampling theorem establishes that exact recovery is possible at fs=2Wf_s = 2W, as the samples fully encode the signal's infinite-precision values without distortion. Unlike frequency-domain proofs relying on spectral replication and aliasing avoidance, Shannon's approach emphasizes the intrinsic information content and finite dimensionality of bandlimited signals, rigorously justifying the Nyquist rate as the minimal sufficient rate for complete representation. This perspective underscores that the theorem is not merely about avoiding overlap in the frequency domain but about efficiently capturing the signal's degrees of freedom with no more or less than necessary.

Applications and Variations

Multidimensional Signals

The Nyquist–Shannon sampling theorem generalizes seamlessly to multidimensional signals, where the principle remains that a bandlimited function in N-dimensional space can be perfectly reconstructed from its samples on a lattice if the sampling density is sufficiently high. This extension, formalized for arbitrary dimensions, applies to signals such as two-dimensional images or three-dimensional volumetric data, whose Fourier transforms are confined to a bounded region in the multidimensional frequency domain. The critical sampling density is determined by the Lebesgue measure (volume) of this spectral support: for a support of measure V in N-dimensional frequency space, the lattice must have a density of at least V points per unit volume in the spatial domain to ensure no information loss. In two dimensions, relevant for imaging applications, the theorem specifies that a signal bandlimited to a region of area A in the (f_x, f_y) frequency plane requires a sampling rate of at least A samples per unit area. This holds regardless of the support's shape, though the optimal lattice geometry varies; for irregular or non-rectangular supports, the minimal density may approach this bound more efficiently than uniform rectangular grids. The adaptation preserves the core idea from one dimension—avoiding through adequate sampling—but the effective Nyquist rate now depends on the geometry of the spectral support, such as its area for isotropic cases or adjusted for anisotropic extensions. For instance, if the bandwidth is anisotropic, with different extents in x and y directions, the sampling lattice must account for the elongated support to prevent distortion. For uniform rectangular grid sampling of two-dimensional images, the condition translates to specific intervals Δx and Δy. Assuming the signal is bandlimited to frequencies |f_x| ≤ B_x and |f_y| ≤ B_y, the sampling must satisfy 1/(Δx Δy) ≥ 4 B_x B_y to capture the full bandwidth without aliasing. Reconstruction is then achieved via a two-dimensional sinc interpolation formula: f(x,y)=m=n=f(mΔx,nΔy)sinc(π(xmΔx)Δx)sinc(π(ynΔy)Δy),f(x, y) = \sum_{m=-\infty}^{\infty} \sum_{n=-\infty}^{\infty} f(m \Delta x, n \Delta y) \cdot \operatorname{sinc}\left( \frac{\pi (x - m \Delta x)}{\Delta x} \right) \cdot \operatorname{sinc}\left( \frac{\pi (y - n \Delta y)}{\Delta y} \right), where the sinc function is defined as \operatorname{sinc}(t) = \sin(t)/t. This separable form leverages the rectangular lattice's alignment with Cartesian coordinates, enabling efficient computation in digital imaging systems. Alternative lattices, such as hexagonal grids, offer improved efficiency for certain bandwidth shapes, particularly circular or isotropic supports common in natural images. A hexagonal arrangement achieves the Nyquist density with approximately 13% fewer samples than a square grid for the same aliasing-free reconstruction, due to its denser packing in the frequency domain that better tiles the spectral replicas. This reduction stems from the hexagonal lattice's fundamental domain covering 86.6% of the area required by square sampling for equivalent bandwidth containment. In practice, hexagonal sampling enhances resolution in applications like remote sensing and computer vision, though it complicates processing compared to rectangular grids. Key applications of multidimensional sampling include digital imaging, where cameras sample scenes on rectangular or hexagonal arrays to meet Nyquist criteria for high-fidelity capture, and magnetic resonance imaging (MRI), where k-space is sampled in two or three dimensions to reconstruct anatomical volumes without aliasing artifacts. In MRI, adherence to the theorem ensures that the field-of-view matches the sampling grid, preventing wrap-around effects in the reconstructed images. Challenges arise with anisotropic bandwidths, where the spectral support is not symmetric, requiring adaptive lattices to maintain the minimal density; suboptimal choices can lead to inefficient oversampling or incomplete coverage. Additionally, aliasing in two dimensions manifests as moiré patterns—interference fringes from overlapping spectral replicas—particularly evident when sampling fine periodic structures like fabrics or grids in photography. These patterns underscore the need for pre-sampling anti-aliasing filters to enforce bandlimiting before discretization.

Non-Baseband Sampling

Non-baseband signals, commonly referred to as bandpass signals, have their frequency content restricted to an interval [f1,f2][f_1, f_2] where f1>0f_1 > 0, and the bandwidth is defined as W=f2f1W = f_2 - f_1. These signals differ from signals, which occupy frequencies from 0 to some maximum fmaxf_{\max}, by having energy concentrated away from DC. The Nyquist–Shannon sampling theorem adapts to bandpass signals by specifying that the minimum sampling rate fsf_s must be at least twice the bandwidth, fs2Wf_s \geq 2W, rather than twice the highest frequency 2f22f_2. This adaptation enables , where fs<2f2f_s < 2f_2, provided the sampling avoids aliasing overlaps between the original spectrum and its replicas shifted by multiples of fsf_s. The condition ensures that the signal can be reconstructed perfectly from samples if the replicas are positioned without intrusion into the signal band. The precise allowable sampling rates are determined by the band position to prevent overlap. For an integer k1k \geq 1, the sampling frequency must satisfy 2f2k+1fs2f1k,\frac{2 f_2}{k+1} \leq f_s \leq \frac{2 f_1}{k}, which creates guard bands around the signal spectrum to isolate it from aliased components. For the lowest-order case (k=1k=1), this yields fs2Wf_s \geq 2W directly, but higher kk allows even lower rates under stricter positioning. These constraints arise from the periodic replication of the spectrum in the frequency domain upon sampling. In practice, bandpass sampling is applied to radio frequency (RF) signals in communication systems, where high carrier frequencies with narrow modulation bandwidths permit efficient digitization at rates far below the carrier frequency, reducing hardware complexity. Another prominent example is quadrature sampling for in-phase (I) and quadrature (Q) demodulation, where the bandpass signal is mixed with cosine and sine carriers to produce lowpass equivalents, each sampled at fs=2Wf_s = 2W; this effectively captures the complex envelope without full-rate sampling of the carrier. However, successful reconstruction demands accurate prior knowledge of the band's location [f1,f2][f_1, f_2] to select an appropriate fsf_s; misalignment can cause replicas to overlap, leading to irreversible aliasing distortion. Sensitivity to band position and noise amplification in reconstruction filters further limits applicability in imprecise environments.

Nonuniform Sampling Methods

Nonuniform sampling refers to the process of acquiring signal values at irregular time instants tnt_n, where the intervals between consecutive samples vary, in contrast to the equally spaced intervals of uniform sampling. For a bandlimited signal with bandwidth BB, perfect reconstruction remains possible provided the average sampling rate satisfies limTN(T)2T2B\lim_{T \to \infty} \frac{N(T)}{2T} \geq 2B, with N(T)N(T) denoting the number of samples in the interval [T,T][-T, T]. This average rate condition ensures that the samples capture sufficient information density, though the irregularity introduces additional considerations for reconstruction stability. Extensions of the Nyquist–Shannon theorem to nonuniform sampling, notably the Beutler-Yao theorem, establish conditions for error-free and stable recovery of bandlimited signals from irregularly spaced samples. Beutler's work demonstrates that reconstruction is feasible if the sampling set forms a "total" set with density exceeding the Nyquist rate, avoiding large gaps that could lead to information loss. Yao and Thomas further analyzed stability, showing that nonuniform expansions converge uniformly under bounded perturbations in sampling locations, provided the average density meets or exceeds 2B2B and the maximum gap between samples remains finite. These results generalize the uniform case by emphasizing density over regularity, enabling reconstruction via series expansions analogous to the cardinal series but adapted for irregular grids. Central to nonuniform sampling is the Landau rate, which defines the minimal average sampling density of 2B2B necessary for stable reconstruction of signals bandlimited to a set of total measure BB. This lower bound, derived from density theorems for entire functions, implies that while nonuniformity allows flexibility, the overall sample count over long intervals must not fall below this threshold to prevent instability. Reconstruction typically employs iterative methods grounded in frame theory, where samples generate a frame for the signal space, allowing stable recovery through least-squares projections or oblique dual frames. Frame-based approaches, applicable in shift-invariant spaces containing bandlimited functions, ensure geometric convergence and robustness to noise when the frame bound exceeds unity. Alternatively, Prony's method can iteratively fit exponential sums to approximate bandlimited signals from nonuniform points, particularly effective for finite-duration or periodic cases. Practical examples of nonuniform sampling arise in analog-to-digital converters (ADCs), where clock jitter induces irregular timing, yet reconstruction is viable if the average rate adheres to the Nyquist criterion. This jitter, often modeled as Gaussian perturbations, serves as a precursor to compressive sensing by enabling sub-Nyquist exploration in sparse scenarios, though here it maintains the full rate on average. Hardware advantages include deliberate nonuniformity in time-interleaved ADCs, which reduces aperture jitter sensitivity and lowers overall system costs by allowing slower individual channels to achieve equivalent high rates through interleaving. Reconstruction from nonuniform samples presents challenges, primarily increased computational complexity due to the need for solving ill-conditioned linear systems or iterative optimizations, as opposed to the closed-form sinc interpolation in uniform cases. Error bounds are tightly linked to measures of irregularity, such as the discrepancy between the empirical sampling distribution and uniform density, or the maximal gap size; larger discrepancies amplify reconstruction errors, with bounds scaling proportionally to the perturbation norm. These factors necessitate careful design to balance density sufficiency with gap control for practical stability.

Undersampling with Constraints

The Nyquist–Shannon sampling theorem traditionally mandates a sampling rate fs2Bf_s \geq 2B for perfect reconstruction of bandlimited signals with bandwidth BB, but undersampling below this rate becomes feasible when additional constraints on the signal structure are imposed, such as sparsity in a known basis or other parametric forms. These priors enable sub-Nyquist sampling by exploiting the signal's low intrinsic dimensionality, allowing recovery through optimization techniques rather than direct interpolation. A prominent approach is compressed sensing, pioneered by Donoho and independently by Candès, Romberg, and Tao, which targets signals that are kk-sparse in an orthonormal basis, meaning they have at most kk nonzero coefficients in a representation of length NN. In this framework, a signal can be recovered from mm linear measurements where mcklog(N/k)m \geq c k \log(N/k) for some constant c>0c > 0, far fewer than the NN samples required without sparsity. Recovery is achieved via , such as 1\ell_1-norm minimization: x^=argminx1subject toy=Φx,\hat{x} = \arg\min \| x \|_1 \quad \text{subject to} \quad y = \Phi x, where yy are the measurements, Φ\Phi is the measurement matrix, and the restricted isometry property ensures stable reconstruction provided the sparsity basis and measurement basis are incoherent. Other methods include sampling of signals with finite rate of innovation (FRI), introduced by Vetterli et al., which applies to parametric signals like sums of Diracs or exponentials with a finite number of degrees of freedom per unit time, say ρ\rho, allowing sampling at rates slightly above 2ρ2\rho using annihilating filters or pronys methods for reconstruction. Multi-coset sampling, developed by Mishali and Eldar, employs multiple parallel samplers with periodic undersampling patterns to capture spectrally sparse multiband signals, enabling sub-Nyquist rates when the signal occupies a small portion of a wide spectrum, with recovery via basis pursuit or greedy algorithms assuming unknown band locations. These techniques find applications in radar systems for sparse target detection and for spectrum sensing of sparse wideband signals, where the assumption of incoherence between the sparsity and measurement domains is crucial for reliable recovery. However, they require precise prior knowledge of the signal model, and reconstruction is generally but not perfectly invertible in the presence of , degrading performance as noise levels increase.

Historical Development

Early Contributions

The foundations of the Nyquist–Shannon sampling theorem emerged in the through mathematical explorations of from discrete points. In 1841, developed a trigonometric interpolation formula for reconstructing periodic functions from equally spaced samples, serving as an early precursor to signal reconstruction techniques, though not framed in terms of frequency limitations. Similarly, Joseph-Louis Lagrange's method, introduced in the , provided a systematic way to approximate continuous functions using a of discrete observations, influencing subsequent approaches to representation in analysis. By the early , the rise of and electrical communications spurred interest in representing signals efficiently, leading to more targeted results on bandlimited functions. In 1915, formulated the cardinal series expansion, demonstrating that certain bandlimited analytic functions could be uniquely interpolated from their values at uniform sampling points spaced at intervals of 1/(2B), where B is the bandwidth, using shifted sinc functions. This work, published in the Proceedings of the Royal Society of , established a mathematical basis for sampling without loss of information for functions of exponential type. In 1933, Vladimir A. Kotelnikov presented the paper "On the of 'Ether' and Wire in Electrical Communications" at the First All-Union Conference on Questions of Communications in Leningrad, proving that a continuous-time signal bandlimited to W can be perfectly reconstructed from uniform samples taken at rate 2W, motivated by limits in transmission systems; however, the Russian-language publication was overlooked in until the . These developments occurred amid growing needs in wired and wireless telegraphy, where discrete pulse representations were essential for efficient signaling, yet lacked a cohesive theorem uniting them. Parallel concepts appeared in optics, as Ernst Abbe's 1873 diffraction limit described the minimal resolvable distance in microscopy as λ/(2NA)—where λ is wavelength and NA is numerical aperture—imposing a spatial frequency constraint akin to the temporal sampling requirement for avoiding aliasing. Nyquist's 1928 analysis of pulse transmission rates in telegraph channels later bridged these mathematical precursors to practical engineering applications.

Attribution and Recognition

The Nyquist–Shannon sampling theorem derives its name from the foundational contributions of and , whose works established the core principles of sampling bandlimited signals. In , Nyquist published "Certain Topics in Telegraph Transmission Theory," where he analyzed the transmission of pulses over telegraph channels and derived that, to avoid distortion, the pulse rate must be at least twice the bandwidth B of the channel, expressed as fs2Bf_s \geq 2B. This result, though applied to discrete pulse signaling rather than continuous waveforms, laid the groundwork for the sampling condition in communication systems. Claude Shannon extended and formalized Nyquist's insight in his 1949 paper "Communication in the Presence of Noise," proving that any bandlimited signal with bandwidth W can be completely reconstructed from samples taken at a rate of at least 2W per second, using the cardinal series (sinc interpolation) for perfect recovery. Shannon's treatment generalized the theorem to arbitrary continuous-time signals, integrating it into information theory and emphasizing its implications for noise-free transmission. Independent discoveries preceded and paralleled these efforts, notably by Vladimir A. Kotelnikov, who in 1933 presented "On the Carrying Capacity of 'Ether' and Wire in Electrical Communications" at a Leningrad conference, rigorously stating the sampling theorem for both lowpass and bandpass signals at twice the highest frequency. Kotelnikov's work, however, remained largely overlooked in the West due to its publication in Russian with limited circulation among a student audience and the geopolitical isolation of Soviet research at the time. Parallel independent discoveries occurred, including by Fumio Someya in Japan around 1940, though similarly overlooked until later due to wartime conditions. These contributions were constrained by wartime secrecy and specialized venues, contributing to their delayed recognition. The naming convention evolved from these origins: "Nyquist rate" emerged in 1930s engineering literature to denote the minimum signaling rate of 2B, popularized in telephony and control systems contexts. Shannon's formulation inspired the term "Shannon sampling theorem" in mathematical and circles by the late 1940s. The hyphenated "Nyquist–Shannon sampling theorem" gained prominence in 1950s textbooks and reviews, reflecting the complementary and theoretical perspectives, such as in S. Goldman's 1953 "Information Theory." Debates over priority have persisted, with claims highlighting Kotelnikov's earlier comprehensive proof and the overlooked contributions from other regions. Kotelnikov's work gained formal recognition in the West starting in the through translations and citations in international literature. Contemporary scholarship emphasizes inclusivity by increasingly citing these diverse origins to foster a more complete historical narrative.

References

Add your contribution
Related Hubs
Contribute something
User Avatar
No comments yet.