Nyquist ISI criterion

When consecutive symbols are transmitted over a channel by a linear modulation (such as ASK, QAM, etc.), the impulse response (or equivalently the frequency response) of the channel causes a transmitted symbol to be spread in the time domain. This causes intersymbol interference because the previously transmitted symbols affect the currently received symbol, thus reducing tolerance for noise. The Nyquist theorem relates this time-domain condition to an equivalent frequency-domain condition.

The Nyquist criterion is closely related to the Nyquist–Shannon sampling theorem, with only a differing point of view.

Nyquist criterion

If we denote the channel impulse response as ${\displaystyle h(t)}$, then the condition for an ISI-free response can be expressed as:

${\displaystyle h(n{T}_s)=\{\begin{array}{ll}1;& n=0\\ 0;& n\ne 0\end{array}}$

for all integers ${\displaystyle n}$, where ${\displaystyle T_s}$ is the symbol period. The Nyquist theorem says that this is equivalent to:

${\displaystyle \frac{1}{T_s}\sum _{k=-\mathrm{\infty }}^{+\mathrm{\infty }}H\left(f-\frac{k}{T_s}\right)=1\text{for all frequencies }f}$,

where ${\displaystyle H(f)}$ is the Fourier transform of ${\displaystyle h(t)}$. This is the Nyquist ISI criterion.

This criterion can be intuitively understood in the following way: frequency-shifted replicas of ${\displaystyle H(f)}$ must add up to a constant value. This condition is satisfied when ${\displaystyle H(f)}$ spectrum has even symmetry, has bandwidth greater than or equal to ${\displaystyle 2/T_s}$, and its single-sideband has odd symmetry at the cutoff frequency ${\displaystyle \pm 1/2T_s}$.

In practice this criterion is applied to baseband filtering by regarding the symbol sequence as weighted impulses (Dirac delta function). When the baseband filters in the communication system satisfy the Nyquist criterion, symbols can be transmitted over a channel with flat response within a limited frequency band, without ISI. Examples of such baseband filters are the raised-cosine filter, or the sinc filter as the ideal case.