Borel–Cantelli lemma

Proof

Let (E_n) be a sequence of events in some probability space.

The sequence of events ${\left\{\bigcup _{n=N}^{\mathrm{\infty }}{E}_n\right\}}_{N=1}^{\mathrm{\infty }}$ is non-increasing: $${\displaystyle \bigcup _{n=1}^{\mathrm{\infty }}{E}_n\supseteq \bigcup _{n=2}^{\mathrm{\infty }}{E}_n\supseteq \cdots \supseteq \bigcup _{n=N}^{\mathrm{\infty }}{E}_n\supseteq \bigcup _{n=N+1}^{\mathrm{\infty }}{E}_n\supseteq \cdots \supseteq \underset{n\to \mathrm{\infty }}{lim sup}{E}_n.}$$By continuity from above, $${\displaystyle Pr(\underset{n\to \mathrm{\infty }}{lim sup}{E}_n)=\underset{N\to \mathrm{\infty }}{lim}Pr\left(\bigcup _{n=N}^{\mathrm{\infty }}{E}_n\right).}$$By subadditivity, $${\displaystyle Pr\left(\bigcup _{n=N}^{\mathrm{\infty }}{E}_n\right)\le \sum _{n=N}^{\mathrm{\infty }}Pr(E_n).}$$By original assumption, $\sum _{n=1}^{\mathrm{\infty }}Pr(E_n)<\mathrm{\infty }.$ As the series $\sum _{n=1}^{\mathrm{\infty }}Pr(E_n)$ converges, $${\displaystyle \underset{N\to \mathrm{\infty }}{lim}\sum _{n=N}^{\mathrm{\infty }}Pr(E_n)=0,}$$ as required.

General measure spaces

For general measure spaces, the Borel–Cantelli lemma takes the following form:

Borel–Cantelli Lemma for measure spaces—Let μ be a (positive) measure on a set X, with σ-algebra F, and let (A_n) be a sequence in F. If $${\displaystyle \sum _{n=1}^{\mathrm{\infty }}\mu (A_n)<\mathrm{\infty },}$$ then $${\displaystyle \mu \left(\underset{n\to \mathrm{\infty }}{lim sup}{A}_n\right)=0.}$$

Converse result

A related result, sometimes called the second Borel–Cantelli lemma, is a partial converse of the first Borel–Cantelli lemma. The lemma states: If the events E_n are independent and the sum of the probabilities of the E_n diverges to infinity, then the probability that infinitely many of them occur is 1. That is:

Second Borel–Cantelli Lemma—If ${\displaystyle \sum _{n=1}^{\mathrm{\infty }}Pr(E_n)=\mathrm{\infty }}$ and the events ${\displaystyle (E_n{)}_{n=1}^{\mathrm{\infty }}}$ are independent, then ${\displaystyle Pr(\underset{n\to \mathrm{\infty }}{lim sup}{E}_n)=1.}$

The assumption of independence can be weakened to pairwise independence, but in that case the proof is more difficult.

The infinite monkey theorem follows from this second lemma.

Example

The lemma can be applied to give a covering theorem in Rⁿ. Specifically Stein (1993, Lemma X.2.1), if E_j is a collection of Lebesgue measurable subsets of a compact set in Rⁿ such that $${\displaystyle \sum _j\mu (E_j)=\mathrm{\infty },}$$ then there is a sequence F_j of translates $${\displaystyle F_j=E_j+x_j}$$ such that $${\displaystyle \underset{j\to \mathrm{\infty }}{lim sup}{F}_j=\bigcap _{n=1}^{\mathrm{\infty }}\bigcup _{k=n}^{\mathrm{\infty }}{F}_k={\mathbb{R}}^n}$$ apart from a set of measure zero.

Proof

Suppose that $\sum _{n=1}^{\mathrm{\infty }}Pr(E_n)=\mathrm{\infty }$ and the events ${\displaystyle (E_n{)}_{n=1}^{\mathrm{\infty }}}$ are independent. It is sufficient to show the event that the E_n's did not occur for infinitely many values of n has probability 0. This is just to say that it is sufficient to show that $${\displaystyle 1-Pr(\underset{n\to \mathrm{\infty }}{lim sup}{E}_n)=0.}$$

Noting that: $${\displaystyle \begin{array}{cc}1-Pr(\underset{n\to \mathrm{\infty }}{lim sup}{E}_n)& =1-Pr\left(\{E_n\text{ i.o.}\}\right)=Pr\left(\{E_n\text{ i.o.}{\}}^c\right)\\ & =Pr\left({\left(\bigcap _{N=1}^{\mathrm{\infty }}\bigcup _{n=N}^{\mathrm{\infty }}{E}_n\right)}^c\right)=Pr\left(\bigcup _{N=1}^{\mathrm{\infty }}\bigcap _{n=N}^{\mathrm{\infty }}{E}_n^c\right)\\ & =Pr\left(\underset{n\to \mathrm{\infty }}{lim inf}{E}_n^c\right)=\underset{N\to \mathrm{\infty }}{lim}Pr\left(\bigcap _{n=N}^{\mathrm{\infty }}{E}_n^c\right),\end{array}}$$ it is enough to show: $Pr\left(\bigcap _{n=N}^{\mathrm{\infty }}{E}_n^c\right)=0$. Since the ${\displaystyle (E_n{)}_{n=1}^{\mathrm{\infty }}}$ are independent: $${\displaystyle \begin{array}{cc}Pr\left(\bigcap _{n=N}^{\mathrm{\infty }}{E}_n^c\right)& =\prod _{n=N}^{\mathrm{\infty }}Pr(E_n^c)\\ & =\prod _{n=N}^{\mathrm{\infty }}(1-Pr(E_n)).\end{array}}$$ The convergence test for infinite products guarantees that the product above is 0, if $\sum _{n=N}^{\mathrm{\infty }}Pr(E_n)$ diverges. This completes the proof.

Generalizations

Renyi–Lamperti lemma

The assumption of independence in the second lemma can be relaxed. The Renyi–Lamperti lemma states that if the events ${\displaystyle (A_n)}$ satisfy ${\displaystyle \sum Pr(A_n)=\mathrm{\infty }}$ and a condition of weak dependence regarding the correlation of the events, specifically: $${\displaystyle \underset{n\to \mathrm{\infty }}{lim inf}\frac{\sum _{1\le i,j\le n}Pr(A_i\cap A_j)}{{\left(\sum _{i=1}^{n}Pr(A_i)\right)}^2}=1,}$$ then ${\displaystyle Pr(A_n\text{ i.o.})=1}$.

This result is related to the Kochen–Stone theorem, which provides a lower bound for the probability of infinitely many events occurring when the limit inferior in the condition above is positive but not necessarily 1.

Conditional Borel–Cantelli lemma

A powerful generalization involving conditional probability is known as the Conditional Borel–Cantelli lemma (or Lévy's extension of the Borel–Cantelli lemma). It connects the occurrence of events to the accumulation of their conditional probabilities given the past.

Let ${\displaystyle ({\mathcal{F}}_n)}$ be a filtration on a probability space, and let ${\displaystyle E_n\in {\mathcal{F}}_n}$ be a sequence of events adapted to the filtration. Then, almost surely: $${\displaystyle \left\{\sum _{n=1}^{\mathrm{\infty }}Pr(E_n\mid {\mathcal{F}}_{n-1})=\mathrm{\infty }\right\}=\{E_n\text{ i.o.}\}.}$$ In other words, the event that ${\displaystyle E_n}$ occurs infinitely often is almost surely equivalent to the event that the sum of the conditional probabilities diverges. This result is a consequence of martingale convergence theorems.

Counterpart

Another related result is the so-called counterpart of the Borel–Cantelli lemma. It is a counterpart of the Lemma in the sense that it gives a necessary and sufficient condition for the limsup to be 1 by replacing the independence assumption by the completely different assumption that ${\displaystyle (A_n)}$ is monotone increasing for sufficiently large indices. This Lemma says:

Let ${\displaystyle (A_n)}$ be such that ${\displaystyle A_k\subseteq A_{k+1}}$, and let ${\displaystyle \overline{A}}$ denote the complement of ${\displaystyle A}$. Then the probability of infinitely many ${\displaystyle A_k}$ occur (that is, at least one ${\displaystyle A_k}$ occurs) is one if and only if there exists a strictly increasing sequence of positive integers ${\displaystyle (t_k)}$ such that $${\displaystyle \sum _{k}Pr(A_{t_{k+1}}\mid {\overline{A}}_{t_k})=\mathrm{\infty }.}$$This simple result can be useful in problems such as for instance those involving hitting probabilities for stochastic process with the choice of the sequence ${\displaystyle (t_k)}$ usually being the essence.

Kochen–Stone

Let ${\displaystyle (A_n)}$ be a sequence of events with $\sum Pr(A_n)=\mathrm{\infty }$ and $\underset{k\to \mathrm{\infty }}{lim sup}\frac{{\left(\sum _{n=1}^{k}Pr(A_n)\right)}^2}{\sum _{1\le m,n\le k}Pr(A_m\cap A_n)}>0.$ Then there is a positive probability that ${\displaystyle A_n}$ occur infinitely often.

Proof

Let ${\displaystyle S_{m,n}=\sum _{i=m}^n{\mathbf{1}}_{A_i}}$. Then, note that $${\displaystyle E[S_{m,n}{]}^2={\left(\sum _{i=m}^{n}Pr(A_i)\right)}^2}$$ and $${\displaystyle E[S_{m,n}^2]=\sum _{1\le i\le j\le n}Pr(A_i\cap A_j).}$$ Hence, we know that ${\displaystyle \underset{n\to \mathrm{\infty }}{lim sup}\frac{\mathbb{E}[S_{1,n}{]}^2}{\mathbb{E}[S_{1,n}^2]}>0.}$ We have that $${\displaystyle Pr\left(\bigcup _{i=m}^n{A}_i\right)=Pr(S_{m,n}>0).}$$ Now, notice that by the Cauchy-Schwarz Inequality, for any random variable ${\displaystyle X\ge 0}$: $${\displaystyle \mathbb{E}[X{]}^2\le \mathbb{E}[X{\mathbf{1}}_{\{X>0\}}{]}^2\le \mathbb{E}[X^2]Pr(X>0),}$$ therefore, $${\displaystyle Pr(S_{m,n}>0)\ge \frac{\mathbb{E}[S_{m,n}{]}^2}{\mathbb{E}[S_{m,n}^2]}.}$$ We then have $${\displaystyle \frac{\mathbb{E}[S_{m,n}{]}^2}{\mathbb{E}[S_{m,n}^2]}\ge \frac{E[S_{1,n}-S_{1,m-1}{]}^2}{E[S_{1,n}^2]}.}$$ Given ${\displaystyle m}$, since ${\displaystyle \underset{n\to \mathrm{\infty }}{lim}\mathbb{E}[S_{1,n}]=\mathrm{\infty }}$, we can find ${\displaystyle n}$ large enough so that $${\displaystyle |\frac{\mathbb{E}[S_{1,n}]-\mathbb{E}[S_{1,m-1}]}{\mathbb{E}[S_{1,n}]}-1|<ϵ,}$$ for any given ${\displaystyle ϵ>0}$. Therefore, $${\displaystyle \underset{m\to \mathrm{\infty }}{lim}\underset{n\ge m}{sup}Pr\left(\bigcup _{i=m}^n{A}_i\right)\ge \underset{m\to \mathrm{\infty }}{lim}\underset{n\ge m}{sup}\frac{E[S_{1,n}{]}^2}{E[S_{1,n}^2]}>0.}$$ But the left side is precisely the probability that the ${\displaystyle A_n}$ occur infinitely often since ${\displaystyle \{A_k\text{ i.o.}\}=\{\omega \in \mathrm{\Omega }:\mathrm{\forall }m,\mathrm{\exists }n\ge m\text{ s.t. }\omega \in A_n\}.}$ We're done now, since we've shown that ${\displaystyle P(A_k\text{ i.o.})>0.}$

Applications

Strong Law of Large Numbers

The Borel–Cantelli lemma is a standard tool used to prove the Strong Law of Large Numbers. In many proofs, Chebyshev's inequality is applied to bound the probability that a sum of random variables deviates from its mean. If these probabilities sum to a finite value (often involving a convergence of ${\displaystyle \sum n^{-2}}$), the first Borel–Cantelli lemma implies that large deviations occur only finitely often, establishing almost sure convergence.

Metric Number Theory

The lemma was originally formulated by Émile Borel in the context of number theory to study the properties of normal numbers. It is central to the metric theory of Diophantine approximation. For instance, the Borel–Bernstein theorem uses the lemma to show that for almost all real numbers ${\displaystyle x}$, the inequality ${\displaystyle \left|x-\frac{p}{q}\right|<\frac{1}{q^2\ln q}}$ holds for infinitely many pairs of coprime integers ${\displaystyle (p,q)}$. Conversely, if the function ${\displaystyle \varphi (q)}$ on the right-hand side is replaced by one where the sum ${\displaystyle \sum \varphi (q)}$ converges, the inequality has only finitely many solutions almost surely.