Neyman–Pearson lemma

Statement

Consider a test with hypotheses $H_{0} : θ = θ_{0}$ and $H_{1} : θ = θ_{1}$ , where the probability density function (or probability mass function) is $ρ (x ∣ θ_{i})$ for $i = 0, 1$ .

For any hypothesis test with rejection set $R$ , and any $α \in [0, 1]$ , we say that it satisfies condition $P_{α}$ if

α=Prθ0(X∈R)
- That is, the test has size $α$ (that is, the probability of falsely rejecting the null hypothesis is $α$ ).
$\exists η \geq 0$ such that

\begin{matrix} x \in & R ∖ A ⟹ ρ (x ∣ θ_{1}) > η ρ (x ∣ θ_{0}) \\ x \in & R^{c} ∖ A ⟹ ρ (x ∣ θ_{1}) < η ρ (x ∣ θ_{0}) \end{matrix}

where

A

is a negligible set in both

θ_{0}

and

θ_{1}

cases:

{Pr}_{θ_{0}} (X \in A) = {Pr}_{θ_{1}} (X \in A) = 0

That is, we have a strict likelihood ratio test, except on a negligible subset.

For any $α \in [0, 1]$ , let the set of level $α$ tests be the set of all hypothesis tests with size at most $α$ . That is, letting its rejection set be $R$ , we have ${Pr}_{θ_{0}} (X \in R) \leq α$ .

Neyman–Pearson lemma—Existence:

If a hypothesis test satisfies $P_{α}$ condition, then it is a uniformly most powerful (UMP) test in the set of level $α$ tests.

Uniqueness: If there exists a hypothesis test $R_{N P}$ that satisfies $P_{α}$ condition, with $η > 0$ , then every UMP test $R$ in the set of level $α$ tests satisfies $P_{α}$ condition with the same $η$ .

Further, the $R_{N P}$ test and the $R$ test agree with probability $1$ whether $θ = θ_{0}$ or $θ = θ_{1}$ .

In practice, the likelihood ratio is often used directly to construct tests — see likelihood-ratio test. However it can also be used to suggest particular test-statistics that might be of interest or to suggest simplified tests — for this, one considers algebraic manipulation of the ratio to see if there are key statistics in it related to the size of the ratio (i.e. whether a large statistic corresponds to a small ratio or to a large one).

Proof

Given any hypothesis test with rejection set $R$ , define its statistical power function $β_{R} (θ) = {Pr}_{θ} (X \in R)$ .

Existence:

Given some hypothesis test that satisfies $P_{α}$ condition, call its rejection region $R_{N P}$ (where NP stands for Neyman–Pearson).

For any level $α$ hypothesis test with rejection region $R$ we have $[1_{R_{N P}} (x) - 1_{R} (x)] [ρ (x ∣ θ_{1}) - η ρ (x ∣ θ_{0})] \geq 0$ except on some ignorable set $A$ .

Then integrate it over $x$ to obtain $0 \leq [β_{R_{N P}} (θ_{1}) - β_{R} (θ_{1})] - η [β_{R_{N P}} (θ_{0}) - β_{R} (θ_{0})] .$

Since $β_{R_{N P}} (θ_{0}) = α$ and $β_{R} (θ_{0}) \leq α$ , we find that $β_{R_{N P}} (θ_{1}) \geq β_{R} (θ_{1})$ .

Thus the $R_{N P}$ rejection test is a UMP test in the set of level $α$ tests.

Uniqueness:

For any other UMP level $α$ test, with rejection region $R$ , we have from Existence part, $[β_{R_{N P}} (θ_{1}) - β_{R} (θ_{1})] \geq η [β_{R_{N P}} (θ_{0}) - β_{R} (θ_{0})]$ .

Since the $R$ test is UMP, the left side must be zero. Since $η > 0$ the right side gives $β_{R} (θ_{0}) = β_{R_{N P}} (θ_{0}) = α$ , so the $R$ test has size $α$ .

Since the integrand $[1_{R_{N P}} (x) - 1_{R} (x)] [ρ (x ∣ θ_{1}) - η ρ (x ∣ θ_{0})]$ is nonnegative, and integrates to zero, it must be exactly zero except on some ignorable set $A$ .

Since the $R_{N P}$ test satisfies $P_{α}$ condition, let the ignorable set in the definition of $P_{α}$ condition be $A_{N P}$ .

$R ∖ (R_{N P} \cup A_{N P})$ is ignorable, since for all $x \in R ∖ (R_{N P} \cup A_{N P})$ , we have $[1_{R_{N P}} (x) - 1_{R} (x)] [ρ (x ∣ θ_{1}) - η ρ (x ∣ θ_{0})] = η ρ (x ∣ θ_{0}) - ρ (x ∣ θ_{1}) > 0$ .

Similarly, $R_{N P} ∖ (R \cup A_{N P})$ is ignorable.

Define $A_{R} := (R Δ R_{N P}) \cup A_{N P}$ (where $Δ$ means symmetric difference). It is the union of three ignorable sets, thus it is an ignorable set.

Then we have $x \in R ∖ A_{R} ⟹ ρ (x ∣ θ_{1}) > η ρ (x ∣ θ_{0})$ and $x \in R^{c} ∖ A_{R} ⟹ ρ (x ∣ θ_{1}) < η ρ (x ∣ θ_{0})$ . So the $R$ rejection test satisfies $P_{α}$ condition with the same $η$ .

Since $A_{R}$ is ignorable, its subset $R Δ R_{N P} \subset A_{R}$ is also ignorable. Consequently, the two tests agree with probability $1$ whether $θ = θ_{0}$ or $θ = θ_{1}$ .

5 more chapters are free with a Lably account.

Example

Let $X_{1}, \dots, X_{n}$ be a random sample from the $N (μ, σ^{2})$ distribution where the mean $μ$ is known, and suppose that we wish to test for $H_{0} : σ^{2} = σ_{0}^{2}$ against $H_{1} : σ^{2} = σ_{1}^{2}$ . The likelihood for this set of normally distributed data is

L (σ^{2} ∣ x) \propto {(σ^{2})}^{- n / 2} \exp {- \frac{\sum_{i = 1}^{n} (x_{i} - μ)^{2}}{2 σ^{2}}} .

We can compute the likelihood ratio to find the key statistic in this test and its effect on the test's outcome:

Λ (x) = \frac{L ({σ_{0}}^{2} ∣ x)}{L ({σ_{1}}^{2} ∣ x)} = {(\frac{σ_{0}^{2}}{σ_{1}^{2}})}^{- n / 2} \exp {- \frac{1}{2} (σ_{0}^{- 2} - σ_{1}^{- 2}) \sum_{i = 1}^{n} (x_{i} - μ)^{2}} .

This ratio only depends on the data through $\sum_{i = 1}^{n} (x_{i} - μ)^{2}$ . Therefore, by the Neyman–Pearson lemma, the most powerful test of this type of hypothesis for this data will depend only on $\sum_{i = 1}^{n} (x_{i} - μ)^{2}$ . Also, by inspection, we can see that if $σ_{1}^{2} > σ_{0}^{2}$ , then $Λ (x)$ is a decreasing function of $\sum_{i = 1}^{n} (x_{i} - μ)^{2}$ . So we should reject $H_{0}$ if $\sum_{i = 1}^{n} (x_{i} - μ)^{2}$ is sufficiently large. The rejection threshold depends on the size of the test. In this example, the test statistic can be shown to be a scaled chi-square distributed random variable and an exact critical value can be obtained.

Application in economics

A variant of the Neyman–Pearson lemma has found an application in the seemingly unrelated domain of the economics of land value. One of the fundamental problems in consumer theory is calculating the demand function of the consumer given the prices. In particular, given a heterogeneous land-estate, a price measure over the land, and a subjective utility measure over the land, the consumer's problem is to calculate the best land parcel that they can buy – i.e. the land parcel with the largest utility, whose price is at most their budget. It turns out that this problem is very similar to the problem of finding the most powerful statistical test, and so the Neyman–Pearson lemma can be used.

Uses in electrical engineering

The Neyman–Pearson lemma is quite useful in electronics engineering, namely in the design and use of radar systems, digital communication systems, and in signal processing systems. In radar systems, the Neyman–Pearson lemma is used in first setting the rate of missed detections to a desired (low) level, and then minimizing the rate of false alarms, or vice versa. Neither false alarms nor missed detections can be set at arbitrarily low rates, including zero. All of the above goes also for many systems in signal processing.

Uses in particle physics

The Neyman–Pearson lemma is applied to the construction of analysis-specific likelihood-ratios, used to e.g. test for signatures of new physics against the nominal Standard Model prediction in proton–proton collision datasets collected at the LHC.

Discovery of the lemma

The work that led to the lemma started around 1927. Neyman later wrote about this in a book chapter:

The general question was how to formulate the problem of statistical tests so it would have a mathematical meaning ... In a voluminous correspondence and in several encounters, in Poland, in Brittany and in London, we [Neyman and Egon Pearson] struggled with the basic question and, in passing, solved a few particular cases. My involvement was complete, bordering on obsession ... Finally, I think it was in 1932, we solved the problem of non-dogmatic theory of testing statistical hypotheses. Once the basic question was properly formulated, the solution came easily. Our main joint paper was communicated to the Royal Society by Karl Pearson and (unexpectedly) favorably refereed by R. A. Fisher. It was published in the Philosophical Transactions in 1933.

Neyman described the actual discovery of the lemma as follows. Paragraph breaks have been inserted.

I can point to the particular moment when I understood how to formulate the undogmatic problem of the most powerful test of a simple statistical hypothesis against a fixed simple alternative. At the present time [probably 1968], the problem appears entirely trivial and within easy reach of a beginning undergraduate. But, with a degree of embarrassment, I must confess that it took something like half a decade of combined effort of E. S. P. [Egon Pearson] and myself to put things straight.

The solution of the particular question mentioned came on an evening when I was sitting alone in my room at the Statistical Laboratory of the School of Agriculture in Warsaw, thinking hard on something that should have been obvious long before. The building was locked up and, at about 8 p.m., I heard voices outside calling me. This was my wife, with some friends, telling me that it was time to go to a movie.

My first reaction was that of annoyance. And then, as I got up from my desk to answer the call, I suddenly understood: for any given critical region and for any given alternative hypothesis, it is possible to calculate the probability of the error of the second kind; it is represented by this particular integral. Once this is done, the optimal critical region would be the one which minimizes this same integral, subject to the side condition concerned with the probability of the error of the first kind. We are faced with a particular problem of the calculus of variation, probably a simple problem.

These thoughts came in a flash, before I reached the window to signal to my wife. The incident is clear in my memory, but I have no recollections about the movie we saw. It may have been Buster Keaton.