The Design of Experiments

Origin of the Null Hypothesis and the Lady Tasting Tea

One of the most notable sections of the book describes the origin of the null hypothesis. Fisher illustrated it with a wager, a woman claimed to be able to tell whether milk or tea had been poured first into a cup. In order to test this, Fisher designed an experiment in which she was presented with eight cups, four of each preparation. This came to be known as the Lady tasting tea experiment. He defined the null hypothesis as the assumption that she has no such ability to make the distinction, meaning her answers would be due to chance. Under this assumption, the probability of guessing all eight cups correctly was calculated to be 1/70. Fisher argued that if she succeeded to distinguish whether milk or tea had been poured first into a cup, the null hypothesis could be rejected. Fisher stated “the null hypothesis is never proved or established, but is possibly disproved, in the course of experimentation. Every experiment may be said to exist only in order to give the facts a chance of disproving the null hypothesis,” thus the concept of a null hypothesis was born. This discussion became central to the philosophy of statistical inference.

Further Developments in Experimental Designs

Beyond establishing the principles of randomization, replication and blocking, The Design of Experiments advanced the statistical understanding of error estimation in field trials. Before Fisher’s work, agricultural experiments often lacked a coherent theory for estimating experimental error, except in the simplest comparisons. Fisher formalized this process, allowing errors to be estimated directly from the data rather than assumed. His methods revealed the unreliability of earlier experiments that failed to account for natural variability, such as differences in soil fertility. Later editions of his related work, Statistical Methods for Research Workers, introduced further refinements, including the analysis of covariance, a technique developed to improve precision by adjusting for variation. Together, these innovations helped organize earlier agricultural practices into a coherent framework for experimental design and analysis.

Controversies

Fisher has faced multiple criticisms and controversies through his decades-long career. His work had established a foundation for significance testing that stressed the indication of evidence against the null hypothesis, using p-values. Ronald Fisher argued that inductive inference was crucial to the progression of scientific knowledge and that this is “experimentally demonstrable when we know how to conduct an experiment which will rarely fail to give us a statistically significant result.” Skeptics, such as Paul L. Morgan and Jacob Cohen, expressed that there were considerable irregularities in Fisher's disagreements with probability. Cohen had a specific critique regarding Fisher, “who emphatically rejected Bayesian theory of inverse probability but slipped into invalid Bayesian interpretations of NHST [null hypothesis significance testing].” Despite Fisher’s statistical legacy, the discourse has argued that his study has contributed to the extensive misinterpretation of NHST in modern research.

In addition to critiques of his work, Fisher has been known to be controversial due to his views on race and history of supporting eugenic policies.

Legacy

After Fisher’s death in 1962, the methods of experimental design and analysis he had established decades earlier had transformed research across the physical, biological and statistical sciences. The Design of Experiments (1935) and its later editions provided a systematic account of the logic of experimentation, as well as the formulation of the null hypothesis into a unified framework for scientific use. The book advanced the use of factorial designs, which allowed researchers to study multiple factors and their interactions simultaneously. While ideas of combining treatments in this way was not entirely new, Fisher demonstrated its logical and statistical advantage over older approaches that favoured observation of one factor at a time. His framework later influenced fields such as psychology and education, where experimental designs gradually replaced purely correlational studies. Fisher’s concepts of randomization, error analysis, and statistical inference remain foundational.

Impact of Fisher's Book and Research

In The Design of Experiments, Ronald A. Fisher has made noteworthy inputs within the field of genetics and biological assay. The book discusses his study of systematically arranged experiments which demonstrated and fully articulated the incorrect assumptions about the independence of plot errors that can be properly eliminated using the process of randomization. This is now commonly known as the principle of randomization.

Since laying the groundwork in experimental design and analysis, many of Fisher’s initial works were advanced by other notable researchers, such as Frank Yates. Rothamsted Research was particularly fond of developing the newer concepts of experimental design and analysis. Fisher’s studies have advanced sampling methods for experimental plots, effectively modernizing yield estimates in agriculture and survey methods in a variety of fields. By 1939, studies involving highly variable data had included these strategies which were adapted by agronomists worldwide. Decades after its publication, The Design of Experiments continues to shape modern research methodology.