Hawthorne effect

Illumination experiment

The illumination experiment was conducted from 1924 to 1927. The purpose was to determine the effect of light variations on worker productivity. The experiment ran in two rooms: the experiment room, in which workers went about their workday under various light levels; and the control room, in which workers did their tasks under normal conditions. The hypothesis was that as the light level was increased in the experiment room, productivity would increase.

However, when the intensity of light was increased in the experiment room, researchers found that productivity had improved in both rooms. The light level in the experiment room was then decreased, and the results were the same: increased productivity in both rooms. Productivity only began to decrease in the experiment room when the light level was reduced to about the level of moonlight, which made it hard to see.

Ultimately it was concluded that illumination did not have any effect on productivity and that there must have been some other variable causing the observed productivity increases in both rooms. Another phase of experiments was needed to pinpoint the cause.

Relay assembly experiments

In 1927, researchers conducted an experiment where they chose two female workers as test subjects and asked them to choose four other women to join the test group. Until 1928, the team of women worked in a separate room, assembling telephone relays.

Output was measured mechanically by counting how many finished relays each worker dropped down a chute. To establish a baseline productivity level, the measurement was begun in secret two weeks before the women were moved to the experiment room, and then continued throughout the study. In the experiment room, a supervisor discussed changes in their productivity.

Some of the variables were:

• Giving the workers two 5-minute breaks (which they said they preferred beforehand) and then switching to two 10-minute breaks. Productivity increased, but when they were given six 5-minute breaks, productivity decreased because many rests broke the workers' flow.
• Providing soup or coffee with a sandwich in the morning and snacks in the evening. This increased productivity.
• Changing the end of the workday from 5:00 to 4:30 and eliminating the Saturday workday. This increased productivity.

Changing a variable usually increased productivity, even if the variable was just a change back to the original condition. It is said that this reflects natural adaptation to the environment without knowing the objective of the experiment. Researchers concluded that the workers worked harder because they thought that they were being monitored individually.

Researchers hypothesized that choosing one's own coworkers, working as a group, being treated as special (as evidenced by working in a separate room), and having a sympathetic supervisor were the real reasons for the productivity increase. One interpretation, mainly due to Elton Mayo's studies, was that "the six individuals became a team and the team gave itself wholeheartedly and spontaneously to cooperation in the experiment". Further, there was a second relay assembly test room study whose results were not as significant as the first experiment.

Mass interviewing program

The program was conducted from 1928 to 1930 and involved 20,000 interviews. The interviews initially used direct questioning, asking questions related to the supervision and policies of the company involved. The drawback of the direct questioning was that the answers were only "yes" or "no", which was unhelpful for finding the root of problems. Therefore, researchers took to indirect questioning, in which the interviewer would listen. This gave valuable insights about workers' behavior, specifically that the behavior of a worker (or individual) is shaped by group behavior.

Bank wiring room experiments

The purpose of the next study was to find out how payment incentives and small groups would affect productivity. The surprising result was that productivity actually decreased. Workers apparently had become suspicious that their productivity may have been boosted to justify firing some of the workers later on. The study was conducted by Elton Mayo and W. Lloyd Warner between 1931 and 1932 on a group of fourteen men who put together telephone switching equipment. The researchers found that although the workers were paid according to individual productivity, productivity decreased because the men were afraid that the company would lower the base rate. Detailed observation of the men revealed the existence of informal groups or "cliques" within the formal groups. These cliques developed informal rules of behavior as well as mechanisms to enforce them. The cliques served to control group members and to manage bosses; when bosses asked questions, clique members gave the same responses, even if they were untrue. These results show that workers were more responsive to the social force of their peer groups than to the control and incentives of management.

Interpretation and criticism

Richard Nisbett has described the Hawthorne effect as "a glorified anecdote", saying that "once you have got the anecdote, you can throw away the data." Other researchers have attempted to explain the effects with various interpretations. J. G. Adair warned of gross factual inaccuracy in most secondary publications on the Hawthorne effect and that many studies failed to find it. He argued that it should be viewed as a variant of Orne's (1973) experimental demand effect. For Adair, the Hawthorne effect depended on the participants' interpretation of the situation. An implication is that manipulation checks are important in social sciences experiments. He advanced the view that awareness of being observed was not the source of the effect, but participants' interpretation of the situation is critical. How did the participants' interpretation of the situation interact with the participants' goals?

Possible explanations for the Hawthorne effect include the impact of feedback and motivation towards the experimenter. Receiving feedback on their performance may improve their skills when an experiment provides this feedback for the first time. Research on the demand effect also suggests that people may be motivated to please the experimenter, at least if it does not conflict with any other motive. They may also be suspicious of the purpose of the experimenter. Therefore, the Hawthorne effect may only occur when there is usable feedback or a change in motivation.

Parsons defined the Hawthorne effect as "the confounding that occurs if experimenters fail to realize how the consequences of subjects' performance affect what subjects do" [i.e. learning effects, both permanent skill improvement and feedback-enabled adjustments to suit current goals]. His key argument was that in the studies where workers dropped their finished goods down chutes, the participants had access to the counters of their work rate.

Mayo contended that the effect was due to the workers reacting to the sympathy and interest of the observers. He discussed the study as demonstrating an experimenter effect as a management effect: how management can make workers perform differently because they feel differently. He suggested that much of the Hawthorne effect concerned the workers feeling free and in control as a group rather than as being supervised. The experimental manipulations were important in convincing the workers to feel that conditions in the special five-person work group were actually different from the conditions on the shop floor. The study was repeated with similar effects on mica-splitting workers.

Clark and Sugrue in a review of educational research reported that uncontrolled novelty effects cause on average 30% of a standard deviation (SD) rise (i.e. 50–63% score rise), with the rise decaying to a much smaller effect after eight weeks. In more detail: 50% of a SD for up to four weeks; 30% of SD for five to eight weeks; and 20% of SD for more than eight weeks, (which is < 1% of the variance).

Harry Braverman pointed out that the Hawthorne tests were based on industrial psychology and the researchers involved were investigating whether workers' performance could be predicted by pre-hire testing. The Hawthorne study showed "that the performance of workers had little relation to their ability and in fact often bore an inverse relation to test scores ...". Braverman argued that the studies really showed that the workplace was not "a system of bureaucratic formal organisation on the Weberian model, nor a system of informal group relations, as in the interpretation of Mayo and his followers but rather a system of power, of class antagonisms". This discovery was a blow to those hoping to apply the behavioral sciences to manipulate workers in the interest of management.

The economists Steven Levitt and John A. List long pursued without success a search for the base data of the original illumination experiments (they were not true experiments but some authors labeled them experiments), before finding it in a microfilm at the University of Wisconsin in Milwaukee in 2011. Re-analysing it, they found slight evidence for the Hawthorne effect over the long-run, but in no way as drastic as suggested initially. This finding supported the analysis of an article by S. R. G. Jones in 1992 examining the relay experiments. Despite the absence of evidence for the Hawthorne effect in the original study, List has said that he remains confident that the effect is genuine.

Gustav Wickström and Tom Bendix (2000) argue that the supposed "Hawthorne effect" is actually ambiguous and disputable, and instead recommend that to evaluate intervention effectiveness, researchers should introduce specific psychological and social variables that may have affected the outcome.

It is also possible that the illumination experiments can be explained by a longitudinal learning effect. Parsons has declined to analyse the illumination experiments, on the grounds that they have not been properly published and so he cannot get at details, whereas he had extensive personal communication with Roethlisberger and Dickson.

Evaluation of the Hawthorne effect continues in the present day. Despite the criticisms, however, the phenomenon is often taken into account when designing studies and their conclusions. Some have also developed ways to avoid it. For instance, there is the case of holding the observation when conducting a field study from a distance, from behind a barrier such as a two-way mirror mirror or using an unobtrusive measure.

Greenwood, Bolton, and Greenwood (1983) interviewed some of the participants in the experiments and found that the participants were paid significantly better. Bolton's archives relevant to his work on the Hawthorne effect are held at West Virginia University.

Trial effect

Various medical scientists have studied possible trial effect (clinical trial effect) in clinical trials. Some postulate that, beyond just attention and observation, there may be other factors involved, such as slightly better care; slightly better compliance/adherence; and selection bias. The latter may have several mechanisms: (1) Physicians may tend to recruit patients who seem to have better adherence potential and lesser likelihood of future loss to follow-up. (2) The inclusion/exclusion criteria of trials often exclude at least some comorbidities; although this is often necessary to prevent confounding, it also means that trials may tend to work with healthier patient subpopulations.

Secondary observer effect

Despite the observer effect as popularized in the Hawthorne experiments being perhaps falsely identified (see above discussion), the popularity and plausibility of the observer effect in theory has led researchers to postulate that this effect could take place at a second level. Thus it has been proposed that there is a secondary observer effect when researchers working with secondary data such as survey data or various indicators may impact the results of their scientific research. Rather than having an effect on the subjects (as with the primary observer effect), the researchers likely have their own idiosyncrasies that influence how they handle the data and even what data they obtain from secondary sources. For one, the researchers may choose seemingly innocuous steps in their statistical analyses that end up causing significantly different results using the same data; e.g. weighting strategies, factor analytic techniques, or choice of estimation. In addition, researchers may use software packages that have different default settings that lead to small but significant fluctuations. Finally, the data that researchers use may not be identical, even though it seems so. For example, the OECD collects and distributes various socio-economic data; however, these data change over time such that a researcher who downloads the Australian GDP data for the year 2000 may have slightly different values than a researcher who downloads the same Australian GDP 2000 data a few years later. The idea of the secondary observer effect was floated by Nate Breznau in a thus far relatively obscure paper.

Although little attention has been paid to this phenomenon, the scientific implications are very large. Evidence of this effect may be seen in recent studies that assign a particular problem to a number of researchers or research teams who then work independently using the same data to try and find a solution. This is a process called crowdsourcing data analysis and was used in a groundbreaking study by Silberzahn, Rafael, Eric Uhlmann, Dan Martin and Brian Nosek et al. (2015) about red cards and player race in football (i.e. soccer).