360-degree feedback

Guidelines

Certain guidelines emphasise establishing trust between raters and ratees to improve rater accountability and feedback accuracy. At the same time, anonymous participation has also been found to result in more accurate feedback, in which case confidentiality among human resources staff and managers should be preserved. The standardisation and optimisation of rating scales and data collection also affects assessment accuracy, including such factors like the time of day.

Issues

Using 360-degree feedback tools for appraisal purposes has been criticised over concerns of performance criteria validity, ability of peers to give accurate feedback, and manipulation of these systems by feedback providers. Employee manipulation of feedback ratings has been reported in some companies who have used 360-degree feedback for performance evaluation, including GE, IBM, and Amazon.

The amount and level of training in 360-degree feedback for both the rater and ratee can affect the level of accuracy of the feedback. If no guidance is given, individual bias may affect the rater's ratings and the ratee's interpretation of the feedback. However, even with training measures in place, unconscious bias may still occur due to factors such as the cultural influences or relationship quality between the rater and ratee. Additionally, if there are potential consequences from rater feedback, rater motivation may shift from providing accurate feedback to providing feedback based on self-motivated reasons such as promoting or harming a particular individual.

A key challenge in 360-degree feedback is role duality as the people evaluate and are evaluated. Research illustrates that people are strategic in their evaluation of others to improve their chances of being positively evaluated.

Some members of the U.S. military have criticized its use of 360-degree feedback programs in employment decisions because of problems with validity and reliability. Other branches of the U.S. government have questioned 360-degree feedback reviews as well. Still, these organizations continue to use and develop their assessments in developmental processes.

A study on the patterns of rater accuracy shows that the length of time that a rater has known the individual being evaluated generally correlates with positive review favorability and lower accuracy of a 360-degree review, apart from raters who have known the individual for less than a year.

It has been suggested that multi-rater assessments often generate conflicting opinions and that there may be no way to determine whose feedback is accurate. Studies have also indicated that self-ratings are generally significantly higher than the ratings given from others.

Results

Several studies indicate that the use of 360-degree feedback helps to improve employee performance because it helps the evaluated see different perspectives of their performance.

In a 5-year study, no improvement in overall rater scores was found from the 1st year to the 2nd, but scores rose with each passing year from 2nd to 4th. A 1996 study found that performance increased between the 1st and 2nd administrations, and sustained this improvement 2 years later. Additional studies show that 360-degree feedback may be predictive of future performance.

Some authors maintain, however, that there are too many confounding variables related to 360-degree evaluations to reliably generalize their effectiveness, arguing that process features are likely to have major effects on creating behavior change. A 1998 study has found that the category of rater affects the reliability of feedback, with direct reports generally the least reliable.

Multiple pieces of research have demonstrated that the scale of responses can have a major effect on the results, and that some response scales are better than others. The evaluated individual following up with raters to discuss their results, which cannot be done when feedback is anonymous, often has a profound impact on results. Other potentially powerful factors affecting behavior change include how raters are selected, manager approval, instrument quality, rater training and orientation, participant training, supervisor training, coaching, integration with HR systems, and accountability.

One group of studies proposed four paradoxes that explain why 360-degree evaluations do not elicit accurate data:

1. The Paradox of Roles, in which an evaluator is conflicted by being both peer and the judge
2. The Paradox of Group Performance, which admits that the vast majority of work done in a corporate setting is done in groups, not individually
3. The Measurement Paradox, which shows that qualitative, or in-person, techniques are much more effective than mere ratings in facilitating change
4. The Paradox of Rewards, which shows that individuals evaluating their peers care more about the rewards associated with finishing the task than the actual content of the evaluation itself.

Additional studies found no correlation between an employee's multi-rater assessment scores and performance appraisal scores provided by supervisors. They advise that although multi-rater feedback can be effectively used for appraisal, care needs to be taken in its implementation or results will be compromised. This research suggests that 360-degree feedback and performance appraisals get at different outcome, leading some executives to argue that traditional performance appraisals and 360-degree feedback should be used in evaluating overall performance.