Bosses have a wide range of beliefs on what makes or doesn’t make performance reviews fair, according to a study set to be published in an upcoming issue of the journal Academy of Management Discoveries.
Researchers found a sharp difference among experienced administrators over how important specific aspects of fairness are to them when judging the objectivity of a performance review.
"The fact that experienced administrators differ sharply in how they evaluate the fairness of the same appraisal suggests why this can be a potential minefield for employers," Hayley German, of the London School of Economics and Political Science, said in a statement. "On the basis of our findings, it comes as no great surprise that annual performance appraisals have been losing favor."
To illustrate the wide disparity in performance reviews, researchers highlighted three examples of the way administrators went about judging the fairness of actual performance appraisals. One placed about three-quarters of the appraisal's weight on whether employees who were being reviewed were provided information they needed in a timely fashion and whether the employees indicated a liking for the employer.
A second discounted those factors and instead relied about 80 percent on whether reviews took into account employees' views and were based on accurate information. In addition to placing some value on timeliness, accuracy and whether the employee liked the organization, the third example placed the most importance on whether employees were treated with dignity in the review process. This was a factor nearly ignored by the other two judges.
Even though they all took different approaches, the study found a reasonable degree of consistency in the weight individual judges assigned to different factors from one review to another. However, when asked to rate how important some factors were in their decision-making process, the administrators' ratings were inconsistent with their actual judgments.
The study's authors said this shows that even though they are consistent in their approach, the managers would have a hard time explaining to others what that approach actually was.
"Although participants adopted a consistent judgment policy across different performance-appraisal situations, they showed little insight into their own judgment policy," the study's authors wrote. "[M]ost seem to have been unaware of the real policies they used."
As part of the study, researchers first asked 56 university administrators to assess an actual performance review they had recently received. They were specifically asked to rate the review, on a scale of 1 to 7, on 10 specific factors of fairness:
- Whether the review reflected their true work effort.
- Whether the review reflected their true work quality.
- It took his or her views into account.
- Whether the review was based on accurate information.
- Whether they were treated with dignity and respect.
- The reviewer refrained from improper remarks.
- They provided reasonable explanations regarding the appraisal process.
- Essential information was provided in a timely fashion.
- Whether they were pleased with the appraisal outcome.
- They liked working for their employer.
These anonymous reviews were then submitted to a group of 49 other university administrators experienced in performance review. They were asked to judge each review on a scale of 0 (totally unfair) to 100 (totally fair). In addition, these administrators were asked to provide their own sense of how they ranked the 10 factors in importance and what percentage each contributed to their estimates of fairness.
Timeliness, accuracy and quality were the highest-rated factors, having a cumulative weight of about 42 percent. The three lowest-weighted factors were being treated with dignity, feeling pleased with the result and refraining from improper remarks, which had a cumulative weight of only 10 percent.
The researchers found the most striking result was the large difference among the judges on which factors were important and which were not in achieving fairness.
"Although some [factors] were generally more salient, there were large individual variations," the study's authors wrote.
While trying to take all 10 factors into consideration might be a burden for managers when giving performance reviews, it may be the fairest way to go about the process, according to the researchers.
"Focusing on only a few known antecedents of fairness seems risky, as managers typically do not know which antecedents may be most salient to which employee," the study's authors wrote. "Importantly, managers need to realize that their own preferences for specific aspects of fairness may not be shared by their employees."
The research also shows that managers would be wise to ask employees which fairness factors are important to them.
"Our study suggests that, aside from the positive effects of being listened to and receiving attention...there is value in fulfilling on the antecedents that employees think are important," the study's authors wrote.
At the same time, however, those giving reviews should understand that even though employees may cite three factors as being most important, it doesn't mean the others are totally unimportant to them.
The study was co-authored by Marion Fortin of the University of Toulouse in France and Daniel Read of the Warwick Business School at the University of Coventry in England.