Performance Ratings - the Research
Performance Ratings – Where are we now?
Other than small businesses where it is easy to manage performance informally on a day to day basis, almost all businesses of any scale use some form of performance management system. Most of these have an annual performance rating. 4 point rating scale performance reviews are common as are 5 point scales. Over 90% of organisations report that this performance management approach is not working. Systems are dysfunctional because they are administrative rather than operational in focus. The annual appraisal is a source of frustration and dissatisfaction for staff, managers and HR alike.
In fact over 25 years of research in organisational psychology finds that formal performance management systems have had no impact on improving either performance or performance management behaviours. 1,2
This is because the mechanics of traditional performance management systems, the way goals are set, the behaviours that are rated, the rating formats, and the form based intermittent approach, are at odds with what is needed for supporting excellence in job performance.
A 2015 Bersin by Deloitte international survey of more than 7,000 business and HR leaders reported that the majority were redesigning their performance management practices. Many said they were reviewing the use of performance ratings.
Is it a good idea to eliminate performance ratings?
Those in favour of eliminating performance ratings claim that they do not accurately measure employee performance. There are a number of reasons for this problem.
1. NON –PERFORMANCE FACTORS THAT INFLUENCE RATINGS
Firstly there are external factors that influence rating – for example the state of the economy and labour market, the culture of the organisation.
The organisational context includes factors such as communication and transparency, trust, leadership style, group dynamics, and most importantly politics.
Research shows that raters consciously or unconsciously, have a number of goals when doing performance appraisals. These goals influence their ratings. 3 Managers may use a lower rating to push staff to perform better, to make an example of them, or to get them to resign.
Alternatively managers may give higher ratings to maintain or improve good relations with their staff, to encourage them, to get them a pay increase, development and promotion opportunities. Often a high rating is given to avoid the discomfort of having to deal with poor performance or a confrontation with the staff member who feels they should have been rated higher.
Supervisors may give higher ratings to make themselves look good and improve their own standing in the organisation. They will also have their own beliefs about how ratings should be awarded, for example some may expect higher standards. Overall these goals tend to result in high ratings for most staff.
Conflicting purposes for performance appraisal affect ratings. When ratings are used for decisions on compensation and promotion the purpose of the ratings is to highlight top and bottom performers – the differences between people. When ratings are used for developmental purposes the purpose is to identify strengths and weaknesses of the individual themselves. Research shows that ratings for the purpose of compensation and promotion will be higher than those that have a developmental purpose. Whilst it is recommended to separate out developmental assessment, in practice this is rarely done.
2. PERFORMANCE RATING FEEDBACK DOES NOT HAVE A CONSISTENT IMPACT ON PERFORMANCE
The assumption is that feedback will motivate employees to continue to perform and or improve their performance.
Many managers are not trained in the art of giving constructive feedback, nor do people like receiving feedback that is different from their own view of their performance. Most people believe themselves to be above average. Nor do employees like being compared to others, as is the case with some rating schemes.
In fact research shows that performance feedback is inconsistent and unreliable, largely because of the many factors that influence ratings. Some research shows that ratings do improve motivation and performance, other research shows an adverse impact.
The adverse impact is more likely where there are differential rewards and in a work environment that requires higher levels of collaboration, rather than individual contribution.
3. TECHNICAL MEASUREMENT CONSIDERATIONS
The format of the rating scale often causes unreliability of measurement. Scales are often too small to offset the effects of restriction of range – the preference of raters not to give ratings at either end of the scale.
In theory 4 point rating scale performance reviews can be used to force a decision between acceptable and non acceptable levels of performance. In practice many are skewed with only 1 unacceptable point and 3 positive points.
Reviewers always tend to favour the more positive end of the rating scale. In performance management it is common to see even seven point scales with rating data skewed towards the top end.
Finally, rating descriptions are interpreted differently by different people. For example if a manager expects high standards they may award ‘meets expectation’ for the same level of performance that another manager gives an ‘exceeds expectation’.
Those against the use of performance rating contend that training in the use of rating scales has not made performance ratings more accurate. This is not altogether true.
There is also consistent evidence of significant disagreement when several raters rate the same employee on the same aspects of performance. Overall the inter-rate agreement is only about 50%. This is taken as further evidence that performance ratings are not accurate measures of performance.
4. RISK FACTORS
Performance ratings can be a potential problem in litigious societies. They may show disparities between ethnic groups which contravene equal opportunity regulations. They may be used in lawsuits over termination, compensation and promotion grievances. In relying on ratings for such decisions the more extensive documentation that is also needed may be lacking.
Nevertheless defending litigation is likely to be more difficult in the absence of evidence based quantitative performance measurements.
Finally both staff and managers would like to see performance ratings eliminated. Workers because they face less uncertainty of employment if their performance is not rated. Managers because they see the change as a reduction in unwarranted bureaucracy. For both staff and managers removing performance ratings removes a process that is difficult and uncomfortable.
Is it feasible to eliminate performance ratings?
Over 90% of companies according to consulting firm Mercer3 link their compensation decisions to performance, to some extent. Approximately 82% link individual performance to compensation. Performance ratings are also the basis for many pay for performance schemes.
Some large organisations, for example Eli Lilly, Adobe, and Gap, Inc. claim to have eliminated annual performance ratings. However many organisations that claim to have eliminated performance ratings have just done away with manager ratings in annual appraisals. In fact their staff are still rated in some way during talent identification discussions or compensation reviews.
One example is Deloitte which is reinventing its performance management system, having discovered that it was consuming approximately 2 million hours or 300 hours per employee per year. This is consistent with the 210 hours for managers and 40 hours for individuals that the Corporate Leadership Council found organisations were using in performance management processes.
Deloitte report they are doing away with cascading goals, 360 feedback, annual appraisal, and the traditional performance rating. However, on further reading they disclose they are still rating staff on 4 factors –eligibility for remuneration increase, promotability, desirability as a team member, and risk of poor performance.
It is impossible to have a performance based rewards system without individual performance ratings.
The argument for keeping performance ratings
The purpose of the performance management process is to improve organisational performance. Since it is individuals who do and organise the work in organisations, being able to evaluate their performance is a fundamental business requirement. Evaluating performance and providing feedback is a fundamental part of a manager’s role.
ACCURACY OF RATINGS
The issue is more how to accurately measure individual performance than whether to measure it. Even in small businesses that don’t have formal performance management systems judgements on individual performance are made – he/she is doing well, he/she needs to focus more.
Performance measurement is unlikely to be more accurate without ratings because all there will be to rely on is unstructured text form data.
It has been claimed that additional raters will improve the accuracy of performance ratings. In fact there appears to be little extra value after the first 3 raters. 9 There may be more cost in administrative time than benefits to performance improvement.
Problems with the accuracy of ratings stem from their subjective nature. Objective measures such as sales, or response time may not be available for all jobs.
It is widely believed that performance management should focus on behaviours or tasks rather than outcomes because of the many external factors that affect business results, and the resulting issues of fairness when rating on outcomes.
However as use of information technology increases objective data is available for an increasing number of roles. Framing objectives and tasks with observable and realistic outcomes for an individual will make assessment less subjective. It is outcomes rather than tasks and behaviours that are linked to organisational success.
IMPROVING PERFORMANCE
Research shows that differenting rewards and recognition, based on performance level differences, does help improve both individual and organisational performance.
The mechanism for this is that top performers are attracted to organisations that recognise and pay for individual achievement. They also like organisations which offer top performers career pathways and rapid advancement 6. They will tend to leave organisations that do not differentially reward achievement – those with fixed and incremental compensation schemes. Recent CEB Corporate Leadership Council research 5 shows that Millennials as a group are more motivated by comparative performance feedback that is able to differentiate their performance.
On the other hand poor performers are more likely to stay with such organisations.
Cost of living increases and step increases based on time in grade do not have much value in motivating improved performance since they have no relationship to performance.
FEEDBACK AND PERFORMANCE CONVERSATIONS
Theoretically if managers engaged in the fundamental performance management behaviours – clear expectations, timely and constructive guidance and feedback, ensuring a supportive work environment and recognition and reward in line with performance, there would be no need for formal performance management systems.
Unfortunately there is abundant evidence that managers lack basic performance management skills. For example a recent study showed that the Lominger competency Growing Talent was consistently rated the lowest of their 67 competencies 7
There is therefore no reason to believe that eliminating ratings will improve the quality and frequency of performance conversations. In today’s space constrained workplaces even finding a private spot for such conversations may be a challenge.
The CEB research 5 also found that top performers were less engaged where differential rating had been abolished, and their productivity was down by 28%, with overall productivity down by 10% compared with organisations that retained ratings.
Whilst the removal of ratings was supposed to lift engagement by removing the unpleasantness, in fact managers reported they felt less connection with their staff, and the perceptions of the quality of the performance conversations that were supposed to replace the ratings also fell.
CAN WE IMPROVE PERFORMANCE RATING?
Most performance ratings are on a scale of 1 to 5 and are supposed to be ratings of the individual’s performance against a provided definition or set of standards. There is no comparison with others.
It has been shown that relative rating scales provide a more accurate performance measure. A relative percentile position is used, for example 70% means performance is better than 70% of people, with 50% representing the average performance.
However studies show that such scales are perceived to be less fair than absolute scales such as BOS and BARS which make assessments against standards only. The perception of fairness is an important factor in motivation and should be considered when deciding on rating scales.
Frame Of Reference scales 8 are a new approach which claims improved rating quality. These scales include descriptions of both positive and negative behaviours or outcomes. The inclusion of more specific indicators in scales provides a frame of reference for the rater. Training raters in the use Frames of Reference has improved rater accuracy.9
One argument for the abolition of performance ratings is the widespread finding of disagreement between raters. This is based on the assumption that all are measuring the same performance. In fact different people see and prioritise different aspects of an individual’s performance according to the nature of their own work and their interactions with that individual. As such a variation in ratings is to be expected and explored as a means to identify strengths and areas for development.
OR IS THIS THE WRONG QUESTION ALTOGETHER.
It is evident that performance management practices have largely ignored the substantial body of research that indicates the type of approach that will drive performance. It is proven that clear individual job specific expectations, day to day quality feedback, guidance and support are the factors that drive individual and team performance. Approaches such as ProMES have shown clear benefits 10
However HR and senior managers are sceptical of the continuous performance management approach because they see it just extending the administrative burden on managers. Many Managers today have been appointed for their technical or functional skills and CEOS identify core leadership skills, including people management as the single biggest problem they face.
Rather than focusing on technical details such as performance ratings, the real question is how can organisations prioritise performance management as the core focus of managers, and make them accountable for it.
In line with an evidence based performance management process this means making expectations of performance management behaviour clear, ensuring managers have the skills required, appropriate tools, monitoring the integrity of the process, and recognising and rewarding those who do a good job of managing and improving performance and capability
REFERENCES
- Pulakos, E. D., Mueller-Hanson, R. A., Arad, S., & Moye, N. (2015). Performance management can be fixed: An on-the-job experiential learning approach for complex behavior change. Industrial and Organizational Psychology: Perspectives on Science and Practice, 8(1), 51–76.
- DeNisi, A., & Smith, C. E. (2014). Performance appraisal, performance management, and firm-level performance. Academy of Management Annals, 8, 127–179.
- Murphy, K. R., Cleveland, J. N., Skattebo, A. L., & Kinney, T. B. (2004). Raters who pursue different goals give different ratings. Journal of Applied Psychology, 89, 158–164.
- Buckingham, M., & Goodall, A. (2015, April). Reinventing performance management. Harvard Business Review. Retrieved from https://hbr.org/2015/04/reinventingperformance-management
- Corporate Leadership Council. (2014). The millennial myth: Three strategies for effectively managing millennials in the workforce. Washington, DC: CEB.
- Cadsby, C. B., Song, F., & Tapon, F. (2007). Sorting and incentive effects of pay for performance: An experimental investigation. Academy of Management Journal, 50, 387–405
- Rock, D., Davis, J., & Jones, B. (2014, August). Kill your performance ratings. Strategy + Business. Retrieved from http://www.strategy-business.com/article/00275
- Hoffman, B. J., Gorman, C. A., Blair, C. A., Meriac, J. P., Overstreet, B. L., & Atchley, E. K. (2012). Evidence for the effectiveness of an alternative multisource performance rating methodology. Personnel Psychology, 65, 531–563.
- David J. Woehr and Sylvia G. Roch (2016). Of Babies and Bathwater: Don’t Throw the Measure Out With the Application. Industrial and Organizational Psychology, 9, pp 357-361 doi:10.1017/iop.2016.25
- Daniel Schmerling and Anne Scaduto (2016). Use the Best; Leave the Rest: The Productivity Measurement and Enhancement System (ProMES) for Performance Ratings. Industrial and Organizational Psychology, 9, pp 305-309 doi:10.1017/iop.2016.15