Multi-point systems for customer feedback — such as five-star ratings for professional services or gig economy jobs — can lead to subtle, unconscious racial bias, according to a study from the University of Toronto.
Instead, a two-point rating scale can reduce bias and the resulting potential financial consequences for non-White workers, researchers said.
“While the objective difference, on average, between white and non-white worker ratings is very small, it matters because of the impact it has on income, highlighting the importance of structure and organizational design for racial equality at work,” Katherine DeCelles, professor of organizational behavior at the University of Toronto’s Rotman School of Management, said in a statement.
DeCelles and colleagues analyzed data from an online home maintenance matching service and found that White workers received higher average ratings than non-White workers. In addition, they received top marks 86.9% of the time, as compared to 83.4% for non-White workers.
Based on the service’s pay system, the lower average ratings reduced the revenue that a worker could make from subsequent jobs. With a five-point system, non-White workers ended up making 91 cents for every dollar earned by White workers, the research team estimated.
When the service switched to a two-choice system — using a thumbs-up or thumbs-down to indicate whether customers would use the contractor again — the racial gaps in top marks nearly disappeared. Beyond that, new workers who joined the service after the switch to a two-point system had no racial differences in earnings for the same job.
In follow-up experiments, the researchers worked with online participants and artificial scenarios to test the differences. Once again, they found reduced racial disparities in two-point performance ratings among participants with subtle racial bias. Notably, participants also told the researchers that their opinions and biases were less likely to influence their performance evaluations when there were only two options.
Since rating systems are common for digital platform-based services, DeCelles and colleagues recommended keeping them simple and focusing the evaluation on whether something was “good” or “bad” to avoid bias. Platforms should also regularly audit their systems to identify systematic variations that could be related to bias and offer other ways for customers to provide more detailed feedback, the researchers suggested.
“People can more clearly evaluate whether someone’s work was good versus not, instead of ‘how good was it?’ which is relatively more subjective and ambiguous — that’s where we’d expect a larger problem with racial bias in evaluations,” DeCelles said.
The findings about client reviews echo what other research has suggested about internal feedback. About 1 in 4 employees responding to a survey said supervisor bias negatively affects their performance reviews, according to a 2023 Syndio report. Asian and LGBTQ+ workers reported the highest likelihood of negative effects due to bias. Instead of traditional annual reviews, Syndio recommended more frequent check-ins, which were associated with lower bias concerns and greater clarity about advancement opportunities.
Employers say there’s room for improvement, too. Only 2% of Fortune 500 CHROs surveyed said their performance management systems work as intended, according to a Gallup report. As employers have scrutinized these systems in recent years, several weaknesses have emerged, particularly with annual reviews. More in-the-moment feedback and separating feedback from pay and promotion conversations can help workers actually improve performance, Gallup reported.