Dive Brief:
- When it comes to evaluating employee feedback, artificial intelligence models are better at working with easily categorized themes and less effective at handling nuance, according to a new study from PYX Labs, a research lab sponsored by Perceptyx.
- The study examined AI’s ability to complete basic assignments, as well as its ability to demonstrate “the judgment required to accurately interpret how employees experience work,” per a news release. The report found that when answers were clear and verifiable, AI models passed between 76% and 82% of the tasks.
- However, when it came to complex results that required the models to interpret and understand open-ended employee feedback and create “a coherent, accurate takeaway,” that percentage dropped to as low as 33%.
Dive Insight:
The benchmark study examined responses from seven AI models from OpenAI, Google, Anthropic and xAI across 84 employee listening tasks, and measured responses against criteria developed by psychologists and organizational behavior specialists. The research concluded that while existing AI models can handle objective work, they’re unreliable when it comes to interpretation and synthesis.
“Organizations are already using AI to interpret employee feedback and generate recommendations that influence real decisions about people,” Joseph Freed, chief product officer at Perceptyx and head of PYX Labs, said in a statement. “The question is not whether these models can produce fluent answers — it’s whether they understand what ‘good’ looks like in the context of the workplace.”
The AI models studied for this report particularly struggled when it came to creating a cohesive accounting from multiple sources and “ambiguous signals.” Synthesis was the lowest-scoring capability across every model, with scores ranging between 14% and 57%, representing a wider gap than for any other task.
“The breakdown happens specifically when they have to weigh incomplete, emotional, or context-dependent signals and resolve them into one clear takeaway,” the report said.
PYX Labs also found “rare but meaningful instances where models produced fabricated statistical outputs or failed to adhere strictly to underlying dataset constraints.” As a result, the report found that the risks associated with using AI models to interpret employee feedback without human oversight were significant.
But just 20% of companies said managers were effective when it came to giving feedback and offering coaching opportunities, according to a 2025 report from WTW. As a result, companies have been using AI to fill in the gaps, with 37% of respondents to the WTW survey saying they use AI tools as part of their performance management process.