Want to Improve Worker Performance With AI? Start by Helping Staff Understand Their Own Strengths and Weaknesses

New Research Shows Self-Awareness Determines Who Benefits Most From AI Assistance

BALTIMORE, Feb. 27, 2026 — Managers looking to improve worker performance are increasingly turning to AI tools to boost productivity and decision quality. But new research published in the INFORMS journal Management Science suggests technology alone won’t determine results. The biggest gains come when workers understand their own strengths and limitations, which helps them know when to rely on AI.

The study, “The ABCs of Who Benefits From Working With AI: Ability, Beliefs, and Calibration,” finds that the productivity benefits of AI depend not only on a worker’s ability but also on whether they are accurately calibrated about their own performance. The study shows AI assistance improves outcomes most for workers who are lower skilled but well calibrated — meaning their confidence matches their actual ability. The research was conducted by Andrew Caplin of New York University; David Deming of Harvard University; Ben Weidmann of University College London; Shangwen Li and Kadachi Jiada Ye of New York University; Daniel Martin of the University of California Santa Barbara; and Philip Marx of Louisiana State University.

In a controlled experiment with 732 participants, the researchers measured three key factors: baseline ability, belief calibration (how well individuals’ confidence matched their actual performance) and the impact of AI assistance. Participants were asked to assess whether individuals in photographs were over 21 years old.

“We found that AI improves performance on average,” said Caplin. “But the amount of that improvement depends on whether people are appropriately aware of their own abilities and limitations.”

On average, AI assistance increased accuracy by nearly 7 percentage points, a more than 10% improvement relative to baseline performance. Still, those gains were not evenly distributed. Lower-ability participants benefited more from AI assistance than higher-ability participants. But the researchers discovered a deciding factor was not simply the use of AI but also how self-aware study participants were about their own skill levels.

“Holding ability constant, better-calibrated individuals benefited significantly more from AI,” said Deming. “The results reveal a striking pattern. Participants with low baseline ability but accurate self-assessments gained the most from AI, nearly 10 percentage points in performance improvement. In contrast, highly skilled but poorly calibrated participants benefited the least.”

The study also found that miscalibration, or inhibited self-awareness, limits AI’s ability to raise a work group’s average performance levels. Although AI assistance reduced performance gaps between top and bottom performers by 34%, a counterfactual simulation showed that if workers were perfectly calibrated, discrepancies between the two ends of the spectrum would shrink by 61%, almost twice as much.

“In our data, the individuals who stand to gain the most from AI — those with lower ability — also tend to be the least self-aware in terms of skills self-assessment,” said Li. “That miscalibration dampens AI’s potential to create better and more consistent work groups.”

Importantly, the findings suggest that improving workers’ self-awareness may enhance the productivity benefits of AI. Unlike baseline cognitive ability, calibration appears to be trainable.

“Calibration is a skill,” said Martin. “Prior research shows that even short interventions can improve how well people align confidence with accuracy. That raises the possibility that combining AI tools with calibration training could significantly amplify gains.”

As organizations invest heavily in AI systems, the study suggests technology alone will not determine productivity outcomes. Human judgment — and the ability to accurately assess one’s own competence — plays a central role in determining who benefits.

“Our results show that AI does not simply reward the most skilled,” Marx said. “It rewards those who understand when to rely on it.”

The full study can be accessed here.

About INFORMS and Management Science

INFORMS is the world’s largest association for professionals and students in operations research, AI, analytics, data science and related disciplines, serving as a global authority in advancing cutting-edge practices and fostering an interdisciplinary community of innovation. Management Science, a leading journal published by INFORMS, publishes research on decision sciences, strategy, innovation and quantitative methods that inform managerial and policy decisions. INFORMS empowers its community to improve organizational performance and drive data-driven decision-making through its journals, conferences and resources. Learn more at www.informs.org or @informs.

Contact

Rebecca Seel

Public Affairs Specialist

(443) 757-3578

[email protected]

###