The Indispensable Human in the Age of AI

Maya Balakrishnana
Asst. Professor, UT Dallas Jindal School of Management

The rise of Artificial Intelligence (AI) and powerful generative AI (genAI) models has sparked considerable excitement, promising to revolutionize fields from supply chain optimization to customer service automation. But beneath the hype, AI’s performance varies wildly. It excels in some tasks but can spectacularly fail in others that seem equally straightforward. This “jagged technological frontier” means that some tasks are easily handled by AI, while others, despite their apparent simplicity, remain beyond its current (and perhaps future) capabilities (Dell’Acqua et al., 2023).

In my AI in Supply Chain Management class, I assign students to create a supply chain-themed crossword puzzle. Half the students are allowed to use genAI while the other half work independently. Students relying on genAI are often shocked by genAI’s struggle with such a seemingly simple task. They obtain disconnected grids, incorrectly spelled words forced into place, and inaccurate letter counts. And these students who initially lean on AI to create these puzzles end up spending more time modifying and correcting the AI’s flawed outputs than those students who skipped using AI altogether, as they were anchored to the AI’s initial, incorrect recommendations.

AI’s Variable Performance across Tasks

This jagged frontier extends beyond creative tasks. AI’s variable performance stems from several factors including:

• Missing Private Information: AI models operate solely on their training data. Humans, however, frequently possess “private information”—valuable, predictive insights inaccessible to the algorithm (Balakrishnan et al., 2025). For instance, a fashion retail manager might observe social media product trends unavailable to their company’s forecasting algorithm, enabling more accurate demand forecasts for new clothing items.
• Data Distribution Shifts: AI models trained on historical data can lose effectiveness when the underlying environment changes and new data no longer matches the distribution of the training data (DosSantos DiSorbo et al., 2025). For instance, a fraud detection AI trained on past transaction patterns might struggle to identify new types of sophisticated scams that emerge after its last training update. This is particularly relevant in dynamic environments like logistics, where sudden shifts in consumer behavior or global events like Covid-19 can render historical demand patterns obsolete.
• Model Collapse: A concerning phenomenon called “model collapse” occurs when AI models are recursively trained on data generated by other AIs, degrading performance over time (Shumailov et al., 2024). Imagine an image generation AI that learns from images predominantly created by other AIs. Over time, the unique characteristics and diversity of human-created content diminish, leading to a flatter, less imaginative output from the AI.
• Bias in Data: AI models are only as good as the data they’re trained on. If that training data contains biases, the AI will learn and perpetuate those biases, leading to inaccurate or unfair outcomes (Chen, 2023). For example, a hiring algorithm trained on historical hiring data that favored a particular demographic might inadvertently screen out qualified candidates from underrepresented groups, regardless of their actual capabilities.

Regardless of the underlying reasons for AI’s inconsistent performance, the takeaway is clear: AI can be incredibly helpful for some tasks yet hinder human performance in others. The challenge is predicting when AI will excel or fail. Thus, a human in the loop is critical. For effective human-AI collaboration we must understand AI’s variable capabilities, learning when to trust its performance and when to leverage our unique human strengths. So, what is our evolving role in the age of AI?

Preparing for the AI Age: Three Paths Forward

For students entering an AI-shaped world, I see three key paths to success and competitiveness:

1. Mastering AI-Proof Professions: This path involves developing expertise in roles that AI cannot (and likely will not) perform, due to cultural, legal, or ethical reasons. Think of professions requiring high emotional intelligence, nuanced human interaction, creativity, or physical dexterity in a sensitive environment (Eloundou et al., 2023). No one, for instance, is likely to want an AI hairdresser with sharp scissors next to their head. These roles emphasize uniquely human capabilities that remain beyond AI’s grasp.
2. Building and Improving AI: Another path is to be on the cutting edge of AI development, building AI models, helping train them, or creating crucial new training data that fuels their learning. This area is particularly vital given issues like model collapse. Addressing these fundamental challenges requires human ingenuity and deep expertise in AI’s construction.
3. Working Effectively with AI: The third, and perhaps most widely applicable, path is mastering effective AI collaboration. This means understanding AI’s strengths and weaknesses and strategically integrating it into your workflow to enhance productivity and decision-making. However, this isn’t always straightforward. Research has uncovered a widespread pitfall: the “naïve advice-weighting” trap (Balakrishnan et al., 2025).

The Naïve Advice-Weighting Trap

The core challenge in effective human-AI collaboration, particularly along the “jagged technological frontier” is that people often struggle to accurately assess AI’s variable performance. They don’t fully internalize that AI, while sometimes excellent, can also spectacularly fail. This misunderstanding directly leads to what our research, published in Management Science, calls “naïve advice-weighting” (Balakrishnan et al., 2025).

We found that unless explicitly prompted, people tend to blend their own predictions with AI’s recommendations using a constant weighted average. This means they apply the same degree of reliance on the AI, regardless of whether or not they possess valuable private information for that specific instance. This is a natural human tendency when seeking advice—if I trust an advisor I should always adhere to each of their suggestions to a high degree—but it’s particularly problematic when the advice comes from an AI with inconsistent capabilities. It means people aren’t differentiating when they should rely more or less on the algorithm.

This seemingly intuitive approach creates significant problems:

• Over-reliance when private information is valuable: When we have valuable private information that could significantly improve a prediction, we often lean too heavily on AI’s recommendation, failing to adequately adjust for our unique insights. This is akin to trusting AI for tasks beyond its capabilities, leading to suboptimal outcomes.
• Not enough trust when private information is not valuable: Conversely, when we lack valuable private information and AI’s recommendation would be highly beneficial, we often under-trust and under-adhere to AI’s predictions. Here, we underutilize AI for tasks within its skillset.
• Worse Decisions: This naïve advice-weighting behavior can lead to a substantial increase in prediction errors, ranging from 20% to 61% in our experiments. Simply having AI assistance doesn’t automatically translate to better outcomes if humans don’t interact optimally.

In essence, naïve advice-weighting causes humans to underutilize AI assistance when their private information is highly valuable, and conversely, overutilize it when it is not (Balakrishnan et al., 2025).

Strategies for Effective Human-AI Collaboration

Figure 1: Leveraging what humans know that AI doesn’t

How can OR/MS professionals and students effectively collaborate with AI and avoid naïve advice-weighting? Our research and other studies point to several actionable strategies:

1. Understand AI’s Inputs: Transparency is Key. One of the most effective ways to mitigate naïve advice-weighting is through feature transparency—explicitly informing humans about the variables the algorithm considers. We found that feature transparency helps users discriminate more effectively when to deviate from AI, leading to a 25% reduction in prediction error (Balakrishnan et al., 2025). This type of transparency helps humans identify what their private information is, when it warrants a substantial deviation, and in which direction, by providing a clearer mental model of the AI’s limitations.
2. Know Your Unique Value: Identify Your Private Information. Recognize your strength often lies in information AI lacks. This “private information” includes qualitative insights, real-time observations, or contextual nuances difficult to quantify or feed into an algorithm. The key is to be acutely aware of and leverage what you know that AI doesn’t.
3. Adjust Strategically: Beyond Advice Weighting. Instead of simply averaging your prediction with AI’s, adopt a more strategic approach. Our research suggests anchoring on the AI’s prediction and adjusting it only based on your private information. This adjusting nudge helps move away from general advice-weighting heuristics, using private information to inform deviations. We found this intervention led to an additional 21% reduction in prediction error over feature transparency alone (Balakrishnan et al., 2025). This demonstrates the power of framing the human’s role as a critical adjuster, intervening only when private information warrants it, leading to more robust and accurate decision-making.

The age of AI isn’t about replacing humans; it’s about redefining our role. Despite its remarkable capabilities, AI isn’t infallible. Its variable performance makes human oversight and intelligent collaboration essential. By understanding the “naïve advice-weighting” bias, embracing transparency around AI’s inputs, recognizing our valuable private information, and strategically adjusting AI’s recommendations, we can unlock the true potential of human-AI collaboration. The future of OR and MS lies in cultivating professionals adept at leveraging AI’s strengths while confidently intervening where human intelligence is irreplaceable. This collaborative approach will lead to more accurate predictions, better decisions, and ensure humans remain central to the decision-making process in an increasingly AI-driven world.

References

Balakrishnan, M., Ferreira, K.J., Tong, J., 2025. Human-algorithm collaboration with private information: Naïve advice-weighting behavior and mitigation. Management Science doi:10.1287/mnsc.2022.03850.

Chen, Z., 2023. Ethics and discrimination in artificial intelligence-enabled recruitment practices. Humanities and Social Sciences Communications 10, 567. doi:10.1057/s41599-023-02079-x.

Dell’Acqua, F., et al., 2023. Navigating the jagged technological frontier: Field experimental evidence of the effects of ai on knowledge worker productivity and quality. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4573321. Harvard Business School Technology & Operations Management Unit Working Paper, (24-013).

DosSantos DiSorbo, M., Ferreira, K.J., Balakrishnan, M., Tong, J., 2025. Warnings and endorsements: Improving human-ai collaboration in the presence of outliers. Manufacturing & Service Operations Management doi:10.1287/msom.2024.0854.

Eloundou, T., Manning, S., Mishkin, P., Rock, D., 2023. Gpts are gpts: An early look at the labor market impact potential of large language models. arXiv preprint arXiv:2303.10130. doi:10.48550/arXiv.2303.10130.

Shumailov, I., Shumaylov, Z., Zhao, Y., et al., 2024. Ai models collapse when trained on recursively generated data. Nature 631, 755–759. doi:10.1038/s41586-024-07566-y.

Acknowledgements: We would like to thank Lingchao Mao for taking time to review this article. Photo credit goes to Cash Macanaya for the header photo and Evgeni Tcherkasski for footer photo. Figure 1 is from the author.