As enterprise leaders struggle to integrate the flood of new AI tools, a more fundamental approach focused on operational redesign is emerging.
Harsha Upadhyay, an e2e Data Architect for IBM's Marketing Data Platform, explains that companies must first build a strong internal data foundation before adopting agentic AI.
To build a resilient agentic enterprise, leaders should first strengthen their data and governance foundations before cautiously implementing new AI systems in a modular, use-case-driven way.
In the wake of a relentless flood of AI releases, enterprises are scrambling to find solid ground. From GPT-5 and 5.1 to Claude 3 Opus, Grok, and Genie, the last few months have made one thing clear for leaders: simply integrating new AI tools as they appear is a strategy destined for failure. Instead, success in this new era requires a more fundamental redesign of how organizations operate. Most of the time, it involves transitioning from traditional enterprise architecture to an "agentic" model, where AI serves as the operational foundation.
For an expert's perspective on this strategic shift, we spoke with Harsha Upadhyay, an e2e Data Architect for IBM's Marketing Data Platform. A leader in enterprise data architecture and digital transformation, her extensive career also includes senior data architect positions at industry giants such as Cisco and AT&T, where she developed data and governance solutions to address complex business problems. According to Upadhyay, before any organization can harness the power of agentic AI, it must first look inward and rebuild its core.
"Agentic solutions and agentic enterprises will fail without a knowledge inventory and a strong data layer," Upadhyay says. In direct opposition to the "move fast and break things" ethos of Silicon Valley, she advocates for a "safe approach" to navigating uncertainty instead. It’s a two-part strategy designed to build resilience while cautiously exploring the frontier, she explains.
First things first: The first part is a no-regrets investment in foundational pillars like governance, executive strategy, and data hygiene. "So even if you think tomorrow, 'This LLM is a flop and we can't do anything,' you are not losing anything."
Start small, learn fast: Only after the foundation is solid should organizations begin adopting agentic systems, Upadhyay says. "Rather than opening a full-fledged initiative, I would go with a smaller, modular approach where you try out different applications step-by-step. Then, through self-learning, the organization will learn from its own shortcomings and apply that to the next one."
Naturally, this use-case-driven approach leads to a diversified portfolio of AI models, Upadhyay continues. For her, this is an inevitability, especially for large corporations. Here, the key is to manage this diversity with a structured, two-layer system: a first set of high-level, general-purpose LLMs, and a second set of customized models specialized for specific business functions, such as a "sales model" or a "marketing model."
Even with a perfect strategy, however, the road to an agentic enterprise is filled with hurdles, Upadhyay says. Admittedly, the industry's concerns around security, privacy, and integration with legacy systems are significant.
Accountability abyss: For Upadhyay, the most profound source of enterprise hesitation often comes from a single, unresolved issue. "The word is 'accountability,' and that is still not very well defined," she says. "Most of the time, a model shows the correct response. But what about that one single time when it did not, though it gave the impression that it was the right response? How will you handle that accountability? It's not really well defined, and though model owners say they will address ethical considerations, a defined process is still not in place."
Despite the challenges, Upadhyay is optimistic about the future. Framing it as a mechanism for individual empowerment, she paints a vivid picture of AI as a superhuman toolkit that enhances human capability. "If I imagine myself ten years back, I could not do so many things because I had to reach out to 10 different people, all to do the same task. Now, I'm able to do it by myself today. Because it has opened so many doors, all you have to do is think, innovate, and use your creativity."
Ultimately, if used responsibly, Upadhyay concludes, this technology could empower individuals to be stronger than they ever thought possible. "That's the power I see. AI is being made available to the general public, and it's helping them do more with it."
