In the fast-paced world of enterprise technology, a single year can feel like an eternity. But in the field of artificial intelligence (AI), the past 12 months have completely rewritten the enterprise architecture playbook. The rise of agentic AI—where systems are given objectives rather than simple prompts—is forcing IT leaders to rethink their infrastructure, data management, and operational costs from the ground up.

Speaking on the sidelines of the Dell Technologies World conference in Las Vegas, Dell’s global chief technology officer, John Roese, outlined a sweeping shift in how enterprises should approach AI. “We have shifted our assumption in that the use of AI is no longer a one-shot task like a chatbot,” Roese said. “It’s about handing objectives to the AI system, and that’s what agents do today.” As an example, he pointed to Google’s redesign of its search engine: “You give it an objective, it does some search stuff, and then it builds a whole page for you. Those are all agents working to accomplish an objective.”

Because the user experience with agentic AI is far superior—the human becomes an instructor rather than a doer—enterprises are ripping up old generative AI use cases to rebuild them as agentic workflows. This transformation touches every layer of the technology stack, from hardware procurement to data architecture and cost management.

Busting the GPU training myth

During the initial AI boom, many enterprises rushed to secure large clusters of graphics processing units (GPUs), believing that massive compute power was essential for success. Roese argues this is a fundamental misunderstanding. “The myth out there is that enterprises need thousands of GPUs,” he said. “Our biggest workload inside of Dell only sits on 16 GPUs and supports 40,000 people. You don’t need thousands of GPUs in an enterprise, because for each workload, agent, or project, you only need a handful of GPUs, sometimes half a GPU.”

The reason is that most enterprise AI estates are focused on inference, not training. Large language models (LLMs) are already pre-trained by hyperscalers and model providers; enterprises simply need to run them to generate answers, make decisions, or control agents. “For agents, you only need inference,” Roese emphasized. “There’s no training for agents.” This realization has profound implications for hardware procurement and capacity planning.

However, the architecture needed for inference workloads is changing as well. When enterprises were building simple chatbots, the computation was heavily GPU-centric, with a very light central processing unit (CPU) load. AI agents, by contrast, rely on external tools, communication protocols, and knowledge graphs—components that do not naturally live in the GPU. “When you move to agentic, it’s almost balanced,” Roese explained. “The number of CPUs and GPUs are very similar, about maybe for every two GPUs you have a CPU. You don’t just build an AI infrastructure with a pile of GPUs—you build it with GPUs and traditional CPU compute.” This balancing act requires a new kind of systems thinking, where storage, networking, and compute must be orchestrated in concert.

Air-gapped frontier models and the edge

Enterprises are also benefiting from changes in how powerful AI models are deployed. A year ago, the most capable frontier models were locked behind cloud application programming interfaces (APIs), forcing organizations to send data externally and incur latency and privacy risks. Today, hyperscalers such as Google Cloud offer services like Google Distributed Cloud, which allow top-tier models to be run on-premise. “You can consume it in a virtual private cloud or your datacentre, and you can air-gap it from everything else,” Roese noted. “We didn’t have any of those options, except the API one, a year ago.”

Simultaneously, AI is moving to the edge in a structured way. The recent emergence of agentic frameworks such as OpenClaw—which run natively on devices and AI PCs—has put structure around edge inference. “Those have finally put some structure around running agents on devices, and that’s incredibly powerful, and not a fad,” Roese said. This shift enables real-time decision-making in manufacturing, retail, and healthcare, where latency and connectivity cannot be guaranteed.

Re-architecting the data layer

Meanwhile, data strategies are evolving in tandem with agentic AI developments. Roese warned that bolting standard data storage systems onto AI compute clusters is no longer sufficient. “One of the performance bottlenecks is you can’t get data fast enough to the GPUs to do the work,” he said. “The GPUs you’re paying for are sitting idle, waiting for data.” To address this, organizations must build knowledge and context layers that include vector databases, graph databases, and data annotation tools. These layers cannot sit isolated; they must be deeply integrated into compute.

Dell’s AI data platform now integrates directly with Nvidia’s Cuda-X interfaces, effectively running data layer services at GPU speed. This reduces latency and maximizes utilization of expensive hardware. The lesson is clear: agentic AI demands a converged architecture where data, compute, and memory are tightly coupled.

Mastering tokenomics and model routing

With more model deployment options available at different pricing mechanisms, IT leaders must also manage the cost of AI consumption—even as the cost per token is expected to decline over time. Because “there’s no path where it becomes cheaper to do AI,” enterprises must treat AI workloads as an arbitrage game, said Roese. Using specification-driven development—where AI writes software based on a markdown document—as an example, he noted that if an agentic framework spawns dozens of coding tasks and blindly sends them to top-tier models, enterprises could end up with a much higher bill.

With model routing, however, enterprises can ensure that complex planning tasks—such as creating software specifications—are sent to expensive frontier models, while routine coding tasks are routed to smaller, on-premise open-source models where energy is the only operational cost. “Building a piece of software and doing spec-driven development might have four or five different economic paths to ultimately get to the best overall economic efficiency,” Roese explained. Mastering model routing, he added, will be a competitive differentiator and help lower the cost of product development.

Tokenomics also includes understanding the cost of context windows and retrieval-augmented generation (RAG). Larger context windows consume more tokens, and frequent queries to vector databases add to latency and expense. Enterprises must carefully design their agentic workflows to minimize unnecessary token usage, balancing accuracy with cost.

The human element

Ultimately, the hardest part of operationalising agentic AI relates to the human element. Roese described the traditional human job as a “container of work” that includes a mix of hygiene, productivity, coordination, and expert tasks. Agents cannot perform an entire job, but they are highly capable of executing specific types of work in that container. Dell itself has audited 6,400 jobs across its own business to see how AI agents would impact its workforce.

“The first thing we realised is every single job in the company is going to change,” said Roese. “I’m taking work out of the job and removing stuff from the container. If the container is now only half full, do I need half the number of people, or do I expand that by half? Am I able to do more expert work?” This question is at the heart of the enterprise transformation. The impact of AI on the workplace is so profound that change management has become a key remit of IT leadership.

Roese admitted that for the last four months, he has spent 50% of his time dealing with human dynamics. “AI has ceased being a technology and an ROI discussion. It’s now very much an organisational and human dynamic discussion. You simply can’t use these things unless you fully understand how you’re going to adapt the human population around them.” Enterprises must invest in reskilling, redesign workflows, and create new roles that combine human judgment with AI efficiency. The people dimension is no longer an afterthought—it is the critical enabler of agentic AI success.

As the pace of innovation accelerates, the enterprises that thrive will be those that embrace not only the technical rearchitecture but also the cultural shift. From GPU myth-busting to tokenomics mastery and human-centric change management, the path forward is complex but clear. The age of agentic AI is here, and it demands a complete rethinking of how businesses operate.

Source:ComputerWeekly.com News