As enterprises race to scale AI, the biggest obstacle to performance and ROI may be the infrastructure moving data, not the hardware processing it.
Enterprise AI has entered a new phase. For the past 18 months, organizations have spent aggressively on GPUs, large language models (LLMs), and AI tooling. Now the focus is shifting from experimentation to operationalization. With that comes a sharper emphasis on ROI.
According to a 2025 IDC Spotlight report, organizations are moving away from one-off AI deployments and toward repeatable, scalable architectures designed to support production workloads. As AI becomes embedded across the business, performance, security, reliability, and operational consistency are becoming just as important as model innovation.
And moving data between storage and compute is becoming increasingly complex. As AI environments grow more distributed, ensuring that data reaches compute resources quickly, securely, and reliably has emerged as a critical infrastructure challenge. These developments are forcing CIOs to confront a difficult question: How do you turn AI investment into measurable business value?
When AI projects underperform, many tech leaders assume they need more compute. They add GPUs, expand clusters, or look for a better model. But according to infrastructure teams operating large-scale AI environments, the problem often lies elsewhere.
That’s because the GPUs aren’t starving for compute. They’re starving for data.
In truth, expensive compute resources are only as effective as the systems feeding them. If data can’t move efficiently, securely, and consistently between storage and compute, even the most powerful GPU clusters end up waiting. And idle GPUs are among the most expensive assets in the data center. The cost of disruption is rising too. According to the Uptime Institute’s Annual outage analysis 2025, more than half of organizations say their most recent significant outage cost more than $100,000 and one in five report costs exceeding $1 million.
Look below the waterline
To understand why AI initiatives stall, it helps to rethink the traditional infrastructure model.
Nirav Shah, senior vice president of product marketing at F5, compares modern AI infrastructure to an iceberg. Above the waterline sits everything executives can see: LLMs, AI applications, orchestration frameworks, and increasingly expensive GPU clusters. This visible layer receives most of the attention, and often most of the investment.
Below the waterline lies the infrastructure that determines whether those investments actually deliver value in production: storage, networking, traffic management, security controls, and the systems responsible for moving data between storage and compute.
“Everybody is focused on the visible 10%,” Shah says. “But it’s the other 90% that determines whether those investments actually work.”
That’s where many organizations discover their real bottleneck.
Modern AI systems depend on enormous volumes of unstructured data stored in Simple Storage Service (S3)–compatible object environments. Training, fine-tuning, retrieval-augmented generation (RAG), and inference workloads all rely on a continuous flow of data between storage systems and GPU environments. When that pipeline becomes constrained, GPU utilization drops.
“The symptom looks like a compute problem. The root cause is often data starvation,” says Mark Menger, solutions architect at F5.
And unlike traditional enterprise applications, AI workloads amplify small infrastructure weaknesses. A latency spike, throughput blockage, or traffic surge that might go unnoticed in a conventional environment can have an outsize impact on AI performance.
That’s why many organizations are beginning to look beyond GPUs and focus instead on the storage-to-compute boundary.
Moving from tight coupling to loose coupling drives flexibility
Many of today’s bottlenecks can be traced back to architectural decisions that made perfect sense before AI arrived. Historically, enterprises connected applications directly to storage environments. The approach was simple, efficient, and easy to manage. At AI scale, however, that simplicity becomes a liability.
Storage systems suddenly find themselves handling far more than storage. They are terminating encrypted connections, managing network traffic, enforcing security policies, and processing enormous volumes of requests from increasingly distributed AI workloads. Every encrypted transaction consumes CPU resources. Every connection creates overhead. Every traffic spike places additional strain on systems designed primarily to store and retrieve data.
The result is a classic infrastructure problem: Storage platforms become busy doing work they were never optimized to perform.
To address this challenge, many organizations are moving toward a loosely coupled architecture.
Instead of connecting compute directly to storage, they insert an application delivery controller (ADC) between the two. Acting as an intelligent control plane, the ADC becomes the storage front door, handling transport layer security (TLS) termination, certificate management, traffic optimization, policy enforcement, and protocol-aware S3 processing.
By moving networking and cryptographic functions into infrastructure specifically designed for those workloads, storage systems can focus on what they do best: serving data. The approach also creates operational flexibility. Storage environments can be upgraded, expanded, or migrated without forcing application changes, a concept infrastructure teams often describe as loose coupling.
Rethinking the ‘bump in the wire’
For decades, infrastructure teams viewed any additional layer in the data path with suspicion. The assumption was simple: Every extra component adds latency.
AI infrastructure is challenging that assumption. Recent independent testing conducted by SecureIQLab evaluated the impact of placing an ADC in front of enterprise object storage. The results showed no meaningful throughput penalty, with performance generally remaining within a narrow variance range compared to direct node access.
More interestingly, the testing found that under real-world network conditions, throughput often held up substantially better when traffic was managed through the control layer. The reason is straightforward. The ADC isn’t simply passing traffic through. It is optimizing connections, managing protocols, offloading cryptographic processing, and intelligently steering requests.
In other words, it is helping the data path perform more efficiently.
When data delivery becomes a business problem
When AI systems struggle, the answer is not always more GPUs. Sometimes it’s better engineering.
A large global financial services organization was preparing to expand its AI infrastructure by using Kubernetes-hosted workloads and S3-based object storage. Its existing environment relied on shared virtual load balancing, creating both performance and reliability challenges as data volumes increased. Rather than investing in additional compute capacity, the organization focused on the storage-to-compute boundary.
It deployed dedicated physical ADC infrastructure in front of its object storage environment, creating a centralized control point for traffic management and S3 optimization. The results were dramatic.
The organization achieved at least a fivefold improvement in object create, read, and delete operations. In some cases, delete latency improved by more than an order of magnitude. Just as importantly, the new architecture introduced no performance regression compared with direct node access.
The three dimensions of resilience
Performance is only part of the story. According to Menger, organizations should evaluate their AI data delivery architecture through three dimensions of resilience: reachability, policy, and delivery.
- Reachability ensures that AI workloads can always access healthy storage resources. If a storage cluster degrades or becomes unavailable, traffic can be redirected automatically without disrupting AI applications.
- Policy protects organizations from self-inflicted disruptions. AI workloads can generate “thundering herd” scenarios, retry storms, and other traffic anomalies capable of overwhelming storage environments. An intelligent control layer can shape traffic, enforce policies, and maintain a consistent security posture without sacrificing performance.
- Delivery focuses on continuity. Storage nodes fail. Hardware is upgraded. Software is patched. A resilient architecture isolates AI clients from that churn, maintaining uninterrupted data flow even when back-end infrastructure changes.
Together, these three capabilities help ensure that GPUs remain productive even when the underlying environment is under stress.
Why ADCs are evolving into ADSPs
As AI environments become larger and more distributed, organizations are discovering that traffic management alone is no longer enough. They also need observability, security, policy enforcement, and operational consistency.
This shift is driving interest in application delivery and security platforms (ADSPs), which combine application delivery, traffic engineering, security controls, and visibility into a unified platform.
“AI broke the model of solving delivery and security as separate problems,” says Shah. “When data is moving constantly between storage, compute, and applications across hybrid multicloud environments, you need one platform that delivers and protects that traffic at the same time.”
The trend mirrors the evolution of web infrastructure two decades ago. What began as simple load balancing eventually evolved into sophisticated application delivery platforms capable of managing increasingly complex environments.
AI infrastructure is following a similar path. As enterprises scale across hybrid cloud, multicloud, edge, and on-premises deployments, integrated control planes become increasingly valuable.
The infrastructure mandate
The AI industry has spent the last two years focused on compute. But enterprise AI is increasingly a data delivery problem. Models and GPUs can’t create value if data can’t move efficiently between storage and compute.
For CIOs under pressure to show AI ROI, the next breakthrough may not come from buying more GPUs. It may come from keeping the ones you already own fed.
Join Nirav Shah at the F5 AI Summit on June 23, where he will outline how to architect AI for success. Mark Menger and Hunter Smit will go deeper into AI data delivery in their session: “Reliability belongs between compute and storage.” Register now.
