- AI adoption could stall if enterprise networks can’t handle more stringent latency, packet loss and jitter demands
- Cisco says AI traffic in campus and branch networks jumped 34% and could nearly double in the next year
- Enterprises are racing toward AI at scale, but brittle networks may be the hidden barrier to production success
After years tinkering, testing and piloting, enterprises think they’re finally ready to push AI into production at scale. But there’s one critical component that – left unaddressed – could sabotage their plans: the network.
“They’re looking at things like compute and how many GPUs they have access to and where they are. But I don’t think that they’re looking nearly enough at the quality of the network between the endpoints, between where their AI is and where their users are,” DE-CIX Peering Consultant Brandon Ross told Fierce.
Here’s why this is such a big deal. Cisco SVP and GM for Campus Networking Michael Dickman told Fierce that AI is dramatically changing the shape of traffic patterns. Not only are flows trending more symmetrical (that is, as much information is flowing out to the AI in a data center as the AI is sending back to the enterprise campus), but there are also more of them and they’re more intense.
Dickman added that while enterprise networks are prepared for North-South traffic, or traffic that flows from the campus to an external destination like a data center, they’re nowhere near as prepared for the expected surge in internal East-West traffic that is expected to come with edge inferencing and physical AI.
“As you have that East-West traffic, it changes and intensifies the performance requirements,” Dickman said, pointing to latency as one example. He added AI also drives the need for more identity driven segmentation to distinguish between people, things and agents and apply security protocols accordingly.
“All of those are exposing gaps in networks that were built for a different era,” Dickman said.
Ross added that solving for latency, packet loss and jitter will be critical to ensuring AI lives up to enterprise expectations. Any kind of real-time AI application “is going to be highly dependent on all of those, especially packet loss and especially latency,” Ross said.
How big is the problem?
A recent Cisco report found that traffic in campus and branch networks tied to AI workloads has increased by an average of 34% over the past 12 months. It is expected to grow by another 95% over the next year.
The same report found 73% of organizations will face campus and branch capacity limitations within the next two years.
DE-CIX, meanwhile, released its own report based on a survey of 400 IT and infrastructure decisionmakers in the U.S. and U.K. That study found that 96% of enterprise leaders say their networks are ready to support future cloud and AI initiative, yet their teams spend an average of 11.5 hours each week troubleshooting network connectivity issues.
As enterprises press ahead with AI adoption, Dickman said the “the nightmare scenario is it doesn’t work because the network isn’t ready.”
What can be done about it?
To avoid inadvertently tanking AI business initiatives, Dickman and Ross said enterprises need to take the time to upgrade their networks and proactively close performance gaps.
Dickman said beefing up security and segmentation capabilities is another important item on the checklist.
Ross flagged enterprise reliance on public internet connections as a potential hurdle, given these only promise “best effort” connectivity. He also noted that enterprises need to pay attention to and test their backup path just as much as their primary path. This is because network operators periodically “groom” paths and that can mean enterprises unwittingly get put on a path that doesn’t work for them. But they won’t know there’s a problem until it’s too late if they don’t test.
And while IT teams have been among the first targets for AI-related layoffs, Ross also stressed the importance of having a robust onsite crew capable of tackling network impairments when they happen.
“I see in the market that level of talent is not necessarily employed by enterprises as much as they could be,” Ross said. “If you need someone in real time when you’re network’s not performing to troubleshoot that, that could be your real weak spot.”
Read more about enterprise AI here:
AI inference strains enterprise infrastructure—F5 says telcos can help
Five enterprises, one lesson: AI runs on the infrastructure you already have
Enterprises are losing control of AI stacks but telcos can help, says IBM
How should enterprises and telcos think about open weight AI?
Enterprises now write $100M checks for AI infrastructure, says VAST Data VP