- Centralized AI compute isn’t going away — but it is becoming infrastructure rather than the center of the AI economy.
- IBM’s demand-led approach exposes the risk in building vast amounts of speculative AI capacity.
- The real AI opportunity is shifting from building bigger models to putting intelligence to work across the economy.
IBM and Together AI have just signed a $240 million deal to build a large Nvidia-powered AI inference cluster on IBM Cloud.
At first glance, this looks like a data point against my argument that the AI industry is building an enormous infrastructure bubble around centralized LLM compute. Here is another nine-figure commitment to GPUs sitting in a data center.
Look closer. It actually helps explain what comes after the bubble.
IBM’s Alan Peacock made one particularly revealing comment about the company’s approach to AI infrastructure:
“I’m not going to go and sign up for massive infrastructure deals to build a lake and then hope somebody comes to drink from it.”
That’s the AI bubble in one sentence.
IBM isn’t saying centralized compute is unnecessary. Quite the opposite: Together AI has identified demand, IBM is supplying infrastructure to meet it and $240 million is changing hands.
What IBM is rejecting is speculative capacity — building enormous amounts of centralized AI infrastructure on the assumption that sufficient demand and economic returns will eventually materialize.
There’s another important distinction. This infrastructure is primarily for inference, rather than simply another gigantic frontier-model training project. And Together AI increasingly positions itself as an infrastructure platform for production inference, particularly around open-weight models.
That’s an important transition.
The first phase of the generative AI boom was dominated by model creation: enormous GPU clusters training ever-larger foundation models. The implicit economic assumption was that the companies controlling those models — and the centralized infrastructure underneath them — would capture an extraordinary proportion of the value created by AI.
It is increasingly obvious that assumption was wrong.
Oracle bet on the bubble
Oracle may be the purest corporate expression of the central assumption behind the AI infrastructure bubble: that demand for centralized frontier-model compute will remain sufficiently scarce, valuable and durable to justify financing an unprecedented infrastructure buildout. The collapse in its share price suggests investors are finally asking whether that assumption was ever economically sound.
Which brings us back to IBM’s Alan Peacock and his lake.
IBM is building AI infrastructure when somebody has already come to drink from the lake. Oracle borrowed billions to build the lake on the assumption that the frontier-model economy would keep drinking.
The IBM/Together deal points toward a different future.
As models proliferate and the cost of running them falls, the competitive question changes from “Who can build the biggest model?” to “Who can put intelligence to work most effectively?” Together itself emphasizes inference optimization designed to extract more usable work from every GPU and reduce cost per token.
And that takes us far beyond the data center.
The biggest economic opportunities for AI ultimately lie in factories, telecom networks, vehicles, robots, hospitals, mines, power grids, warehouses and countless other physical and digital systems (see the figure).
AI therefore becomes a distributed infrastructure problem.
Some inference will happen in enormous centralized data centers. Some in sovereign and regional clouds. Some in enterprise infrastructure. Increasing amounts will happen at the network edge and inside machines and devices. IBM itself describes cloud, edge and on-device inference as distinct deployment models, with edge inference particularly suited to applications such as factory sensors and hospital monitoring where latency and privacy matter.
IBM’s participation is revealing. Its advantage isn’t that it has suddenly produced the world’s most powerful LLM. IBM brings enterprise relationships, hybrid infrastructure, security, governance and integration. Indeed, IBM Research says enterprises moving from LLM experimentation into production are increasingly choosing on-premises deployment to retain control of their infrastructure, data and costs.
Those capabilities become more important as AI moves out into the economy.
The mistake at the heart of the AI bubble isn’t believing we’ll need enormous data centers. We will. It’s believing that the data center is where most of the economic value created by AI will ultimately reside — and building infrastructure speculatively on that assumption. The Internet required enormous investments in servers, routers, fiber and data centers. But the economic value created by the Internet didn’t remain inside Cisco routers or Sun servers.
AI will follow the same pattern. Centralized compute isn’t going away. It’s just becoming infrastructure. And the real battle is shifting to whoever can connect that intelligence to the systems on which the economy actually runs.
Learn more about AI at Fierce Network
IBM lends Together AI its enterprise street cred in $240M compute deal
Opinion: AI's Great Divide—Bubble risks and untapped frontiers
Stephen M. Saunders MBE is a communications analyst and USPTO-registered inventor examining how digital infrastructure — 5G, cloud and AI — is reshaping industry, power and society, as well as underpinning the emerging, ubiquitous global digital economy. As anchor of FNTV and a longtime industry insider, he focuses less on growth narratives and more on execution, risk and how hyperscale technology is distorting markets, governance and society at scale.
Opinion pieces from industry experts, analysts or our editorial staff do not represent the opinions of Fierce Network.