SambaNova: Inference bubble? What inference bubble?

AI costs
"Inference is maybe the greatest business model of all time," SambaNova's Anton McGonnell told Fierce. (Art by Midjourney for Fierce Network)
  • SambaNova says AI inference is already profitable, dismissing fears of an inference bubble
  • SambaNova claims customers can recoup AI chip investments in about six months
  • Despite Gartner's demand concerns, SambaNova sees enterprise AI adoption accelerating

Worries that AI infrastructure spending is running far ahead of revenue are growing, but SambaNova thinks the economics of inference tell a different story.

"Everybody who's doing inference well, running inference well, is making money today and paying back their investment," Anton McGonnell, VP of Product and Core Tech at chipmaker SambaNova, told Fierce. "That's so clear across the board." 

So why does Wall Street have cold feet? Well, many of the leading AI companies are private, so it’s hard to get a real sense of their financial picture. Details only recently started to emerge when Anthropic made its public IPO filing; OpenAI’s filing, which was made in June, remains confidential.

McGonnell acknowledged this mystery, coupled with the hefty upfront capital required to build an inferencing business, could give investors the jitters. But he insisted even factoring in all the expenses, GPUs and SambaNova’s specialist AI chips “are very, very easy to make a payback on when running inference.” 

“This narrative is provably wrong. Inference is maybe the greatest business model of all time. It will be the most profitable revenue generating business of all time," he said. 

Payback in six months

Without aggressive assumptions, he said, customers can recoup their capital investment in about six months on a system with a five-year useful life. After that, they are mostly paying operating costs, which he described as small relative to capital costs, and "everything else is just pure bottom line."

McGonnell held up Anthropic's S-1 as proof that inference is a strong business. When pressed on the company's sizable losses (including a $693 million operating loss and an $80.7 billion net loss), he argued the losses reflect investment in growth and training, not weak inference economics. He said inference gross margins are above 90%, "better than SaaS.”

The demand question

As an AI chipmaker, SambaNova obviously has a vested interest in the success of inference use cases. Inferencing workloads are already on the rise and McKinsey has forecast that inferencing will surpass training as the dominant AI workload in 2027 and could account for as much as 40% of overall data center demand by 2030.

The harder question is whether demand for inference will hold. In October, Gartner Senior Director Analyst Will Sommer told Fierce that the firm’s research shows enterprises struggle to get value from AI and consumers are reluctant to pay for it.

McGonnell countered that "someone's paying" OpenAI, Anthropic and other providers "historic amounts of revenue" at an unprecedented growth rate. 

Enterprise adoption may lag AI-native firms, he said, but he sees financial services and other industries now adopting aggressively. "Any enterprise that doesn't adopt this will be left behind,” he concluded. 

Read more about SambaNova and AI capex on Fierce Network:

Gartner warns AI data center capex is ‘unsustainable’ as $3.5T write-down risk looms

SambaNova targets AI inference boom with chips built for existing data centers

Agentic AI needs more CPUs. But does it need special ones?

Hyperscalers face growing investor doubts over AI data center spending spree