- Salience Labs says optical circuit switching can help solve AI inference latency
- The startup’s on-chip OCS design targets faster, lower-power data center networking
- Salience has AMD and Intel in its corner as AI back-end switch sales skyrocket
As AI workloads increasingly shift from training to inference, the industry has been working to tackle a key issue: latency. In the silicon realm, the data movement problem has been solved for by the likes of Groq (whose technology was acquired by Nvidia), SambaNova, Cerebras and Rebellions. But Salience Labs CEO Vaysh Kewada told Fierce the issue hasn’t yet been resolved on the network side.
“We're still using the same network that you would use for high performance computing and [that] has a lot of features that are redundant and add latency. That’s now very expensive because the main thing that's mattering to all of our customers who are spending hundreds of millions of dollars on AI capex is how can they get a good return out of it,” she said.
That’s where optical circuit switching (OCS) comes in. OCS eliminates the need for optical-to-electrical signal conversion by establishing light-based pathways. This not only yields power savings but also reduces latency.
Salience plays in the scale-up realm – that is, the connectivity layer that connects servers in a data center rack. But as the size of GPU clusters increases from dozens to hundreds or even thousands of chips, Kewada said the lines between scale-up and scale-out (rack-to-rack) networking are blurring. That makes the low latency OCS provides all the more important, she said.
What’s happening in the OCS market
Most of the OCS solutions in the market are free space optics and MEMs-based technology, like those offered by Lumentum, Coherent, Omnitron, Telescent and the like. But Salience wanted to do something a bit different.
“We wanted to be the first to develop a fully integrated on-chip optical circuit switch,” Kewada said. That’s because Salience believes a chip-based solution will scale better from a manufacturing perspective, is cost-effective and a solid-state solution, which Kewada said is important from a reliability perspective.
Its two-chip design comprises a switch chip and an amplifier chip to address optical loss.
There’s good reason vendors are chasing this market. Dell’Oro Group has forecast that cumulative back-end switch sales between 2026 and 2030 will approach $1 trillion. The scale-up segment is expected to account for more than half the market by the end of the forecast period and will “more than double the total addressable market for switches deployed in AI back-end networks,” Dell’Oro VP Sameh Boujelbene said in a statement.
Moving at lightspeed
Founded in 2021, Salience Labs has raised around $180 million to date. That includes a $125 million round closed last week which included AMD and Intel as investors.
The company demoed a 32-port OCS solution in March and followed up with a 64-port version in September. It is working toward 128- and 256-port versions of its tech using its latest batch of funding.
Early simulation results have been promising. Kewada said its OCS technology can increase tokens per second per user by up to 52%, depending on the model.
Salience currently has around 60 employees, but Kewada said it’s planning to scale both its headcount and manufacturing operations “pretty aggressively” over the next 12 to 18 months.
It has already announced Tower Semiconductor as its foundry partner and has strategically arranged for half of its chips to be fabricated in the United States. The rest of its supply chain, Kewada said, is entirely outside of China.
“I think we're in a very critical point in this market. Decisions will be made in ’27 for early deployments in ’28 and volume [in] ’29, 2030,” Kewada concluded. “We’re moving into an era of more optical connectivity and more heterogeneous architectures, and this I think means it becomes inevitable that optical circuit switches in some format will be deployed in the network. And given the size of the market, that is a very significant change.”
Read more about optical networking on Fierce Network:
Optical circuit switching is about to storm the data center market
Optical startups to watch as AI pushes optics deeper into the network
Oriole Networks pushes pure photonic networking for AI data centers
Arista gets into scale-up game as accelerator landscape diversifies