- Scaling up compute for AI requires more distance that traditional copper interconnects can provide
- Optical interconnects are the obvious answer, with Lightmatter and a growing number of players jockeying for position in the market
- Adoption has hitherto been held back by concerns about reliability and supply chain shortages
Lightmatter thinks it has the cure for one of the biggest bottlenecks in data center networking today: scale-up. It’s one of a growing cohort of companies trying to secure a slice of an AI networking market that could top $1 trillion by the early 2030s.
Founded in 2017 and backed by companies including Google, Comcast and HPE, Lightmatter is looking to bring optical technology to the scale-up domain. Its Passage suite of interconnect products includes near-packaged (NPO) and co-packaged optics (CPO) and an optical interposer (more on that in a moment). It also makes its own optical lasers, called Guide.
CEO Nick Harris told Fierce Lightmatter’s work is all about finding ways to move more data between processors and do it faster.
“We are focused on the future of computing as a company. And right now what that means isn’t speeding up the processor. It’s about improving I/O and building the lasers that power it,” he said.
What’s the problem with data center networking?
Harris and 650 Group Co-Founder and Principal Analyst Alan Weckel explained that data center networking has hit a wall in the scale-up domain even as AI drives demand for more compute power.
It all comes back to Moore’s Law and the way compute is scaled today. There are two ways to scale: make the chips bigger or connect more of them together. Chips are now as big as they can individually get, so the only answer to scaling lies in connecting more GPUs together – way more than can fit in a single rack.
That means scale-up (connecting chip-to-chip) is bleeding into scale-out (rack-to-rack), but traditional copper-based scale-up solutions simply can’t go the distance needed to connect multiple racks. Enter optics.
“The only way to get these accelerators interconnected and working in the future is to move from copper domain to the optics domain,” Weckel told Fierce. “This is one of the first times we’ve ever had an inflection both in the technology as well as the architecture. Usually it’s been incremental steps – what’s the next speed? Now we’re talking a completely different architecture and mechanism to implement it.”
Near-packaged and co-packaged optics, which place the optical engine on the same board and on the same silicon substrate as the processor, respectively, are nearer-term options. Optical interposers, which feature an optical interconnect layer laying beneath a processor (like a multi-chip sandwich), will surface in the longer-term, Weckel said.
Harris and Weckel explained interposers will prove critical over time because data is sent from the edges of chips today. Assuming a finite chip size, that means there are limits on the bandwidth that can be achieved over time. Interposers unlock another layer of bandwidth by allowing the entire surface of a chip to be used for communication rather than just the edges.
Lightmatter’s NPOs are already in manufacturing and it has taped out five CPO designs for customers already this year. It has also been developing its interposer technology for years and is working on two designs with customers right now, Harris said.
Lightmatter has publicly said it is working with the likes of Nvidia and Qualcomm. But Futurum Group Research Director Brendan Burke said all hyperscalers are actively considering their options for optical networking and are actively pushing their vendors to support new technologies.
Why hasn’t the industry already adopted optical for scale-up?
If optics are the obvious answer, it’s worth asking why this transition didn’t happen years ago. The answer is multi-faceted.
Weckel and Burke told Fierce the industry has hesitated for years in adopting optical for scale-up due to concerns about the reliability and repairability of the lasers used to make optical transmission work.
“The performance benefits and speed ups are well established, but serviceability concerns remain the barrier to adoption. With embedded lasers and fiber optics within a chip tray, those can have a high rate of failure and be very difficult to replace,” Burke explained.
Lightmatter, though, is already solving for these concerns with its design, adding redundant lasers on its chip that can be used to reroute traffic around failures when they happen. Harris added Lightmatter’s technology allows it to reprogram failed lasers and bring them back online, making its solution self-healing.
Weckel said redundant lasers will likely become the de-facto standard in the industry to address reliability concerns.
Then there’s the question of manufacturing and supply chain. Weckel said given the hyperscale customer base, vendors need the ability to rapidly ramp manufacturing to hundreds of thousands or millions of units immediately to win with optical. “This isn’t something you can slowly ramp in,” he said.
Here, too, Lightmatter is trying to get ahead of the curve via partnerships with the likes of TSMC and silicon providers to make ramping more manageable. Harris said its NPO will be shipping and ramping to high volume in early 2027. Its CPO will follow by the end of 2028 or early 2029.
“It’s a supply chain thing,” Harris said. “There’s a bunch of shortages…There are simply not enough lasers to power all of optical scale-up for all the companies that want to do it. There’s not enough optical fiber…I don’t think they’re going to be able to make that next year. It’s going to take a couple years of building factory capacity and tooling.”
Where is the data center optics market is headed?
Despite these challenges, there’s a reason vendors like Lightmatter continue to jockey for position in this space.
Burke tipped the data center optics market for scale-up and scale-out to hit $100 billion by 2030 and pegged the broader AI networking TAM at $350 billion.
Weckel said the AI networking market (including scale-up, out, across and traditional switching) is worth around $100 billion today and will grow to around $400 billion by the end of the decade. By the early 2030s, Weckel said the market will be worth “well north of $1 trillion.”
“It’s exciting for systems vendors. It’s exciting for silicon vendors. It’s exciting for startups like Lightmatter because that’s a tremendous opportunity you’re selling into,” Weckel said.
But the push for optical doesn’t mean copper technology will disappear overnight, Burke said.
“Copper has an enduring role in data centers and especially with different modalities between training and inference, there’s going to be copper based systems going forward. Optical really comes into its own with multi-rack scale up,” Burke said. “It’s unlikely there’s a wholesale transition to this technology, but it will be very useful in those cases where training needs to be scaled across multiple racks and also large model inference. So, there could be a mass market that still depends on copper even in five years.”
Read more about optical networking on Fierce Network
Salience Labs thinks optical switching can break AI’s latency bottleneck
CScale bets AI factories can ride out laser failures at gigawatt scale
Optical circuit switching is about to storm the data center market
Optical startups to watch as AI pushes optics deeper into the network
Oriole Networks pushes pure photonic networking for AI data centers