Optical network infrastructure has played an important role in datacenter architectures since the turn of this century. It serves as the backbone for long-distance connectivity, delivering needed bandwidth for mission-critical applications and operations at enterprise scale. However, over the past two decades, bandwidth demands have increased by a factor of 1,000 times while energy per bit has declined by only 100 times.
These are staggering statistics, and AI is poised to place further pressure on transport efficiency and performance given the rise of generative AI and the management of the associated large language models—with their corresponding data load. Of paramount concern is energy consumption, as most organizations are focused on aggressive net-zero and sustainability goals. Historically, networking has constituted roughly 5% of total datacenter power consumption. With ever-increasing workload demands, this could more than double unless there are significant architectural changes.
To address these challenges, there are three optical networking trends that Moor Insights & Strategy believes that Marvell can capitalize on in the coming quarters with its underlying silicon designs. These apply to three areas: datacenter interconnects, rack-to-rack connections, and inside-rack connectivity.
Datacenter interconnects
For datacenter-to-datacenter interconnects, the amount of data and the requirements for throughput make optical networking a no-brainer. Long-distance optical connections have historically been served by silicon and infrastructure solutions from companies such as Ciena. However, since 2015, most cloud service providers have chosen pluggable modules based on the ZR and ZR+ formats; these are powered by coherent digital signal processors (DSPs), a product category originally invented for undersea fiber optic cables. The advantages of this approach are compelling, because it reduces capex up to 75% thanks to a smaller footprint while also cutting opex costs for power and space. The availability of merchant coherent DSPs also helped cultivate an ecosystem of module manufacturers that expanded choice for customers.
Both 100 Gbps and 400 Gbps pluggable modules have been broadly adopted by cloud providers and telecommunication providers—so much so that, in terms of units, modules are expected to soon surpass traditional equipment. In September 2023, Marvell introduced Orion, the first coherent DSP to support the ZR/ZR+ format, as well as COLORZ 800, a module for datacenter interconnects. This effectively extends the range of module-based connections from ~120km to ~500km at full bandwidth and nearly 1,200km at 400 Gbps. Orion and COLORZ, along with Marvell’s recently announced plans for 1.6T, could position the company for future infrastructure design-in success given Orion’s higher capacity and longer-term investment protection.
Rack to rack
Increasing data density and applications that span disparate racks drove the need for optical interconnects. Databases that extend outside a specific rack are a good example, as is high-performance computing. In this context, Marvell introduced the Nova PAM4 DSP electro-optics platform and the Teralynx 10 Ethernet switch in 2023. Like coherent pluggable modules, PAM4 DSPs and modules used inside datacenters have seen rapid adoption inside clouds, with bandwidth doubling every two to three years. Most connections over five meters today are made with optical modules.
In its latest iteration, Nova supports 1.6T performance. One of the big debates in this sector is the use of linear-drive pluggable optics (LPO) vs. digital signal processor (DSP) modules. The difference: LPO modules don’t employ a DSP, so they reduce power consumption by 20% to 50%, depending on whom you ask. On the other hand, the lack of a DSP means that connections must be engineered to avoid performance degradation. Even LPO advocates say the technology might be best suited for short reach (5m or less) connections or when a single OEM controls equipment on both ends of the connection. LPO also lags in performance. Most LPO modules—and the only place to see these modules is at trade show demos—top out at 800 Gpbs.
The specifics of this are important enough for us to dig into the technical aspects for a moment. History says that customers will opt for speed of deployment and bandwidth. With the rapid pace of AI adoption, the familiarity of PAM4 modules is appealing. Another advantage: competition inside the module ecosystem leads to steady improvements. Silicon photonics, a process for making optical components with CMOS, will migrate from coherents to PAM4s to cut power and costs. Process manufacturing advances and the use of 2.5-D and 3zD chiplet packaging—all technologies under examination for modules—will lower cost and power consumption even further. Module energy could drop from 14 picojoules/bit to 10pj/b and below.
As a result, the potential energy savings from LPO get minimized. An easy rule to follow is that modules are half of networking power, and current DSPs are half of module power. So, if networking is 5% of total power consumption in a datacenter, current modules represent 2.5%. That means LPO might lower total module power down to 1.5% or 1.25%. But a traditional module incorporating SiPho and 3-D packaging might also be able to bring it down into the 2.0% to 1.7% range—within a percentage point of LPO even without any extra engineering tricks.
Inside rack
Over time, the data throughput and power efficiency challenge shifted from datacenter-to-datacenter connections to rack-to-rack connections and eventually surfaced inside the individual rack. Copper had been doing a fine job interconnecting compute, but memory and JBOD storage connections needed more optical interconnects to deal with AI and HPC applications.
To replace legacy copper cabling within the rack, Marvell is producing DSPs optimized for cloud workloads. The DSP essentially turbocharges the cable so more data can be sent for longer distances. The Perseus portfolio represents an industry first by Marvell in this regard. It is a compelling concept, given that Perseus has the potential to support higher data throughput within a lower power envelope relative to traditional copper.
Final thoughts
Advances in optical networks are needed, especially given the momentum of AI workload adoption. The underlying silicon that is integrated into datacenter networking infrastructure will play a critical role in improving performance and reducing power consumption. Datacenter interconnects, rack-to-rack, and inside-rack connectivity are three areas of opportunity for improvement. Moor Insights & Strategy believes that Marvell is well positioned to capitalize, given its silicon portfolio strength and high market share position in both coherent modules and PAM4 DSPs.
