Broadcom Routs Nvidia InfiniBand From AI Backend Networks: Implications
It Is Time To Reset Nvidia NVL72 Shipment Expectations
As predicted nearly two years ago, Broadcom’s Tomahawk 5 is starting to replace Nvidia (NVDA) InfiniBand technology at the back end of AI networks. Despite much sell side research to the contrary, the contest is not even close. Other than a single exception, Ethernet now appears to have routed InfiniBand in all major hyperscaler designs. You can get some evidence of this from Broadcom (AVGO) public comments. CEO Hock Tan said in the most recent earnings call:
“In Broadcom, we have the deepest and broadest understanding of what it takes for complex, large workloads to be scaled out in an AI fabric. Proof in point, seven of the largest eight AI clusters in deployment today use Broadcom Ethernet solutions. Next year, we expect all mega-scale GPU deployments to be on Ethernet. We expect the strength in AI to continue. And because of that, we now expect networking revenue to grow 40% year-on-year compared to our prior guidance of over 35% growth.”
Arista Networks (ANET) CEO Jayashree Ullal has been saying something similar of late. In the most recent earnings call she said:
“If you recall, in February, I shared with you that we are progressing well in four major AI Ethernet clusters that we won versus InfiniBand recently. In all four cases, we are now migrating from trials to pilots, connecting thousands of GPUs this year. And we expect production in the range of 10K to 100K GPUs in 2025. Ethernet at scale is becoming the de facto network and premier choice for scale-out AI training workloads. “
What is important to note is that not only Nvidia is losing out on InfiniBand sockets but in most cases, it is not even winning with its Spectrum-X Ethernet offerings. There are several reasons for this. Some of the key ones are:
Contrary to Nvidia claims, there is no evidence that InfiniBand is a superior network. Data from Meta (META) and Broadcom shows that opposite and suggests that Ethernet is not only cheaper at a system level but has about a 10% edge across a wide range of measurements. Nvidia has yet to counter Meta and Broadcom claims.
Customers do not want to pay the high premiums for InfiniBand offerings – more so since the technology has fallen behind Ethernet in offering good TCO.
Customers do not want to be single sourced to Nvidia offerings especially given Nvidia’s ruthless MO and tendency to compete with its customers and partners.
Going to Ethernet means customers can pick and choose networking and connectivity hardware suppliers that best fit their strategic and tactical needs. This improves the TCO while also being in-line with customer strategies.
And, finally, to the extent Nvidia offers Ethernet solutions, these fall short of Broadcom’s alternatives. Also, Nvidia’s long history of trying to create a customer lock with proprietary features does not sit well with the hyperscalers.
Given these compelling reasons, Nvidia is losing InfiniBand sockets and not gaining meaningful traction on the Ethernet side, thus effectively reducing Nvidia’s role in the networking business. With the leverage from networking gone, Nvidia is also finding it difficult to push its proprietary NVL72 rack system which was the centerpiece of Nvidia’s systems pitch at Blackwell launch. Once you choose to stay away from Nvidia for networking, there is less of a reason to choose Nvidia’s proprietary NVL72 rack. The Information has reported that Microsoft (MSFT) has refused to use Nvidia’s proprietary rack even when Nvidia pressured Microsoft saying that early Blackwell shipments will be only available to customers who buy the entire NVL72 system. The Information has also reported that Google (GOOG) (GOOGL) and Amazon (AMZN) AWS have refused to use Nvidia’s rack. From the open-source history of Meta (META), we can be assured that even Meta will not be using Nvidia’s racks. That means Nvidia is not batting 0 out of 4 at the top 4 hyperscalers with NVL72.
Keep reading with a 7-day free trial
Subscribe to Beyond The Hype - Looking Past Management & Wall Street Hype to keep reading this post and get 7 days of free access to the full post archives.