When the only company you can beat is yourself, what do you do? After all, you are the leader, right? And nobody else is even suited up and on the court. Some companies might rest on their laurels, saving money and slowing down R&D. NVIDIA is not one of those companies.
Two years ago, the industry was set to begin a “Cambrian Explosion” of new silicon to accelerate AI. Indeed, several companies launched impressive new platforms, including Cerebras, Graphcore, Google and Amazon AWS. However, none have been able to rival NVIDIA in per-chip performance for data center AI. Consequently, NVIDIA enjoys a massive lead in the critical ecosystem of AI solutions. For example, NVIDIA hardware was used by seven out of ten finalists for this year’s Gordon Bell awards (essentially the Emmys for high tech breakthroughs).
This week NVIDIA announced three new products in markets where it is already the undisputed leader: GPUs for AI, Workstations for AI development workgroups and ultra-fast networking with 400G Infiniband. To put it bluntly, I do not see any semiconductor company executing as well as NVIDIA across the data center.
What did NVIDIA announce?
Six months after NVIDIA replaced the industry-leading V100 with the industry-leading A100, the company has already doubled (or by some measures, even tripled) its GPU performance for AI and doubled the performance of Mellanox Infiniband. Today, NVIDIA announced a new variant of the A100 GPU, which doubles the amount of High Bandwidth Memory to 80GB using HBM2e, while increasing bandwidth to 2 TB/Second. The largest beneficiary will be recommendation engines, which have an insatiable appetite for fast memory.
The new A100 80GB platform is already shipping to customers, including the University of Florida’s previously announced Cineca AI Exaflop system, which, at ten exaflops, will be the fastest AI supercomputer in the world. It will also power the upcoming NVIDIA-owned Cambridge-1 Supercomputer announced last month. As usual, NVIDIA did not disclose pricing, but the A100 80 GB will definitely command a premium; nothing comes even close.
Figure 1: The new A100 80GB platform comes on an SXM4 package. It is only available in the DGX andimage: NVIDIA
Second, NVIDIA announced the availability of end-to-end Infiniband 400G, again doubling the performance of the leading networking solution for performance and latency-sensitive applications in finance and HPC. After all, if server performance keeps doubling or tripling, the network must keep up to feed the computation engines.
Figure 2: The new 400G Infiniband is available across the spectrum, including NICs, DPUs, Switches,image: NVIDIA
Lastly, NVIDIA announced the world’s most powerful desk-side workstation for AI and HPC workgroups, the DGX Station A100. It features fully integrated liquid cooling, allowing it to plug into any office environment and silently power development groups with a supercomputer’s performance. I left a note to Santa, but I think this is beyond his budget for me. The platform includes 4 of the new A100 GPUs, 3rd Generation NVLink, a 64-core AMD EPYC CPU and 7 TB of NVME storage. This platform demonstrates NVIDIA’s strategic commitment to become a full-stack supplier to demanding data center customers, complementing the HGX platform for cloud providers and DGX for short pod installations.
Figure 3: The new NVIDIA DGX STatrion A100 for AI and HPC workgroups supports four A100 80GB GPUs. image: NVIDIA
Today, just as AMD announced its entrant to data center GPUs, NVIDIA shifted the goalposts for AI performance yet again. While NVIDIA blows everyone out of the water on AI speed, I believe the new AMD platform, which I covered here, will find a home in more price-sensitive supercomputers who don’t need bleeding-edge AI. NVIDIA’s new GPU can interconnect via the new Mellanox 400G networking platform and be configured by channel partners using the HGX board, the DGX server or the new DGX Station.
NVIDIA just keeps raising the bar, and I don’t see any other companies lined up to clear it at this time.
Disclosure: My firm, Cambrian AI Research, like all research and analyst firms, provides or has provided research, analysis, advising, and/or consulting to many high-tech companies in the industry, including AMD and NVIDIA. I do not hold any equity positions with any companies cited in this column.