Nvidia Promises The Pervasiveness Of AI And A New Data Center Architecture

Just after coming off two major industry announcements - the introduction of the highest performing consumer graphic cards and the proposed acquisition of Arm - Nvidia launched its largest virtual Graphics Technology Conference (GTC) on October 5, 2020. Unlike the spring 2020 GTC, which was changed to a virtual conference at the last moment due to COVID-19, this one resembled a more traditional GTC with the kickoff by CEO Jenson Huang revealing a flurry of product and technology announcements. Dressed in his trademark leather jacket and standing in the middle of his kitchen (again), Mr. Huang provided a glimpse into Nvidia’s new solutions for the data center, edge AI, healthcare, and a new suite of collaborative tools.

There were two key areas of focus for this GTC - the growing pervasiveness of AI and a proposed change to the data center architecture. The latter leads to the announcement of a new platform to offload the I/O and security functions in data center servers. Nvidia has developed a new SoC dubbed the Data Processing Unit (DPU) to offload the data management and security functions, which have increasingly become software functions, from the main server CPU to an intelligent Network Interface Card (NIC). The SoC sits on a half-length PCIe card (the Bluefield-2) and is paired with a software platform called DOCA to enable what Nvidia calls the “Data Center Infrastructure-on-a-chip.” The combination delivers a programmable data center platform for the software defined data center.

These types of offload engines are nothing new and have been tried before with limited success. However, they often lacked the software and many of the functions were still driven by hardware at the time. With many functions having already transitioned to software and Nvidia’s continued success in the data center with both GPUs and Mellanox NICs, Nvidia may have success where others have failed. Nvidia demonstrated how offloading these functions reduces the demand and operating costs of the host CPUs. Combine this with the shift toward the use of accelerators like GPUs for workload processing and you drastically reduce the performance requirements and/or the number of CPUs required in future servers. If you go a step further and combine this with the increased core count, I/O, and memory bandwidth available on some of the latest CPUs like AMD’s EPYC processors, and the server architecture begins to look drastically different than the CPU-centric platforms in use today. 

Nvidia even has a key platform partner for the new intelligent NICs in VMware, which is supporting the concept with its software. Nvidia went so far as to offer a roadmap for three generations of BlueField chips. And even before the next generation is available, Nvidia will also be offering a version dubbed the BlueField-2X, a full-length PCIe card that has the BlueField-2 DPU paired with an Ampere GPU for what Nvidia calls “Bluefield-4 performance.”

Other than the Bluefield announcements, the heart of Huang’s presentation was focused on what Nvidia has done and is continuing to do to enable the pervasiveness of AI. For the data center, Nvidia announced the open beta availability of Omniverse, Jarvis, and Merlin. In addition, Nvidia announced two new AI application frameworks - Maxine and Clara Discovery. While the first three were announced earlier, it’s worth a quick recap. Omniverse is a cloud-based collaborative environment that allows designers/creators to collaborate on the same project using different tools like Creative Suite and Maya simultaneously while doing real-time rendering. Jarvis is a Natural Language Processing (NLP) personal assistant platform that can respond not only with human sounding response within a few hundred milliseconds, but also with natural facial expressions. And Merlin is a framework that handles the complex task of modeling information about products and services with the models of individual users. 

Maxine provides a new AI framework to improve the video conferencing experience through a number of methods, such as modifying the focus of users to provide a more person-to-person interaction, providing active noise cancellation, real-time translation and subtitles, and even an avatar with natural facial expressions similar to Jarvis. The Clara Discovery framework is designed around the continued investment in the research and development of new drugs. Maxine and Clara Discover join a growing list of Nvidia applications frameworks. 

Nvidia also announced the investment in a new DGX SuperPod deployment in the UK. The new supercomputer will be dubbed Cambridge-1 and will be located somewhere in or around London. At 400 Petaflops, the new Cambridge-1 supercomputer will be the highest performance supercomputer in the UK, the 3rd highest performance in the Green 500, and in the top 30 of all supercomputers in the world. Cambridge-1 represents a continued investment with partners and its commitment to invest in the UK as part of the company’s proposed acquisition of Arm. Nvidia also announced a number of data center-related partnerships, including one with GSK for the development of a drug research laboratory, VMware for the support of AI in the VMware AI Cloud, with Cloudera to accelerate data engineering, with Microsoft for AI support in Microsoft Office on Azure, and with American Express for fraud detection. The company also announced that NGC, Nvidia’s cloud for containerized stack, will be available in the Microsoft Azure, Amazon AWS, and Google GCP marketplaces.

For edge computing, Nvidia announced the EGX - a complete AI platform for the edge of the network. The EGX combines a Bluefield-2 DPU with an Ampere GPU, a similar configuration as the BlueField-2X, in a PCIe module form factor with system software, AI frameworks, and connectivity through Nvidia’s cloud-based Fleet Command platform. The EGX can operate in any standard server allowing anyone to deploy an AI platform at the edge.

Nvidia also announced the latest addition to the company’s Jetson platform. The Nano 2GB is a cost optimized version of the Nano that was released earlier this year. The Nano 2GB uses the same Jetson SoC as the Nano but with fewer I/O ports and half the memory making it more accessible for education at just $59 . The Jetson platform has become extremely popular for both education and the development of a wide variety of applications like robotics. So far, over 3,000 companies have ordered production Jetson products for use in commercial applications. 

Nvidia also continues its investment in other industrial applications like automotive through its Drive platform and Isaac for robotics. Mercedes has committed to using the Drive platform in all its vehicles beginning in 2024 and BMW is using the Isaacs platform for materials processing. Additionally, Nvidia committed to accelerating all Arm platforms from the edge to the cloud.

This GTC is considerably different from the original GTC that focused heavily on gaming, at which I presented. However, they all feature one thing missing from many of the technology conferences today - a vision. 20 years ago, leaders like Bill Gates and Craig Barrett would keynote the conferences with a vision of the future. While most of those predictions did not come to fruition, at least not in the timeframe they predicted, it provided aspirations for the industry. Mr. Huang hails from that era and continues the trend with one slight difference, he not only provides a vision for how pervasive AI will be in society, he provides incremental proof points with each GTC, stepping stones to that goal of an automated society where “everything that moves will be automated” and “even machines will be writing software.” When you see the continued efforts by Nvidia and its partners to enable AI technology in every form of electronic platform, the predictions don’t seem farfetched. Several years ago, Tirias Research predicted that by the end of 2025, almost every new electronic device would be using some form of AI either on the device, in the cloud, or in some hybrid fashion - a prediction that is proving accurate. The pace of technological innovation is accelerating, and it will have dramatic implications for society