Qualcomm not only bested the entire field for power efficiency, a 16-card server was the fastest tested.
Qualcomm, perhaps best known for its leadership Snapdragon mobile platform, has further enhanced its AI bona fides with V1.1 MLPerf inference benchmark suite released in September. Not only did the company run a clean sweep against all other platforms in terms of power efficiency (e.g., images/second/watt), the Cloud AI100 won the best overall performance rank for object detection. Let’s take a closer look.
Specifically, Qualcomm was able to deliver the highest data center inference performance for image processing, the highest performance per watt across the board and the fastest edge performance. Realizing up to 34% improvement in software performance since the last MLPerf V1.0 release, Qualcomm has now vaulted from a niche player to a broad provider of AI acceleration platforms. Qualcomm submitted 82 benchmarks results including 36 power results and expanded coverage including language processing (BERT).
The Qualcomm Cloud AI 100 for Edge and Edge Clouds
Qualcomm submitted results for online and offline runs of image processing with ResNet50, SSD Large and Small as well as BERT Large. Qualcomm delivered the highest performance efficiency of all entrants in all models and platforms. In addition, a Gigabyte server with 16 Cloud AI100 cards delivered the highest inference performance of all submissions, beating an Inspur and NVIDIA DGX server with 8 A100 GPUs each.
The Cloud AI 100 consumes only 15-75 watts, compared to 300-500 watts of power consumed by each GPU. So, on a chip-to-chip basis, the Qualcomm AI 100 delivers 50% of the performance at only 15% of the power. Compared to a Dell server with Nvidia A100 GPU, the Qualcomm advantage rises to 2.6 X better performance per watt at a complete system level, not just at a chip level.
Leadership Edge Performance, Latency and Efficiency
While power efficiency in the data center is important, it is vitally so on the edge and edge server configurations, and here the Qualcomm technology has no peer. Looking at the AEDK DM.2e development platform (using a 15-watt version of the Cloud AI100), computing applications realize a full four times better power efficiency for edge devices, and twice the efficiency in edge cloud server applications tested.
Qualcomm is also delivering the lowest latency at the lowest power, critical for many computer vision applications such as autonomous guidance, on-premise safety and security applications that demand the lowest response time to be able to react to rapidly changing scenes.
While power efficiency in the data center is important, it is vitally so on the edge and edge server configurations, and here the Qualcomm technology has no peer. Looking at the AI Edge Development Kit (AEDK) DM.2e development platform (using a 15-watt version of the Cloud AI 100 chip), computing applications realize a full four times better power efficiency for edge devices, and twice the efficiency in edge cloud server applications tested.
Qualcomm is also delivering the lowest latency at the lowest power, critical for many computer vision applications such as autonomous guidance, on-premise safety and security applications that demand the lowest response time to be able to react to rapidly changing scenes.
Conclusions
With industry leading advancements in performance density and performance per watt, the Qualcomm Cloud AI 100 platforms are leading in all the latest benchmarks. This is a major shift from a world dominated by a single vendor, and the addition of power metrics, measured at the wall, is a great step forward for clients wanting to understand TCO and those who have taken a Carbon Pledge. Qualcomm has significantly expanded its submission to MLPerf benchmarks, doubling the number of platform submissions from edge to cloud and expanding coverage to include Language processing (BERT) and SSD MobileNet to Vision networks. We look forward to seeing additional submissions in the future, as well as any early indications of Qualcomm’s plans for future Cloud AI platforms.
When it comes to mobile and edge AI, we have no doubt that Qualcomm is in it to win it.