A few months ago, I published a blog that highlighted Qualcomm’s plans to enter the data center market with the Cloud AI100 chip sometime next year. While preparing the blog, our founder and principal analyst, Patrick Moorhead, called to point out that Qualcomm , not NVIDIA , probably has the largest market share in AI chip volume thanks to its leadership in devices for smartphones. Turns out, we were both right; it just depends on what you are counting. In the mobile and embedded space, Qualcomm powers hundreds of consumer and embedded devices running AI; it has shipped well over one billion Snapdragons and counting, all which support some level of AI today. In the data center, however, NVIDIA likely has well over 90% share of the market for training. Meanwhile, Intel rightly claims the lion’s share of the chips for inference processing in the world’s largest data centers. I’ve written extensively about NVIDIA and Intel, so let’s take a look into Qualcomm Technology Inc. (QTI).
The path to distributed intelligence
Smartphones today are the pervasive interface for nearly 3 billion people to communicate, take photos and videos, and access personal data and applications. You knew that, but you may be unaware of how much AI those phones are processing. If you have ever taken a photo using an Android phone, you have probably used AI from QTI.
Qualcomm intends to use its leadership in power-efficient mobile processing to expand beyond the handset to create “Distributed Intelligence,” where AI processing can be performed as close to the user as possible, offloading to the cloud when needed. So, the processing or pre-processing can occur on the device, in the cloud edge and/or in the data center, depending on power, data and latency requirements. By interconnecting these three processing tiers via 5G networking, each tier can collaborate to improve understanding and deliver advanced functionality for the user.
The QTI AI engine overview
QTI takes a heterogeneous computing approach to deliver AI and application performance at low power and cost. It places its Hexagon Processor, Adreno GPU and Kryo CPU, as well as the modem, security processor and other logic, on the same die. The latest update to Hexagon includes a dedicated Tensor accelerator, akin to the TensorCores found on NVIDIA’s latest GPUs. A few AI-enabled applications on QTI-equipped mobile phones that make use of these cores include Dual and Single-camera Bokeh, Secure 3D Face Authentication, Scene Detection, Super Resolution and myriad of computational photography enhancements. Benchmarks published by Anandtech and PCMag verify that the Snapdragon 855, which can deliver over 7 trillion operations per second (TOPS), outperforms the Huawei Kirin and Samsung Exynos mobile processors.
Software: frameworks, tools and libraries
Of course, any chip needs software to be useful, and QTI built a robust stack for AI applications. This includes the popular neural network frameworks like PyTorch and TensorFlow, and also the frameworks developed by Microsoft , Amazon , Facebook and Baidu . QTI supports the Open Neural Network Exchange (ONNX), a common data format for importing neural networks, and runs them on the processors found in Snapdragon. This illustrates an important strategic thrust for QTI: the company wants to support practically every style of AI directly on Snapdragon and has the software to meet the needs of a diverse development community.
Use cases for AI in 5G
5G will utilize AI processing for network optimization and enable new AI-enabled applications, thanks to faster processors and the technology’s 1ms-or-less latency, higher bandwidth at 1 Gbps and massive connectivity. QTI believes the 5G future will both require and enable “distributed intelligence.”
AI application in 5G wireless transmission and management is in its infancy but will become essential in optimizing these networks. AI will assist in transitioning the management of wireless networks from a human-centric model to an automated model, improving signal quality and service levels. Specific applications that reside in base stations include the following:
- Predicting base-station switching handoffs to minimize quality degradation and dropped calls
- Planning and provisioning beams for frequently traveled paths to optimize signal strength and quality of service
- Enabling beamforming in massive MIMO arrays and millimeter-wave antennae to optimize transmission quality by identifying the most efficient data delivery route for a particular user (this provides a higher quality of service at lower power and bandwidth consumption)
Conclusions
QTI believes that in order for AI to realize its potential as a transformative technology, AI must become pervasive and collaborative across mobile, edge and cloud computing resources. The company embraces this strategy as “Distributed Intelligence” and recognizes that this will require new products to extend its technology footprint far beyond mobile and embedded devices. To accomplish this, the company must do the following:
- Continue to innovate in AI acceleration and software on mobile devices
- Expand that technology in edge device markets, including smart IoT and self-guided devices such as autonomous vehicles, drones, etc.
- Develop platforms for edge and cloud computing service providers to complement and extend the intelligence that requires more processing power than what is available on the device
Going forward, the company is investing in the technologies and market development programs that will help it expand further into intelligent edge cloud and data center environments. In fact, the stated goal for the AI Cloud 100 of reaching 350 TOPS may well put it at or near the lead position when it comes to market in 2020, assuming the company can translate that performance potential into real application benefits (real apps and the mlPerf benchmarks, for example).
Of course, QTI will face challenges in this market expansion into the data center and cloud edge—from established companies like Intel, NVIDIA and Xilinx, as well as from a vast array of startups. In fact, NVIDIA has been making significant progress in extending its lead in training to the inference market, as I noted in this blog last week. Moor Insights & Strategy believes that when QTI launches its cloud and next-generation 5G products, the company will emerge as a significant player in the AI computing revolution. We look forward to hearing more details of the company’s plans and products at the annual QTI Tech Summit in December.