While advances in AI accelerators have improved silicon performance by over a thousand-fold, it’s software that turns bits of silicon into useful capabilities for consumers and businesses. After all, none of us even think about the underlying chip when we take a low-light photo on our phone, play a VR game, or ask our digital assistant to set a reminder. The underlying AI that enables these functions is hidden by software developed by thousands of engineers around the globe. These application ninjas need software tools that are closely linked to the silicon to be productive and be able to produce performant code. The ultimate goal is for AI engineers to develop, optimize, deploy and accelerate their models on a specific hardware. The developers also want multiple layers of access within a software stack that works with their own design workflow, and to write this code once and run it on any hardware, regardless of the presence or absence of specific acceleration features.
Qualcomm Technologies, Inc. (QTI) has recently redesigned and enhanced its extensive suite of AI software to help these developers gain multiple access points across software layers and move their code more freely across the company’s products and tiers. The company’s new AI software stack is also powering domain-specific software development kits (SDKs) that extend capabilities across a broad range of QTI hardware platforms. It can also be used as a standalone development kit if you just want to, for example, develop on the Snapdragon mobile platform.
What is the QTI AI Stack?
QTI’s goal in creating this stack was to unify and simplify their array of software tools for OEMs and Developers to create, optimize and deploy AI applications on Qualcomm Technologies’ products while fully leveraging the Qualcomm® AI Engine’s performance and efficiency.
Qualcomm AI Stack supports a range of popular different AI frameworks including TensorFlow, PyTorch, and runtimes including TFLite and ONNXRT. The AI Stack is comprised of underlying developer libraries and services, system software, tools, and compilers so that any AI feature developed for one device can easily be deployed on others. The existing Qualcomm® Neural Processing SDK, the popular Qualcomm® AI Model Efficiency Toolkit (AIMET) Pro, the AIMET Model Zoo, model analyzers, and Neural Architecture Search (NAS) have all been included. In addition, Qualcomm recently ported the Qualcomm Neural Processing SDK to Microsoft Windows, while the Qualcomm AI Engine direct, an advanced AI runtime, has been extended across all Qualcomm Technologies’ products including the Qualcomm® Cloud AI 100 inference processor. This latter enhancement helps developers fulfill the write-once-run-anywhere goal QTI has set, enabling developers to deploy existing models directly to the AI accelerators on Qualcomm Technologies’ platforms.
On top of Qualcomm AI Software Stack sit three domain-specific SDKs for autonomous vehicles (Snapdragon Ride™ SDK), Intelligent Multimedia SDK for robotics and IoT and virtual reality (Snapdragon Spaces™ SDK). Once again, building these SDK’s on a common foundation helps developers support the entire portfolio of QTI hardware implementations in diverse segments, including the Qualcomm Cloud AI 100.
Qualcomm AI Stack unifies different AI frameworks and popular runtimes including TensorFlow, PyTorch and ONNX. These frameworks are built upon the underlying developer libraries and services, system software, tools, and compilers so that any AI feature developed for one device can easily be deployed on others. The existing Qualcomm Neural Processing SDK, the popular AI Model Efficiency Toolkit (AIMET), the AIMET Model Zoo, model analyzers, and Neural Architecture Search (NAS) have all been included. In addition, Qualcomm has recently ported the Neural Processing SDK to Microsoft Windows, while the AI Engine Direct has been extended across all Qualcomm products including the Cloud AI 100 inference processor. This latter enhancement helps developers fulfill the write-once-run-anywhere goal QTI has set enabling developers to deploy existing models directly to the AI accelerators on Qualcomm Technologies platforms.
On top of the Software Stack sits three domain-specific SDKs for autonomous vehicles (Snapdragon Ride), Intelligent Multimedia, and virtual reality (Snapdragon Spaces). Once again, building these SDK’s on a common foundation helps developers support the entire portfolio of QTI hardware implementations in diverse segments, including the Cloud AI 100.
Conclusions
Unifying Qualcomm Technologies’ extensive array of AI software will enable developers to build apps more easily using the same set of tools regardless of the target deployment platform, including mobile, automotive, XR, compute, IoT, and cloud platforms. As QTI expands its reach into the connected intelligent edge, the criticality of a unified software stack has only increased, and the company has now met that challenge head-on with Qualcomm AI Stack and will have the ability to bring more AI development tools and more domain specific SDKs in the future.