The AI research group at Qualcomm Technologies has pioneered innovations from on-device learning to wireless AI, and has ambitious plans for the next set of “firsts”.
Whenever people take photos or speak to a digital assistant using a mobile phone, they often don’t realize that they just took advantage of Artificial Intelligence (AI). If they think of AI at all, it is typically in the context of Autonomous Vehicles or perhaps Facebook’s (Meta’s) massive data centers. While AI is becoming ubiquitous and distributed across edge devices and cloud servers, many challenges remain to realize the connected intelligent edge vision CEO Cristiano Amon has for AI to enable automated perception, reasoning, and action.
For AI to enable the levels of automation and personalization Qualcomm AI Research VP Jilei Hou believes that AI hardware and software must become much smaller, faster, more efficient, lower power, and able to learn continuously at the edge in the real world. This provides the perfect complement to remote processing in the cloud, whose reach has been further advanced through Qualcomm’s 5G technology. These are not just engineering challenges; progress and breakthroughs must come from fundamental scientific, applied, and platform research.
AI Firsts from Qualcomm AI Research
Qualcomm AI Research recently held a webinar where Jilei Hou, VP of engineering and the head of AI R&D at Qualcomm AI Research, outlined “firsts” in eight research areas that they are justifiably proud of. Their novel AI research and full-stack AI optimizations have pushed the AI industry forward and enabled first-ever proof-of-concept demonstrations on commercial mobile devices.
Their AI firsts include 8-bit model quantization leading to the best power-efficiency toolkit in the industry (the AI Model Efficiency Toolkit, or AIMET), on-device learning demonstrating a 30% improvement in keyword detection, federated learning with an end-to-end software framework, video semantic segmentation in real-time, group equivariant CNNs, AI for wireless with the first weakly supervised method for passive RF sensing, video super-resolution at 4K at 100 FPS on mobile, and neural video compression with the first real-time HD decoding on mobile. We interviewed Hou recently and learned a great deal about the challenges and solutions of mobile AI.
Artificial Intelligence at Qualcomm, like most everything else in the company, begins with mobile. And that heritage sets a high bar, since mobile devices are battery-powered, lightweight consumer devices with limited processing, memory, and I/O. The ongoing research underway at Qualcomm AI Research is centered around three primary domains: fundamental research, applied research, and platform research. The latter covers topics such as power efficiency and on-device learning, which are essential in mobile but have also benefited Qualcomm in other vertical markets like IOT, XR, and automotive. But efficiency doesn’t stop with hardware; Qualcomm AI Research is also advancing model design, compression, quantization, algorithms, and software tools to attack the problem through full-stack research.
Efficiency comes in part from using less data in the model and quantization. If one can run a model with 4-bit integer math, it will be up to 64 times more efficient than using 32-bit floating point. But all that efficiency cannot come at the cost of reduced accuracy. To that point, Qualcomm AI Research has demonstrated a state-of-the-art 8-bit transformer model with less than 1% accuracy degradation.
Today, some AI processing has to be offloaded from the mobile device to a cloud service. However, faster response times, improved privacy, better personalization, and improved understanding within the context of the request will require on-device learning.
Looking to the future
The researchers at Qualcomm AI Research are pursuing additional new “firsts” that could result in significant value and differentiation for customers and partners. Targeting improvements in AI efficiency on mobile devices and the rest of the connected intelligent edge can pave the way for improved personalization and automation of many tasks, enabling new use cases and improving the overall user experience.
Increased AI processing on edge devices is both inevitable and extremely valuable to improve user experience and device functionality. The challenges remain daunting, given the limited amount of processing and memory capacity on mobile platforms. Qualcomm Technologies Inc is rising to the challenge, and these 8 “AI Firsts” demonstrate that the organization is already making significant progress, with more innovations in sight.