IBM Research Says Analog AI Will Be 100X More Efficient. Yes, 100X.

by | Sep 23, 2021 | In the News

IBM AI Hardware Research Center has delivered significant digital AI logic, and now turns their attention to solving AI problems in an entirely new way.

The IBM AI Hardware Research Center is located in the TJ Watson Center near Yorktown Heights, New York. IBM

Gary Fritz, Cambrian-AI Research Analyst, contributed to this article.

AI is showing up in nearly every aspect of business. Larger and more complex Deep Neural Nets (DNNs) keep delivering ever-more-remarkable results. The challenge, as always, is power and performance.

NVIDIA has been the leader to beat for years in the data center, with Qualcomm and Apple leading the way in mobile. NVIDIA got an early start when they realized their multi-core graphics cards were a perfect match for the massive amounts of calculations required to train and execute DNNs. NVIDIA’s tech has spurred huge growth in the sector; NVIDIA chalked up just over $2B last quarter in data center revenue, and AI accounts for a large (although unknown) portion of that high-margin treasure trove.

Here comes Analog Computing

It’s tough to beat NVIDIA at their own game, so several vendors are taking a different approach. Mythic, a Silicon Valley startup, has already released their first analog computation engine, and IBM Research is investing in an analog computation roadmap. But before we dive further into the deep end of the pool, just what is analog computation?

Traditional computers use digital storage and digital math. Data values are stored as binary representations. The typical computer architecture has a compute section (one or more CPUs or GPUs) and a memory bank. The CPU shuffles data into the CPU/GPU for calculation, then shuffles the results back out to memory. This constant data motion greatly increases the performance overhead and energy cost of the operation.

Analog computation is an entirely different approach. Numeric values are represented by continuously variable circuit values (voltage levels, charge level, or other mechanisms) in analog memory cells. Analog calculations are handled by the analog circuitry in the memory array. Each cell is “programmed” with analog circuitry, and the resulting analog value represents the desired answer. Calculations are performed directly in the memory cell, so there is no need to move data back and forth to a CPU. The massively parallel calculations possible with this approach are a perfect match for the enormous matrix calculations required to train or execute a DNN.

IBM Research Sees Analog as the Next Big Thing in AI

IBM’s analog implementation uses memristive technology. Memristors are the fourth fundamental circuit component type in addition to resistors, capacitors, and inductors. IBM uses memristive Phase-Change Memory (PCM) or Resistive Memory (ReRAM) to store analog DNN synaptic weights. Circuits are built on the chip to do the desired calculations with the analog values. This includes forward propagation for DNN inference, and additional backward propagation for weight updates for training.

IBM plans to integrate analog compute engines alongside traditional digital calculations. An analog in-memory calculation engine could handle the large-scale DNN calculations, working in partnership with traditional CPU models.

IBM Research typically delivers technology through two channels. The first, of course, is to license the Intellectual Property to tech companies. The second is to turn their inventions into innovations in their own products. From a hardware standpoint, we could envision IBM building multi-chip modules that attach one or more Analog AI accelerators to systems, possibly using IBM’s future DBHi, or Direct Bonded Heterogeneous Integration, to interconnect the accelerator to a CPU. Also note that IBM recently announced on-die digital AI accelerators as part of the next generation Z system’s Telum processor. The reduced-precision arithmetic core was derived from the technology developed by the IBM Research AI Hardware Center.

Conclusions

This movie isn’t over yet. There remains significant work to do, and a lot of invention especially if IBM wants to train neural networks in analog. But IBM must feel fairly confident in their prospects to start writing blogs about the technology’s prospects. Data Center power efficiency is becoming a really big deal, with some projections forecasting a 5X increase in 10 years, to 10% of worldwide power consumption. We cannot afford that, and analog could make a huge dent in reducing that.

Significant challenges remain, but analog technology has terrific potential. For more information, see our Research Note here.