Mythic claims 400 TOPS with 1.28B parameters on 16-chip PCIe Card
Please welcome new Cambrian-AI Analyst Gary Fritz, who contributed to this article.
Artificial Intelligence applications are starting to show up in everything from cell phones to supertankers. But at the edge, they are running into the same roadblocks that traditional applications have fought for years: they need more speed. More capacity. Lower power. What’s a burgeoning neural net to do? To make matters worse, machine learning models are growing at an exponential rate, doubling in size every 3.5 months. In these models, traditional digital electronics may struggle to deliver the necessary performance at low power with adequate memory resources, especially for large models running at the edge.
Enter Mythic
Consequently, some cutting-edge chip developers are pursuing an entirely new approach: in-memory analog computation. Mythic, based in Redwood City, CA and in Austin, TX, is the first to market with such a device. The company has just announced its new M1076 Analog Matrix Processor (Mythic AMP), a single-chip analog computation device that can deliver impressive compute resources at very low power. Each AMP chip can pump out up to 35 TOPS (trillions of operations per second) with a 3W power profile. At ten TOPS per watt, that would put Mythic in the lead of the growing cadre of edge AI platforms. The company will initially focus on edge AI deployments where power and performance are critical, and large or multiple models are required.
Of course, as always, we anxiously await real application benchmarks to validate the company’s claims. Mike Henry, the company’s founder and CEO, says he intends to submit results to MLPerf, which is very good news indeed. (Our interview with Mike can be found here.)
The M1076 AMP scales from single-chip edge endpoints to datacenter applications, and is available in a compact PCIe M.2 A+E Key card for space-constrained embedded edge AI applications. For more demanding applications that require many streams, multiple large, deep neural networks, and higher resolutions and frame rates, 16 AMPs can be combined onto a single PCIe card. Mr. Henry indicated that a single chip can also support multiple simultaneous models. The 15-chip card delivers up to 400 TOPS and 1.28 billion neural net weights – and draws only 75W of power.
As for software, Mythic says over 60% of the company’s engineers have been preparing a full stack of development and optimization tools, and is. Initially, the company will focus on image processing, where many of the early evaluations are now taking place. The company is confident it can begin to establish and nurture an ecosystem for this new class of computing, but we all know that AI software can be much tougher than startups realize.
But What is Analog Computing?
Analog computers use variable-range voltages to represent desired values. You could compare them to a stereo amp; the “inputs” are the music source, the volume knob, and so on, while the “output of the calculation” is the volume that comes out of the speakers.
In-memory computation uses flash memory cells as analog circuitry. By tuning the properties of the individual memory cells, you can construct a circuit that represents the desired calculation. An array of memory cells can do massively parallel matrix operations – exactly what you need to drive a neural net. Since the calculation happens directly in memory, there is no performance penalty to move the data back and forth. And flash memory is very power-efficient, so such a compute device sips power.
Conclusions
While the big companies like IBM and Intel have been researching analog computation for many years, Mythic has been able to solve some of the thorniest barriers, including extending analog compute from two bits to four or even eight with stable results, and get a production platform ready for market. The amount of on-chip memory available on the M1076 AMP could be a game changer, especially for applications that need multiple neural networks to run concurrently, such as in advanced surveillance cameras.
Having recently raised $75M in additional venture capital, Mythic certainly bears watching, and so does analog computing in general.