Our relationship with computers will never look the same. Here are the winners and losers.
The web, the media, and my email are entirely full of ChatGPT missives and questions. I’ve spent hours with scores of investors, all wanting to understand the impact on semiconductors and cloud service providers. The hot chatbot’s impact will impact everything over the next five years. Surgery? Yep. Marketing? Absolutely! Knowledge workers like me? Especially!
But what about closer-in opportunities for investors? Who stands to benefit? Who is likely to be disrupted, and why? Let’s dive in.
ChatGPT: The Little Bot that Could…
… could pass the Wharton Business School MBA exam. …could pass the US Medical Licensing Exam. …could get an A+ on a high school essay. …could create totally non-sensical answers to trick questions. … could be abused. But here are a few crazy things it did it the first week of being live, thanks to Monica J. White:
Explain it to me slowly. Author and Midjourney
As Andrew Feldman, Founder and CEO of Cerebras, told me when I asked about ChatGPT results: “There are two camps out there. Those who are stunned that it isn’t garbage, and those who are stunned by how imperfect it can be.” Sometimes, we have to remember; ChatGPT has no knowledge, no context, no judgement. It can only rely the frequency of word combinations to generate what looks to be intelligent prose or poetry.
Who wins, who loses?
Clearly, the venture-backed group of companies building generative models, or using open source models to create applications, are well positioned here, including Cohere, Jasper, Stability, Bloom.ai, and others. Note that none of these companies were mentioned in the list that ChatGPT generated below. Nor was Microsoft!
This lapse shows a glaring issue with ChatGPT and the GPT3 model upon which it based: the is a long lag between training the network and using it. Frequent retraining will be required if Microsoft and Google plan to incorporate Large Language Models into search. This could be a nice knock-on effect for NVIDIA and other hardware vendors who provide the heavy lifting for training DNNs.
Here’s ChatGPT’s list of companies that are deeply involved in creating or using generative models. Note that this is based on 2-year-old data! ChatGPT
As for losers, much has been written about the potential impact on Search revenue at Google, however we would point out that nobody has more AI engineers and scientists than Google, so it would surprise us if they fail to productize LLMs. And Google search is very sticky; many of us just type a search into our browser url pane. That being said, we think Microsoft has a strong lead in figuring out how to embed LLMs into the company’s products and should have a time to market advantage.
An Absurd Amount of Compute
Let’s talk about hardware requirements and providers. Unlike the inference processing of, say convolutional neural networks used in image processing which can be done on a fast Intel CPU, Large Language Models demand a massive amount of processing with adequate memory capacity & bandwidth, far more than a single GPU can handle. From what we hear, it takes 8 NVIDIA A100 GPU’s to contain the model and answer a single query, at a current cost of something like a penny to OpenAI. At 1 million users, thats about $3M per month. As adoption continues to explode, it could take thousands of GPUs to meet the demand. The gold rush is clearly on; who will get the gold?
Clearly, NVIDIA is the leader today, having supplied the 10,000 GPU supercomputer to Microsoft to host for OpenAI. And the upcoming Grace/Hopper combination was designed specifically for large models, with shared memory connectivity, NVLink, and a Transformer Engine expected to provide a big boost, perhaps 6X, over the A100 used today.
But Intel and AMD want a piece of the action as well, and the upcoming Instinct MI300, which we covered here, could be a serious contender. Intel says Gaudi 3 is coming soon, and of course startups like Cerebras see this as a huge opportunity. While we do not yet know if the MI300 has anything like a Transformer Engine on board, we have heard that interest in the MI300 in OpenAI helped motivate the development of the “Triton” software, reducing or eliminating the dependency on NVIDIA CUDA for OpenAI, and speeding the adoption of new hardware. We also would note that Meta’s new AI Template, which also eases the porting of new models, already supports AMD Instinct MI200.
Conclusions
From our perspective, NVIDIA stands the most to gain in the next 2-3 years as ChatGPT , but we suspect AMD will get a solid shot at at least a partial design win. NVIDIA could see an incremental billions in revenue for its fast data center GPU’s and Grace-Hopper could get a nice tailwind in demand as it begins to ship later this year.
But let’s see how ChatGPT would finish this article. I asked ChatGPT to make it humorous; not bad!
ChatGPT, the world’s most advanced language model, is like a superhero that can generate text faster than a speeding bullet, more powerful than a locomotive and able to leap tall buildings in a single bound. But even a superhero needs a powerful sidekick and that’s where GPUs come in!
NVIDIA GPUs are like the trusty sidekicks that help ChatGPT to be the best version of itself. They help ChatGPT to train faster than a speeding flash, generate responses more powerful than a steam engine and process data taller than any building. But without them, ChatGPT would be like a superhero without a cape, just a regular Joe.
In short, if you want your ChatGPT to be the hero that saves the day and impresses everyone, you better make sure it’s got a powerful NVIDIA GPU as its trusty sidekick. Because let’s face it, who wants a language model that runs as slow as a snail and generates text as dull as dishwater? No one, that’s who.