Intel has a multi-prong AI strategy. The company announced the AI PC and Intel Xeon Gen5 with excellent AI performance and TCO.
Intel announced new products for desktops and servers, and the focus for both was AI, where Intel has a commanding lead over the competition. Intel’s presentation included over 40 slides out of 70 that mention AI.
The venue in New York City was undoubtedly chosen for Wall Street, and the stock is up ~10% since it opened on Thursday. Investors must like what they hear. First, Intel is on track with its “5 process nodes in 4 years” design and manufacturing strategy. Second, the AIPC chip will arrive just in time for Microsoft Copilot sales in 2024. Third, the newest Xeon cranks out 42% more AI performance than Sapphire Rapids, the previous generation; nobody comes close to Intel for AI performance on a CPU.
The AI PC
Last Fall, Intel announced the “AI PC.” The company launched the first Core chip tailored for AI at this event, the “Core Ultra.” While the name implies lower-end AI PCs will come in the future (Core Not-quite-ultra?), the new chip includes a dedicated multi-streaming Neural Processing Unit (NPU) in addition to a new compute tile with up to 8 cores, 2 of which are the power-efficient E-cores. Intel believes that 80% of all PCs shipped in 2028 will be some sort of AI PC. Qualcomm is also targeting this opportunity with its Snapdragon X Elite, available in mid-2024 laptops, as is AMD with its Ryzen AI. The noise you are hearing is probably the gathering benchmarking storm.
Intel says the multi-chiplet Ultra is the “largest architectural change in 40 years.” In addition to 64 new cores (far fewer than AMD or many Arm designs) and the NPU, it includes integrated Wi-FI 6E, CPU frequencies up to 5.1 GHz in Turbo mode and a large 320MB cache.
The 5th Generation Intel Xeon
There are two primary values for the 5th Gen Scalable Xeon: upgrading Xeons to save TCO for older workloads and higher AL performance for the new.
For upgrading last-decade Gen1 Xeon, Intel claims data centers can save up to 77% in purchase and operating costs, excluding software licensing, which can increase a lot with higher core counts. According to Intel, some 120 Gen1 Xeons (3 racks of 2U servers) can be replaced with a single Gen5 server based on Intel’s claim of a 94% reduction in server count.
Unfortunately for Intel, data center spend has shifted to GPUs for AI. Nvidia now garners some 68% of the entire data center spend on processors (yeah, I couldn’t believe the chart, so click it), so owners/operators have less capital to upgrade old equipment to save on operating costs. But if Gen 5 can save you 77% TCO AND get fast CPU AI performance, that’s something Intel can sell.
Intel has the best server CPU for AI, and they are capitalizing on it. Gen5 is the second generation of Xeon to implement AMX, introduced in Sapphire Rapids (4th gen) Xeon in January ‘23. AMX provides 4-5 times faster BF16/INT8 matrix multiplication to speed up workloads like AI. But like vector instructions, AVX, software developers must change their code to exploit it. For example, Microsoft has updated ONNX, the AI runtime exchange format, to support AMX and deliver better performance. It will take an ongoing effort to convince and incentivize app providers to use AMX, which Intel is making.
Conclusions
Since data centers are jumping on the AI moment, many will consider moving to the Gen5 Xeon as they add new capacity, often connected to accelerators. This momentum could entice software developers to adopt the AMX programming extension to deliver the best AI performance.
However, many will choose to wait for the next (or next next) generation. Sierra Forrest with e-cores will ship in the 1st half of next year, targeting the cloud-first design of many slower cores (144 cores per chiplet, 2 chiplets per die and per socket), but no AMX, multithreading, …) and Granite Rapids with P cores later in the year.
So, as we said in the beginning, investors are happy to see Intel stick to plan, and produce increasingly competitive products against AMD. But the battlefront is shifting to AI accelerators, where the Intel Gaudi3 is expected to challenge AMD and Nvidia next year. That is the battle that matters most.