The idea of chiplets is simple: develop the best semiconductors for the needed functions using the most proper manufacturing process. Then combine an assortment of chiplets on a multi-die package, and voila! A lower-cost approach to advanced semiconductors.
The concept also implies using pre-existing semiconductor components for standard functions and focusing new design efforts on the specific functionality needed for the target market or use case. The package can be less costly since design teams can use the most appropriate manufacturing technology for a particular function. For example, many I/O functions do not need, nor benefit from, advanced process nodes and can be manufactured in older, less expensive geometries.
This article examines two recently unveiled enabling technologies: a system chiplet from Cadence Design and Arm Holdings and new interconnect technologies from Intel that provide the glue needed to assemble the puzzle into a compelling solution. We will also examine the benefits of chiplet-based designs and some reasons why they may not make sense. For example, Nvidia does not use chiplets—at least not yet. We explore why that is the case.
Why chiplets?
Chip designers constantly fine-tune the performance/power/area (PPA) mix to meet the project’s business goals. Chiplets and 3D manufacturing apply a new paradigm to meet these needs, but they also add complexity that must be understood and managed, from design through manufacturing and packaging. Advocates tout the benefits of chiplet-based designs.
Monolithic silicon implementations require a team to design the entire logic on a single chip and tape it out to their manufacturer, like TSMC. Assuming this is a high-value chip, such as a chip to accelerate AI computations or power self-driving vehicles, the cost of an advanced high-performance manufacturing process node burdens functions that do not benefit from expensive manufacturing. With chiplets, an engineering team can focus on unique value-added functions while outsourcing or reusing other logic, improving time-to-market and costs.
Chiplets can enable the development of systems with reusable components combined with custom IP, allowing scalable solutions from low- to high-end. Chiplets also enable the building more extensive multi-chip solutions, using chiplet-to-chiplet communications technologies (UCIE) to extend beyond the die.
Source: CADENCE
Chiplets allow a design team to use or reuse functions for current and next-generation products. Chiplets also afford the development of a new ecosystem of chiplet providers that license their IP as a complete chiplet or sell chiplets as off-the-shelf products.
An arm-based “system chiplet”
Realizing the opportunity, Cadence Design recently taped out a system chiplet, co-designed with Arm Holdings, to provide a reference design with the Arm CPU and logic to manage the resources of a multi-chiplet SoC.
This design includes two UCIe controllers and PHYs to interconnect with other chiplets, a network-on-chip (NOC) for on-chiplet communications, a single LPDDR5/5X controller, and system control and management functions. The architecture is flexible; customers can add functions like I/O, the desired number of Arm cores and more to produce a custom system chiplet solution to meet their needs.
Source: CADENCE
Cadence is initially targeting the automotive market for this system chiplet, where OEMs are seeking to add brand-specific semiconductors for functions like infotainment and control. However, the chiplet and concept are appropriate for various SoCs, such as robotics and drones.
Why automotive? Cadence shared its view of the auto electronics market, which they see as exploding with innovations and reaching $386 billion in revenue by 2030 when there will be over 200 sensors on average in each vehicle. For automotive companies, using pre-existing and tested chiplets results in faster time-to-market.
Source: CADENCE
Below is an example for advanced driver assistance systems (ADAS), which envisions an SoC to support a family of automotive solutions. CPUs, GPUs, Neo AI, and Cadence Tensilica DSP chiplets are connected to a system chiplet to provide memory, control, and I/O subsystems. One can imagine how this approach can extend the lifespan of components, speed time-to-market and maximize component reuse with an extensible architecture for different solutions.
Source: CADENCE
Intel Foundry’s new chiplet-enabling technologies
Intel Foundry scientists will present seven papers at the 2024 IEEE International Electron Devices Meeting (IEDM) conference. One will be key for enabling advanced packaging for SoCs with interconnected chiplets. Intel is also announcing something called Quasi Monolithic Multi-Chiplet Interposer, as well as a next-gen fine pitch EMIB. These innovations should increase bandwidth density between chiplets.
Source: INTEL
Intel is also unveiling a new heterogeneous integration solution called Selective Layer Transfer, which the company says enables “ultra-fast” assembly of chiplets thinner than 1um—17× thinner than a human hair. Intel claims this can deliver 100× better chip-to-chip bandwidth.
Source: INTEL
So, why would you not use chiplets?
When asked about his company’s thoughts regarding chiplets, Nvidia CEO Jensen Huang said at the company’s GTC event this year that bigger monolithic chips are simply faster and that he sees no reason (at this time) to break his GPUs into chiplets.
To his point, chiplet interconnects (UCIe) take up the die area needed for more logic or SRAM. And UCIe, while fast, introduces more latencies that slow down the performance compared to a monolithic die implementation. The trade-off just does not make sense for a company known to produce the fastest AI GPUs. Nvidia’s plans could change in the future, of course, as technologies improve and as Nvidia pushes past the reticle limits.
To be sure, we have already seen a few awful failures along the chiplet journey. Most notably, Intel’s Ponte Vecchio GPU, used in the Argon National Labs Aurora supercomputer, featured 47 chiplets using three manufacturing nodes: Intel 7, TSMC N5 and TSMC N7. While a technical marvel, the chip’s complexity probably created production delays and yield issues. The chip’s performance was disappointing compared to the AMD Instinct MI250 GPU used in the DOE’s 37,888 GPU Frontier supercomputer at Oakridge National Labs. By the time Ponte Vecchio shipped in January 2023, it was competing against more advanced products like Nvidia’s H100 GPU.
Intel has sunset Ponte Vecchio and is now focusing on Gaudi 3 and the next generation Falcon Shores GPU, which will incorporate some of the Gaudi architectural features for AI.
Source: INTEL
Final thoughts
Chiplets certainly offer compelling benefits, but one must be willing to trade these off against some of the costs of complexity, die area and latencies. The most interesting development may be emerging business opportunities and models. A system chiplet can offer design teams a core set of functionalities that nearly every SoC will require, allowing teams to focus on their unique value-add.
Another example is the Tenstorrent RISC-V chiplet, which the startup has sold to several clients, including LG Electronics and Hyundai. Jeff Bezos has announced his participation in Tenstorrent’s latest funding round of $693 million, valuing the company at approximately $2.6 billion. Other chiplet companies include Celestial AI for photonic fabrics, Elyian for more flexible memory controllers, Credo Semiconductor for I/O and Alphawave Semi for wired connectivity.
IDTechEx projects that the chiplet market will reach $411 billion by 2035, with growth driven primarily by AI, automotive and high-performance computing segments. With that much opportunity at stake, semiconductor makers should evaluate when and where to deploy chiplet-based designs, not “if.”