A hot potato: HPC GPUs designed to accelerate AI training cost a lot of money, but TSMC won't be able to satisfy the increasingly high customer demand yet. The world's leading chip manufacturer is still lacking the kind of advanced packaging capability the market is requesting right now.
AI servers are expected to explode to an estimated market value of $150 billion by 2027, and Nvidia currently makes the most sought-after GPUs designed to accelerate algorithmic training for chatbots and generative AI services. TSMC, which is tasked with actually making the GPU-based boards the aforementioned AI servers will employ, won't have enough manufacturing capacity to satisfy market demand for at least one and a half years.
Speaking with Nikkei, TSMC chairman Mark Liu said that the AI accelerator shortage isn't caused by a lack of AI chips designed by Nvidia. The current market situation mostly depends on TSMC's own limited capacity with CoWoS packaging. The Taiwanese company cannot fulfill 100% of its customers' orders right now, Liu stated, but "we try to support about 80%" of manufacturing requests.
Chip-on-wafer-on-substrate (CoWoS) technology is an advanced packaging platform that provides "best-in-breed performance and highest integration density" for HPC hardware applications, TSMC explains. The interposer-based, wafer-level system integration process supports a wide range of HBM cubes and package sizes, the company says.
TSMC currently makes the overwhelming majority of GPUs, FPGAs and other specialized chips designed to accelerate AI computations. Those accelerators employ HBM memory to provide the largest bandwidth possible for AI training, and they are manufactured with the CoWoS packaging technique. Other chipmaking companies also provide similar packaging capacities, but TSMC is likely getting the largest orders coming from the most reputable chip companies including Nvidia and AMD.
According to industry analysts, advanced packaging technologies like CoWoS are costly and financially risky to implement. Therefore, smaller manufacturing companies are likely less motivated to invest all the money they would need to increase and refine their packaging capabilities.
TSMC is spending nearly $3 billion on a new packaging fab that should come online by 2027. The company is committed to increasing its packaging capacity "as quickly as possible," Liu confirmed to investors. The "tightness" in manufacturing output is expected to be released by the end of the next year, TSMC's chairman said, with CoWoS wafers doubling in 2024 compared to 2023.