The coming AI chip shortage

Artificial intelligence is booming.

Tools like ChatGPT are getting more capable at an impressive rate as companies race to plug them into new areas of the economy.

But the burgeoning demand for AI computing power faces a big constraint: the graphics processing units, or GPUs, needed to train and deploy these models.

These specialized, costly GPUs are almost entirely made by one company — Nvidia — at one manufacturer in Taiwan, according to Chris Miller, a professor of history at Tufts University and author of “Chip War: The Fight for the World’s Most Critical Technology.”

Marketplace’s Meghan McCarty Carino spoke with Miller about the potential shortage of chips to fuel the AI boom.

The following is an edited transcript of their conversation.

Chris Miller: The chip industry overall is facing a slowdown — as the economy slows, people are buying fewer smartphones, companies are spending less money updating their data centers. But for the specific types of chips that are used for AI, there’s actually a boom and some shortages that are already becoming visible. And it seems like the demand for these types of chips is only set to grow.

Meghan McCarty Carino: And what about the risk? As you know, we have seen in many other cases of technology of having the supply chain for this be so concentrated.

Miller: Well, it certainly is a risk. But there aren’t that many other options for most firms in the short run. The number of designers and producers of GPUs is quite limited. And not all GPUs are the same. They have different technical specifications, and so if you’re used to using one type, it’s not necessarily very easy to switch to a different type.

McCarty Carino: What about the CHIPS [and Science] Act and these hundreds of billions of dollars in funding going to increase development and production for chips domestically? Could that help here?

Miller: Well, most of the CHIPS act funding is not going to be for a specific type of chip. I think we’re likely to see CHIPS act funding being deployed to facilities that can manufacture GPUs and other types of chips that are used in generative AI, but I don’t think we’re going to see either the government or any of the firms that are applying for funding necessarily focus on generative AI as a primary use case because the reality is that we need lots of different types of chips. And this is one high-profile example. But across the economy, there are many different demands for more capacity for chipmaking.

McCarty Carino: What are some of the challenges when it comes to transitioning to more domestic manufacturing of these types of semiconductors?

Miller: Well, the types of chips that are used to train AI systems, including these advanced GPUs, are generally produced using some of the most cutting-edge manufacturing processes that exist. And there are just a couple of companies in the world that have the requisite capability to make these chips. And so there’s just a couple of companies that we’re focusing on, [specifically] their decisions about where to build new facilities. And they’re making their decisions based partly on cost dynamics, they’re looking at incentives, at tax policy, workforce dynamics. Can they hire all the workers that they need? And there’s really intense competition between leading companies in this segment of the industry in the U.S. and Taiwan and [South] Korea for talent, but also for support from governments. And that’s why the U.S. is having to try hard to attract chipmaking investment in the United States.

McCarty Carino: Recently, we’ve seen Europe also making moves kind of in line with the CHIPS act to shore up their manufacturing of this type of technology. Fundamentally, what do you see as the big challenges for really making this a sustainable transition for countries like the U.S. or in Europe?

Miller: Well, I think you have different dynamics that are present in the U.S. versus Europe versus Japan. For example, the U.S. has traditionally been a leader in many types of chipmaking technologies. In addition to designing advanced chips like GPUs, there’s also a fair amount of advanced chipmaking production in the U.S., and there has been for some time. In Europe and Japan, there’s less advanced chipmaking today. And so it’s going to be harder for those countries to try to scale up at the absolute cutting edge. And I think that’s why we are already seeing Japan, for example, focus not on the most cutting-edge, but on slightly less advanced chips. And I suspect that as different European countries begin to map out their plans for their chip industry, they may well end up focusing not on the most cutting-edge, but on slightly less advanced chips that have broader applications.

McCarty Carino: Overall, how do you see, you know, what the AI boom could mean for domestic chipmaking?

Miller: Well, it’s ultimately good news for the entire semiconductor industry because it increases demand for semiconductors. It puts computing power at the front and center of the transition in many businesses’ models for how they’re planning to develop new products. And it illustrates ongoing U.S. strengths because it’s U.S. firms that designed the chips underneath many of these AI systems. It’s also U.S. firms that are designing the AI systems themselves. And so it’s a success story for the U.S. tech industry.

McCarty Carino: What are you watching as we start to see this funding make its way into the world and, you know, some guidelines coming across from the Commerce Department on how this money will be spent?

Miller: I think the key question going forward is going to be now that we know how the government plans to spend much of the CHIPS funding, how will the companies respond? And how will chipmakers’ customers respond? Because ultimately, the companies that make semiconductors are trying to make chips in a way that will appeal to their customers. And so a major factor going forward is going to be when we look at the big customers for chipmaking companies like Nvidia or Apple or [Advanced Micro Devices], where are they going to want to buy their semiconductors from? From the U.S.? From Taiwan? From Korea? From other geographies? And that’s something where we’ve seen some initial indications from companies, but right now it’s still unclear as to how they’ll plan to procure the chips that they need over the coming years.

Related links: More insight from Meghan McCarty Carino

Over on our partner show, “Marketplace,” Kai Ryssdal interviewed Commerce Secretary Gina Raimondo last week about how she’s thinking about the rollout of CHIPS act funds as not just an economic policy but a matter of national security.

And last week, Nvidia announced a new innovation in the AI space — not hardware, but software called NeMo Guardrails, which can supposedly prevent chatbots from hallucinating or saying inappropriate things, in part by getting a second large language model to fact-check what the first large language model says.

But what about the second large language model? Who watches the watchman?