The memory aspect of semiconductors today has gotten so extreme. Stuff is so expensive that people are simply not able to make lower-end equipment or devices anymore. And this is like killing everything, right?
Hello listeners. Welcome to another Semi Doped podcast. I'm Austin Lyons of Chipstrat and with me is Vik Sekar from Vik's Newsletter. Hey Vik, what should we talk about today?
I don't know. There's so much going on in the semi world. Like my head was reeling trying to think of what to say here. I mean, we don't want to sit here all evening and like chat about this stuff, but yeah, like some of the big things that have happened is that memory is continuing to get out of hand. I think we should just talk about it at least because it's kind of well known for now.
Totally. It feels maybe it's just because we're paying more attention than ever or maybe things are as crazy as ever. Memory, optics, CPUs, logic, everything. It feels as crazy. But we haven't talked a ton about memory. So, let's dive in. Let's talk memory.
Yeah, let's do memory. In a sense, the memory aspect of semiconductors today has gotten so extreme. Stuff is so expensive that people are simply not able to make lower-end equipment or devices anymore and this is like killing everything, right? That's so the one of the things that has been in the news. I'm not entirely sure if it's true or not, so I'm going to preface that by saying it could be a rumor, but it's possible that Nvidia might not announce a new gaming GPU generation in 2026.
And that by that I mean we may not see an RTX 60 series. We will see stick with a variation maybe of the 5090 Ti. Somebody online said it's like maybe coming through in Q3. This is kind of a big deal because as I was reading in the last 30 years of Nvidia's history, they have never not announced something GPU related every year because they are fundamentally a GPU company and they have always had something to offer consumers. And when we spoke about Nvidia at CES, we were joking like where's the consumer stuff here? Like I can't buy any of the stuff Jensen's showing us, right?
Yeah. That's really interesting. And you know, to be clear for listeners, when Vik is talking about GPUs here, he's obviously talking about graphics cards, consumer-oriented. And zooming out, it really feels like there's this tension, and we'll get into this with the impact to memory between data center, AI data center spending and consumer spending, both GPUs and with CPUs, right? So we see from a memory perspective, and we can jump into this, we see a lot of memory companies like Micron for example wanting to invest more and more of their capacity and their money into HBM DRAM and the trade-off of making consumer memory for example.
Yeah. And just for context, if anybody isn't aware of why HBM and DRAM or why GPUs and AI accelerators are related, it's because both of them use DRAM and the AI accelerators use a high performance version of DRAM called high bandwidth memory where they just stack these DRAM chips and they are able to sell those stacked DRAM chips called high bandwidth memory for a lot more money. Of course they are a lot harder to make which is why only SK Hynix, Samsung and Micron can even make HBM. They're only three companies but yeah so this is why so all these companies, these memory manufacturers take their wafer supply and make HBM for AI accelerators then nothing is left behind in terms of capacity for like regular consumer gamers who just want to play a game when they get back from a hard day's work, you know.
Right, totally. And in the same breath, Nvidia is naturally incentivized to sell more AI data center chips, right? I think you had a note here that gaming GPUs are 8% of Nvidia's revenue down from 35% of revenue in 2022 and I think that really illustrates that like yes, obviously Nvidia loves gamers, that is the name of the company, like I mean that is like the core of the company for the last 30 years but just naturally from if you're running a business you're incentivized to chase after the biggest markets and unfortunately for us consumers, you know, and all the gamers who helped build Nvidia into who they are today, like they're just not the biggest market today.
Yeah. And AI chips make like 65% operating margins and gaming does like 40%. And the 8% of the revenue only comes from GPUs now. So obviously from a business perspective it doesn't really make sense to put too much effort into GPUs which is kind of sad, you know, because what happened to the rest of us, you know, everything is like AI.
Yeah, the AI chips have the better operating margins and therefore it's the same thing trickles down to these memory companies. You know, it's like to your point, the HBM, the sort of AI memory. You're just going to make more selling wafers full of that than you are wafers full of the memory needed for gaming GPUs.
Yeah. And every once in a while, something comes up in the news and everybody's like, "Oh, that's the HBM killer. That's it." Like no more HBM and everybody can get like regular DRAM again. One such incident was like the context memory storage which came online and everybody was like worried that this is going to kill HBM because you don't need to store that many weights I suppose or like context on HBM and because you could store it all relatively cheap in this massive NAND storage system for context memory and KV cache. However, you know, you can't just get rid of HBM. It is still the prime real estate for holding weights and doing inference and all that. There is no way you can just like lose HBM. And if you see all these like newer technologies that are newer chips and GPUs that are coming out, they all are increasing the HBM content. Nobody's putting lesser HBM. HBM4 is now here and all the GPUs are going to use that. They're going to put in like a terabyte of HBM in a single GPU or something. Yeah. So it's not going down anyway. So there's no way like I see in the immediate future of how this DRAM supply is going to even like bounce back because if everybody's adding more HBM, you know, then there was some talk about how people are starting to use a combination of optics and DRAM to like pool memory together and bypass HBM again. I don't know, it's a cool idea, yes I get it, but HBM performance is still unbeatable so it's always going to be a premium memory and given how much AI capex spend there's going to be in 2026, which we are going to get to in this podcast, everybody's going to still want to buy GPUs and HBM.
Okay, so just to give you an idea of really how bad things are, right, how bad is this memory thing. So I pulled up some numbers by like looking around and it seems like the GDDR7 price has like 4x'd in the last year. It's gone up four times. It's kind of ridiculous. And I saw this article from Tom's Hardware that estimated like what is the cost of like a GPU,
a chip itself. And the calculation showed that it was like something like $300 to make it, like the core GPU, but the memory around it was like more than that, you know.
Because of the 4x'ing of GDDR prices. So the so-called video RAM or VRAM on a GPU is like now 80% of a gaming GPU's bill of materials.
How on earth are you supposed to make money selling GPU cards now? Right.
Right. Oh man, that's painful.
Yeah. And then I was looking around a little bit more to see like, okay, like TrendForce estimates that apparently DRAM contract prices have doubled in a single quarter. That's like a 100% increase in one quarter. Like from last Q4 to now in one quarter, it's like twice as expensive. And if anybody listening isn't like sure exactly what spot versus contract pricing is, spot market is basically like what price is on the open market. Like if you were to go buy a stick now, you know, what is the price of RAM? That's kind of spot market and that usually varies significantly all over the place, you know, based on demand, supply, all kinds of things day-to-day. But contract prices are different, which contract prices are basically memory makers making a deal with you know the HPs and the Dells and all these people that use DRAM in their products, laptops, and they have a volume estimate and they say okay look we're going to make like I don't know a million laptops this year and so we need so much memory and the memory companies will go all right given that you have this volume here is our contract pricing. So it's kind of like the commitments have been made beforehand and in spite of that if you know they are able to price it 2x in a single quarter it's getting out of hand seriously.
Yeah indeed and this would be so contract pricing is would be why Apple had a really good quarter and you could tell that demand and margins weren't yet impacted by the memory price increases because they were still on these contracts that had been locked in in the past. But that's also the big headwind for Apple going forward, which is the question is how repeatable is that? Because we know that it's going to catch up with them. They're going to have to renegotiate and have new contracts and yeah, memory prices even at the volume that they buy. Right now is a good time. Memory is very cyclical and it's hard business to be in, but now is a great time to walk into your customers and say we're raising prices 100%.
100%. You know, and that's the thing. Actually Apple is a good example because I think they got into a contract pricing with Kioxia like well in advance. Like I don't know how they saw it coming. It was kind of amazing that they were able to sign in a deal at lower prices compared to anybody else and that really saved a few quarters of business for them. But like you say, yeah, it's going to come back soon enough and they're going to be like
No, nobody can outrun memory.
That's right. And okay, we always talk about these like big memory players, right? Like because of HBM, you see these three big companies always around. But now I saw that Nanya Technology, which is a Taiwanese maker of memory, posted a 600% year-on-year revenue for DRAM. Okay. And they don't really make HBM or anything fancy for that reason. They make standard DDR4. They don't even make much DDR5. It's a small fraction and I think it's about 10% of their overall revenue. But even they are making money. So you know if you have spare cash lying around I think we should just start like a memory fab like anybody will buy our stuff right now if we make memory. It's that much.
I think I'll have a hard time convincing my wife to invest our money into a memory fab. But it's compelling, right? Like obviously and you know this is why the memory market is cyclical because everyone looks at you know how do you get 600% year-on-year revenue. You can increase your prices 600% or you can try to sell 600% more chips at the same price or some combination therein. So obviously there's going to be a temptation for everyone to say there is a low supply, massive demand, we should increase capacity, we should build more fabs. Then of course you get to the point where lots of capacity comes online, prices you know you can't raise prices as much and then it all starts to cycle from there.
Yeah. You know Nanya's price increases are the equivalent of like selling a can of baked beans for $100. I'm like this is baked beans. Why are we selling this for a hundred bucks? You know it's like that. It's standard DDR4 memory is not even the latest generation. It's ridiculous.
So it's not only about NAND. I mean it's not only about RAM because we should talk about NAND flash because ever since context memory has come online you know there's this whole rush towards NAND memory and storage systems because as Jensen announced the Vera Rubin platform has in the storage rack 1,152 terabytes of NAND per rack and Morgan Stanley's estimates are that in 2026 they're going to ship 70,000 Rubin racks each with 1,000 terabytes of NAND. So, you can imagine how much NAND supply that's going to be in 2027. And Rubin alone, apparently, according to Morgan Stanley, is going to take up 13% of the global NAND supply just for Rubin.
Wow. Talk about AI driving everything.
Yeah. It's ridiculous. But you know I think context memory is fascinating. We had a podcast with the WAC CIO, the chief AI officer Val Mirkovic, which is one of the previous episodes if anybody wants to check that out. And also I wrote a Substack article on why this is going to drive the price of inferencing down although it doesn't seem like it. Opus 4.6 in fast mode is like burning tokens and people's bank accounts at the same time. But you know I think overall eventually when KV cache can be stored on mass I think the cache hits will improve and inference costs should go down. That's the long-term trend I see but I don't think we're there yet.
Yeah. No, I agree with you. I think my takeaway from listening to that pod was there's to your point like if we can intelligently store the KV cache correctly so that we have cache hits and not cache misses then we can let the GPUs you know be lazy and not have to recompute things right and so we can increase GPU utilization decrease token economics. Yet at the same time it just says to me we're going to need more memory. Like at every step of the hierarchy, we're just going to continue to want more, you know, because then that's going to open up the cheaper that gets and being intelligent there with offloading as much context as you can, it's just going to drive, like Jevons paradox is going to drive people to want to put in more and more context, you know, into their prompts. I still struggle today where I'll work with Claude Code or something and I'll feed it, you know, a bunch of PDFs and be like, hey, analyze all these earnings calls or whatever and it'll still be like, ah, that's too much. It's too big. Or it takes too long. And you can still see that we're hitting the limits today. So, I think the more and more we do, the more desire will be because at some point I'm going to be like, "Oh, great. Now, I want my agents to analyze these 10 earnings calls that all happened in the last two weeks and I want to give you, you know, six quarters worth of context for every agent. Go do all this, you know, and there'll be a point where it can, but it will only be able to do that with innovations in where we store the KV cache and how we do that in such a way that it doesn't cost me a thousand dollars to do it, right? But maybe $10, you know, of API credits."
Yeah. And not to throw a spanner in the works here, but the whole agentic AI, OpenClaw, you know, hype cycle that we're in. And it's not just hype cycle. I think the agentic AI era is about to become the dominant way we use inferencing because the whole idea of typing stuff into a chat window and then getting responses is not sustainable. Like you want to give broad instructions and then it goes off and does its thing and gives you exactly what you want, that's agentic AI in a nutshell, right? But for all this to actually work we need CPUs now and you mentioned that earlier, these CPUs if they have to agentically operate need DRAM again, right?
So memory is not going away, yeah.
Yeah, everywhere you look, yes, the GPUs they want tons of HBM, they want even more SRAM or the XPUs. The CPUs, they want more DRAM. We're going to do more compute in the network. So, the DPUs are going to need more memory, right? Just everywhere you look. Yeah. So, the Vera Rubin platform is supporting over one terabyte of DRAM, 1 to 1.5 I would say in these memories, pluggable memory slots. They're not soldered on anymore. So they want to fill up as much DRAM as possible in these servers because it is becoming more CPU heavy workloads are not just GPU heavy and GPU and HBM heavy they're also CPU and DRAM heavy because of all the orchestration that needs to happen between the agents and then everybody like you were saying wants to have a lot of context they want to put in the big PDFs they want to put in all their Google Drive documents and ask it to give like a whole summary of various things. All of this takes like large large scale storage. So, you know, we're in an era of finding a use case for something that just requires so much memory. This I don't see it changing in the immediate future. And with that, you know, there's some proof to what I'm saying too because Western Digital is booked up in long-term agreements all the way up to 2028, 29. Some people are even talking about 2030 long-term agreements. So, think about that. You're like almost we're talking about almost half a decade into the future and like people are signing up supply agreements for storage. It's just ridiculous, right?
Totally. It feels, you know, unheard of. And then on the other hand, you know, you could say like, oh, it feels like a bubble, right? People are just grabbing for everything they can. But I would say when I think about the value created by agents even in my own life like oh I'll spend 20 minutes planning things and let the machines go do work for me versus me just manually doing all this stuff like it's just a step change in my user experience and I wouldn't go back. So even if it feels like a bubble, there's a mad rush, more capacity will come online. Will people always, you know, want to buy supply for the next three, four, five, six years, maybe, maybe not. But the fundamental value that I'm getting isn't going to go away. And I'm not going to stop. And I think there's a lot of the world that has yet to experience that value, right? A lot of people in their day-to-day jobs that have yet to because we're kind of on the bleeding edge, early adopters, tinkering with this stuff. I think most workplaces have yet to realize how much of their workflow they could automate, how they could work differently, think differently, delegate. And so it's very hard to even if there is a bubble and even if things back there's an overbuild fundamentally, it's just hard to see like we live in a different world now. You know,
We're on the forefront of it. Maybe we're sort of two years in the future, if you will, but it's going to come to the rest of the world. So it feels crazy to say like there won't be sustained demand for memory.
Yeah. We'll revisit this conversation at future checkpoints. Maybe next year or two years down the line. We'll find out if this whole podcast aged like wine or aged like milk.
