The fact that we are stacking logic on logic has two implications. The first one is it is entirely centered around hybrid bonding because that is the secret sauce that allows them to increase transistor density.
Welcome to another Semi Doped episode. I'm Austin Lyons with Chip Strat, and with me is Vik Sekar from Vik's Newsletter. Hey Vik, what's up?
It's going well. Yeah, it's a nice time to chat. We usually do this later in the week. So usually we record on a Thursday, but today we're like doing Tuesday. And it's perfect because we were actually going to talk about something else, but then there is this paper that dropped from Huawei that said something to the effect of, 'Oh, we're bypassing EUV. We're going to be in like TSMC's 14 angstrom equivalent by 2031 and everything online is all at once.' This is hitting me in my face saying Huawei has, you know, circumvented EUV, ASML is dead in the water, US dominance is gone in leading edge nodes. I mean, at least that's the sentiment I get. And considering we recorded the deep dive on lithography last week, so I know I texted you and was like, 'Okay, like, can we just talk about this early this week because it's so fresh in my mind, we should just do this.'
Yes. Yes. So, this is perfect. Let's talk about it. And in fact, it was Memorial Day in the United States yesterday, so I was like hardly even online, but the internet was blowing up. So, I'm glad that you texted me. I'm glad that you read it. I look forward to learning from you through this conversation. And I will say I wonder how the United States government, anyone in those circles, like the Commerce Department stuff, how they're feeling because of course EUV lithography is the big geopolitical chip on the table. And I bet some of them have to be freaking out if they heard or saw an X like, 'Oh yeah, that you know what you used for controls is now rendered useless.' They have to be also like, 'Wait a minute. Is this real or is this marketing?' So that's the goal of this is to unpack what actually did Huawei announce and was the marketing speak and the interpretation different than the technical paper and the technical innovations here.
Yeah. So we wrote this down in the Semi Doped newsletter. This is kind of where our first reactions go. So if you're not signed up to that, you know, as the listener, you should. It's free on Substack. You should just go to semi-dop.com and just like sign up because whatever news comes our way, we just try to like write up quickly the first thought that comes to mind. Usually it's the most natural response. But yeah, after that, I think I got a few questions saying, 'Hey, what do you think of this piece of news?' And then I responded and said, 'Yeah, this is really interesting that you know Huawei has this approach of trying to improve performance of chips overall but without trying to go to the fancy machines which they can't get a hold of because they're all under export control.' So without much ado, I think we should first explain what the whole claim was. So there is this conference called ISCAS 2026 that I believe is being held in Shanghai this week and He Tingbo, who is like a Huawei director and the head of HiSilicon, which is the chip arm of Huawei, I think gave a talk. So was that... I did not hear the talk. Do you know if He Tingbo is a he or a she? Because I might totally get this wrong.
Um, I saw a picture of a woman. So I think the she gave... whoever gave the talk was a woman, I'm pretty sure.
Okay. Okay. I'm glad I asked. But anyway, the point is the whole idea is that there is this new guiding principle called the TOAO scaling law, and the spiel is at least that it's going to replace Moore's law because you know, over time Moore's law has stopped scaling and we've been, you know, squeaking along ever since we hit kind of the EUV nodes below 7 nanometer. And the whole idea is like, okay, let's look at something else. And so this is where I actually like this framing. Actually, the TOAO scaling law has a much more fundamental scaling that I truly appreciate. And I need to explain why. Why this TOAO thing. First of all, TOAO is a measure of delay. It could be delay on the chip, it could be delay through the interconnect, it could be delay between racks, it could be delay, I don't know, between entire data centers or anything. So the whole idea of basically going to smaller transistors was essentially to minimize this delay. The smaller you made a transistor, the delay got, you know, lesser from the input to the output of the transistor and that only meant it went faster. So for the longest time the only way to make things go faster was to reduce the delay. So Huawei's interpretation is to stop thinking about dimensions of the transistor which they really can't scale without EUV machines. But why not go down one further level and ask what was the transistor solving anyway? And the answer was delay. So then the next logical question is, okay, we can't improve the transistor delay anymore because we don't have the machines. So where else can we improve the delay? Because it's not like delay comes only from the chip or the GPU or the CPU, whatever it is. Delay is everywhere. Delay is in software. Delay is in interconnects and how you hook up memory. What memory protocols and handshakes you do. So they were like, okay, let's reinvent everything and think of this from a whole system perspective. So this is what they call their TOAO scaling law. We will now scale down TOAO at the system level. Not so much at the transistor level that has been done historically, but over the entire system. It could be a phone to begin with or an entire AI data center. We are now scaling down delay.
Okay. Yeah, that makes sense. I mean, so summarizing it, they're basically saying, hey, Moore's law is dead. And by the way, even if it's not dead, we can't shrink transistors anyway because we can't get our hands on EUV tools. So if we want to continue to increase performance and we can't reduce the geometric footprint of each transistor, how can we increase performance? And so they zoomed out and said, well, wait a minute, maybe the geometric scaling of transistors was actually ultimately about reducing delay. And so then they're trying to reorient around TOAO this time, delay, resistance, capacitance product, and say, okay, fine, we have one knob that we can't turn, but what are all the other knobs that we could turn to continue to reduce delay? And I do like the point of not just delay at the transistor level but extreme co-optimization or STCO, DTCO, which we could talk about. I saw you had a tweet about this, which is just saying how can we look up and down the whole stack from transistors and devices to circuits to systems to racks to interconnects to the whole data center to software on top of it and how can we try to co-optimize amongst all of those.
Yeah. So whenever they say this is kind of a law, it's always nice to see some equation. And I read the whole paper actually. It's an easy read for a paper actually. And they had this nice equation which says the TOAO of the transistor, TOAO of the system is basically the delay through the transistor, delay through the circuit, delay through the chip and delay through the system, right? And what they want to do at every subsequent generation, like the TOAO of the next generation is the TOAO at this generation divided by some factor alpha where they think that that alpha factor is like 1.3x a year for mobile and 1.5x for auto and maybe even 10x for AI workloads. So think about that, like optimizing across the system. They are thinking that they can reduce the delay by 10x for AI workloads. That is a significant improvement and that is why they feel like they can get a 1.4 nanometer class performance by tweaking other parts of the system, not just the transistor.
Okay. So this law, like you know, most of these laws are not actual physical laws but they're observations. So they must have had in their paper, were they showing just like chips that they created and measured these constants and that's where they're seeing the scaling? Or where did this data come from in this like empirical observation?
It's not anywhere I think. So they have some silicon. Okay. So let's get to that in a bit. But their optimizations were interesting because they happen basically across, from what I could tell, across three dimensions. The first dimension was that they just want to make transistor density more, right? That's the whole thing that EUV does. EUV lets you pack in more transistors per unit area of the chip by making transistors smaller. So they stepped back and like asked, okay, we can't make transistors smaller. So how do we scale up the number of transistors in a chip? So they decided, okay, fine. We'll just take two chips and stack them one on top of the other and like hybrid bond it. Hybrid bonding is a packaging technique that's very interesting because you can have like millions of connections between these two logic chips. And the way it works is that you just heat them and like put pressure and they literally stick to each other in the most simplest way. That's what hybrid bonding is. So you can have like very, very, very fine connections that are like closely spaced, like the pitch between connections is something like 1.5 micron across a massive area of a chip. Think about that, right? So hybrid bonding is a very fine pitch packaging technology which is probably the most advanced packaging technology you can get. So their first dimension was, okay, let's just stack two chips together. So in the space of one chip we get now two chips. Hooray. You know, that's one way to get transistor density. That's a kind of cheating because now you also use two times the silicon area because you got to sandwich two wafers together, but you know, considering the cost of EUV, which we discussed in the last podcast episode, maybe it's not a big deal. Just saying.
Yeah. Yeah. Yeah. Totally. Okay. So there, so you're saying they can't... so they've got a chip, they can't increase the transistor density because they're at their fundamental limits with DUV and multi-patterning and whatnot. And historically, by the way, how the industry is kind of quote-unquote getting around Moore's law is like systems of chips, you know, so it's like, oh, whether you're using chiplets or whatever, so it's like, okay, well, let's use like 2.5D integration. Let's put chips next to each other and then we'll have CoWoS, you know, or interposers and connecting things. But you're saying that Huawei said, 'Oh, no, wait a minute. Well, what if we, like, instead of putting those transistors far apart and have increased delay because now we've got to route between them. What if we just try to decrease the delay by stacking them in three dimensions to sort of increase the density, if you will, in a unit volume really?'
Yeah, they call it logic folding. Okay. Which is a nice name but it's really, if you think of it, no different I feel compared to what Intel Foveros is or how AMD stacked SRAM with 3D V-Cache. I guess it's not technically, you know, logic-to-logic stacking when you're talking about AMD's 3D V-Cache because they put SRAM on top of a logic wafer to boost like L3 cache on it, but in principle, yeah, they hybrid bonded a SRAM wafer onto a logic chip and that's kind of what this is all about. So logic-to-logic stacking is not easy, right? Because how are we going to get heat out of this thing? That's one example, like thermals are quite challenging. So there are a lot of challenges to doing this stuff actually. And you know, you got to align it. Like think about it, like the connections are like 1.5 micron pitch, actually is ridiculously tiny and so the alignment between the bonds needs to be perfect. And hybrid bonding itself is a crazy packaging technology because the surface you're bonding needs to be very flat and defect free and all that because when you squeeze it together, there's like a dust particle between the two of them or whatever, like you know what happens, like now you've got an open connection between the two, you know, sandwich chips and that's bad. So it's a challenging process and that's the whole question is like, does China actually have the equipment to do this? Yeah, funnily they do. They do because for two reasons. One, they do have this expertise because they have been doing memory stacking for NAND at YMTC using wafer-to-wafer stacking and hybrid bonding of chips. That's how NAND chips work. They have even done hundreds of layers of NAND. So they are familiar with it but memory is a little bit of an easier problem because memory has so much redundancy that you can kind of route around stuff, like have failovers in the memory architecture and stuff like that. So stacking is a different problem in memory than it is for logic. Stacking two GPUs on top of each other is a significantly harder problem than trying to stack 400 layers of NAND memory, you know.
Totally, totally. So okay, so you're saying historically stacking things is not a new concept. Even stacking logic is not necessarily a new concept but normally when we're stacking first, like let's say logic on an interposer, that interposer is passive. So it's not as big of a deal. You're just routing through it. And then even if you're stacking like memory on logic, that's a little... and of course memory and NAND and HBM, a lot of these things are already three-dimensional. So we're already used to figuring out how to create things in three dimensions and stack them. But it is a bit of a different beast when we stack logic on top of logic because they're both active, they're both powered, they're both giving off thermals, and you have to make sure all the connections are correct. And there's like today the way these things are built, there's not this built-in redundancy where like, oh, if something fails just route around it. But conceptually it's still to the industry, logic on logic is not a new thing that Huawei has invented.
No, it's not and it's been around in principle. So that's what makes it interesting, like it's a challenging problem and it's impressive that they do have silicon that... it's called the Kirin 2026, this is a mobile SoC processor and they actually have this implemented and they have plans to keep going in the future. So they've already, you know, let's say I don't know how the paper has all these numbers... oh yeah, here I have them, I think, yeah, they've managed to like double their transistor count. Yeah, obviously by stacking. And so they kind of jump nodes in principle because you know, we spoke about this, there is no such thing as like 5 nanometers or 2 nanometers anymore because the transistor architectures have changed. So this is just a nomenclature now anyway. So you can go to the same class node by doing other things like gate-all-around was one of those other things like you could do to go to 2 nanometers. So Huawei's approach is like, yeah, we'll just stack transistors and we'll get the same transistor, you know, density. In principle this is like, you know, CFET maybe, like a complementary FET where people were like, why should I put an NMOS and a PMOS transistor next to each other? Because for a complementary MOS you require P-type and N-type transistors. What if I put them on top of each other, you know, I can save space. So this is along the same inspired lines, not exactly the same thing, but basically you can... why not put a whole transistor wafer on top and stack them like that and you can jump generations forward. So this is... they've done it actually and this is very impressive engineering. All kudos to them. I'm not going to take away from their engineering achievement here. So that is the whole logic stacking aspect of it. There are two more things that I think are very useful but I want to get to that after you ask me this question.
Yes, thank you for letting me graciously interrupt. So on the logic stacking, so one of the things this reminds me of, of course, is DeepSeek in that they could not get enough compute. They could not get enough compute and enough like memory bandwidth with the chips that they were given. Allegedly, okay, maybe they did find their way to some H100s, but allegedly they had these stripped down which caused the DeepSeek team to have to innovate in other dimensions because they were constrained on one. And so then they got to be the first to think deeply about other things like, you know, how do we offload some communication, overlap some communication and compute, do other little tricks so that we still unlock the right performance. And so what I'm thinking about is, okay, if Huawei is constrained to not use EUV and therefore they're thinking about like, okay, how can we reduce delay in other parts of our system and one of them is it's forcing them to go to logic-to-logic stacking maybe sooner than the rest of the industry feels that they have to. My question is who is manufacturing it and is this giving them an advantage in just getting more practice manufacturing logic on logic? Like will they be able to sort of run ahead a little bit because they are forced to build this for their customer sooner than say a TSMC?
Yes. This brings... actually it's a good question and I'm glad you asked this now before I went on to talk about something else because the fact that we are stacking logic on logic has two implications. The first one is it is entirely centered around hybrid bonding because that is the secret sauce that allows them to increase transistor density. So you can increase the density two times. Can you stack it three times? I don't know. Can you stack it four times? I don't know. Like, so what is the limit on hybrid bonding here? How many layers? That's something I don't know. But remember, it gets like extremely difficult because if you are stacking logic on logic, you would... I think you would want to do die-to-wafer stacking, not wafer-to-wafer stacking because you will get wrecked on yield. You know, as it is these logic chips are kind of big and yield is such an important thing because you know, you don't get all that much as people imagine especially at the very cutting edge but maybe 7 nanometer nodes is okay. So that is one aspect of it that it's entirely based on hybrid bonding and the question is are they capable of it and the answer to that is at this point... memory stacking is all like wafer-to-wafer but logic needs die-to-wafer and that is kind of new even to like Besi and these companies that specialize in hybrid bonding and even they have product releases that are like very recent. Okay, so die-to-wafer bonding in logic chips is very cutting edge and luckily from what I was looking at this, there is no export restriction on hybrid bonding machines. You have extremely high limitations on what you can do in EUV but not so much on hybrid bonding machines. So that's one thing. So they do have the machines that they can do this with and if you look at like Besi's business, 35% of all their business is actually China-based if you see the last quarter. So that is actually a large fraction of their machines actually do go to China. So I'm pretty sure they've stacked up on some Besi machines before they let this cat out of the bag. Because if you already have a piece of silicon that's stacked up and working as in the Kirin mobile SoCs, you can believe that they've been working on this for years. Okay. This is not an overnight achievement.
