These parasitics, when it became flip chip, became lesser, and it was important for the RF work I was doing. Flip chip saved a lot of design work because when you design the chip, it works just as is. You don't have to account for the package as much if it's a flip chip; it is the chip. So I think now there is no chip without the packaging.
Hello everyone and welcome to another Semi Doped episode. I'm Austin Lyons with Chipstrat, and with me is Vik Sekar from Vik's newsletter. Hey Vik, what's going on, man?
I don't know. Are you a trillionaire yet? Because I know somebody who's going to become a trillionaire right about sometime soon. I don't know how much money he has. I'm talking about the SpaceX IPO.
Yes, yes. It's Friday when we're recording this, Friday, June 12th for listeners, and right now, today is SpaceX IPO day. And so Elon Musk is going to graduate from billionaire to trillionaire.
Apparently, I'm not sure really how close to a trillionaire he actually is. It's just too many zeros. You know, usually we record stuff pretty close to when we actually publish it, but since I'm going to be traveling next week, we decided to move this up a little bit. So if you're hearing the SpaceX news and like, 'Wait, wasn't that like last week? What's he even talking about?' Yeah, that's because we're recording it early this time.
Yes, totally. Yes, summer is for travel and vacations. Yeah. So, trillionaire, you know, I'm just a thousandaire, so it must be nice.
Just adding zeros. How hard can it be? Come on.
Yeah, yeah, right. Just add a zero. That's what I'll tell my kids when you grow up. You just got to add a couple zeros. That's your goal.
Yeah, but it's nice though. Like, I think it's valued at what, like 75 billion or something. This is one of the biggest IPOs in history. And so it's exciting. It's listed under the ticker SPCX.
So now we have this publicly traded company that does rockets, that does chips, that does AI, self-driving cars, you name it, right? And yeah, what's interesting compared to what we'll talk about today is Elon Musk's link to the Terrafab project, which is closely tied with Intel.
Of which we'll be talking a lot about today. So it all checks out. SpaceX is tangentially related to what we're talking about today, which is advanced packaging. You know, there's an interesting thing to noodle on, which is SpaceX and all its projects is like the culmination of deep tech is cool again, deep tech is valuable, right? So it's like satellites, space internet, electromagnetic waves, they make their own PCBs, so fabrication, manufacturing, and then like Terrafab. But it does go all the way up to they have xAI and they do self-driving in cars, so it does go all the way to physical AI and actually to AI. So it kind of spans the hard tech stack all the way from atoms all the way to intelligent bits.
And space, and yes, the vast universe beyond Earth, totally.
It's cool. I don't know, I like all these projects are technically very interesting, and the way you put it is like, oh yeah, you're right, he's going all the way from chips all the way to space, and there's everything in between, robotics and self-driving cars, and that's amazing. Like that's a lot of cool tech in one place. That's all I have to say about it.
Totally. Right. And then you know, he does the Neuralink stuff and the Boring Company. So he's got his hands in all sorts of interesting stuff. But okay, we'll save Elon for, we should give him a whole post sometime or a whole episode, you know. But yeah, we should do SpaceX maybe at some point. Let the IPO settle down. Yeah.
Yeah, yeah, yeah. They'll have an earnings call, so we'll get to listen and learn a lot. And I haven't even read like the IPO prospectus or anything, but we'll talk SpaceX sometime. But today, we're going to talk advanced packaging. But first, a quick message from our sponsor. Today's episode is sponsored by SambaNova. If you're running inference at scale, you know the hardware trade-offs between memory capacity, chip count, and quantization. SambaNova Cloud sidesteps these constraints entirely. You get API access to frontier models including full precision DeepSeek R1 at 250 tokens per second with no hardware lead times. Texas Advanced Computing Center, OVH Cloud, and Hume are already on the platform. Try it today at the SambaNova dashboard. The link is in the show notes.
Yeah, very important. You know, advanced packaging is basically what drives AI chips these days, and it is such a bottleneck. And TSMC has been the typical provider of advanced packaging. Now we have news that Intel is getting in some orders for their EMIB processing, which is also advanced packaging. And I should just expand it right up front: embedded multi-die interconnect bridge, EMIB, right? That's EMIB, which is a way of connecting different chips together on some kind of a substrate. We'll talk about all the differences between what TSMC's chip-on-wafer-on-substrate is. And within chip-on-wafer-on-substrate, which we will refer to as CoWoS, there are many variations and flavors of that. And so we should bring all those fine details to light, and then we'll talk about EMIB, what's different from that, and then really at the end we should hit on the whole Google's commitment to EMIB technology going forward. It's very interesting. So yeah, that's going to come in the end. So stick around for that one.
Perfect. So friends, you might first want to start actually with advanced packaging. What does that even mean? What is advanced about packaging, and what even is not advanced packaging? What is like simple packaging?
So let's start just at the very basics. So packaging, you know, we talk a lot about wafers, TSMC making wafers and making logic chips on wafers, and we've talked about the size of these wafers and being like dinner plates, and it's these big silicon wafers and everything that's patterned. We've talked about lithography before, but interestingly, when you think about your computer or your phone, there's not necessarily an exposed piece of silicon that people see, right? The silicon that gets made on this wafer has to get diced into individual chips, and then they need to be packaged, which is basically everything that happens to the wafer after it leaves the fab. You need connections from the die to the outside world for power, for signals, to get the heat out. You need mechanical protection, plastic or ceramic or something, so that the die actually survives the real world. So if you break open your phone, you don't just see this beautiful silicon thing sitting there. You got packaging, and you'd have to cut off the packaging to actually get to the silicon. So at the highest level, packaging is taking the wafer, dicing it up, taking individual dies, and then actually connecting them to the surrounding environment, whether it's a PCB or whatever. Anything else in your experience as an electrical engineer to say very high level about packaging?
The silicon itself, actually, apart from its interconnectivity to the outside world, needs to be protected from the elements, and it also needs to be protected from electrostatic discharge and things like this. All of this means that packaging is important. You need to protect a chip inside, and while doing so, also be able to connect to it, because you just can't put a piece of silicon in a phone and expect it to do anything. So the leads or the connection points into the chip are all done through packaging. So it's an essential step that has been around for a very long time. Right? So the earliest simple packaging approach was wire bonding, which is just like a piece of metal that comes from your PCB maybe to the chip that is sitting on the PCB. It's a very crude example, but yeah, it's just a metal wire that goes up and over and connects into the chip. This has been around for a very long time, and believe me, even in the age of AI and all of this, wire bonds do exist. Okay? They're still being used for a whole variety of products because not all this advanced packaging can handle things like, I don't know, power chips. Power chips require wire bonds because they need to carry a lot of current, and so these big honking wires that are like old school still work well. So simple packaging isn't dead. It's not like something that's in the past and it's like we forgot about it. Oh, those were the days. No, it's still there today, which some applications require. So the other way to do this simple packaging is like you flip the chip upside down and put solder balls there, and then you put it onto a PCB, and you melt the solder balls a little so that it sticks. That is the flip chip approach. That was a game changer in, I'd say, the '90s because the connection distance between the chip and the external world just got a whole lot shorter. Using those wires are like so long, but if you can put a solder ball and flip the chip upside down, it was amazing. Like it changed, it blew people's minds, and it was like a revolution in packaging in the '90s because now performance and the parasitics, the resistances and the capacitances, all got lower, and the chips could work so much better. And everybody was like, 'Oh yeah, flip chip is the best.' And then ultimately it comes down to variations of that in the future as it improved. They made the distance between these solder balls closer and closer and closer. So you could put like more connections into the chip. Like if you wanted to put a thousand connections into a chip, then you need these solder balls to have like a short distance between them so that you could put a lot of wires into the chip like that. So that became a big thing for how to make these things smaller and smaller. So that is the evolution of basically wire bonding and flip chip. So just for somebody who's never seen the world of packaging before or anything like that.
Yes, totally. And so again, just hitting it again, you know, wire bonding. Think like the simplest thing is if someone literally, like if you're in just a lab trying to build something for a class and you had a wafer, you would probably just literally like glue it onto a PCB and then attach wires like from the die, like you know, at the leads, the edges of the die or something, to your PCB. And you can imagine that is both very labor intensive. It's very error-prone, and then you kind of have these weird like exposed wires, and like Vik said, they're long and they have resistance, and it just feels fragile, doesn't it? And so of course, this flip chip is nice because now instead of having the top of the wafer where you have to attach wires, you just turn that upside down and you just have these short direct connections. And now it's actually the back of the wafer that's kind of exposed to the elements, if you will. And the connections are made by sandwiching these together. And you have these little microbumps. And so you have like still electrical conductors to sort of help you attach and connect the die to the PCB below it. But you can just see how ultimately once you figure out how to do that with high yield and at scale, then that is a little bit less labor intensive, more complex, but less labor intensive and also like more reliable. And so you know, but for those early decades, the packaging is sort of like the unglamorous part, like the manufacturing, the logic chip and everything that's like the front end of the process, and it's like where the value was captured, and it's complicated and very capex intensive. And this packaging is really like mechanical assembly and even some like human assembly. It's capex light, but it's also like lower margin, sort of lower value add. And so actually the industry structure, you know, way back in the day, people used to manufacture their own wafers, and then some would do their own packaging, but eventually actually some of that packaging got outsourced to OSAT, which literally stands for, I think you have to double check this, Outsourced Semiconductor Assembly and Test. So if you ever hear the word OSAT, that's what that stands for, or the term OSAT. And these think of the companies like Amkor, ASE, SPIL, Powertech, I think. So the foundry would ship finished wafers. The OSAT dices them, wire bonds or flip chips it, molds it, you know, puts it in its protective casing or whatever. And then usually they do the testing as well to show that the final package still works as you'd expect. So yeah, that's just for people like rule of thumb to know like fab companies design it, a foundry fabs it, and OSAT packages it. But of course you could also have companies like Intel and IBM where they do some or all of it in-house.
Yeah. So that's the whole basic coverage of what packaging is, and this has been around a very long time. As a personal anecdote, you know, even when I started working, I did run into a lot of the packaging guys. Those were just like, it seemed like it was a necessary evil. Oh yeah, we got to send it for packaging. You know, there's nothing fancy about it. There is nothing. It's just grunt and boring work because you got to make sure that the package is designed well enough so the chip can sit inside it, and they do all these tests like whether it survives the heat and it survives the temperature. You know, it's like dotting the i's and crossing the t's of the semiconductor world. It did become a performance limiter at some point even in my own work that I have, you know, early in my career itself because these parasitics, when it became flip chip, became lesser, and it was important for the RF work I was doing. Those things matter. Wire bonds are still terrible. So because my work was so parasitic sensitive and RF and all that, it was important, and flip chip saved a lot of design work because when you design the chip, it works just as is. You don't have to account for the package as much if it's a flip chip. However, as things got more complicated, it became a part of the design process itself. It eventually became part of the chip or an extension of the chip. And throughout time now, you know, if you look at all the AI chips and all that, it is the chip. There, without advanced packaging as we would call it now, it is the chip. So I think now there is no chip without the packaging, and it's good to define now how we transition into advanced packaging, right?
Yes, yes, totally. So if what we talked about was simple packaging, then what is advanced packaging? What puts the advanced in advanced packaging? Well, ultimately now, so the goal is always in a dream world, we would just have huge monolithic silicon chips, and because you would have like the fastest communication and the lowest latency between, you know, dies or whatever. But as we've talked about before and we can hit on again, there's something called the reticle limit. And so you can only make silicon dies so big. Then we probably talked about in the past too, and we can talk more again. There's also this concept of chiplets, which is like, wait a minute, some things don't scale when you make transistors smaller, like SRAM or maybe even like IO doesn't improve as much. And oh yeah, we talked about this last week on our Intel as we talked about Clearwater Forest where Intel says, 'Hey, yes, let's make the compute dies using 18A with our smallest transistors possible, but let's have other Intel 3 dies and Intel 7 dies for IO and memory and whatnot.' Ultimately, those need to get connected together. And even if you're Nvidia making GPUs where your GPU is the reticle limit, they make it as big as they possibly can, and they actually want even more compute. So they take two of those. With Grace Blackwell, there's two GPUs, and they actually have two huge reticle size GPU silicon dies, but they want them to behave. The goal is always, can we make this behave almost as if it's still just one big piece of silicon? So that's ultimately the goal. And you can't do that with wire bonding, and you can't really, a step in the right direction would be flip chip, but ultimately can we do even better? And so advanced packaging is saying, hey, could we actually use the same semiconductor processes as the chip itself, the logic chip itself? Can we use things like deposition, etch, and lithography to actually pattern metal interconnect at like micron scale pitches instead of big old wires? And could that help us get, you know, a lower latency, smaller resistance? And going back to tau, like tau being the time delay of signals communicating, resistance times capacitance. Could we try to shrink features even with the packaging as much as possible to, you know, limit tau? And really just all make it seem as if these dies that are being stitched together are more like as if they're on one piece of silicon.
Okay. Yeah. So that's the fundamental basis for the two technologies we mentioned in the beginning, which is like TSMC's CoWoS and Intel's EMIB. These are technologies designed to hold multiple chips together. Whether they are memory like HBM and a GPU, or like multiple GPUs like you were saying on some of these other chip platforms like the Blackwells or the Rubins, they have multiple chips, and those chips are really huge chips. So since you can't make one infinitely large GPU die, you have to stay to the reticle limit, which is actually 858 mm squared. It's a 26 mm by 33 mm reticle. That's about as big as you can make a chip. So the GPUs are at that limit. And to make it any bigger and put more transistors and have more compute, you've got to put multiple such reticle limited GPUs into one substrate, which we will see. I just generically call it substrate for now. And then you hook them up. You hook them up with very, very, very fine wires because the number of connections going between either the GPU to GPU or GPU to memory are way too many. There are like just an incredible number of wires. So the connection that happens between them also is not just on a single level. It's usually multiple levels. So whatever the interconnecting mechanism is underneath these chips has to support pretty complex routing that are all very finely featured. Right? That is the key that makes this advanced packaging. These are not like crude traces like a PCB is. These are almost chip level features that are being printed on substrates so that you can almost make one die an extension of the other.
Yes. Yes. You make a really good point about the wiring interconnect geometry. And so HBM3, I think it had something like 1,024 signals coming out or something. There's a lot. So you're obviously having to route all of these to the GPU, and you're trying to do it in small dimensions. And so yes, this, you know, think of it as like an ultra mini PCB, but you're right, you're like even as small as possible. You're wiring all this, and you're, you know, you're needing to route around things. It's non-trivial at this point.
Yeah. And HBM4 took it to 2,048 parallel lanes. So that's an example of why you need very, very fine interconnects, very fine metal lines that go between these chips on this interconnection substrate, and that has become pretty much the stronghold of maybe one company, maybe two in the world today, right? Yes.
TSMC dominates the CoWoS packaging that is required for all AI chips today, and they are extremely capacity limited because the way TSMC does it is actually a wafer fab process. It's not something you can do in an external third-party factory of some kind. You need sophisticated fab equipment to be able to make these things happen. And so that's where the whole advancedness of it comes in. And that's why somebody like a TSMC who knows how to make this with high yields and have a long history of making chips and running a foundry comes in, and that's why not everybody can get into this game. Yes.
Yes. So this is a really interesting point. Let's talk about this for a second. You know, going back in history, we talked about, okay, everyone used to be their own IDM, and then eventually there were like people said, oh, let's outsource the back end of this and let these OSATs do some of the processing. And along the way, foundries came up, and so even TSMC, which was a foundry, said, we deliver the wafer and someone else can package it for us. Not huge value there, but now we're talking about advanced packaging, and it takes the same know-how as a foundry, it takes the same tooling, the know-how. And so now all of a sudden it's like, wait a minute, the simple packaging is still totally used high volume, whether it's from cheap components to expensive components, but there's this advanced packaging bit, and that's actually these fabs that can make logic chips or make memory chips are actually capable of doing the advanced packaging. And so TSMC actually entered advanced packaging, you know, I think in like the late 2000s, maybe 2010s or so, where they started looking ahead and saying, oh, this advanced packaging will be a thing, and this is something we could do. So it could actually either create a new market if you want to think about it that way, or maybe bookend what OSATs can do. And you know, so it's like TSMC, there's OSATs, but now there's this advanced packaging that TSMC could do again. And so TSMC started doing R&D, and they, you know, came up with CoWoS, which we'll talk about. They also, I think, did some early work on InFO, integrated fan-out packaging, which we probably won't talk about much today. But I just wanted to point out that actually now OSATs are trying to say, we want to get into advanced packaging. So OSATs are trying to expand beyond the simple packaging and actually move their way into advanced packaging. So for example, you can go listen to Amkor's earnings call, and you'll hear about this because that's where a lot of the value is accruing and is going to be more and more and more important in the years going forward. So if you're a forward-thinking OSAT, you're saying like, hey, I don't want to be left out. I actually want to try to get into the advanced packaging game too.
Yeah, that's amazing. Right now what happens is that once advanced packaging has gone into a fab, and you know these traditional OSAT packaging houses like Amkor want to get a piece of the bigger pie by going and doing what a fab does and doing advanced packaging. Anybody doing this now starts to run into the same limitations of wafer manufacturing and or chip, right? So to put this in perspective, I think we should just quickly step back and see, okay, we spoke about the reticle limit being 858 mm squared. Now the H100 was already at that limit really, right? And you mentioned that the Blackwell die is basically two reticle size dies stitched into one GPU using CoWoS, right? So what's the next one? How many chips are we going to put in next? Like we got two.
Yeah, I believe for Rubin there's supposed to be four dies, four GPU dies stitched together.
So now what happens, yeah, so with the four die thing, it starts getting complicated because remember that the manufacturing process for the packaging that holds these dies together is the same thing that was used for the chip itself. Right? So what that means is now you couldn't make chips that are bigger than the reticle limit, which is why it stopped there. But now you need to make packages that are bigger than the reticle limit. Otherwise, how are you going to put two GPUs into one and stitch them together? These packages that you are running in a wafer fab process now need to be, let's say, two reticle sized to hold two chips or usually more, right? Because you need to put HBM, you need to put other stuff. So maybe three, you know. So the first generation of CoWoS was actually something like this. It was maybe 3.3x the reticle size that you could make. And so that somehow started becoming the limiting factor now. And the question is, how come you can make packaging substrates that are like bigger than reticle limits but not ICs themselves? I think I'm not entirely sure of the right answer, but I'm going to take an educated guess because these wafer fabs that run packaging substrates are actually much simpler. You don't actually make transistors, and so that means that you can probably stitch together a few reticle shots worth and make a larger reticle substrate that can hold maybe two or four GPUs at one time. Right? So that's where it comes from. What do you think? Do you think that's the right answer as to why you can make advanced packaging that's 3x reticle size?
Yeah. Yeah. Well, which by definition advanced packaging has to be bigger than reticle size because like you said, you're taking reticle size like dies and you're wiring them together. So I think that's just the key is that ultimately advanced packaging is still just drawing wires. So you're not trying to in one shot draw a bunch of tiny resistors, and you're not necessarily using this like 18A style like pitch and tools. You're using semiconductor tools, but it's still bigger. You're still drawing wires ultimately, like metal wires, if you will. So I think conceptually it's more about you're taking these dies and you're able to pattern, use lithography and pattern, but you're drawing wires. Now, I don't think you're necessarily, we should, I need to learn this. It's a great question. I don't think you're drawing all the wires in one shot, you know? So you're probably like putting the wires in over here and over there and over here and over here, still like kind of like stepping around.
Yeah. So this whole chip-on-wafer-on-substrate or CoWoS technology, and we will get right now very quickly, we're going to get into what the wafer and substrate is, but is typically referred to as a 2.5D packaging technology. It's only 2.5D because you have an active piece of silicon that is with transistors packaged over, let's say, a passive piece of silicon without transistors, but it's still silicon, right? So that is the chip-on-wafer process. So the chip with active elements like transistors is packaged over silicon on wafer. So chip-on-wafer, and that is 2.5D. Right now, if you, the 3D version of this would be stacking a logic chip on top of a logic chip or active on active. So 2.5D is active on passive, 3D is active on active. So that is a packaging technology that is around in all of these fabs too, but we won't get into it too much today because that is really not an advanced packaging technique. It's a method of stacking active circuits together.
Yes. So let me, I'm going to say it again because it's like marking terminology and people will get weirded out by it, but this is just industry standard. So there's 2D, 2.5D, and 3D. And yeah, the 3D is really cool and we should talk about it sometime. Stacking logic chips, but we won't talk about it a ton today. But just to define these things because people are, you know, in your brain you're probably thinking like two dimensions, okay, I get it, like maybe it's like a piece of paper, like the surface of it, and three dimensions, like I get that we live in a three-dimensional world. What the heck is 2.5D? And again, just industry conventions, sorry, semiconductor industry is weird. Just like we talked about 18A doesn't mean things are 1.8 nanometers or 18 nanometers or whatever, or 18 angstroms, 1.8 nanometers. That's kind of a lie, and 2.5D is also a lie, but it's just the naming convention. So engineers, don't overthink this. So basically think of when we talked about flip chip or hybrid bonding, just think of that as like 2D, like you've got a die and you slap it on substrate and you wire it together, and just think like the die is sort of like 2D. It's just a die. Like you're looking at the surface of it, whatever. 2D. 3D is what Vik said is the far other end of the spectrum, which is like you're trying to stack things, right? You know, this is like you're building the skyscraper, whatever. It has multiple stories, if you will. So that would be 3D. And then 2.5D is like, hey, we're going to increase the size of this house, but instead of building many stories, we're just going to put a bunch of little houses next to each other, and then we're going to like build a hallway that connects them all, right? So this is 2.5D. So think of like, you know, you've got a compute die and you've got another compute die, like two GPUs, for example, and then you're stitching them together. That would be 2.5D, or even technically HBM memory connected to a GPU is also considered 2.5D, which can get a little confusing because technically internally within HBM it is actually 3D, it is stacked, right? Like these HBM, it can be like 12 high or 8 high or whatever, but now I'm getting way too, I'm giving you way too much information because you're probably getting confused. But generally when you think about 2.5D, just think that like it's one story, but it's like chips next to each other and they're connected. And like Vik said, there's a key point which is in 2.5D, and we're going to get into the details. We've buried the lead. We will finally explain to you what CoWoS is. But when you connect these things, there's usually a passive interposer as we'll talk about in 2.5D. So okay, Vik, let's talk about chip-on-wafer-on-substrate.
Okay. Yeah, we got to get to it right at some point. But all this background is important because otherwise if we just dive right into it, CoWoS is this and that, it just like it loses people who are not familiar with this stuff, just like lose context immediately. And I felt it in the past, and I think this background so far was useful, and I hope everybody feels the same way.
Chip-on-wafer-on-substrate. Okay. Chip is the GPU, right? It's sitting on a wafer. The wafer in this case could be a piece of silicon, right? It's a silicon wafer, but it doesn't have any transistors on it. It just has metal lines that can connect to another chip sitting nearby. So that's a chip-on-wafer. And this stack of the chip sitting on a piece of silicon now sits on top of a wafer. So it's like a three-layer stack, right? And that is what is CoWoS. And that is the big thing that TSMC made, you know, go into production around the early 2010s with FPGAs, I think initially.
