There is a memory in almost every chip that nobody talks about, and it holds a few kilobytes to a few megabytes. It has no spot price or a super cycle, but without it, most chips — analog especially — can't work or even meet the specs on its data sheet. So for 30 years, that memory has been embedded flash, but as we get to smaller technology nodes, it really stops working. So today's guest designed embedded flash and then spent the rest of his career on what comes after it.
Hey, welcome to another episode of the Semi Doped podcast. I am Vik Sekar, the founder of Semi Exponent, where I do boutique research on semiconductors and AI infrastructure. I also write Vik's Newsletter on Substack, one of the top 20 technology newsletters on the platform covering AI and semis. I come from two decades of working in the semiconductor industry, and my work now is taking that engineering background and explaining what matters in the technology and why it matters to the markets. Check out Vik's Newsletter on Substack, and for technology research, reach out to me at Semi Exponent. And our guest today is Ilan Sever, chief memory architect at Weebit Nano. He has spent over 30 years in non-volatile memory, semiconductor IP, and SoC design. He was previously group CTO at Dolphin Design, director of IP and libraries at Tower Semi, and design manager of flash memory at ST Microelectronics. Ilan, welcome to the podcast.
Awesome. So I guess one of the things that we don't often talk about is what exactly are non-volatile memories, because when I mentioned flash memory, that's there pretty much everywhere, but it is so ubiquitous that people have even stopped thinking about it in the mainstream. It is assumed to be there, but it's still a very important part of all of our electronics. And it's not just like DRAM and SRAM — that's the whole problem, right? So you've worked on this stuff before, so let me start by going through your journey. What made you get started on working on flash memory?
Yes, thanks, Vik. So yes, non-volatile memory is basically a fundamental element of any electronic system. It's where you store your software code and your parameters and anything that you would like to stay even when the power goes away. So we all have systems in which the battery can run out or we shut it down, but when we power it back up, everything goes back to life. So where is this stored? All this information is stored in non-volatile memory, meaning a memory that doesn't need the power in order to retain the data. So you mentioned flash memory, the technology of choice even today in what we call NAND storage, like your phone storage, is what we call NAND flash. I'm not talking about this technology; we are now referring to technologies which are embedded within system-on-chips on silicon, and this was previously served with embedded flash, which is an N-type flash. Basically, flash technology is storing charge on some floating gates. So there are some capacitors which are charged, and it remains there even if you shut the power down, and that's how flash works. As you mentioned, flash doesn't scale well. So in geometries below 28 or 22 nanometers, flash reached its physical limit, actually. So physically, you cannot implement embedded flash anymore, and the industry had been looking for alternative technologies for a few decades now, and we will talk today about one of these technologies that is now gaining really market acceptance, which is resistive RAM.
Awesome, that's a great introduction. So for people listening, we always talk about static RAM, SRAM, which holds the state, but it holds information without having to be refreshed. However, if you turn off the power to that SRAM, you're going to lose everything in the SRAM. Like you switch off your computer, your OS is removed from the SRAM and all of that stuff, right? So on dynamic RAM, I think it's more like you have to keep refreshing it. But of course, you take power out, that goes out too. Non-volatile memory that you were mentioning, Ilan, is that it's the stuff that stays all the time, you know, on, and even if you turn off your phone, it holds information, and that's very important because your devices just can't forget stuff all the time. So we'll definitely get to why flash stopped scaling as we went to smaller nodes and what resistive RAM means. But I just want to step back just a little bit and say, exactly what is the role of non-volatile memory in a device? It could be an embedded device, or a sensor, or a power controller, or an automotive chip, whatever it is — what is the role of this non-volatile memory? What does it actually hold?
Yeah, that's a very good question, Vik. So fundamentally, I would say the base usage or the most wide usage is for any embedded MCU or microprocessor — it has its software. And if you want to have a really embedded device, whether it's an IoT device, a medical device, your hearing aid, or whatever device that needs to be compact, low power, fully embedded without additional external chips, you would like to have your code inside the same system-on-chip. You have an MCU, whether it's any MCU architecture, it would need to run some firmware, which is software.
Like a microcontroller, right? Like MCU is a microcontroller unit, and it has to run some code. The question is, where does it get this code from?
So of course, if you want the code to be there even when someone is replacing batteries, or maybe the battery is out, or even applications that don't have power all the time — you know, there are applications today which do energy harvesting and they just get energy from ambient sources and they are not always connected to any power source. So in any of these cases, you want the software stored. So the main usage or the first usage, I would say, is to store your software. But then in addition, just think about sensors. So you have some IoT device or medical device. It is logging maybe the temperature, maybe your heartbeat, maybe some ambient data from some sensors, and you want to have a logging of this data. So think about a sensor. It's logging what happened, and maybe after a while it is transmitting the information to some system, whether it's an industrial sensor, medical sensor, any type of these devices. So they always need, in addition to the code storage that we discussed before, also some data logging storage. Moreover, most of these systems, these mixed-signal embedded systems, they have also some analog parts. And for the analog parts, usually you would need to have some kind of trimming. So these are a kind of normally one-time programmed data that you do upon production to just put all the analog values in place. So you use trimming.
That's another usage for non-volatile memory. So you just want to kind of burn this data into the chip so it returns throughout the lifetime of the chip. And last but not least, on security applications, there is always storage of some secret key. So encryption keys are normally stored on some kind of non-volatile memory on security applications.
Awesome. Yeah, that's a great introduction because I think what most people don't realize is, take the example of the analog part you mentioned. You know, my own background is in analog semiconductors. So it's amazing that when you're making a billion of these little analog sensors, or any analog power supply unit, or a power distribution chip, or whatever it is, as you manufacture a billion of them, right, what you're saying is like not all of them are going to come out the same — no way — because there is process variation. And if you say that this thing has to output like 1 volt, it's going to output 1.02, the next chip will output 0.99. You know, this is not usable because let's say you want to power a GPU with this power controller chip. It can't be putting out these random variables of voltages. And so the trimming aspect is just an electronic knob that is inside the chip that when you can write in something like a code, like, hey, adjust the voltage by 0.02 volts down, and you can write code that in a one-time fashion and then center everything. Like the alternative in the digital world is you have to kind of keep binning chips — like Intel will take, oh, these are the fastest chips, I'll sell them as i9, these are the slowest chips, I'll sell them as i5, you know — but in the analog world they don't do that. They just trim and calibrate things so that they all work the same. So this is like a very unspoken part of the analog world which is extremely important.
Yes, this is correct, and this is again one of the use cases. In this case, we are talking about a very small amount of data that needs to be retained throughout the lifetime, and it also is programmed only once or maybe very few times. So when we discuss non-volatile memory, it's important to state one very important term, which is what we call endurance or programming cycles. So for example, if you program a software, then yes, you would need to change it, right? The software versions people do, even what is called firmware update over the air today. So your system can update code even in the field or over the air. So all this requires reprogramming of the data, unlike this streaming data or what we just mentioned that is more like production data that you stored once. In any case, any type of non-volatile memory is able to reprogram data, not as many times as you can write to SRAM or DRAM, which are the volatile technologies that you mentioned that obviously you can read and write to them almost infinitely.
Exactly. Okay. This is great because you're setting this non-volatile memory to be a kind of memory that you mostly write one time or a few times and read many, many times in its lifetime.
Yes. So any type of non-volatile memory will need to be reprogrammed. And one of the parameters that we look at when we look at NVM technology is the number of cycles, the endurance of this memory. So some technologies can only work up to 10k, 10,000 reprogramming cycles. Advanced non-volatile memory is expected to already allow in the order of 100k, 100,000 to a million reprogramming cycles, and this means it's a very good technology that allows a wide usage for many types of applications.
Awesome. That's helpful to know because we will obviously talk about endurance and reprogrammability when we actually talk about what resistive RAM is. One other important point that you mentioned I think we should touch on is the capacity of this thing. When you talk about — you mentioned NAND in the beginning — you can store terabytes on NAND flash, and in DRAM you can store gigabytes, in SRAM you can store hundreds of megabytes. So what is the range of non-volatile memory in terms of storage capacity?
The capacity of the non-volatile memory depends on the application, obviously. So we mentioned embedded MCUs having some firmware. This is normally counted in kilobytes, but I would say tens, hundreds of kilobytes to single megabytes. So maybe one, two, or four megabytes is a large NVM capacity for embedded MCU — maybe four megabytes.
So we're talking about single-digit megabits, right? Like 1, 8, maybe 16 megabits, 32 megabits for code storage. Of course, all the other use cases like data logging or trimming that I mentioned are much smaller — we're talking about kilobytes.
Now, in more advanced MCUs, so maybe in lower geometries where people embed non-volatile memory with high-performance microprocessors for any type of applications — automotive, industrial — it can reach, I would say, ranges of 16, 32 megabytes. But we are definitely still not in the gigabit or gigabyte terrain that is implemented in other types of technologies like the ones you mentioned, DRAM or NAND flash.
Yeah. So that's also very important to understand about these non-volatile memories because you want to essentially, let's say, reprogram them over the air. What you're actually reprogramming is only a few megabytes or a few kilobytes. And when you drive your Tesla into your driveway and it wirelessly connects and updates its firmware, this is where it is actually stored. It is stored in a non-volatile memory, probably on the larger capacity side than you would have in a little sensor. I would assume an automotive NVM — non-volatile memory — with like embedded flash or whatever is like a larger capacity than a sensor, which would, or for the trimming applications, it's going to be really small because you only need to correct for, like, you know, put in a code, that's it. So it has to store a very little amount of memory compared to what an automotive application would use, but it's definitely not in the gigabits, gigabytes range and all that, right? So that's the whole thing here. So typically we've used eFlash for doing this, and now it seems like flash has hit some kind of a limitation as we go to smaller nodes. What exactly is eFlash, and why does it not scale to lower technology nodes, say below 22 nanometers?
That's a good question. So embedded flash — basically flash memory is storing some charge in an insulated floating gate. So it's like a transistor in which the gate has some charge stored, or a floating gate, and it changes the conductance or the threshold of the transistor. That's the way the data is stored. So you can have charge stored on that floating gate or taken away, and it changes the behavior of the transistor. Now, the amount of charge that you store there, when the technology scales to low geometries like 28 nanometer, 22 — and I'm not even talking about single-digit nanometer technologies that are available today — the dimensions of this transistor and the dimension of this so-called floating gate are so tiny, and the amount of charge that is stored there is just moving physically to almost single electrons. You know, a very small amount of charge, and this is just a physical limit. And of course, holding this charge over there without any leakage, without the charge going away by all kinds of physical mechanisms, is becoming really impossible in the smallest geometries, and that's why flash cannot scale to these smaller geometries.
Okay. Yeah. Yeah. That's very helpful. Actually, I wanted to compare this to what, like, for example, DRAM looks like. You know, a modern DRAM actually uses the equivalent of, let's say, a 10-nanometer transistor, but the charge that is stored is never stored within the transistor itself because there is a capacitor on top of the DRAM that actually stores the charge. And the whole problem of DRAM and capacity and DRAM density is how to get as much capacity by digging a deep trench in the silicon and connecting a transistor to it, and how to maximize that capacitance without having to rely on the transistor to hold that charge. But what you're saying is like in the flash world, the transistor has to hold a charge, and then you make the transistor so small that you can hold basically, you know, a few electrons of charge in it, and that's about it because it's so small it physically stops scaling. You need to hold more charge than that, for which you need larger transistors, period. And that's why flash does not scale beyond 22 nanometers. So there's no flash memory in FinFET transistors or below.
Yeah, this is a very good point because you mentioned all kinds of vertical capacitors and trench into the silicon, the TSV in discrete DRAM chips. When we are talking about the embedded world, it's very important to note that the whole idea of embedding a system-on-chip with memories and with analog is that you would like to keep a plain CMOS technology that allows to integrate on the same chip with the same basic technology features. You would like to have analog components. You would like to have digital components. You would like to have SRAM, because there will always be SRAM, and you would like to have an NVM. But you will never invest a special trench capacitor, or you will never invest a lot of special technology, because it will make the whole system-on-chip too expensive. So another key parameter that we look at when selecting a non-volatile technology — I mentioned before endurance — and the performance of the memory is how much does it interfere, how much does it add on top of the baseline CMOS technology. In the end of the day, semiconductors is very cost sensitive, so it doesn't seem like that, but just imagine how much is the selling price for such an embedded MCU. If one would need to add too many processing layers, which eventually translate to money and to time of manufacturing the silicon wafer, that will make the whole system-on-chip too expensive. Imagine there is a large system-on-chip that has everything I mentioned — digital, analog — and it has one NVM component, one NVM code storage. If you add more layers of processing, special structures, you pay this additional cost over the whole area of the chip, which is making the whole system-on-chip more expensive. So a key parameter in adding any technology is how many masks we count. Basically, in semiconductor processing, we count masks. So how many masks does the technology add on top of the plain CMOS, you know, plain vanilla technology? And that's another key element because in the end of the day, this means how much are you paying premium for having a system-on-chip with embedded NVM.
So when you mention, for example, NAND flash, that's a totally different technology with, you know, 3D integration and many levels. And when you mentioned, for example, discrete DRAM with the trench capacitors, these are expensive technologies, but they are optimized for a single use case, which is a NAND flash chip or a DRAM chip. In embedded, you need to have a plain vanilla technology with the minimum additional layers to support all the types of memories that you need.
Yes. Okay. It's very important because you're saying that, look, the cost of these microcontroller chips that require these non-volatile memories is already very low. Now, if you want to add like a whole lot of masks into the semiconductor process just because you have to implement this non-volatile technology, it better be very inexpensive because otherwise you can't have the whole bill of materials for the chip go up just because you want to add NVM materials inside. So non-volatile memories should come with as few masks as possible. So with that being said, how many masks does a typical NVM embedded flash process add?
Yes. So important to mention. So embedded flash adds another, I would say, polysilicon layer, a floating gate layer on top of the transistor. In some cases there is split gate technologies. There are all kinds of flavors of that, but they add a lot of processing, and this processing is done in what we call the front end of line. So in the transistor, very close to the silicon substrate, where you actually implement your transistor. And for making that, we are talking about in the range of 10 additional masks, maybe eight to 10 additional masks just to process these additional features required for embedded flash.
Okay, that's — what percentage of the total mask count would you think that is? A typical CMOS process has, would you say, like 70, 80 masks?
Even less. I mean, typical CMOS will have even less masks, and so yeah, this can add 25, 30% add-on, and it can become very significant in the cost.
That's a problem. That's what we're actually trying to solve here. And when you're trying to go to smaller transistor nodes anyway, you can't store enough charge and you can't add so many layers. If you have to add 30% more layers, that's not a cost-optimal feature. So I think this is a good time to get into — we've set the stage exactly as to what is non-volatile memory, why is it used, where is it used, what are its features, and what is required from a non-volatile memory in terms of cost, simplicity of implementation, capacity, and everything. So I think we have set the foundation of non-volatile memory and where it lies. I think the next interesting step is to go into what Weebit Nano actually does, which is resistive RAM. Like, this is a nice time to get into what that is and how it solves many of these problems that we just talked about.
So indeed, resistive RAM is a technology that is aiming at replacing embedded flash. It has several key benefits related to our previous discussion. First of all, it is very low cost. It is adding just two additional masks to the technology. It is embedded only in what we call the back end of line. So we actually build the structures between the metal layers, way above the silicon substrate and transistors. Now, why is this important? We mentioned before all kinds of mixed-signal chips, analog chips that require some non-volatile memory. These technologies have passed through a lot of optimizations of their unique analog characteristics. For example, some technologies have BCD transistors, which are high-voltage transistors. And when you invested so much in an analog technology to have the perfect analog properties and for power management and other types of applications, if you had to now embed what we call a front-end-of-line technology, a technology that sits at the transistor level, it would interfere with the other analog transistors that you spent years optimizing. So another key importance of a technology that you would like to add on top of an existing, let's say, analog or mixed-signal platform is that putting it in the back end of line means it is totally decoupled from the transistors. So you are not impacting the transistor behavior, and you are not impacting any analog characteristics because the memory processing is done actually after, way after you finished processing all your CMOS technology. It is actually done in the top layers, top routing layers, metal layers above the silicon.
