The Compute 100: Austin Lyons, Analyst & Writer, Chipstrat

Awesome. Yes, thank you for having me.

Sure. Totally. Yes. So, I'm older than I look. So, it might be longer than people will think, but yeah, so I got trained as an electrical engineer in undergrad and in grad school. I did my master's at the University of Illinois. And then, interestingly, when I was at the University of Illinois, I was actually in the PhD program, but I started a startup with some friends. This was back like in the Kickstarter days, and some other grad students, and we launched something on Kickstarter. Had a lot of fun with that and it really like opened my aperture for what's possible. And so I was kind of like, 'Oh man, I got to go get into industry and maybe get into startups.' And anyway, so I actually went straight to Intel out of grad school down in Austin, Texas, and was working on a startup at night. Not in semiconductors. This was more of like the, you know, IoT, iPhone apps, B2B SaaS, early days era. And so I was doing hardware engineering by day. I was doing software engineering by night and that kind of set me down a path of thinking beyond just engineering but more about like who is the customer, what do they really want, is this useful. Ultimately we raised some money, we built a product and it turned out the customer didn't really want it, so you kind of learned early on the hard way. And so yeah, fast forward, I ended up eventually moving into product management and getting an MBA and was just really sort of like impacted by those entrepreneurial experiences. And so I kind of sit at this nexus of I understand the deep technology, I can think like a product manager and ask, you know, the questions about roadmap, value prop, pricing, that kind of thing, and then I have a bit of an entrepreneurial spirit myself. So of course I always love to study startups as well.

Yes. Yes. Yes. So I was working as a product manager by day and of course the world was changing, you know, ChatGPT was launched and all of a sudden like semiconductors, which was kind of my original first love, became super interesting. Early on of course it was just from the perspective of logic, but now it's so much more, you know, it's logic, memory, optics, power, foundry, all the things. But you know, I was spending a lot of time listening to, consuming media, like I was training for marathons, so consuming podcasts, reading people's things, trying to pay attention, and I just saw like a gap in the market from like an education content perspective. Either there was either like very technical things or sort of very non-technical kind of fintech type things, and I just thought like we're missing like a Ben Thompson for the semiconductor industry here. And I felt like, you know, my background gave me a particular take on certain things, and so I just decided to start writing, you know, because it was as accessible as ever, and then it kind of snowballed from there. You know, you start writing and essentially you start like putting your public thoughts out there in the court of public opinion, but it turns out most people are really nice and really interested, and you find like the other nerds like you who want to talk about this stuff, and then you know, as you guys know how it goes, then it just kind of snowballs from there.

Sure. So some of my early takes, very early on, there's an AI accelerator startup, Etched, and I started looking at them. This was, you know, early on, it was just the Nvidia sort of gravy train. Everyone realized like, holy cow, GPUs are the future of AI and Nvidia is obviously dominating in training and we weren't even into the inference area yet. And I saw that Etched was out there and I started asking questions around like, oh, could there be companies in this space other than Nvidia? So Etched was one of them that kind of got me into thinking about like, well, what would it take for a startup to compete with Nvidia? Would the model labs trust and buy from a startup? Would the hyperscalers trust and buy from a startup? And so kind of looking at first principles of like, well, what's wrong with the GPU, you know, and what's right with the GPU? And so I started kind of pulling on threads and realizing like, hey, GPUs are super flexible, that's great for training, that's great for inference, but at the same time once you like start to fast forward, if transformer-based inference is like the defining workload of our time, which has turned out to be true, then it would make a lot of sense for hyperscalers, who actually know their workloads, to actually invest in something that might be a little bit less flexible, but as a trade-off it's very, you know, hyper-designed for the workload at hand, for, you know, attention and LLMs and KV cache. And so, you know, I had, early on I started saying like, hey, I think there's something here. I think these AI accelerator companies compete. And then similarly early on I was actually pretty bullish AMD, and I think actually that was one of my first posts. I was like a year in that started to get a lot of traction. Was just saying like, hey, you know, I think everyone at the time was talking about this moat, and part of the moat early on was the fact that Nvidia had written software so that anyone in almost any domain could accelerate their workload, from scientific computing to oil and gas to quantitative finance and so on and so forth. And so anyone who's trying to compete, you know, like say AMD was trying to compete, people would say, 'Oh, well, you have to write libraries in all of these different domains and it's going to take you hundreds of thousands of man-years.' But once all of a sudden it became that the TAM for LLM inference was going to be way bigger than any of these other high-performance computing things, all of a sudden that CUDA moat kind of shrank, which was like, well, to compete for all these dollars maybe AMD doesn't have to even worry about all those other markets. Maybe if they just focus on making their Instinct chip and their ROCm software competitive for LLM inference and start there, they could actually compete. And yeah, so those are actually kind of two very similar takes, which is, you know, I think startups can compete in this market and I think AMD can compete. So I'll just pause there, but it was very focused on logic early on.

Yeah, at this point. At this point.

So, Intel, so early on, you know, once Nvidia started selling all the GPUs and making a ton of money, everyone was saying, 'Where's Intel? Where's Intel's GPU?' At the time I kind of said like, you know, hold the phone. Intel is a CPU company and frankly like at the time this was, you know, back in the Pat Gelsinger days when they were even trying to figure out like, are we going to have an external foundry or not? And what is our strategy? And Intel was behind. Basically the advantage that Intel always had was they could do their own manufacturing and for many, many years that gave them a performance benefit compared to everyone else because they could get to the next node from a manufacturing process faster and therefore they could reap that and bring more performant, more power-efficient products to the market. Again, mostly CPUs, both for client and for server. Then there was a period of time around 10 nanometer, 7 nanometers where actually Intel fell behind for various reasons, one being not getting on EUV fast enough. But then all of a sudden that kind of exposed their... basically you can think of Intel as manufacturing and product and they're an integrated device manufacturer. They do both. They don't use TSMC. Manufacturing used to really benefit the product team. Once manufacturing fell behind, all of a sudden that really started to drag down the product team and sort of expose if you have the same technology as everyone else or worse transistor technology, are your designs actually as performant and do they compete? So manufacturing started to fall down and then the product team started to fall down. And meanwhile everyone's starting to say like, okay, Intel's struggling, but oh, they just need to make GPUs to catch up. And my question was, should they make GPUs or should they not make GPUs and not make all these other like networking chips and stuff and actually just focus on fixing both those things? Make better CPUs like they used to. And of course AMD is starting to eat their lunch. ARM was starting to eat their lunch and figure out the manufacturing thing. Not only get back to being competitive, but then of course the problem with manufacturing, everyone used to manufacture their own chips because there's actually a benefit if you can co-design and design for manufacturing. If you can do that together, not just design the chip, but also design the manufacturing, you can like optimize up and down the stack. But of course with every generation of foundry it becomes more and more expensive to the point where most companies didn't have the volume to fully utilize and sort of amortize all of those huge capex investments. So, you know, people kept falling off, but Intel had enough volume. So it started to beg the question, but even they were starting to get the limits, which was if their manufacturing could catch back up and if they could take on external companies, could they kind of get back into the game and compete with TSMC both on their performance, but also financially, like could it actually be competitive from a cost perspective? And so early on, my Intel hot take was simply they shouldn't focus on GPUs. They should actually just right the ship and make sure their CPUs are performant and they should like double down on Foundry. And now I will say I didn't expect it, that has paid off for Intel. CPUs have exploded with agentic AI and I didn't necessarily expect that as early on, but it has sort of worked out for them in that their CPUs have been in high demand and can help continue to fund the business as they're starting to bring Foundry up. And of course, you know, within the past like six months, we started to hear a lot about Intel Foundry and external customers. And so they are, you know, on stable ground with CPUs and with manufacturing. It still begs the question, what is their AI strategy? What is their GPU strategy? But I think now they have, like, they've put enough sort of sturdy ground, foundational ground underneath their business that they have kind of earned the right to expand into GPUs.

Yes. So, the question for Intel is why didn't people just raise their hand when everyone wanted to make GPUs and TSMC said we are supply limited and it kind of depends on where you're in line and you get allocated so many wafers and Intel said, hey, we're going to get into the game. Why didn't people rush over to Intel? Well, obviously there's like a big cost in saying like, now we have to redesign everything for a different foundry, but ultimately it's cultural. So when you are an IDM, integrated device manufacturer, your foundry ultimately serves your product team and you do kind of have this internal group think about this is the way we do things and this is the best and that is not a customer-centric mindset. Like, you know, your customer is the product team but you're all the same company. And so Intel, when they said that they were going to become an external foundry and they started working even with small customers, it did give them an opportunity to start trying to build that muscle. And probably the best thing that has happened at Intel is they brought in external hires who've worked at other foundries, whether it was GlobalFoundries or even like Micron or Samsung memory companies who already have that correct orientation of I work in a service business because that's what manufacturing is. And so yes, I think Intel has had to undergo a lot of cultural change, but they seem to have worked through this and ultimately it will end up benefiting Intel products because as Intel manufacturing team now they have to make the decision who should get the capacity, who should get the wafers, should it be our internal products team or should it be our external customers? And that's actually going to be great for both of them because now the products team can't be sloppy and just be like, ah, we screwed up here, do a hot run and just run a couple more wafers for us. It's like, no, no, no, you have to behave like an external customer, you know, you need to plan in advance and there might be times where you have to go to the end of the line, you know, and wait your turn.

Yes, totally. Yes, it's a brand new world and of course again none of us could have predicted all the geopolitical stuff that's going on that also has, you know, indirectly benefits Intel Foundry as customers are incentivized to say, I'll give you a chance.

So, packaging used to be a fairly boring space and sort of like low value, low margin in that obviously if you like go way back in time, packaging was literally like taking discrete transistors and kind of like wiring them together and then it was like taking discrete chips on a board and like printed circuit boards, you know, and just like big copper traces in between them. But nowadays we've moved into advanced packaging where we're taking chiplets or dies and trying to connect them in three dimensions. And that's like way more sophisticated from the perspective of how do you route everything electrically. You know, what are all the... it's complicated from both a design and a manufacturing perspective. From a design perspective you're thinking about like crosstalk and noise and electrically routing all these signals to the right places. And then...

Yeah, that's a good analogy. Yes. You know, historically it was just like you had a chip and that was it. It's like you got a house and it's in the country, whatever. We're not even dealing with roads. You know, and then eventually there's like this 2.5D where it was sort of like, yes, let's start to put buildings near each other and now we have to put roads and alleys and connections between them. And then as we're headed into 3D, like for example, high bandwidth memory is 3D stacked DRAM chips. And that is, yes, you're literally, you know, stacking them up and you have to think about like, yeah, the internal plumbing of, you know, I guess if you're thinking about like a high-rise building, like how do I get heating and cooling to all of these, get the waste out, you know, electricity. And then yeah, we will continue in the future. There's actually companies who are working on, you know, actually stacking like logic chips. You could either have like logic chips with memory on top or logic chips stacked on logic chips. So yes, now we do have to think about the infrastructure in three dimensions which ultimately takes it from... it used to be OSAT that did this packaging and that stands for like outsourced testing and assembly. Yeah. Outsource semiconductor assembly and test. Yes. Outsource semiconductor assembly and test. Yes. And it was literally, I mean it says outsource like right there, you can tell it's like low value. But now it's obviously getting higher and higher value and getting more and more complicated. Like really if you at the end of the day it's very similar to what foundries do when they're fabricating logic chips or memory chips. It's like that level of deposition and etch and patterning. Now the OSATs are trying to move into that space because they want to accrue the value at the OSAT layer and they want to say like, no, no, we can do CoWoS now. So if you can't get it from TSMC, we can do that too. But of course, it's also an opportunity for TSMC and Intel to expand from just making logic chips to doing more of the full-on logic, dicing, packaging, you know, everything.

Yeah. Yeah. This is tough because you have to come in, you know, one of the hard parts when you first look at the semiconductor industry is it feels like a lot like everywhere in the value chain there's like two or three or four companies and they're huge and so you have to ask like, okay, if I am a PhD student and I'm studying some sort of really interesting like new material or something like how do I take on, you know, Applied Materials or someone like that? I don't have scale but what could I do? Obviously one thing that you're thinking about right away is could I build some interesting technology that they would want to acquire? But if you want to be a full-on company like let's say, let's talk advanced packaging. Obviously like as this is getting smaller and smaller and you know the power consumed by these dies continues to increase. There's really interesting questions around like how do you deliver power there? If power used to come in just like straight from way off to the side, like is there ways to integrate it much closer on the package or even like as a die as part of this like 3D stack? So you can think about like are there power delivery opportunities, are there power dissipation opportunities? Interestingly I think probably what doesn't get appreciated enough is from an EDA or a design tool perspective. There used to be people that would design and think about like the electrical characteristics and maybe different like mechanical engineer type people thinking about the thermal characteristics. And now we live in a world where you kind of have to always be doing both and thinking about both because it's just such small places with such hot dies that are interconnected, you know, in three dimensions. And so I think there's opportunities to ask, are there ways that we could help solve this or create a new user experience from a design perspective where like the electrical design and the thermal design is taken into consideration? But there are definitely opportunities. It's just about like that seam of how do you take your innovation and bring it to market and sort of get in here.

Both of those because both are important. From the logic perspective, there was a period of years where... one thing that people don't appreciate about Intel is they actually do a lot of the transistor research and development that enables future generations that the whole industry uses, TSMC included. And so Intel's always been really good at that early R&D and so they have innovations like from the early days of like strained silicon and using different materials as oxides and things like this. And so they've kind of always really pushed forward on stuff, but there was a period where they took some bets on some materials that didn't work and they ended up delaying jumping to EUV. I can't remember off the top of my head exactly why, if it was a cost concern.

That actually aligns with the idea that they had some more financially oriented folks who were driving the ship during those times and not engineering-centric folks. And so Intel, yeah, they sort of slowed down there, but it's also worth pointing out... so basically they were stuck at a particular node, 10, 7, and while TSMC was able to catch up and keep going and then of course everyone, AMD for example, is using TSMC and they're getting on, you know, smaller and smaller transistors and Intel was essentially stuck. At the same time, Intel has always been innovating on packaging and EMIB is one of their packaging processes that they use. But the idea is once you start to make chips that are really big and you hit the reticle limit, you might want to make two of them and tie them together. TSMC, their approach is CoWoS, chip on wafer on substrate. And right there you can hear it. There's like three levels there. Chip, wafer, substrate. So there's this interposer where the connections happen and Intel has historically had EMIB where basically just in the substrate itself they embed these little bridges and so they basically just put the connections in between the chips kind of right where you need them. The rest of it is a substrate and it hasn't gotten much attention at all until very recently. But one of the benefits of being an integrated device manufacturer is when your technology works and you have the scale of Intel's product team making CPUs for tons of laptops in every company worldwide, you have the opportunity to even take your packaging and put it in production and run it at scale. And so now when Intel Foundry is talking to potential customers who either just can't get enough CoWoS capacity, ultimately they might be gated by the amount of wafers they can produce at TSMC, or as CoWoS, there's different flavors of CoWoS, but some of the early flavors that were adopted they have particular trade-offs. And even as TSMC has come up with something called CoWoS-L which is local silicon interconnect and it's fairly similar to EMIB, there's still the three layers versus two, but there are some scaling problems with the way that CoWoS is implemented and Intel Foundry can, you know, essentially stand on the sideline and say, hey guys, we've got this thing called EMIB, it scales really well, we are already making it in volume, we're servicing all of our Intel products team and all of their products that they're shipping. So we know it works and we have spare capacity. So yes, as we move to these AI accelerators that want to just be as big as freaking possible and have all this HBM that needs to be, you know, interconnected, it's kind of the perfect timing for EMIB and Intel Foundry.

Yeah, these are awesome questions. So, you know, where do I get my information? I mean, I of course try to consume all the public information that I can, earnings reports, whatever. I try to when I can get my hands on like expert interviews, those are interesting. You always have to take those with a grain of salt. And then I have the opportunity in my day job with Creative Strategies as an analyst to talk to a lot of these like merchant silicon companies and talk with their executives. So sometimes I'll bring them on and publicly interview them or other times I'll get to talk to them behind closed doors and just ask them what's top of mind. So that is, you know, an awesome privilege. But I also am always trying to like talk to startups, see what people are saying on X, you know, again a lot of it public conversation. I wish I had, if there's one thing I could do more of it's continue to try to talk to people who are in the trenches as much as possible. Like obviously, you know, going to like OCP or conferences is like a really good place to, you know, meet people and have the opportunity to just ask them because there's, you know, I try to balance like, okay, what is the market narrative around something? What is the fintech narrative? What are the thought leaders like, what is Gavin Baker saying? But what am I hearing from companies, like what are they saying? And then thinking about like questions where it's like, well, the company said this, but did they actually mean this or they specifically didn't talk about this? So trying to find those gaps and noodle on some of those. And then I will say one of my advantages, I think, is that I don't live in Silicon Valley. And so I'm kind of like outside of the thought bubble of like, yeah, everyone's thinking this or saying this. You know, it's like I live in the Midwest and so we have a particular perspective on how the world works. And so, you know, I've worked in, and as you guys are experienced with as well, worked in industries outside of this and so just that kind of lived experience also I think gives you an ability to think critically and differently about certain things. And then I guess to Brian, to your question of the master plan, you know, I think what is really fun for me, so like when I was a product manager, what was awesome is like, ah, I like to think about strategy. I like to think about technology. This is so fun. Like I get to make a plan. What's our one-year plan? What's our three-year plan? And then you think about it for a little bit and then you just spend the rest of your life implementing it. So it's like a little bit of strategy and like a lot of execution. What I like about my current role working as an analyst writing Chipstrat and then I also host a podcast called Semi-Doped with Vic Shaker is I just like to be on the forefront as new information comes out and think strategically like, oh, what does this mean? You know, China's going to shut down access to indium phosphide. Oh no, is that real? What does it mean? So a lot of just that like thinking strategically. And then fortunately I've been able to build up relationships with executives and people at these companies to start to ask like, I think this means this for your company, like what is your answer to that?

Yes. So, you know, when it comes to disaggregation, we'll start with prefill-decode disaggregation because everyone knows that, of course, it actually makes a lot of sense. There's sort of these two subtasks and they have two different memory constraints or profiles — how much memory capacity and memory bandwidth they need. Prefill doesn't need much and decode needs a lot. So obviously right there you're like, wait a minute, we could have two different SKUs in our profile and then we could run part of the workload on one and part of the workload on the other. But then to your point, Gabby, now that means you do add some complexity — you need to orchestrate across them, you need to send information across them. And so I think right away that starts to point to like we're not talking about chip design, we're talking about rack design and ultimately data center design, and start to ask questions about like, well, what is the most power-efficient, most cost-effective, fastest way to send information back and forth? And you know, how should — you could go so far as to ask like how should we even architect the data center? Do you have like racks of prefill chips, racks of decode chips, or do you have a rack that has prefill and decode in it so we can talk back and forth? And that even goes to — that's like co-design even with like the model companies of like how are they taking their mixture of experts and splitting it across, you know, we've got these layers on this GPU and these layers on that GPU, this expert on this GPU, this expert. And I do think that like that's why it pushes Nvidia to just say like, oh, I want to try to get 576 GPUs in the same rack even if the power's crazy, if the cooling's crazy, if the manufacturing's crazy, because it's like you're essentially trying to keep as much of the compute as close as possible to make it easier to send everything over copper if you can. But it does start to become a problem and it opens up again opportunities to rethink like, oh yeah, but what about long context size and KV caching and a cache miss versus a cache hit? And oh, should we get clever and interesting with like having a rack right next door that's just a ton of storage so you can store the KV cache as close as possible? And I think that so we're obviously disaggregating things and saying like how can we have the proper skew for the proper workload need, but we're also realizing like, crap, there's trade-offs, and how can we also kind of aggregate it as much as possible and get them as close as possible. And so I think, you know, time will tell where other interesting innovations happen from networking to memory to cooling to continue to let us get more done faster but not waste as much power sending bits all over the place.

I'm most excited about physical AI, about embodied AI. You know, if you think about it, like yes, in the last 30 years, an increasing amount of our world's GDP is impacted by digital things — Netflix and Facebook and the internet and Riverside.fm and what have you. But so much of the world is still physical. Everything that you buy, everything that's manufactured — Apple, Nvidia — like it's all supply chains, it's all physical hardware, it's all getting moved, it's all getting shipped. Of course, I live in places where people are growing corn and soybeans and hogs and things, and just so much of everything everywhere you look. Obviously, anyone who eats anything like that had to get physically moved. And most of that stuff is still not touched by AI. It's barely touched by digital. You know, there's some logistics and things that go on, but literally like — I mean, I was at a large grocery retailer and we were using mainframes still, and this was like not a decade ago, right? And they will continue to. And you know, there's places where people are still using pen and paper. And so it's crazy to think — obviously just getting digital is one thing, but like, man, how can physical AI just unlock even more productivity? And it's not just like making what you're doing more productive — like saying like, oh, here's the way we work today, how can I be more productive — but like one of the cool things with anything digital that's AI-enabled, it enables some, you know, solopreneur to do so much more than they never could before. You know, oh, hey, I'm a domain expert. Like, I'm a dentist and I've always thought our software sucks, but now with AI, I can just write better dental software because I'm the domain expert and I can just do it. Send Fable off for a week — my oldest is in middle school and he's been spending this summer like coding an indie game and it's like super cool. And he knows how to code but not like that good, but Fable can just go off and do it and then he can spend a lot of time thinking about like the story and the graphics and the art, you know, and stuff like this. Now, imagine if we could take that and put it in the physical world. Like, I don't know, what if I could realize that I hate this particular part of my car? And of course, I could 3D print it and stuff, but like what if I could actually manufacture and assemble it and be like, 'Hey, I'm going to make this bespoke add-on piece to cars and start selling it kind of Etsy style or something, but it's actually something physical.' Or like in Iowa where I live, there's actually lots of small towns that are supported by like local manufacturing. It might be like snowmobiles or basketball hoops or whatever. But they suffer from brain drain like everyone else does, which is, you know, everyone going off to the big city and they don't want to come back and live there and work on it. But like, I don't know, what if you could use robotics and AI innovations, physical AI innovations somehow to make it so the same amount of people could get more physical work done, or they could graduate to managing fleets of humanoids instead of being the person doing the thing, you know. And it's obviously very fuzzy, but what I'm excited about is just like, man, it feels like there's so much like human productivity that's not unlocked yet. And there's so many like interesting capable people that have very interesting like mechanical or physical ideas, but we just don't have the tools for them to just build, you know.