Hello listeners and welcome to the third episode of the Semi Doped podcast. I am Austin Lyons with Chipstrat, and with me is Vik Sekar from Vik's newsletter. All right. So, this week was CES, and of course we had a big long presentation from Nvidia as always, and we want to unpack that and just dive deep, give you our thoughts, some interesting strategy insights and technical insights. And I'm going to start with Vik, our technical guru. So, Vik, what stood out to you?
Hey, hey, Austin. Yeah, this CES was like, "Where's the consumer equipment here? There's like nothing that we can use at home. Why is it a consumer electronic show? You're showing us all these like Vera Rubin platforms that I can't put in my house. I can't even buy one of those Grace Blackwell chips or Vera Rubin chips. I mean, why are you showing this to me?" Anyway, it was like the first time in 5 years they haven't introduced any GPUs at CES. So that was interesting. However, what I took away from the whole thing was when Jensen brought on those like cute robots, and I'm like, how much for one of those things? I just want one of those so I can just like wander around my house like super cute, you know? They're like Jensen pets. Nvidia NV pets, you know. I want some NV pets for myself.
Yeah. There's consumer electronics for you.
Yeah. Yeah. Yeah. I mean, and then he goes on to declare that this is the ChatGPT moment for physical AI. And I'm like, yes, give me a cute robot and I'm good to go. Anyway, before we get into talking about all this enterprise Vera Rubin stuff, what did you see that was even remotely consumer-based?
So, something interesting that I've been noodling on, and I just wrote about this. The DGX Spark is within the consumer price range or the home tinkerer. And for anyone who hasn't been following along super closely, you may remember Project DIGITS a year back or so where Nvidia said, "Hey, we're going to take a server-grade, cloud-grade AI supercomputer like Grace Blackwell and we're going to bring it into desktop form so it can actually sit on your desk." They ended up having a naming contest and they ended up naming it the DGX Spark, and this is a GB10 chip is what they call it. Grace Blackwell 10. So, it's definitely slimmed down from the GB100, but it is truly bringing Nvidia's server architecture to the desktop. And so this could be thought of a little bit as consumer electronics in that it may be the first time you could actually have true AI PC server really within reach. So, that's the closest I can get to a consumer electronics play. What do you think that a person who is a consumer, an enthusiast, would want an AI PC at home for?
In the CES presentation, Nvidia did have a video and I thought it was really awesome, and it was a dev sitting down at his desk and hacking on a personal project, which by the way, the first thing that stood out to me right there is if you look closely, he was on a MacBook and the DGX is sitting on his desk and it's plugged in. Of course, it's air cooled and it's connected to the internet, so it's not like a huge liquid cooled beast. But he was using his laptop as a thin client. And he was running a server, basically like a little chat interface server, so he could have his own little local chatbot. And then he set up a router where he could direct queries to different models. So like text queries to a fast text model, but he also connected it to a robot so he could ask questions and have it routed to the robot, which of course has a camera, so he could ask questions about his environment, like, "Hey, where's my dog at? Is it on the couch?" And what he did was he set up on the DGX Spark this router that could say, "Oh, that's asking an environmental question that needs visual information. I'll send it to the robot, have it grab a picture, and then give that image to a vision language model, a VLM running on this DGX Spark, and I would be able to answer the question like, 'Hey, is your dog on the couch?' And he shouldn't be." And then, of course, you could give it a command like, "Hey, tell my dog to get off the couch, right?" So, you could route, again, the DGX Spark could route it to the robot, and the robot could use its speaker essentially to talk to the dog. And what I thought was really cool about this is like, "Oh, this could be what the AI PC actually is, where you're hacking around creating really interesting things." Honestly, it reminds me of IoT when everyone was like, "Hey, now I can make my lights turn on and my garage door close and stuff like that when I wake up or when I go to bed." But you can imagine it being even more interesting and even easier to program because you can just use language models to talk to it. And anyway, so I thought this was actually a really cool demo of where I could see like I could imagine people hacking at home again, whether it's IoT or other interesting things, or definitely at work having access to this and being able to say like I truly want to automate some of my workflow. How can I use a DGX Spark or something like that to help me do that? So yeah, what did you think about the DGX Spark?
It was cool. Cool. I saw the pictures on like X. It had this like, you know, picture of this chip with these like liquid cooling in it and all that. It's not like the server-grade thing, but anyway, it was a very sleek black looking thing. That was awesome. And I was like, wait, what is the power consumption? Do you know any numbers of how much this would run at home? Because this is going to be a power bill nightmare otherwise.
So, okay, good question. So, there's two different products here. There's the DGX Spark which is this little tiny form factor mini PC thing, and then there's the DGX Station, and that's what you're referring to. And like Dell had one, and there's pictures on like go to Michael Dell, you know, on X and you can find it where the side is taken off and it literally looks like a desktop computer of old, but yes you can see like copper cold plates in there and like what appears to be maybe tubing, and so like it might be liquid cooled. Which now that product by the way has some sort of GB300 chip in it. There wasn't many details. Nvidia's DGX Station was like a sort of a reference design. Then Dell took that and made the Dell Pro Max. And for listeners, this is your GB300 chip. So straight up, this is server-grade CPU, Blackwell GPU, has HBM. So it was like in Dell's instantiation of it, it was 784 GB of unified memory. It was 288 gigs of HBM3 and then 496 gigs of LPDDR5. It's the NVLink chip-to-chip coherent memory. You know, like literally if you're a developer who used to have to do all of your local development, if you will, actually just remoting into a cloud to access a GB300 there, now you literally can have that at your desktop. But I will say to your question, Vik, I'm thinking like, okay, if this is like 1,500 watts or something crazy of power dissipation, which by the way, the DGX Spark is like way less. It was on the order of like 140 watts. If we're talking like 1,500 watts of heat dissipation, and if that's truly liquid cooling, I'm not sure how that's going to work at my desk at home.
Yeah. You know, so I looked it up while you were explaining this. I saw all the specs of this thing and you hit the nail on the head. The system power for the DGX Station is 1,600 W.
So it is quite the beast, and I think it's aimed more towards an enterprise-level deployment where you could have a smallish inference server running that is running HBM, serving multiple users and things like that. Maybe it's not really meant for the home hobbyist, but that's what the DGX Spark is for.
Yes. Yes. Okay. So, okay, I was writing about this, too, and I've been thinking about this, and I think you're exactly right. DGX Spark, you know, that's going to be on the order of like $4,000 or something. That's, you know, definitely like high-end hobbyist or still useful at work, like totally useful in the enterprise. DGX Station. The way I'm framing this and thinking about this, it reminded me back of the 1960s era where people had big IBM mainframes that cost, you know, million dollars plus. And at the time, and remember this is 1960s, right? So at the time there weren't that many actual enterprises who could afford it, but the ones who could, wow, you got access to frontier compute. That's amazing. Then we started to see the rise of the minicomputer which was like 10x cheaper mainframe-class compute, but now all of a sudden it's like 100k, and that actually expanded the total compute because now suddenly large enterprises, midsize enterprises could buy this. And of course as these form factors, they're still big but not mainframe big, but as the form factor shrunk over time and the cost came down a little bit over time with competition, you could literally have a mainframe at your desk. You could literally have individual teams that had their own minicomputer instead of one big mainframe shared across the whole company. And so when I've been thinking about the DGX Station, I've been thinking about that as an analogy of like, hey, today if you want a rack of AI compute, I hope you have 3 million plus, you know, and again, maybe also a liquid-cooled environment to put it in or whatever, you know, like hyperscalers do and some very large customers do, but a lot of people don't. So they're still stuck renting GPUs or they're just locked out of having AI supercompute, but surely there's latent demand there. And so when I think about the DGX Station, I think about you can literally get a Grace Blackwell and instead of needing seven, you can have one. You can have one, and I don't know how much it's going to cost. Is it $50,000 for the Dell Pro Max? Is it 100,000? I don't know. But you can take, you can unleash the supercomputer from the hyperscaler cloud and you can actually put it in your building. And I don't even think if it's liquid cooled like to the point the form factor looks like a desktop that you could put at your desk, but maybe you're still actually just putting it in your on-premises environment where you already have some cloud or like not cloud but some local AI compute racks with like CPU servers or something, and so maybe it's like in a closet or in a back room somewhere, but you're taking the AI supercomputer, unleashing it and putting it in your own business. So now all of a sudden maybe midsize enterprises can afford one or multiple. And to your point, I envision like why not run a bunch of local inference on it. Like I don't think of it as necessarily only like a very beefy, very expensive single-dev ML researcher workstation. I actually think these, you could call them like an AI appliance, if you will, like local AI appliance, could sit in a room and actually serve all sorts of interesting inference for local use cases. Especially if you want to have your own private data or you just don't want it maybe accessing the outside world or whatever you want, you just want to have control over it, why not just run that locally? And by the way, of course, I think CFOs would be super interested in this because now they're thinking opex of renting GPUs converts into capex. Like there's all sorts of interesting benefits here.
Yeah, absolutely. And that's the nice thing about things that were mainframes becoming a computer on your desk today, because the phones that we have today are an extension of that, because these mainframe-like computing powers are now in our phones and pockets. So in that same analogy, maybe in the future we'll have phones that can run complete inference or watches that can run complete inference. Maybe that is the use case for edge AI, and maybe we're not just there yet and it takes time to get there. But you see how your argument, when stretched to the extreme on the AI inference front, can go to edge AI devices in a very nice way.
Totally. And you know, in the interim, someone's going to ask, "Oh, okay. Well, the counterargument is like, oh, are you going to actually put HBM in your watch or your phone? That sounds pretty expensive." But I do think we could live in a world right now where maybe you don't need to, because maybe if we have access to these like inference servers at work, I mean, maybe we can again use our laptop or our phone as a thin client. And as they showed in this demo, which everyone should go watch, there's like a 2-minute video you can find on YouTube. And until it becomes like cost acceptable or the thermals get figured out and all those implications, maybe the AI that we run locally on our laptops and on our phones is still more of like the real-time speech recognition, real-time stuff. And then maybe this heavier agentic stuff, you can still launch it from your phone or your laptop, but it's actually still running on some plugged-in compute, but it just happens to be cheaper and locally accessible.
Yes, folks, check out DGX Spark. Check out DGX Station. Go read my article on Chipstrat. Noodle on it because it is super interesting to think about this additional compute TAM by bringing the supercomputer to the desk. And you might ask, is this going to cannibalize some cloud use cases? And I would actually say in the aggregate, no. Of course, yes, there will be some, but I actually think there's a lot of interesting new workflows that could run locally that are just accretive to everything you're going to already do in the cloud. Because think about it, the cloud is always going to be frontier compute and it's going to run the frontier model. And what we run at the edge is actually like the frontier model from 5 years ago as it has trickled all the way down. So to do the most interesting, most demanding stuff, long-form video generation, whatever, you're surely going to still need racks of GPUs to do it. But over time, we will continue to improve in the amount of AI that can be done locally. And so I think there's just new interesting use cases waiting to be unlocked. Now, enough there. Let's move on. So consumer electronics. One other thing that you could start to call consumer electronics would be the car, right? Is the car not like a Rivian or a Tesla? Is it not just a computer on wheels? So, I'm stretching here a little bit, but was there anything from Nvidia around autonomy or automobiles? Vik.
Yes. Alameo. So, Alameo is Nvidia's answer to fully self-driving vehicles, kind of the ones that you find on Tesla vehicles kind of. It is what they describe an open-source autonomous vehicle platform that uses something called a vision language action model. So the way a vision language action model works is basically it takes sensor inputs from the environment and then it processes it through kind of a language model in the sense of it's going to do reasoning and chain-of-thought reasoning, and I'll explain a little bit more of what these kinds of things do, and then it takes action. So the action is basically in the form of trajectories or the list of possible actions that it can take in the future. So those are, you know, the kind of decisions it needs to make based on the vision inputs it gets from its sensors. Right? So the VLA model is actually a 10 billion parameter model. The way it works is it takes all the sensor inputs and it uses the LLM under the hood to kind of think through how a human would think through the whole situation, and there was like a couple of like videos on this as well that was really nice at the keynote, and it basically thinks through with the way you and I would, you know, maybe if we speak out loud while driving, this is how we would be like, or telling our, you know, if when our kids get old enough to drive, they'll be like, okay, this is what we would tell our kids: hey, look out for the dog that's on the left. Make sure that you are ready for when it runs across. Or hey, slow down. There's somebody crossing the road. Or accelerate because the light just turned yellow and it's safer to accelerate than slow down. So, you know, these are the instructions that you would probably give with language to somebody you're teaching or somebody you are, or even maybe to yourself if that's how you drive. But that's what the VLA model does. This whole thing is going to be first tried out with the Mercedes-Benz CLA and is apparently hitting the roads in Q1 2026. So, I don't know if you see one, let me know. It's kind of cool. In response to all of this, I think Elon said like, "Oh, this is great." And actually to Jensen's credit, he did acknowledge Tesla as like the most advanced driving model on the road today. And so Elon was like, "Yes, this is good." But it's the getting it to 99% that is the easy part, but handling all the long tails of self-driving, which is when you say long tails, if people aren't familiar with the phrase, it's all the unknown unknowns. Basically, you just don't know what's going to go wrong on the road. So there is a whole bunch of scenarios which you just cannot account for. And that's the hardest part of self-driving. And Elon says that is the part that has taken the most time to actually get right, and that's at least a few years away. So people are like saying okay, Alameo is still like maybe a few years away from where Tesla self-driving is today.
Yeah, definitely there's a ton that we'll have to unpack here in its own episode. But I definitely was interested to hear that there was this Mercedes-Benz car that's going to hit the road. So obviously, oh, and one other thing somewhere, I can't remember if it, I think it was in the CES presentation, Jensen said something like they have a quarter of their company working on, it was some large number, maybe it was just like 7,000 people, but some very large amount of people working on autonomous vehicles. I'll have to go find that somewhere to figure out what the exact number was, but it was a surprising amount of Nvidians, if you will, working on autonomous vehicles. So, I think we'll only continue to hear more on this front, especially if Jensen thinks, you know, it's the ChatGPT era of physical AI and self-driving is nothing but physical AI.
Yeah, absolutely. I think we will hear a lot more about this going forward. And they have this whole thing that is the interesting thing about all this is all open source, and I was reading the website and they said, you know, they have this entire data set of all, you know, driving scenarios and vehicle dynamics, all of this stuff is actually on GitHub. So if you go to nvab/alpasim you will see like there's like a whole lot of stuff that you can use to do your own kind of like work on autonomous driving if people are into that kind of research. Not saying I can particularly do it, but it's amazing that they put all this out there for people to work on and improve. Obviously, it's a pretty hard problem.
Yep. Get yourself a DGX Station, download this GitHub repo, and off you go.
All right. Should we move on to the Vera Rubin platform even though it's not consumer electronics? Yes, we should do it because you know the bulk of the whole keynote talk was the Vera Rubin platform and it was pretty amazing honestly I have to say. The Vera Rubin platform actually consists of six chips, right? It has the Vera CPU, the Rubin GPU, and they have a data processing unit or DPU which is on its generation 4 BlueField DPUs. They also have an NVLink switch which is now on version 6 for scale-up networking. NIC, which is network cards for, you know, networking within and outside the rack and the chips and all that. And then you have the Spectrum 6 photonics, the first photonics, you know, Ethernet switch for scale-out and in between rack. So all of this, like six of these things make up the Vera Rubin platform, and the fundamental claim is that the Vera Rubin platform provides a cost per token of one-tenth that of Grace Blackwell. Let me quickly add in one thing. What's interesting is that Nvidia used to say they made seven chips. So they say hey, we're not just a GPU company. We make seven chips and write software of course, and now they're saying six chips, and the one that's missing is the InfiniBand switch. So normally it was CPU, GPU, DPU, you know, NIC, and then it was we make two switches, one is Ethernet Spectrum X and one is InfiniBand switch, and here they specifically did not call out the InfiniBand switch. I just found that was interesting. I don't necessarily know that that means that they are going to stop making InfiniBand, but I think it does go to show that their customers have all been opting for Ethernet-based switches for scale-out because people are familiar with Ethernet, and therefore at a minimum from a marketing perspective they're just going to focus the conversation on that. So there's just an interesting observation.
That's a great one actually because maybe this is a subtle hint of where things are going in the future.
Yeah. Let's talk about some of these chips. What do you think?
Yeah. I mean let's dive in for people. We'll get a little technical here, but we can explain why it matters. But yeah, what did you hear from Vera and Rubin and maybe how did that compare to Grace Blackwell?
Yeah, the Vera CPU actually has more cores than the Grace CPU. And interestingly, this is the first CPU from Nvidia that uses their custom Olympus cores which have been like in-house customized. It's ARM based but it's ARM compatible but it's not an off-the-shelf ARM core like the previous generation Grace CPUs which actually used 72 ARM Neoverse cores. So there are 88 custom cores now versus the 72 ARM cores in Grace. So that's the one big difference. And this Olympus core actually supports spatial multi-threading which means it takes in two threads per core. And therefore now you effectively have like, you know, this is basically the multi-threading thing we saw with CPUs like in the 1990s or 2000s or whenever that came out. So now you have 176 threads running actually. So the real jump is from 72 to 176 which is kind of significant when it comes to computing power on the Vera CPU.
Interesting. Why does that matter for a GPU? Because at the end of the day all the matrix multiplications are happening on the GPU. So but what's going on on the CPU and why is even more simultaneous compute useful?
