So check this out. This week was actually the release of Grokbot, which is an agentic AI platform. So you think about it like an OpenClaw, but extremely easy to use, so that everybody can run agents all the time right now, even if your computer is closed. So this is a very important development, I think, because it's not so much for what Grokbot is as a product, but what it means from an underlying substrate of compute, and specifically what it means for CPUs.
All right, hello listeners. Welcome to another Semi Doped episode. I'm Austin from Chipstrat, and this is Vik from Vik's Newsletters. And if you're watching this on YouTube or something, you'll notice Vik is in a different location. So Vik, tell us where are you? And then I also saw that you are trying this Grokbot, which I saw on X, but I have not tried. I honestly don't even know anything about it. So I want to hear where are you, and then tell me about Grokbot because I know you've been playing with it.
Awesome. I am in the Bay Area. I came here for like Hot Chips next week, which is very exciting. A lot of people are coming down for this conference, but I decided to show up a week early because I was actually in Taipei attending the Open Compute Project APAC, like the Asia-Pacific region version of this conference. And I was like, yeah, I think it makes just sense to come here a little bit early because I've got so many people to meet in the Bay Area. It's been amazing. I've met so many amazing people and so many amazing things have happened that we should eventually, it'll all come out at some point and we'll discuss it on the podcast or whatever when the context is right. But yeah, it's been an amazing week and so I am in the Bay Area. I got my little camera rig with me, but I don't have my Semi Doped sign, which I'm very bummed about. So I'm going to travel with my neon sign next time.
Yeah, so like Grokbot, Grokbot is an interesting thing because this is an agentic AI software. Like I don't know, they call it an agentic harness is the term for it. It's a piece of software you just download off the website and you install it and you got an interface where it's pretty much everything works out of the box, right? So what you do is the first thing that comes up and you go, oh, hey, here's the, you know, what do you want to do? And I said, hey, I want you to be a chief agent and then you're the chief of my agents and you're going to talk to all the other agents. And then it's like, oh, what other agents do you want? I'm like, oh, I want an agent that does this. I want an assistant agent. I want a, I don't know, a news monitoring agent, something like that. And it just spins it all up automatically, right? It's fancy. It's really nice. So I think this will open up the doors to a lot of people starting to use agents now because the whole OpenClaw approach and the Hermes approach was like a little bit difficult. Like you got to install it and you got to go into the command line and you've got to set up an API key and you've got to set up all of this stuff. It is a little bit involved for everybody to use, but Grokbot takes care of everything.
Nice. Amazing. Okay. So like OpenClaw and Hermes, however it's pronounced, those ran locally on like your own machine, right? And this Grokbot, this must, how much of it runs locally on the machine versus like in the cloud? You said you downloaded something.
Yeah, this is just a piece of software that essentially communicates with a computer in the cloud. This is the nice part of it, right? Which is what I really like about this is it is actually running your own little dedicated computer in some server somewhere. So you have like a little virtual machine that's running somewhere and everything is within that box. All your tools are installed there. All your sign-ins are installed there. Like you don't need all these MCP connections and these plugins and those plugins. It's just think about it as your computer but somewhere in the cloud. It's a VM. It's a virtual machine, right? The nice thing about this is that all of the security practices and everything that matters for these agents to run in its sandbox environment is all contained within there. So if your agent has like signed in into one tool, it is available easily across all your agents and everything because it's a local computer. That's the difference compared to like OpenClaw and Hermes.
Nice. Amazing. That does sound so convenient. So in this, would you say it's better than just like having a bunch of cloud code agents running locally on your machine that you have to manage?
So it's a give and take, right? There's a lot of control when you have your own cloud code agents. You can specify a whole lot of stuff to the finest detail, like which model you should use for doing what, and you can token optimize and be like saying, hey, for these queries you can do, you know, so you can set up OpenRouter as a model provider through the cloud code interface and you can in OpenRouter you can say, okay, use the auto router or use this hierarchy of models such that these requests are routed here and those requests are routed there. This takes all that out of the equation. Like you just, you don't know what's happening. You just have tokens and you use those tokens.
That's it. You don't know what's being used.
You know, I think the easy button is how it's going to go. Agents will go to the masses where they don't need to go buy a Mac Mini. They don't have to install something. They don't have to know what OpenRouter is, or which maybe is getting acquired by Stripe now. Yeah, but they don't have to configure and token max or, you know, get token efficient, token min. You know, they can just hit the easy button. So then does that mean I should buy a bunch of CPU stocks if Grok is going to spin up all these VMs?
We'll get to the not financial advice part really quickly. But you mentioned Mac Mini. So I just wanted to touch upon that. Like there was this when agents were rolled out not so long ago. It was like only December last year that we even heard of this thing. It was such a rush to get compute that even Intel was caught unawares of the demand for CPUs. Right? In their earnings call in February, they said, 'Oh, we have a lot of CPU demand. We had no idea it was coming.' So the question is why did all that happen? If people observed keenly enough at the time, you could kind of tell why what's happening because the moment agents showed up, everybody ran and bought a Mac Mini. So what exactly were they buying? Were they buying a Mac Mini for its memory? Were they buying it for its NPU-based, you know, the graphics processing they have in there? Or were they buying it for just compute or a CPU? Right? So that question if you revisit and ask yourself why people bought the Mac Mini, that will tell you something. And the reason is people didn't want to run OpenClaw on their existing laptops because you're giving it access to your whole computer and that's dangerous because you don't know what it's accessing. You don't know what it's going to do with the information on your computer. Does it access sensitive information, medical records? You have no control. So the idea is what if I could buy a separate machine, a Mac Mini, keep it only for this purpose, install OpenClaw on it, and then have that as my agent computer. And then there I don't put all the sensitive information. I give it only what it wants and then it works all the time. Now the problem with that setup, and this is familiar to anybody who's run a home server setup like I've run one for like 10 years, but keeping that up and running is a real big pain because you are responsible for power, right? Like if there is an outage or if your computer's, I don't know, like some part, the PSU blows up or, you know, something happens, power goes down, or if your internet is out, that's a problem because you don't have connectivity into your own machine at home. But you have to set up networking. Like how are you going to access your PC at home when you're traveling like I am right now? So you've got to set up some kind of a VPN network. Like we, you know, you and I set up Tailscale for all this stuff all the time. It's not that simple for everybody to go off and be like, 'Hey, if I tell my mom, hey, mom, all you need to buy is this Mac Mini and then set up Tailscale and, you know, set up a UPS so power doesn't go down and have some failover internet so nothing happens.' She'll be like, 'I'm out. I can't use this stuff.' Okay, you keep your agentic PC. I'll just go back to my regular laptop.
Yeah, 100%. Yes, it is all about the easy button for 99% of people. Seriously.
Yeah. So that was the problem. So people were buying these computers just to sandbox their environment and have its own, you know, area for agents and stuff to run. At that time itself I thought about it and thought like, wait, what is the right way to do this? Like do you have to buy a PC and go do this? No. At the time too, what you could do was you could get on a VPS, you could rent out a little virtual machine on a server and you could install OpenClaw on that one and then you just go off and run it on the cloud, which is a much better solution because you don't actually have to buy a, I don't know, $600, $800 machine, which got so much more attention because like everybody's buying it. So it was like a craze and prices went up and all that. Assuming you don't want to run models locally because some people actually bought the Mac Mini, the high version with the maxed memory version. Great decision in hindsight. I think you could pull out the memory and sell it for a profit compared to the cost of the machine itself now. But consider it an investment. But yeah, if you wanted to run models locally, you could do it, right? But I don't think models could all be run locally. They're really big. Say you want to run a frontier model like the Fable-class model with your agents, there's no way you could do that, you know.
Right. Right. Yes. I have an agent computer by the way and I do run a few models locally but it's obviously smaller things for the non-frontier tasks including like audio, basically like transcription, you know, voice-to-text. Like so I'll run this podcast through on that when I post it to our Semi Doped newsletter to give the transcript to people. That's like the simple kind of thing that can run on a local box. But for most people when they want to spin up agents and they want to do things, I think that there's a lot of work that you would want an Opus or a Fable quality, you know, frontier model.
Yeah, it depends on the workload. You know, you might want a better quality model. So what is the right way to do this right now? That's what Grokbot does. All of this. It does all of this for you because you don't actually have to when you get Grokbot. This is not a sales pitch and they're not really a sponsor or anything, but what problem it solves is all I'm trying to articulate. I'm not suggesting anybody go and I'm not shilling this product, okay? So don't get it. Actually, it's $200 a month. Most people probably won't get it, okay? It's very expensive. So not saying you should. But what Grokbot solves is you just get the virtual machine with all the integration tied in. It asks you when you install like, hey, what integration do you want? I said, I want my Google Calendar. I want my Google Drive. You know, some of these things you can just say add, add, and all those things are natively installed and ready to go for you. You don't have to do anything. Like there's Canva, there's Figma, there's so many tools out there. You don't have to set up each one. So you set it up there into your Grokbot instance and it's available across all your agents and then it runs on a virtual machine and you don't have to be responsible for anything including the security of that virtual machine because that's Grok's problem. I hope they have experts taking care of that. I don't have to become like a cybersecurity expert now.
Right? I mean, I don't know. I thought Elon fired all the people at X or slimmed it way down, but anyway, carry on. I'm sure they've got their zero.
Maybe they don't. Okay, fine. Then deploy Fable to be your security expert.
Got to be a creative problem solver. Come on. Yeah, whatever. Like all this is, no, you don't have to worry about anything. You get a virtual machine that sandbox security is supposedly insured. Then you've got all your tools installed. You run your agents and it's amazing. So now we go to the question of CPUs. So do we need them?
Right? Like let's now go into the question of why do we really need CPUs? Because we never even mentioned GPUs in all of this stuff. Yeah, sure. For doing the inference, we want to do GPUs, but we really haven't even touched upon what CPUs have really done for us in the last 6 months since agents showed up and I thought that's what we should dig into now.
Let's dig into it because from your example, okay, so we talked about running cloud code on like your laptop. So there's going to be work done on the CPU there as well as calls to a GPU in the cloud. We talked about running OpenClaw on a Mac Mini. So now you've got another CPU on your desk. It's not your laptop. It's your Mac Mini. It's in its own little contained environment. It's got a CPU and yet it's still going to the cloud for the GPU. And then of course now you're talking about using Grokbot, which is actually spinning up a VM in the cloud. So a CPU in the cloud. And so it's a CPU in the cloud talking to a GPU in the cloud and then communicating results back down to you. So there's actually, and we'll get into this, but what I'm alluding to is there's different work happening. In some scenarios it's happening on your local CPU, some it's on your home server CPU and others it's on a rented cloud CPU. And so there's obviously interesting implications, but yeah, let's get into it. Let's first talk about what do the CPUs even do versus what is the GPU doing?
Yeah. Yeah. Before the show, we were talking about a nice analogy we could come up for this. I'll let you explain that analogy because this is a very nice framework for which people can think about where GPUs lie, where CPUs lie and how we should think about this going forward.
Yes. Okay. So ultimately I think of a good way of where the value happens is the GPU. The GPU is like the genius, you know, now it's like the genius that has a PhD in everything which of course you don't always need. But the GPU is like the genius and then at the end of the day once the brain, the genius thinks of all the work that needs to be done, which I mean this is what I'm doing with Fable is I'm trying to say like, you know, give me your very thoughtful approach or Opus 2. And then once it is decided like what work needs to be done, ultimately most of this work, or it depends on the workload, but a lot of the work can actually be given to like assistants, like little doers, little task carry-outers. So ultimately, you know, I think about the GPU as like the genius that is doing all the thinking and delegating work and then ultimately the CPU as the assistants that once told what to do, they can go carry out all the work that the genius told them to do.
Yeah. So ideally, you don't want a genius doing all the work. Like the genius is thinking about it. The genius has paid a lot of money per hour to come up with these brilliant ideas and you don't want the genius going off and, I don't know, putting paperwork in the drawers or you don't want the genius looking up some information from a file that's in the drawers or you don't even want the genius going out to fetch the mail. Like ideally in an ideal world you want to have the absolute genius, aka the GPU, just doing what it does best. Actually there's one, we can break this analogy down into a little bit more detail. So if all the workers are like CPUs, like you know, CPU cores doing their thing for example, I think we can break down what kind of worker that is too because there is something called the head node CPU and the analogy to the head node CPU is an assistant to the genius. The assistant to the genius, which is the GPU, the job of this assistant is to keep handling the genius' work. You don't, you know, if the GPU, if the genius needs coffee, you give the genius coffee. If the genius needs water, you give, the genius doesn't stop working because it's a lost, you know, you're losing money if the genius stops working. So you want to make sure that once the genius is done with this task, the next folder of materials is ready for the genius' work to be done. And the host node CPU will be like, 'Here you go, sir. This is what you got to do next.' The genius is like, let's go, let's do this. So that's the host node and it has to have a, I'd say it has to have a pretty fast per-core performance because it's constantly monitoring what the GPU is doing. Are you done? Are you done? No. Okay. Like let's get this ready. Let's keep it ready. As soon as the GPU is done, this has to hand it in like, you know, no delays, no wasting time. Do not waste the genius' time. So that's the host node CPU.
Yes. Yeah. I love it. I love it. It shows that you want like a high clock speed or a very responsive CPU but also obviously you want like the communication between the CPU and the GPU, between the assistant and the genius, to be as fast as possible, right? So that, you know, there's no like, so for example, ideally you wouldn't have the assistant like be in a different building and then they have to come over and check in and then walk back to their building and then come back and check in, right? You want them like in the same room talking with each other. And so maybe to take it like a layer deeper, there's differences between a coherent host and a standard host with these host nodes. So like a coherent host would be like Grace Blackwell where you've got like the GPU and CPU and they can talk over a shared really fast network. So like NVLink C2C versus like a standard host where, you know, maybe you have just some Hoppers and you take an x86 processor and you put it on the board but it's still talking over PCIe. So that would be a little bit more of an equivalent to like, you know, they're in separate buildings or separate rooms.
So the chip-to-chip, the C2C protocol that the GPU and the CPU uses, the coherent protocol, the coherent CPU would be somebody who's standing right next to the genius and looking over. I'm like, oh, here you go, here you go, you know.
And if they, yeah, exactly, and if they have like memory coherency and then it's like I can see the, I'm the assistant and I can see the genius' notes and I could like jot down on his notes too or something.
Yeah. So I was thinking about HBM is in this analogy and this is where we push it too far and we shouldn't but we'll do it anyway. But I think of HBM as the stack of papers next to the genius' desk, right? The genius is not going to get up off the chair and go and get it from the cupboard or anything. HBM will be the stack of papers on the desk and the genius can just take it and keep working on it and putting it back on or whatever. The memory coherency comes from the assistant having access to the same pile of papers on the desk because they're in the same room. So if the assistant wants, can also access the memory directly.
Exactly. Exactly. When you zoom out to your point, this CPU assistant, this host node, it has one job and it's keep the GPU fed. And that's ultimately what the host CPU needs to do. Keep the GPU fed. So then it starts to raise the interesting question that made a lot of sense in the ChatGPT era where everything's just a chatbot and it's just like, yo, I'm asking questions and it's just like, just keep the genius fed so he can respond to questions. Now all of a sudden when there's all this extra work to be done, like the genius is spewing out code and you need to compile the code, see if it compiles. Maybe you need to run the code. Maybe the genius is like, I need information from 50 different sources, go, you know, hit SEC filings and go Google the web or search the web, you know, and go do all this stuff. Now all of a sudden, the question is, can that CPU assistant go fetch all these filings? Can it take all this data? Does it have the memory capacity and bandwidth to analyze all this data and keep the GPU fed or is this too much work for that assistant standing right next to him and do we need an army of other CPUs to help?
Yeah. Yeah. So this brings us to the concept of what is it? I don't know. Is there a thing called an agentic CPU? I don't know. That's like a marketing term probably, but I think it makes sense in this context.
