The Abundance Economy Is Coming Faster Than Anyone Expected w/ Emad Mostaque

You're telling me that running a lot of these agents and programs is going to require very little memory and compute, right? Like you're comparing to a video game. Does this mean we will not need stronger personal computers anymore? So if you got a MacBook, the amount of tokens that you're producing at home, it's fast enough because you're usually like 4 billion, 3 billion, 8 billion active parameters. So a billion parameters is like a gigabyte. So if you got 8 GB of RAM, you got an 8 billion parameter model, 7 billion parameter model in there as an example. This is a short hand, right? It's running at probably 30 to 40 tokens a second, which is about as fast as you can read, and if that's crunching around the clock and you can be offline, that's all that you ever need. Even for a base level, if you're running on a 5090, then it's going to run 10 times faster, right? So the throughput speed is fine. The main issue has been the RAM, and this is why you saw such a big boom in the RAM stocks, because what happens is the models used to be what was known as dense, so you had a 27 billion parameter model that needed 27 GB of RAM. That's again approximate, right? We figured out how to route them through something called mixtures of experts. So something like a GLM model, the new 5.2 that people have been talking about, is like 650 billion parameters. So it needs that much RAM, but the active parameters is only 50. Normally those two would match. And the more parameters you get, the slower it gets, whereas now you're seeing like 200 tokens a second or something like base 10. So this means that you'll probably be RAM constrained versus chip throughput speed constrained, because again you don't really need tokens faster than you can read for most things. But right now we still haven't figured out how to get these sparse models that need lots of RAM down to little RAM, because the performance drops as you kind of take that extra routing out.

Well, the processing speed of those particular active parameters is fast enough now even for consumer devices.

Um, it's just the extra RAM needed to hold all those extra things that you might need to route it. But then when you get to specific use cases, like last year we trained a medical model that outperforms human doctors that works on a Raspberry Pi. You don't need that much knowledge to actually be a doctor model. You don't need Reddit data, you don't need to know about philosophy or Taylor Swift or anything like that, right? So again, these specialized models can get really efficient, and then you have generalized models and then you have coordinators around them. So there'll be all sorts of things. But the base thing is you don't need faster silicon. Even with silicon, you see the Vera Rubin chips coming from Nvidia and all these other chips. The example I give to all listeners is, if you go to chatjimmy.ai, have you ever seen chatjimmy.ai?

Uh, it's always funny. You can type that in.

Okay. Yeah.

All right.

And then you hit enter.

It's instant.

Yeah, it generates basically the generation speed is slowed down by the speed of light of the font going back and forth. So, chat Jimmy is by a company called Talis that etch the models onto the silicon. So, you have these general purpose compute things, but really models are just a series of probability weights. So, something like a Claude goes at like 50 tokens a second. This goes at 16,000 tokens a second. And so you'll start to see these types of chips emerge next year and it's much cheaper because you don't have all the moving parts but you're locked down to a specific model. So as models get good enough fast enough and cheap enough and people be talking about this from like, well how long can we run a GLM 4.2, 5.2? Do you need better than a tablet? You'll see it actually drop down to the cost of silicon almost right to run these models and that gets a bit crazy when you think about it.

Yeah, that's Llama 8B models, so it's not a smart model.

Um, but again, they're going to do larger and larger models. So, you'll have GLM on that speed. So, 16,000 tokens a second GLM 5.2, Fable Opus level model. That's insane. Again, you can't even absorb that speed. We read at most like 60.

So the inference providers will be providing like one zillion tokens a second. Your local computer is providing like 40, and that kind of covers most tasks between them.

Yeah. I mean it's one of the interesting things in that you look at the top performers who use these models like they're not working less. You actually get into this weird kind of mania where you're constantly reading and iterating especially if you got ADHD. It's very tiring but you're not reading the base stuff anymore. Instead you're kind of going one abstraction level above. And it's kind of like when you've got a team as a company, right? you start hiring people and they do all the base reading and then you see the chewed up version of it and if you're not exercising certain muscles they atrophy and then other muscles adapt. So the muscle that adapts I think I kind of view this as type one versus type two thinking you know in Kahneman style. So you got one type of thinking which is like I'm really thinking from first principles and I'm thinking things through and then you're like oh there's a freaking tiger in the bush you know like I think that this is pushing our thinking towards there's a freaking tiger in the bush because you can rely on the AI to do much of the basic crunch work and thinking and analysis but then you lose those skills and you become almost a constant pattern recognition person and I'm not sure if that's going to be that healthy to be honest because we'll rely more and more in it. And I think the instructive example of this is like how many people can even read a map anymore? You know, you just use your Google maps or your Apple maps and whatever.

This is true. You know, it is true. A lot of people never actually thought properly, you know, like.

Human human moments right nostalgic.

Well, you know, like what is the meaning of life, right? It's a nice easy one. Life is kind of what you make of it. And I offer some suggestions there about what won't work. Like, hey, I'm an accountant. Oh, that's not going to be good, right? Like if you have that as an identity. Um you can see all sorts of other things. But I mean ultimately we're moving from scarcity economics to abundance economics. But the abundance can really slap people down if we don't allow people to be a part of this. This is the key thing because I think what you need to articulate is how do we build a good Star Trek type future, you know, not Star Wars. So Star Wars literally has trade wars, you know, imagine that, it's got tariffs and scarcity all over the place. Whereas Star Trek is this abundant great future where you're exploring and you've decided where you want to be. Like why should anyone go hungry? This is a coordination issue. We have great coordinators coming. You know, the future of the economy is a billion zillion robots. Like I bet you have an extension that you want to build. You can't be bothered to get construction folk in. An army of robots comes in by drone, builds the damn thing, and then disappears, you know? Like this is going to be cool. Long cement. Because you can use that and create all sorts of things. So I think that ultimately meaning goes back to what it was, which is, you know, if you're part of a strong community, religious, cultural, family, or otherwise, you're not going to be as affected by this as opposed to if you're like, I am more intelligent, smarter, capable than other people. Because you're going to have a new entity that is more intelligent, capable, smarter than you kind of coming. And the isolationist kind of nuclear family that we've had is going to be very difficult to do that. As well as there is no safety net that we can have for the type displacement that we've got. The practical example I give of this is the truck driver in America. So there's a million truck drivers upon whom two to three million jobs rely. They will be replaced by a Tesla Optimus robot opening a truck door and getting in and driving the truck away. And that Optimus will be a buck fifty an hour and boom, you got a couple hundred billion dollar economy gone. What does that truck driver do for identity? Well, if they've got a strong church community, otherwise then you know they're good. If they can be given shelter, that's good. But they're not going to retrain to be a programmer when programming doesn't even exist. Like what do they turn to? So I think long term we have to even really rethink how does value flow in a society and what do we award? Because the classical kind of capital structure had this link to labor that is now broken. Because in a few years time it'd be like why do I need to hire a digital worker? I got a literal digital worker. You know, why do I need to hire a human to do this? I got my robot. And so yeah, this is the real danger and this is why we need to have big discussions about this type of stuff.