DeepSeek's R1 Breakthrough and Implications | Sharp Tech with Ben Thompson

Takeaway number two: DeepSeek employed a new training method for DeepSeek-R1. R1-Zero could represent a paradigm shift in how we think about model development going forward. We discussed that R1-Zero is more important than R1 because R1-Zero developed the ability to think independent of human guidance. It is just pure reinforcement learning. Again, the limitation with R1-Zero is not that it's not a good model; it's that it wasn't very human legible. It had poor readability.

And so one of the things they did with R1, for example, is they introduced a reward function to make sure it always stayed in the same language, because R1-Zero will switch between English and Chinese and all this sort of thing, which is a readability problem. That came at a performance cost—it made it perform worse in some respects. There is a real bit where making these models human accessible actually makes them worse. Think about it in the context of: the more RLHF you do—and this has been documented for years—the more you decrease model performance, because in some respects you're asking it not to follow its core learnings, its core weights.

And so, just in general, R1-Zero and this idea that everyone's been trying to do pure RL for these sorts of things, but they did get it to work. That, to me, in the long run really reinforces this aspect. The whole distillation thing reinforces the power of synthetic data, if you like—the signal amidst all the noise. Number two is this RL, which is the model can make itself better without human intervention, and that means much more scalability in the long run in terms of improvement, much more of a virtuous cycle in terms of AI making AI better. Again, it's very appropriate: we've spent the last few months talking about the Bitter Lesson. It's a reinforcement that the Bitter Lesson holds even for thinking.

The big takeaway here, again, we've been on it: the economics of the absolute cutting-edge model has been questionable for a while because of the distillation issue. It's much cheaper to serve. Who is going to actually create the leading-edge model if everyone else is going to make money off it except for them, and you don't get a return on your core initial investment? We talked about that a lot with Nat and Daniel last week. There's a bit where just the belief and pursuit of AGI and that it's going to break away is critical and essential, because the economics of model serving are kind of falling apart if the way it works is one team invests tons of money to build a model, and everyone else distills it and serves it way cheaper.

I think that's part of Microsoft's bit about bowing out of the training bit, which is: look, we're here to serve our customers. You go off, OpenAI; if SoftBank wants to fund it, that's fine. We're just going to distill it and serve it, and that's going to be fine from our perspective. It's a very rational thing for them to do. So that's the real question there. If you think about that, that's good for Microsoft stock. Cheaper models and more efficiency is a positive thing. The big hangover, if anything, for these companies is how much they're spending on capex. If that capex is used more efficiently, or they—

It's great for Amazon. Amazon is the other big winner here. Who cares if they can't make a model? They don't need to make a model. They can just serve all these other models, and they can do it even more efficiently than you thought. This is good for Meta. It's kind of bad right now—Llama's not super efficient. It's probably a wake-up call for their team that they've been spending way too much money and they could probably be doing this much more efficiently. But in the long run, Facebook's costs are going to be dominated by inference, and inference being cheaper and models being cheaper is good for them.

Now, the big question is Nvidia. Nvidia has two big moats. Moat number one is CUDA, which is the language everyone uses to program this stuff and make it work. I think that is still pretty strong. Number two has been their memory bandwidth infrastructure—the idea that you can loop all the... They bought Mellanox, they have InfiniBand, they have NVLink, they have all these things that let you treat a massive fleet of GPUs as one GPU, and it handles all these sorts of things. That is the precise efficiency that DeepSeek found. And so, at least in inference to start, the plausibility of other chips—if you could run all this on just one chip, suddenly the memory bandwidth isn't as important. Maybe for the absolute cutting edge it really matters, but does it matter as much for the trailing edge? Are other possibilities that maybe aren't as good as linking all their GPUs together as Nvidia is, or as seamless as Nvidia is, more plausible? That is a fair concern.