AI, Learning, and Podcasting | Dwarkesh Patel | Ep. 19

First of all, we just didn't realize how much we didn't know about human evolution. Just like the story you learned in high school, all of it is like at least somewhat false about how, when, where, who.

A big chunk of it didn't.

Thanks for having me on.

I'm in a similar position as you where I've also interviewed a lot of people who are breathlessly anticipating what's coming with AI, sometimes in a very optimistic way of the AI researchers, in other cases they're worried that like the world's going to end in two years. And I think what's changed my mind around how soon we're going to get to these super transformative outcomes is just trying to use these AIs to help me with very simple like script kitty kind of tasks for my own podcast. And so I have a lot of friends who think, look, the reason the Fortune 500 isn't using AI all across their stack right now is because the management is too stodgy. They're like just not being creative enough about how to get O3 into their workflows. And look, I'm thinking a lot about how to use AI in my podcast post-production setup. I've tried for 100 hours to get it to be useful for me and it hasn't been that useful. And I think that's because it's just genuinely hard to get human-like labor out of these models fundamentally because these models can't learn on the job in a way a human can. So if you think about a human employee, probably for the first three to six months they're not even useful, especially when it comes to knowledge work. The reason they become more useful over time is not mainly their raw intellect, although obviously raw intellect matters, but it's rather their ability to build up context and to learn from their failures in a very rich way and to interrogate them. And the models currently, you just get like whatever they can do in a session. You talk to them for 30 minutes and then they totally lose awareness or understanding of how your business works, what your preferences are, etc. And a lot of tasks just require you to do a five out of ten job at something, then you talk to your boss, you go out to the consumer, and then you learn what didn't go wrong, you ask yourself what didn't go well, and you just keep iterating on that. And they just can't do this on-the-job kind of training, which I think is what makes humans valuable.

But even there, so offline we were having a conversation about how to write a tweet for something for your podcast, right? And then we were discussing, oh well, one strategy might be like write it for a group chat. Maybe you can add that to... like you would think this is exactly what other ones are for, right? It's like write the tweet. This is like language in, language out. Here's a system prompt. But like why is it, I assume you aren't getting that much use out of just like having the AI write your tweets for you. And why is that? Not because, right, by the way, I'm focusing on this not because writing tweets is like the most important thing in the economy.

Well, it's actually a reasonably high... So like think about you posting something on Twitter or Substack or whatever. Like that bar for you is actually going to be pretty high. There's a lower bar of thing where you know language models are really useful, like customer support tickets for example. You know it works really well there because you don't need the language to be perfect. You don't need insane nuance and it's actually okay for a certain class of thing if it's right 97% of the time and if it knows how to, you know, say, 'Hey, I'm wrong here.' So I do think that what we're describing here still does drive a lot of GDP growth in certain areas at least because you know there's you don't need the high bar everywhere.

Yeah. I mean the venture view against that would be that it's like the beginning of this massively exponential curve and these things are like, you know, in the first half of the first inning but all the curves are like this and so three years from now it's going to show up.

Yeah, of course.

There is one thing really interesting to observe which is that like they have cracked reasoning. It just so happens that reasoning ended up being much easier than something we take for granted which is just that day in day out you're going to be picking up information in your workplace. But you know like you go back to Aristotle and his big take was like look the thing that makes humans special from other animals is that we can reason and the other animals can't. And it's sort of funny that like these models just aren't that useful yet. They can't do almost anything except for the one thing they can do is reason. Given the fact that these sort of like ambiguous capabilities do come online, that makes me think that like continual learning might also be something that like in a 10 years it's also useful to remember deep learning is like not that old. 13, 14 years old or at least like sorry AlexNet is like that old when we started training these models. Like it's very possible to me that in a decade or two we find a solution to this problem.

Yeah. No 100%. I'm putting it more in the context of the real potential for AGI, just like a genuine replacement for human labor. I expect that to cause like a 10x increase in the level of growth. And so if it's on the scale of the internet, I'm like, 'Oh wow, this is so much shorter of what AGI could be.' So where this is clearly not that useful yet and a sort of more tangible reason to expect this kind of change. But one is just that the amount of labor supply just dramatically increases. So I think people often, especially in tech, focus on how it will make a specific industry more productive. This like narrow productivity improvements where you just... no, imagine like a trillion people in the world who are each specializing and each discovering new knowledge or we just get all these gains from comparative advantage as a result. But another is that because these minds are digital they have advantages even if they're the same amount of intelligence, specifically advantages in collaboration that humans just can't have because of the way our minds work. One example of this is okay suppose this problem is solved where you can like actually learn on the job. Now like a human can on the job learn from their work and then so over the course of 20 years they become master of their craft. They're incredibly valuable. You're one such person right, you've like picked up all this context in the tech industry from running companies from investing in companies. If we get models that have this human-like capability, not only could they learn on a single job, copies of the model are deployed all through the economy, they can amalgamate the learnings from basically doing every single job in the economy at the same time. And at that point, even if you don't have further algorithmic innovations, you would still have something that's functionally becoming a super intelligence, but you have this like broadly deployed intelligence explosion. That's just one of the many ways in which the fact that they are digital just gives them like emergent intelligence like ants or something.

I think a good way to think about this is maybe China. So, okay, why has China been so successful in not only catching up in science technology, but in fact in many ways surpassing America in a lot of key domains? And obviously China is full of lots of brilliant people. And so that does lend credence to your argument that like look they have to have, I think the intelligence is a big part of it. But I think more fundamentally just like once you've hit a benchmark of intelligence the scale is what makes China so successful right? You have just within manufacturing there's 100 million people who have built up, who are working in manufacturing in China, who have built up all this process knowledge in whatever subdomain is relevant to whatever is being built. So that scale I think is like if China is graduating I think like tens of millions of STEM graduates every single year. It's not that any one of them is super brilliant. It's just that each of them can specialize, right, in whatever radar technology that BYD needs or whatever production technology is needed. I'm thinking about this out loud, but it opens up the question of would more impact happen from a trillion more super connected, super collaborative human level intelligence people or from one just like demigod level intelligence who could like figure stuff out and tell us all what to do.

Yes. I haven't seen them up close as you have, but it seems to me I agree that they've had a huge impact. People like Elon and Steve Jobs. It seems to me their impact has been more so a product of like just like you will do this otherwise I will throw a tantrum. Which is good, right? Like you should throw a tantrum and I'm going to sleep in the office for years on end and just like get people in the right place at the right time. But it's less so like only Elon can come up with how the fins on the SpaceX rocket should be designed and therefore this his like uber intelligence allows him to like design like across five different hardware verticals.

Yes. But I mean this is actually another way to illustrate why digital minds are such an upgrade. Elon has obviously been super successful across so many different areas of technology. But of course he's just like one person to the extent that you think there was something unique about him or about the small teams he had assembled like early SpaceX, early Tesla. If they were digital, you could just replicate them like early SpaceX team, replicate them 10,000 times, throw them at a thousand different harder verticals and see what happens because you can't scale that with humans.

Yeah, I think he'd have a much easier time moving digital. This is another one of the key advantages of being digital. Because right now Elon has the same 10 to the 15 flops in his head that every single other human has. Now this could be negative as well, right? Like Xi Jinping also only has 10 to 15 flops in his head that every other human has. The ratio of compute or deliberation happening at the top versus through the hierarchy is just so lopsided and that requires a lot of delegation. Now that's good in a sort of like free society sense. You don't want the president to have that much control over your life. But if you within these specific domains where we want like companies have this outer loop where if they fail they can just go down unlike a country. So there it might make sense to have more of a, have the company be the product of a single coherent vision.

I think in the near term it will be like the latter but over the long term obviously like if AI... be AI CEOs basically in the long term.

Yeah, I think the biggest consideration there is the progress for the last 10 years in AI. I mean more than 10 years actually but even going back to the 60s the progress in AI period has been driven by increases in compute and especially in the deep learning era it's been like stupendous increases in compute dedicated to frontier systems. If you just look at public announcements of how big the compute runs are it looks like the trend since I think 2012, 2016 has been 4x per year. So there over four years is 160x the biggest system trained from the one before in terms of the compute used that physically cannot continue past this decade from how much energy is used to like what fraction of advanced chips at TSMC you need to procure to even like raw fraction of GDP. So then you would need just need progress from other, the progress would just have to come from algorithmic progress or I mean it would just have to be that and because this key input into AI progress would stop after the next five years there is this dynamic that the yearly probability of AGI is like quite high now, not that in the sense that like it will happen but there's like a decent chance every year until 2030 and then it just has sort of craters because then you're just like okay we'll just have to think hard about what's missing. We can't just throw more compute.

Did you see the METR uplift study from the other day? Oh, it was super interesting. METR is this organization that does evaluations on basically how AI is progressing. They had a very interesting result the other day. So they had open-source developers who are working in repositories that have tens of thousands of stars. They did a randomized control trial with these people where they would issue them a random pull request that was open in these repositories and they'd work on it in one case just by themselves in another case with the help of like Cursor and Claude 3.7 and then they measured in the case where they're working with the AI one how much do you think you were sped up and then two how much were they actually sped up and the developers thought that they were 20% more productive as a result of AI. They were actually 19% less productive as a result of AI.

It was really interesting to read the threads of the developers who participated trying to explain. So people who have looked at this who are even more bullish on AI are like they think their experimental design was extremely robust.

Especially the senior engineers.

And the senior engineers had from what I remember the biggest decreases in productivity.

So these are people who are experiencing this. They've been working on it for decades. And there's a couple explanations. One is just that I think in a lot of domains there is this common failure mode in intellectual work where you default to procrastinating by doing a thing which seems productive but is not moving the ball forward that much. So the classic example of this is in college instead of like rereading the textbook, you should just do the practice problems and maybe using these AI tools and then like going on social media for 30 minutes while you're waiting for the completion to complete is another example of this. Why did this happen? So at least for now to go back to your original question, it's like not obvious to me that it's making us smarter. It's interesting. The version for everybody is I think a large number of people are having ChatGPT sort of like give them guidance in life at this point, you know, I don't mean that in some like huge philosophical way although maybe in some cases but even just like day-to-day people are like here's my whole setup this is everything about me like what should I do and that is a big influence and so it's sort of we kind of to the extent that these models have quietly gripped a lot of people either through their decision, through their engineering work, through whatever else. It's like pretty important that these get really good.

I use it some. I think I use it less than a lot of my friends. I think more I'm commenting that I think broadly a lot of people do use it for that.

Yeah. Which is probably good in a lot of cases like I think it is very smart and people know how to use it and things like that. But it just speaks to even with engineering and then also on you know people using it in sort of like search and question asking for personal use that's got a lot of influence.

A little bit. Yeah. I feel like the bullishness that a lot of people in biology have for AI's ability to do like drug discovery and things like that seems very promising to me. I don't know what you found as you like spent time in biology. One interesting question I had for these people is in biology we can either employ models which think in thought space. So just like humans they can come up with hypotheses and so forth or models which think in protein space or DNA space or capsid space. You know, like humans just aren't, we can't like, you know, I think G sounds really good here. Then let's do T next. And so I'm curious which one they think is a more promising or more useful complement to the current progress in biology. Is it having better models that can think in, you know, like the AlphaFold type stuff or is it just like have O3 come up with hypotheses and just like write them out in English? At least George seemed to think it was the biospace like thinking in proteins or DNA or so forth because this is while we have millions of life science PhDs who can like come up with ideas being able to prune through them in simulation.

Exactly. Yeah. It is like the more useful complement there.

Yeah. I think in the long run I sort of worry about the fact that we know ways in which things can just go horribly wrong. Like we have the equivalent of nuclear weapons but in different domains. So mirror life in biology, apparently if you have life with the opposite chirality there's just no defense like plausibly it could render many life forms unviable on Earth. And so George Church was one of these people who wrote this letter saying like look this thing exists let's not work on it. But like I don't know over the course of hundred years what's the equilibrium here. In physics I interviewed this physicist, one of the things he's worked on is thinking about this thing called vacuum decay and the TLDR is basically like it might be plausible to just like literally destroy the universe.

Look I'm a podcaster so you're asking a content creator about physics and so I'll do my best. Apparently in quantum field theory we're in a sort of what's called a metastable state where if it's like sort of having a huge valley and then a little bit of a hill and then we're in this like little rump here. It's possible to throw such a huge amount of energy and what would happen is that this like bubble would expand at the speed of light which would just be like total destruction. It sounds like some wild sci-fi thing, but...

I'm interested in what the year 2050 looks like. Obviously, you need to understand AI in order to understand what happens in 2050. But throughout history, there's never been a case where there's just been a single technology which explains why say the industrial revolution happened, right? It wasn't just that we made better textile machines. You have improvements in sector after sector which are enabled by key innovations in specific sectors. But like for example, yeah, it's important to learn about what's happening in bio and robotics etc. So I want to get into those fields. Also I've just been interested in the fact that we are finally getting to a pace of change that we actually have seen before in history but not for a long time. I most recently interviewed this biographer of Stalin, Stephen Kotkin. And I think Stalin is born in the 1870s. And you just think about his life from the 1870s onwards. Railway, planes, airplanes, steamships, radio, telegraph, light bulbs, combustion. I mean, World War I is a crazy example where you start off the war. I think there were the Wright brothers had flown but there weren't like many, there were like on the order of hundreds of planes in the world and there were no tanks like tanks was not a thing. And World War I ends with like it's a tank war it's a plane war in not that many years.

Wow. Over the course of four years. And planes.

So I mean the reason Germany thought it was going to win is because it had this like railway network and there was this plan of how you could do this two-front war and knock out both France and Russia at the same time just by like you would be amazing at railway logistics and I think the von Moltke or whoever the leader of the German command was said at the end of the war I like we lost because of trucks right like we didn't anticipate that like there was another way to ship enemy combatants to the front. Yeah, we're just going to see like this level of change across so many different sectors.

Given what I do, I'm actually quite pessimistic about like being able to learn from other fields. I just know people who will like read some philosopher in the 19th century and they think like oh this like explains how Silicon Valley works or like this explains AI and I'm like no I think you just like have to read the papers about AI. I just think it's very hard to generalize.

Yeah. I think just people have this idea that like I'll come up with my grand theory of history.

Yes. But I just like in any domain it's just very hard to have this like... I know a couple of people who can do this and I find it really impressive. But what I noticed about them they're just like not hand wavy at all. So there are people who like for example if you're trying to model how AI will impact economic growth. One way is just to like read the sort of like firsthand accounts of people going through in the 1500s and like oh like let's read the biography of the Medici and so forth. And then there's other people who was like okay let's look at the growth rates going back 10,000 years. What is like the long run secular trend, what actually explains what changed. Well, there's the endogenous growth theory where the key change is that population growth and more people come up with more ideas. Okay well AI is more people they'll come up with more ideas there'll be more specialization. So you just like there's this very different mode of learning from other fields which is empirical and empirical is maybe the wrong word but just like very falsifiable and grounded versus I'm just going to read I'm going to go a library and just like read a bunch of random books.

Honestly, it's just a super bespoke whatever I happen to be interested in that week. If I'm reading an interesting book...

Reading. I think there's a couple of people you learn a lot from talking to in general. I've just sort of been disappointed about like... I mean given the fact that you're talking like some of the smartest people though.

So in some domains people can be super generative outside their domains. I've sort of been disappointed about the fact that look you might hope that you could talk to some historian about World War I or about the history of oil or something and then they'd have insights about like how this applies to AI. But really, those connections will likely come from you and not from them. When I was interviewing Daniel Yergin, who's the author of The Prize, it's this book about the 200-year history of oil. One thing I thought was really interesting is that Drake discovers the first oil well in Pennsylvania, I think in the 1850s. And then the Model T car or I think it's like 1905 or something that like you finally have cars with internal combustion engines that people are using to transport all around the world. And this is a sort of industrial use case of oil. Before that, most of oil was just wasted. It was only the kerosene component. So all of Rockefeller, all of that history that you sort of think about as a Gilded Age, like oil baron and stuff, that's happening when a small fraction of oil is being used just for lighting before the electric light bulb was invented. And in fact, when the light bulb was invented, I might be getting my dates wrong on the Model T and the light bulb, but it's like around that area. When the light bulb was invented, people are like, 'Oh, Standard Oil is going to go bust because what's the other use case of oil?' And so I do find it interesting that it took more than 50 years to go from we have discovered limitless energy in the earth to here's a way to use billions of gallons of this stuff. And I think it has maybe interesting implications for AI where look we have like basically it's sort of shocking how cheap AI is. I think that's why I always find it confusing that people are like optimizing on cost like do I want like 2 cents per million tokens or 2 cents per million tokens. So we have this like commodity that we could potentially use at an industrial scale. And we don't know how to like... part of it just technological. We don't know how to like get those tokens to be more valuable and part of it is just like yeah. What do we do? Where's what's the internal combustion engine equivalent for AI?