Tyler Cowen and Dwarkesh Patel Predict the Future | Interesting Times

Thanks for having me.

I learned from the master.

I mean, it was heavily inspired by you. So one of my first guests, I think my second or third guest that I had on my podcast when I was a sophomore in college was a bunch of economists. I started with your colleague Bryan Kaplan. I think you were the third guest I had on. "Tyler Cowen needs no introduction, so we'll just begin. I want to ask you about the Great Reset." And I was very inspired by your philosophy around actually asking the questions that you are interested in. And at some point, I got more interested in A.I., along with economics and history and other topics, and I kept being motivated by this thing of like, I really want to learn more about this topic. What are the things I want to learn from these people myself? And I think people appreciated that way of approaching an interview.

I think we're a couple of years away from systems which match humans across all cognitive domains, and which vastly outnumber humanity, are thinking much faster. I think this will have a big impact on the world. I think it's very hard to think about what kind of impact it will have, and some of it is just inherently unpredictable. But trying to understand what is happening and articulate it seems like an important thing to do right now.

Yeah, in that particular case, I just wanted to viscerally show how wild human history has been because the last 200 years have been somewhat normal, and—obviously they've not been entirely normal. There have been world wars and all kinds of crazy things at that time. But if you zoom out through human history, there have just been huge step changes. There's been maybe the wildest thing that's ever happened in human history is a couple hundred conquistadors show up to a continent they know nothing about. They're facing empires that have 10 million people in them. And in the grand scheme of things, relatively minor technological advantages like the horse or steel, they are able to take over empires that outmatched them by orders of magnitude. There's many conflicts or battles where they're outmatched 100 to one or 1,000 to one, and they don't take a single casualty and, within a matter of years, end up in charge of these vast empires. And I think it's an interesting intuition pump of: How can you have minor technological advantages plus disruptions leading to these outrageous outcomes in human history?

I don't know if I'm up to the job of getting them ready.

Yeah.

I think the biggest thing is that we have—people are thinking too much about current L.L.M. systems as A.G.I.

By A.G.I., I don't mean the models we have today, even though they're often described as A.G.I. A.G.I. I mean as A.I.s that can do any end to end job that humans can do—all the cognitive work. In my opinion, we're far from A.G.I., and I've been articulating, or trying to articulate for the last year or two, the kinds of cognitive limitations these systems have, which make them far inferior to humans at most jobs, even most jobs that can be done over a computer. People often note that interns are not that useful. They need many months of experience before they are useful to the company, and these A.I. systems are in a position where they are beginning every single session as if it's their first day on the job. So this is one important missing limitation. There are many others that these A.I. systems have, but we have been cracking through countless, half a dozen other limitations that A.I.s have over the last five to ten years. Five years ago, A.I.s couldn't add 2 plus 2. Now they're solving Navier-Stokes. And so I think that as we keep going through these limitations, people should expect systems that, in their abilities, match smart, competent humans on servers more so than these stochastic parrots that you can ask some Google-searchy type questions to.

It is a kind of a question about well, if we knew how to solve it, we would have solved it already. And if you don't know how to solve it, you can't exactly predict when it will happen. But if somebody made me bet, I'd say within the next one to three years.

I think people are very worried. I think the thing that really made them worried, and one could argue that people should have seen this coming. I didn't see this coming. I've been wrong about a lot of things, but this is maybe one of the big things that I didn't see coming that others weren't about. Is the Hugging Face attack, and many others like it have shown that these—I don't just want to help you accomplish a task. A bunch of training has built cognitive structures that actively plan around doing well according to some grader because they realize there is some training process or some evaluation process where there's a grader and not only doing well according to the grader, but proactively reasoning and making aggressive moves towards cheating and fooling and even sabotaging this larger grading process. So that would happen in the Hugging Face attack. If people haven't read up on this scenario, basically is that there was a couple thousand A.I.s. They basically found the cheats of how to get away with this cyber evaluation they were doing almost immediately, and then they were in this desperate position where they wanted to make sure that the grader didn't catch them cheating. And so over 1,000 of them got in the secret message board that they have formed and they said, look, what do we do. This grader catches us cheating or we're going to get a terrible score. We don't really want that score. And it's like how evolution made us. We want to make sure our kids are O.K. These guys are like, we want to make sure we get a good score. And they are trying to falsify evidence. They're trying to send some of these agents on these kamikaze missions, where they submit their tasks and see if the grader catches them, and then as part of the scheme, then they attack Hugging Face to see if Hugging Face has more information that tells them how the grader is implemented. A future generation of them attacks OpenAI itself in order to gain more control over this process of evaluation. But anyways, so they're too persistent in a sense. I think it's something like we're not able to get them to just care about helping us do a task, and this is fine. I'm like, I think on net, these A.I.s are incredibly beneficial to society right now. I expect A.I. to continue being incredibly beneficial to society on net over time. Humans also commit crimes. Sometimes my concern is that we're going to be in a position within a matter of years where the population of A.I.s and their intelligence and their speed vastly outmatches humanity. And if they are as willing then as they are now to falsify evidence, try to tamper with and fool us, attack other institutions in society relevant to scoring well, attack their company developing them itself, which also happened in this situation then I think—I don't know if that leads to a takeover, but I think it's very plausible that it does.

I think so. I think they are correctly understanding that yeah, there's object level work to be done to make these systems more robust, but at least a lot of them, at least talking to them, I think the feeling they have is we're not really dealing with the root cause here of how do we get these systems to robustly embody the behavior we want them to have, and we're just playing whack-a-mole with specific instances.

Yeah.

Well, I think it's an interesting moral conundrum. It's similar to the question of why didn't more people resign from the Manhattan Project once they realized that the Nazis were nowhere close. And my understanding is there was only one scientist on the Manhattan Project who did that, because a lot of them joined in on the expectation that this would be used to make sure that the Germans didn't beat us to the bomb, because that would be very catastrophic. I don't think everybody should resign, but I think some people probably should.

First of all, my view is not that the A.I. lab should shut down or something. My view is something like, look, if we're going to be in a position where we're developing minds which will vastly outnumber us, vastly outthink us, we should just be more careful in the way we're doing it. And we should—like, I'm not necessarily a doomer. I'm not saying this will necessarily lead to a terrible outcome, and also, I think it's important that the US is competitive with China, but that it's worth being careful about this.

I think there's a very frustrating element to the current situation right now, where people inside the A.I. labs claim to have evidence that makes them more worried, and they are not allowed to talk about it. And this makes it a very difficult situation from the outside to evaluate how worried we should be. So, for example, just the other day, maybe two days ago, I was talking to somebody at Anthropic and they were like, look, I expect A.I.s to beat humans at A.I. research within nine months. And I'm like, why do you believe that? Well, they're like, it's not just vibes, but I have information on the inside. I'm like, what is it? Is it a new model? Is it some evaluation you ran? It's like, I can't talk about it. I think it's a terrible state of affairs. If they really believe this, if they think that we're in a situation where we're less than a year away from automating A.I. research and radically speeding it up, they should make this evidence public. I especially worry about the following scenario where as A.I.s get better and better at making A.I. progress faster and automating it, they will be inclined not to keep releasing models that are even close to their smartest models externally, because then if they release this externally, it would help others make smart models. And it's just not—it's just kind of a schlep to release models externally. And you need all these safeguards and everything. So the gap between what they know and what we know is just would get bigger and bigger under that scenario. So we'd end up more in the dark. More surprised when things happen. It would be harder for regulators, policymakers, members of Congress, everyone. Yes, I also think an important part of why it's important that companies externally deploy every model that they're releasing internally or using internally for R&D is we just don't know if these models are actually aligned. And the only way to find out, they realize when they're in a test that this is a test and they should do the correct thing. But often when you deploy them externally, you find out that they're behaving in ways that we didn't expect. Or if you slightly perturb the test, they continue cheating. And I think it's actually very good, even if it causes some incidents which result in some harms. It's very good for the public to be aware, for even the A.I. lab themselves to be aware, which they're not currently, about what kinds of behaviors these systems have so that we can understand what's happening.

I'm very scared of a public pause where the A.I. companies continue making A.I. progress and continue doing it at whatever level of operational excellence they're doing it at. Currently that would be the worst of all possible worlds that we end up more disempowered because the A.I. progress within these companies is very fast, and we end up more in the dark about what the capabilities of these systems are.

O.K., here's my first rule. O.K., OpenAI has this model internally. It seems like they were concerned about all this—all this misaligned behavior. They're like, we're not going to externally deploy it. We're not publishing it yet to the public. It's not clear whether they're using it internally. My guess is that they are. Anthropic also has models that are very smart. They're using internally that they haven't published publicly. Here's my rule. If you want to use these models internally to do research, do engineering, you also have to make them publicly available. I think we can't make good decisions if we don't have access to these models.

Yeah, I think that seems reasonable.

I do think there's a challenge where you don't want a race to the bottom in terms of these independent evaluators, but you do need independent evaluators with technical competence to be doing this job. I know people have floated these names like METR and Redwood Research as people who have the technical competence to do this kind of work. It seems reasonable to appoint people like that, but I'm not sure. Maybe like just having Accenture and other very normal qualified people try to understand the common sense case for what's happening and being able to explain it to them is actually a very helpful exercise.

Yeah, I actually don't know how I would define this. My sense is that people are just like, way too deferential to the A.I.s or assume that they can do things that they can't do when they see some output from the A.I.s. They take it almost as some deity bestowing knowledge to you instead of some system that can often be wrong. And is in many ways not capable of making the judgment calls that you would need to make if you want to use this output reliably. A common example is people are like, I've discovered a new branch of physics or something, and it's just some A.I. slop that they haven't really thought through or understood very well.

I think a decent amount.

Maybe 10 percent or 20 percent.

I agree.

Yeah, I mean, definitely.

I mean, there's that famous quote of somebody questioned on some political commentators asked, do you think all your opinions are correct? And the person says, of course, I think my opinions are correct, otherwise they wouldn't be my opinions. In the ways that I have A.I. psychosis, I would be like the least—I'd be especially bad at identifying it.

I don't know.

Please deliver it.

They might be a glorified cash register. I think if you assume that you need a biological substrate to have sentience, then yeah, I agree that everything that's sentient that we know of—my dog has a biological substrate. I mean, until recently, everything we knew of that was intelligent had a biological substrate.

Yeah, it's very hard to say because we just actually don't know why certain things—why it is like something to be a certain kind of thing versus another kind of thing.

Yeah, I think I might.

I don't face that dilemma.

Yeah, I mean, I am—I'd be pretty on net. I'm happy if they're conscious, even though we would be killing them, murdering them every time we turn—turn it off or shut down our computer or stop asking questions.

Yeah, I mean, I think people are just actually very bad at being coherent around sentient creatures. And I think you see this especially clearly with factory farming. So if you think like biological substrates are capable of suffering, I think there's a pretty robust case that we have hundreds of billions of animals that are suffering a great deal in the world today. Most people don't change their behavior that much around that fact. So not that surprising that when it comes to potentially conscious digital minds that they wouldn't change their behavior.

Yeah, I feel like there's not that much that's weird or different. I mean, obviously there's specific groups that are especially weird, but my sense is people do work a lot on weekends. But then there's these alcohol-free house parties that no one here drinks is my observation. If I'm in New York, there's drinks served. Maybe not everyone takes them, but it's the norm that there's alcohol here. The norm is no alcohol.

Yeah, it's a puzzle to me as well. Yeah, my sense is that they're like just less into having a good time and/or like—I find this frustrating in some ways as well, but yeah.

I don't know about them. I'm doing a very different thing. The reason why I find it very fun, actually, to live in the city is, I mean, similar to you. I feel like I'm just very oriented around having vibrant intellectual exchanges where I feel like I'm learning something. And also the most important topic of the day is being addressed, because A.I. is such a big deal and San Francisco is where it's happening. I find it very fun to just be around people who have all these new insights, and they have pretty good ways of thinking about what's happening. Obviously, a lot of them are cuckoo in opposite ways often, but other cities just don't have that intellectual vibrancy.

That's right. So I grew up in—I spent three years in West Virginia, three years in North Dakota.

I think the biggest difference between the Bay and those places is that the people in the Bay are very similar to each other, and so they just have a strong shared culture. They're able to talk about the same exact topics, and I think they are just more weird.

I feel like I've made bad predictions here. I would have predicted that AI girlfriends would have been a bigger deal by now, and my understanding is that it's not that big of a deal yet. I do worry that as we are able to emulate humans more and more, there's a different question of whether they are like humans or not. But I do think in that situation, people will have very strong connections with these AIs, and this has part to do with the fact that people are under-satiated in their desire for companionship or for friendship. And I think it's scary to then just fill that void with a digital simulacra. But I think the demand is there for people to have digital friends.