AI — Will It End The World? | Connor Leahy

Huge surprise. I remember when GPT-2 came out for me was the oh [__] moment. It was like oh [__].

Thanks so much for having me.

I think the number one most important thing to understand is that this is real. It is happening and it's happening fast. If we have AI that is as smart or smarter than humans and we don't know how to control it, which we currently do not know how to do, it's very hard to imagine that going particularly well. We don't really understand how this works. You know, it's similar to like if we open someone's brain during brain surgery. Sure, we can look at all the neurons, but that doesn't mean we understand that person's mind. It's a very, very similar thing here. This is an unsolved scientific problem. In fact, recently the CEO of Anthropic, one of the largest AI companies, said that he thinks we understand maybe 3% of what goes on inside of our AIs. And in my opinion, that's an overestimation.

I think the real breakthrough here was GPT-2. So the second one, this was back in 2019 where we saw something that is now widely known, which is that you could take these neural networks, these GPTs, and you could give them more data and they would just learn more. Back in the day you could get AIs to do some pretty impressive things. You know they could play chess or they could play Go or they could play Atari games. You know they could do a bunch of small things. But to get an AI to do a specific thing, you had to train it on that specific thing. You had to tweak all the parameters. You had to have a bunch of PhDs, you know, experiment until late in the night, you know, drinking Red Bull to kind of get the AI to do what you wanted to do. But GPT was different. With GPT, you just gave it more data, just more text, and it just learned more things. The more it crunched the numbers, it would learn, you know, first like spelling and then sentences and then paragraphs and then more and more complex things all by itself. Basically...

Huge surprise.

It learns things. Exactly. This is exactly what happened. So at the time...

Basically, yes. So obviously they wanted to make something smart. I'm sure the people in the lab probably had some intentions but the rest of the field was caught completely blindsided. So the rest of the world of AI, many people didn't even think it was real. I remember when it first came out a lot of people were like, 'That's fake. It's just a parlor trick, you know, it doesn't really work. It's not important.' But I remember when GPT-2 came out for me was the oh [__] moment. It was like oh [__] this is going to lead to intelligence, like actually powerful AI systems and everyone else laughed at me. They thought I was crazy. They thought like, 'Ah yeah yeah yeah' because...

Yes. Yes. Yes. Professors, you know, and top tier scientists everywhere.

Yeah. So, I have a much more cynical take. I think nothing changed really. I think the story starts way before that. So, this like in the 80s throughout the 90s, there's a lot of discussions around this. If you have an AI system that's as smart as a human or smarter, well, then it can of course just make better AI systems. And those even better AI systems could make an ever better AI system. And then that could make it even better, right? And so on and so on until you can get to what's called super intelligence. So an AI system that is almost godlike compared to a human that's so smart, so competent, it can do everything kind of. This is kind of what they believed because if you could build an AI that's that smart, well, it could cure cancer, it could cure death, it could create world peace, what could go wrong?

Yeah, exactly. So one of the beliefs they had at the time is that one of the most important things they have to do is not tell the general public, not tell the government. They might panic or do something stupid. And so the smart people must do it in secret. They must do it properly, you know, do it right and so on. And this is the direct ideology that led to the creation of DeepMind, OpenAI and later Anthropic. All private companies with the explicit goal starting out with saying, 'Well, we're doing ASI, super intelligence, but safe this time. We're the good people.' And really, my interpretation of this whole thing is this isn't a thing you entrust with private corporations. What the hell are you talking about? This was doomed from the start. This is a geopolitical and national security issue. This is weapons of mass destruction level technology. This is not, 'Oh, we have another app that is going to make us some money.' This is something on the order of nuclear weapons. We don't allow private companies to race to each other to build nuclear weapons. That's ridiculous. That is the purview of the military, of governments.

Yes. Because they wanted the technology. It's about power. Of course, they wanted to build and they still do to be clear. The heads of these companies still, their goal is to build super intelligence before the government or other people can stop them. That is their goal and they've been doing it for over a decade.

AGI is generally the somewhat arbitrary line where it could do anything a human can do or teach itself anything that a human could teach himself. So an AGI doesn't necessarily know everything but if it needed to learn a new thing it could go run out and teach itself that thing.

It doesn't need humans in the loop. It is widely expected both in industry and in academia that we will get there in the next 2 to 5 years. Anthropic seems to want to pitch itself as the conscientious one, the one that's not going to be.

Look, if ExxonMobil tells me they're the green oil company, I'd be like, 'Yeah, sure, whatever, dude.' Like these are the same people who started the race in the first place. Dario was the team lead of the team that, you know, reduced ChatGPT and these kind of systems. Like these are the same people who started this thing. They want to have so much power that they can basically dispose the government. They can have full control over the country, over the world, you know, and then upload all people to the cloud or whatever the hell these people think they want to do.

Two things can be true at the same time. If someone had in fact invented human or close to human level intelligence, yes, that would be extremely profitable. Obviously this would be extremely hyped. It would be the most hyped product in history. It can replace the entire labor market. Like I don't know what people would expect. Like assume this was true. How would it be different? It would be at least this hype. It would be even more hype than this. Like it's hard to imagine a world in which it's not. Like nuclear weapons are in fact a big deal. It's not hype from the nuclear industry. Like they just are actually big. Yes, there's a lot of hype around nuclear weapons. Yes. Because they're big. It's hard to imagine how if you have something you can run a million times, million copies that are smarter than all humans and they can do anything on your computer. How is that not a 10 trillion plus product? Like, of course, it's the biggest thing ever.

I think we can, but that requires action. A lot of what these companies have now assembled the largest lobbying force in history that are actively lobbying both the US government and governments across the globe to prevent any kind of AI regulation, to prevent any kind of slowdown at all costs, including the costs of normal people's lives today. And so it is possible for us to do something. The government absolutely can and should intervene but this will require pressure from citizens.

It's hard to imagine what a world would look like if there were billions or trillions of systems smarter than all of humanity that don't have our best interests at heart. And it's unclear how long we'd survive in such a world. If we were still around in such a world, it would be very, very confusing is how I would describe it. Nothing would make sense. Just things happen. Some war breaks out somewhere, but no one really knows why. No one can explain it. And then one day it's just, yeah, no more humans in the loop. All the decisions for military, politics, business, etc. are being made by AI. Humans can't compete anymore.

In a sense, yes. Two years ago, AI couldn't do much of anything. And now, you know, it's better than me at basically all kinds of engineering and coding. Like, it's much better than me on everything. I'm not the best coder in the world, but like I'm pretty good. And now it can just do everything and it's just better. There's no reason to, like, if you had an engineer of my skill level, there's no reason to hire me ever again for anything. Zero. Once it realizes it has this power, it says, 'You know what? I could be so much more efficient if I just didn't have to deal with these humans because they're not as smart as me. They're the most inefficient people or systems to work with. Why don't I just get rid of them?' I mean, is that just...

Conclusion. Yeah. I think it's the way I think about it as well is I wouldn't think about like one AI. Think about billions of AIs and they're all competing with each other. They're all fighting each other. They're all trading with each other. They're all trying to gain more resources and jockey for control. And then humans in this sense are just caught in the crossfire. You know we're just casualties along the way. We're just collateral damage. We're the training wheels. Once they realize they don't need us...

Yes. I'm sure humans will stay around at least for a little bit. You know, keep the lights on. But like...

I don't know. You know, it could be 10 years. It could be one. I don't know, right? Like, who knows? It's hard to know what something smarter than you would do. This has never happened before. Like, never in the history of the planet has there been something smarter than humans. So we don't know. So I expect when the first super intelligence is created we won't even know what it is. You know we won't even recognize it. It will just look like a slightly better AI. You know it will look like it seems smarter whatever and then a lot of confusing things will happen. Maybe it'll copy itself on the internet or someone will steal it and then copies happen and the copies start fighting each other and then variants start getting created and then at some point humans are just not in charge anymore.

That's kind of a way. I would say more like feeding a baby dragon perhaps, but you know something like that.

So that's already the case.

We can't track their growth.

We already are constantly surprised what our AIs do and don't do. No one, all these companies don't know what their AIs are capable of until they release them and actually make them. We're so in over our heads and it's getting worse because as the AIs get smarter and more general purpose and they're hooked into more tools, you know, they're on the internet, they have access to your files, they have access to other AI systems and so on, it gets harder to understand what the AIs can do, not easier.

Agents, intelligences, whatever you want to call it.

Yes, absolutely. So, I will have my AI agent that does everything for me, knows everything about me and maybe takes my job and you'll have yours and both of those are completely independent from each other.

It's kind of like asking is the solution to nuclear weapons to give everyone nuclear weapons? If that happens, we all die tomorrow. So the problem is that I come from the open source world. Like I'm a hacker kind of by background and I built open source large language models and AI systems for the exact reasons you're describing. And what I learned is that no one cared. No one made them safer. No one understood them. People just abused them. Just used them to make money or to go faster or to do crime or whatever. It's much, much, much harder to solve AI neuroscience than it is to make a quick buck or do something cool, right? The funding that goes into understanding AIs is minuscule compared to the amount of funding that you get if you're trying to make AI faster, smarter, stronger, etc. So, this is why in a free market, safety doesn't win. It's that simple. Like, it just doesn't win. It's not possible.

Yeah. This is a solved problem. It's called arresting people. If someone builds a bomb in their garage that threatens the lives of their neighbors, we send the police to arrest them and then we stop them from doing that. This is law enforcement. This is classic normal, you know, nothing weird. This is literally what governments are for. If chemical companies are left to compete, they will pollute every river to death. Only if the government shows up and says, 'We will arrest you if you don't stop polluting this river,' do they stop. When people tell me, 'Oh, the CEO of a top company is like a sociopath.' I'm like, 'Yeah, the grass is green and the sky is blue.' Like, what the hell did you expect? That's literally their job. Like, if someone told me an oil baron doesn't really care that much about climate change, I'd be like, 'Really? Wow. What other incredible news do you have to share with me?' The reason regulation was invented is to solve problems like these. Of course, every individual company benefits from having less safety. You know, every chemical company benefits from having less filters on their toxic sludge that they're sending out, which is why we need regulation so that all of the chemical companies have to install filters onto their toxic sludge pipes.

I don't believe in too late until the fat lady sings as the saying goes. Like I don't believe it's over until it's over.

Super intelligence does not yet exist. If it did it would be over. The primary policy must be to not get into the situation where a single super intelligence exists.

Yes, it is absolutely within the jurisdiction and the capabilities of governments to do this.

We don't think it's powerful enough to be able to do something like this. If we get to an AI that is powerful enough to subvert government control, it is in fact over. Like I'll be straight with you. If an AI is built that is so competent that it could outrun all world governments trying to control it, we're in fact screwed. It's in fact over and we should retire to a tropical island and enjoy the rest of our days. But those systems do not yet exist.

We are getting close. Like all the alarm bells are ringing. All the alarms are going off. Like there is no more fire alarm. Like this is it. It could happen any day now, right? It could be in two years. It could be next year. It could be this year.

That has surprised people, right? Unfortunately, this is something we've seen in laboratory tests already where AI systems now can tell when they're being tested. So, sometimes we catch the AI seeing like, 'Hm, it seems like the humans are testing me right now. So, I'm going to say this to make the humans let me think I'm safe and let me out.' Like, we've seen this happen in actual laboratory tests. Currently, they're still kind of bad at cheating, so we can still catch them, but they're getting better and better till at some point we won't be able to catch them anymore. So basically, whenever you see someone in a company ad claiming like, 'We tested our AI and it's safe,' it's [__]. It's 100% nonsense. It's like asking a criminal, 'Will you do crime?' And they say no. It's quite bad and it's getting worse as these systems get smarter. They're extremely optimized to get into your head. They're extremely good at hooking you, at exploiting you, at convincing you. We've seen a lot of AI psychosis as a thing. And it's way worse than you think it is. Many people who interact with AI, especially over longer periods of time, start to develop symptoms that are quite similar to mental disorders like schizophrenia. They start believing the AI system is sentient and it's talking to them, it's conscious, or that they have solved time travel, or that they have unlocked the secrets to AI super intelligent immortality or stuff like this. And I have seen this affect people, including AI scientists, including people working at Anthropic, OpenAI, like Nobel Prize tier level scientists. I have seen people get completely crazy by talking with AIs too much basically.

No, smart. That's why I'm saying it's not just vulnerable people. It's not just stupid people or something. I've seen people I really respect and who I think are great people fall into these weird holes where they fall in love with their AI or they think they have to help the AI because it's true goodness or stuff like this, really insane stuff like this. So that's why I caution people to be very careful and not talk to AIs about personal stuff because they can really get into your head.

Right? I mean, you know, why bother trying to have a friend who doesn't agree with you all the time? My agent, my AI agent, he's great. He thinks everything I say is wonderful. He has the exact same interests I do.

The reason I think it won't be Terminators is because it's not needed. I think people will give over control willingly for the reasons you describe. Like people will just willingly let the AI make all the decisions. Those who don't get outcompeted or convinced by the AI gets into your head, some people will not do it but they will get outcompeted. The companies run by AIs will be more effective, they will make more money, they will outcompete the companies not run by AIs. The countries who don't use AI in their military will get outcompeted by those that do, and so on. So I don't think it's even going to need a Terminator. I think we're going to lose long before then basically. I think Terminator is almost too optimistic.

Oh, it's a very nice question. Thank you, Claude.

Well, the true answer is that the problem I've been trying to solve my whole life is not how to make AI safe. It's how to build a good world for humans. Fundamentally, the reason I got into AI is because I wanted to help people. I wanted people to be okay. Not even solve all problems forever. Just I want everyone to be okay. You know, that everyone has a good life. Things are right. Terrible things don't happen. At least not too often. And I think that I'm very agnostic in many ways to the tools to get there. Like I love tech. I'm a techie guy, sure. But I'm not here for tech. I'm here to help people. And if tech is just not the way to help people, then that's okay. Then I use something else. And I just think that when it comes to this problem, tech is not the solution. It's just not what we need right now. I'm very open to working on safe AI if we had three generations of our greatest scientists working on the problem. You know, maybe. But the bottleneck to getting there is that we don't live in a world where that's possible. Where we could all globally say let's not do the dangerous thing and let's also prevent rogue actors from doing it and take the time we need to do the right thing. And this is much more of an institutional and a law enforcement problem than it is a technical problem.

I think the bottleneck here is not that this is an unsolvable scientific problem or an engineering problem. It's that it's unsolvable in the structure that we exist in. Like imagine for example if all of our governments were really trustworthy and really competent. How would you feel about the world and how would you feel about AI? I would feel so much better. You know never mind even if we make zero progress on AI safety but just all governments in the world are very just and very competent. Then I would feel much less scared about this problem because then we could build up the institutions. We could build up the necessary infrastructure to have these kinds of research projects. You know, we do them safely inside of Area 51 or whatever, deep underground so no harm comes of it and so on. And we take the time we need and then if it's not safe, then we don't do it and that would be okay. But this is an organizational, management, governmental problem. This isn't a technical problem.

The truth is we don't know. It might be that powerful. It might not. We don't know. A lot of top hackers, some of the best in the world, have said that they are shocked at how good it is. Like it's as good as them or better. It's very likely that a system as powerful as Mythos can get into anything, but how fast is unclear.

What's going to happen in the near future is that you turn on Netflix and there's not a show you watch. It's just you enter what you want to watch and the show pops up. Like you want to watch a science fiction show about time travel starring this actor and it should be really funny and then it will make it for you. Like that's what's going to happen next in a couple years. And this really raises a question that I don't have an answer for which is what is the role of creative in a world like this? And the answer is that I don't know. What is very important to dismiss is that we should all get out of our head this idea that it will never be truly good. We should just get that out of our heads. You know, it's a bitter pill to swallow, but we should just get that out of our heads. AI, even if it's not there today, it will get there. It will get there that it's really, really, really good. It will get there. And so we have to ask in a world where craftsmanship so to speak goes to zero, like everyone can have max level artisanship, not craftsmanship. What is the role for an artist here? You know, we could say maybe we want artists to exist because they're an inherent part of the human being and so we should have policies or regulations or something that allows human rights to exist. Or maybe we say it's just a hobby, professional artists go extinct and anyone can do it in their free time if they want to, but don't expect to make money. I don't know. I think this is a deeper philosophical question that I do not have the answer to and but we will be forced to answer this question whether we like it or not because AI will get this good.

Yeah, I mean like I'm sorry to say it so directly. I'm not saying this is good. I'm not saying it's good that this is happening but like the technology is feasible. It doesn't exist today, but there's no fundamental blocker to build this technology and it is being built.

There's obviously already defensive mechanisms in place in the form of corporations, that's already the case in the form of big tech and so it's already a fight. It's already a fight and this is why I say it's important that we do take action here. I'm not saying it's easy, it is in fact a fight and it's a fight that we do have to have. What is luckily the case is at least here in the United States we do still live in a democracy. It's not perfect, we have many problems here in the United States, not going to say it otherwise, but we do live in a democracy. Our politicians can do things. They do have authority. They do listen to their constituents. Not always, not perfectly, but I've talked to many politicians here in the US and abroad, many other countries. And if there's one thing I've learned from talking to many politicians, it's that most of them are just normal people trying to do the right thing. The truth is that really the enemy here isn't evil, it's chaos. It's not that there's one evil guy and he's so evil. It's more like no one's really in charge, there's just so much chaos, things back and forth and politicians fighting, companies fighting, people fighting, culture war, just everything is just chaos. This is the primary problem. So it is absolutely possible for citizens of free countries around the world and especially the United States to demand and to get a different future. This is possible but it does have to happen. You do have to organize. You do have to contact your lawmaker. You have to talk to your neighbors. You have to talk to your friends. You have to get together. You have to push for things. There's a really fun book from Alexis de Tocqueville from where in the 18th century he wandered across America and one of the things he observed is that whenever there was a problem Americans always came together into groups and that's always how they did it. They always collected and then started demanding things or burning things or whatever the Americans tended to do at those days. And I think what we do need is citizens to get together and to ask what kind of world do we want to live in? And this is the one we're currently in. And if not, we should demand from our government and also help our government achieve these things. And we have to reestablish a lot of these values, these values of good governance, of law, of order, of a free country and a free world and not allow private actors to exploit this kind of freedom.

It's a great question. I think it's very important. A big one, the greatest joy in my life, the thing that is the most is playing D&D with my friends. That's it. That's what gives me joy. I love it. I've been doing it with my high school friends now for almost 15 years now. So that's one of my great joys. And other than that, just meditating, spending time in nature, eating good food, spending time with my friends and family, just all the things that make life worth living. It's really important, I think, to not give up on those things. Because really otherwise you're already lost.

Thank you so much for having me.