I think a lot of the cyber issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering. The next phase, which I think carries the most risk, is closed RSI, closed loop without the human reviewer process, and that will obviously be faster, but I think is also a very dangerous way to get to full AGI. And our international institutions are very, very weak, as we all know, just at the time when we need them to be strongest in a way. So that's a bit unfortunate. But by the way, British 'a bit unfortunate' is 'holy'. Just for translation. Another translation. Yeah. It's like tea time and dinner time. Exactly. A bit unfortunate.
Please welcome Demis Hassabis and Reid Hoffman. Demis, thank you for joining us.
Great to be here. Right. Thanks for having me.
We've known each other for, I think, well over 10 years now, maybe approaching 12 or something like that. And it was an honor when you asked me to be on the AI ethics board for Google, for, you know, for our various funny meetings. Yes, and it's been great. So one of the things we've been trying to do at the summit is to be envisioning the world that we, as we'd want to make it. The theme has been agency, and you know, as you have been a leader in and knew well before most people were thinking about AI that one of the key things for thinking about AI for human benefit is medicine and medical. Say a little bit about how you envision this, how this helps, you know, people say, 'Oh, if I could be getting that, I understand why you AI guys are actually trying to build something that creates a longer human experience, a more quality of experience, something that's not...' it's, you call it anti-aging.
Yeah, for sure. I mean, if we, if we take a step back for a moment, it's like, um, big picture, and, and you know this, having been sort of, um, knowing us from quite early in our DeepMind journey, that the, the whole point of me working on AI for my whole career was to use it as a kind of like the ultimate tool to, to advance science and, and medicine and our understanding of the world around us, but also improving human health. I've always said that the number one thing we should use these amazing tools that we've now built and we were, you know, dreaming about 20 years ago is to improve the human condition and improve human health. And I've tried to do that certainly with my own, uh, kind of more personal projects, uh, obviously things like AlphaFold, um, but also, uh, the, the spinout drug discovery company Isomorphic, um, I've tried to push it really hard in that direction, and as I know you have too more recently, and I think it's exactly the sorts of things we need to be showing the world that AI can do. I think, um, you know, what better thing than to, you know, use AI to help us cure terrible diseases. So some of the protein folding was this kind of intersection of the idea of like we have really good data, we could use scale learning because the whole learning, like the, the cognitive revolution from learning starting with even Pong, but the, you know, and self-play and all the rest is like, okay, that's, this would be a really good way of doing it. How does, um, like the kind of the AlphaFold lead to medicines? Like what's part of the arc? And, and you know, because one of the things is a question I get frequently is like, yeah, yeah, you're talking about that, but is it true? Is it going to happen anytime soon? You know, D. So I'm, you know, I'm, I'm passing along the question I get to you.
Yes. Sure. Well, look, biology and, um, is the most complex, you know, systems in the universe, right? So that's why it's so hard because of the emergent behavior, um, the interactions, and proteins are the building blocks of life. That's how, um, I've always seen them, and, and the protein folding problem, which is what we, you know, tried to solve with AlphaFold, was considered to be like a 50-year kind of grand challenge in biology. It's sometimes described to me as, as like Fermat's last theorem but for biology, and it's critical to understand the structure of a protein from its genetic sequence, its amino acid sequence, to understand its function, and that's really important to understand basic biological mechanisms, but also to understand, um, you know, what goes wrong in disease and what things to target. You know, if you imagine you're designing a drug, then you need to know what the 3D surface of the protein looks like so you know which part of that protein, uh, to, to bind into. So it was always a critical but not obviously only still a small part of the entire drug discovery process, but it's absolutely critical part of that, and we were able to, you know, with AlphaFold accelerate it, you know, by, by orders of magnitude, um, um, by building, um, AlphaFold. And if you think about it, we finished AlphaFold 2 back in 2020, which is, you know, six plus years ago now, and it's still probably the best example of, concrete example of AI applied to a really hard scientific problem, uh, that was thought to be sort of uncrackable. Um, and so that just shows you how far ahead of its time it was. And now, you know, we're really proud of, of the impact it's made. Like 4 million researchers around the world use it. Pretty much every biology and medical researcher in the world, almost every drug in future will have made use of AlphaFold, all the structures. But it's not enough. We want to expand it to biochemistry and, um, also predicting things like toxicity or ADMET properties, all of these other things that you need to know about before you can have a drug that's safe to use and that's efficacious, you know, efficient in the, in, in whatever it is you're trying to tackle.
Indeed. Um, so speaking of accelerations, uh, let's go to one of the hot topics of the day, uh, recursive self-improvement. Uh, what is your kind of theory of the game? And I say this because you, you know, are a master games person, so I use that as kind of deliberate. And if you don't know what I'm referring to, either look at the documentary or look it up on Wikipedia. Uh, so like RSI, opportunities, concerns.
Yeah. Well, look, RSI, uh, it's obviously an extremely, uh, useful technique to make progress. It mostly works best in what I call verifiable domains, what's called verifiable, viable domains. So, um, math is one, which is why we're seeing so much advance there. Coding's another, um, but originally it was games. Actually, one of the first sort of famous recursive self-improving systems was one we invented quite a lot of years ago now called AlphaZero, which was off the back, built off the back of AlphaGo, which was the kind of, you know, famous program that beat the world champion at Go back in 20, you know, 10 years ago now, 2016, and kind of was the beginning, I think, of the modern AI era. And then we built a second version that was in some ways more consequential, AlphaZero, that didn't need to use any human data anymore. And, um, it just basically played against itself, started at random, and then, you know, around a dozen to two dozen different epochs, you know, it'd play like a million games against itself. Uh, it would then train a new version of itself, uh, uh, that was slightly better than the old version, and then that would play a million games against itself, and so on. And, and for example, and then it could play any game, not just Go, it could play any two-player perfect information game. And for me it was especially visceral as a chess player because I remember watching it play chess, and within, um, you know, 12 hours basically, or less than 12 hours, it would start from random play in the morning to, you know, you'd have your coffee break at 11:00 a.m. and I could still just about beat it. Then by lunchtime it was grandmaster level, and then by, by tea time it was better than, um, the, any, any chess playing entity. Time is dinner time for the US computer, time to, you know, or computer, you know, kind of handcrafted chess computer is better than all of them. So we sort of saw the power of that recursive self-improvement, and obviously the thing that all these domains have in common is you can verify the answer: is it cor-, is it a correct answer, is it an optimal answer, is the equation correct, does the code run, did you win the game. And the important thing about a verifiable domain is you can verify the answer, um, so you know when you've, you've, you've, you've hit the right point. Um, but also that means you can create almost infinite synthetic data because you can just use the current best versions you have to set new problems or make new moves, and then, um, uh, just sort of verify, uh, uh, whether the synthetic data was correct. The problem with synthetic data a lot of the time is you can't verify that it's right, um, from the right distribution, but in these cases you can. So we've seen the enormous progress with maths and with, um, coding, and coding's obviously become enormously valuable and everyone is having to do that now. But I do worry about it because in the limit, so there's different forms of RSI right now. We're in the, the early days of RSI, I would say, where what's happening is, and you're probably all experiencing this as well with your teams, is, um, you know, it's making a human engineer maybe three, four, five times more productive. Okay. So, uh, but the, but the human is very much still in the loop and reviewing code and architecting things and, um, and, and, and writing code still. Um, but it's all just enhanced. Um, the next phase, which I think carries the most risk, is closed RSI, closed loop, where you basically manage to somehow figure out how to do it without the human reviewer process, uh, in this, in the, in the whole procedure, and that, um, will obviously be faster, but I think is also a very dangerous way to get to full AGI because we're likely, for two reasons. It's, it's sometimes used to be referred to 20 years ago as like an intelligence explosion because you don't quite know, uh, how, you know, how do you control, keep control of such a system, uh, such a process, and then the end result will likely be quite alien. It's going to be like an alien intelligence. There's no reason to think it would be sort of in the similar kind of mind space that, uh, humans are in. So that will make it very hard for it to understand, for us to understand it, and maybe for it to understand us. So I think it's, you know, we should be very cautious as we get closer towards that closed loop form of RSI, and I think all the AI researchers sort of know that, but there is a sort of race to the, there's this sort of intense, obviously, uh, uh, uh, competitive race to improve your coding systems. So presume that the, the incentives to be in a race continue, and you know, you and I, some people really know this, you know, have tried to figure out if there's various ways you can get global collaboration to... You've dedicated a huge amount of energy to that, so thank you. Um, uh, it's a hard game, as you know, but presume that the competitive pressure for continuing to do RSI and coding continues.
In that environment, what are the best ways that we can potentially navigate that? Because we have to see kind of where RSI is going, what it's pointed at. What are the kinds of things that the, you know, the leading labs such as your own and the others, what should they be doing to try to navigate this environment?
Well, what we're doing, and I think other leading labs are trying to do, is you got to, uh, so there's, there's more prosaic things like make sure, uh, uh, cyber defense is accelerating faster than cyber attack, and making sure you give access to trusted cyber defenders first on, on maybe slightly fine-tuned versions of your general models that are fine-tuned for cyber defense. Um, and so you keep the defenders ahead of the attackers. It's a pretty tricky problem to do in general. Um, also building much more hardened sandboxes. I think a lot of the, uh, cyber kind of issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering. I think better, you need really amazing monitoring systems. You know, it shouldn't be possible for agents to be running for, for weeks without anyone realizing. You know, these kinds of things that we've seen some of our, uh, some of our competitors and others maybe, you know, it's just the beginning of this happening. So I think everyone will get rapidly better, but they have to as well because otherwise something bad's going to happen inadvertently. Some kind of incident, you know, I sometimes call it like an oil spill, is going to happen on the current trajectory if we're not, if we don't get, uh, collectively, if, if everyone doesn't pay a lot more attention and put a lot more resources on these types of systems. Eventually you kind of want formal guarantees, maybe some bounds around what these systems can do, uh, test how robust, like the things, like the guardrails are. This is actually something I encourage a lot of top mathematicians that I know well who are maybe, you know, pretty sad about the state of pure maths, but they, their skills would be incredible, uh, if they pivoted to these types of domains, more like complexity theory, game theory. How do you, uh, and of course when they're going to have to start dealing with swarms of these agents, potentially collaborating in some way. Um, how does one keep track of all of that? And should there be ways that the labs are encouraged, required, permitted to collaborate on this? And if so, what should those, what should some of those ways be? Because it obviously gets stronger if there is sharing of intelligence, methodology. Yeah. You know, mutual accountability.
Yeah. I mean, there's, there's been lots of proposals, and I've been working on these things a lot. And you, you, you know, you saw my essay over the summer, maybe some of you read it, about potential for like an SRO, industry-run standards body with some kind of government, uh, oversight over it. Um, I think something like that is going to be needed, and then ideally it becomes international. You know, it's US-led, and then obviously the state of geopolitics today makes it, and, and our international institutions are very, very weak, as we all know, uh, just at the time when we need them to be strongest in, in a way. So that's a bit unfortunate, but we got to start somewhere. By the way, British 'a bit unfortunate' is 'holy'. Just for translation. Yeah. It's like tea time and dinner time. Yes. Exactly. A bit unfortunate. And, um, and, and so we're trying to, you know, various of us are trying to do things about that, and, and that essay did have an impact, and, uh, you know, in the White House and in the administration and, and with our colleagues, and eventually, you know, I think some of that energy became the accord that you, the famous letter everyone signed, which is, which is better than nothing. I don't mean it goes far enough, but it's a start, and at least it acknowledges from all of the main players, uh, uh, the issues that are there and, and that they need to be addressed. Doesn't say too much about what to do about them, but, you know, that has to come next, and at least it gives air cover, I think, for the teams to work together on, you know, what sorts of benchmarks, what are safe, best practices on safety, what are you seeing, you know, uh, uh, in terms of analysis of what things could happen with these types of systems. And I do think there are some things that generally all the leading labs do agree on, like, um, the idea of third-party auditors or evaluators coming in. Obviously there's a question mark which third parties, and, and I think we need a healthy kind of ecosystem of that, that actually, you know, you could have good startups and companies doing that, that, that, that there's enough revenue there for them, and there should be given the amount of investment going into AI as a whole, a bit like there are auditors, you know, in, in the finance industry or any of these industries. So I could see that happening, um, and some of the big players are moving into this, like Accenture we're working with, other leaders are working with. I think they're a little bit ahead of the game because they've acquired, you know, a good startup, good friend of ours, Mark Warner's startup, so they have some expertise internally, but I think we need more of that. And then other things like, you know, the leading labs maybe red-teaming each other, uh, before, uh, general release of these systems.
