All right everybody. We have an amazing crowd here today. We're going to be live streaming this. So let's hear you. Make some noise so everybody can hear that you're here. Let's go. I'm Alex Caneritz, I'm the host of Big Technology podcast, and I'm here to speak with you about the frontiers of AI with two amazing guests: Demis Hassabis, the CEO of DeepMind, is here. Google DeepMind. Good to see you, Demis. Good to see you too. And we have a special guest: Sergey Brin, the co-founder of Google, is also here.
All right. So this is going to be fun. Let's start with the frontier models. Demis, this is for you. With what we know today about frontier models, how much improvement is there left to be unlocked? And why do you think so many smart people are saying that the gains are about to level off?
I think we're seeing incredible progress. You've all seen it today, all the amazing stuff we showed in the I/O keynote. So I think we're seeing incredible gains with the existing techniques, pushing them to the limit, but we're also inventing new things all the time as well. And I think to get all the way to something like AGI, I think may require one or two more new breakthroughs. And I think we have lots of promising ideas that we're cooking up and we hope to bring into the main branch of the Gemini branch.
All right. And so there's been this discussion about scale. Is scale the star or is it a supporting actor?
I think I've always been of the opinion you need both. You need to scale to the maximum the techniques that you know about. You want to exploit them to the limit, whether that's data or compute scale. And at the same time, you want to spend a bunch of effort on what's coming next, maybe six months, a year down the line, so you have the next innovation that might do a 10x leap in some way to intersect with the scale. So you want both in my opinion. But I don't know, Sergey, what do you think?
I mean, I agree it takes both. You can have algorithmic improvements and simply compute improvements: better chips, more chips, more power, bigger data centers. I think that historically, if you look at things like the N-body problem and simulating gravitational bodies, as you plot it, the algorithmic advances have actually beaten out the computational advances even with Moore's law. If I had to guess, I would say the algorithmic advances are probably going to be even more significant than the computational advances. But both of them are coming up now, so we're kind of getting the benefits of both.
And Demis, do you think the majority of your improvement is coming from building bigger data centers and using more chips? Like there's talk about how the world will be just wallpapered with data centers. Is that your vision?
Well, no. Look, we're definitely going to need a lot more data centers. It's amazing that, you know, it still amazes me from a scientific point of view: we turn sand into thinking machines. It's pretty incredible. But actually, it's not just for the training. It's now we've got these models that everyone wants to use. We're seeing incredible demand for 2.5 Pro and I think Flash, we're really excited about how performant that is for the incredible low cost. I think the whole world's going to want to use these things, and so we're going to need a lot of data centers for serving and also for inference time compute. You saw Deep Think today: 2.5 Pro Deep Think, more time you give it, the better it will be. And certain tasks, very high value, very difficult tasks, it will be worth letting it think for a very long time. And we're thinking about how to push that even further. And again, that's going to require a lot of chips at runtime.
Okay. So you brought up test time compute. We've been about a year into this reasoning paradigm, and you and I have spoken about it twice in the past as something that you might be able to add on to traditional LLMs to get gains. So I think this is a pretty good time for me to be like, what's happening? Can you help us contextualize the magnitude of improvement we're seeing from reasoning?
Look, we've always been big believers in what we're now calling this thinking paradigm. If you go back to our very early work on things like AlphaGo and AlphaZero, agent work on playing games, they all had this type of attribute of a thinking system on top of a model. And actually, you can quantify how much difference that makes. If you look at a game like chess or Go, we had versions of AlphaGo and AlphaZero with the thinking turned off, so it was just the model telling you its first idea. It's not bad, maybe master level. But then if you turn the thinking on, it's way beyond world champion level. It's like a 600 ELO plus difference between the two versions. So you can see that in games, let alone for the real world which is way more complicated. And I think the gains will be potentially even bigger by adding this thinking type of paradigm on top. Of course, the challenge is that your models need to be a kind of world model, and that's much harder than building a model of a simple game. It has errors in it, and those can compound over longer term plans. But I think we're making really good progress on all those fronts.
Okay. Yeah, look, I mean, as Demis said, DeepMind really pioneered a lot of this reinforcement learning work. And what they did with AlphaGo and AlphaZero, it showed, as I recall, something like you would take 5,000 times as much training to match what you were able to do with still a lot of training and the inference time compute that you were doing with Go. So it's obviously a huge advantage. And obviously, most of us get some benefit by thinking before we speak, although not always. I always get reminded to do that. But I think that the AIs are much stronger once you add that capability, and I think we're just at the tip of the iceberg right now in that sense. It's been less than a year that these models have really been around. Especially if you think about, with an AI during its thinking process, it can also use a bunch of tools or even other AIs during that thinking process to improve what the final output is. So I think it's going to be an incredibly powerful paradigm. Deep Think is very interesting. It's basically a bunch of parallel reasoning processes working and then checking each other. It's like reasoning on steroids.
Now Demis, you mentioned that the industry needs a couple more advances to get to AGI. Where would you put this type of mechanism? Is this one of those that might get the industry closer?
I think so. I think it's maybe part of one. Should we say? And there are others too that we need. Maybe this can be part of improving reasoning. Where does true invention come from? Where you're not just solving a mass conjecture, you're actually proposing one or hypothesizing a new theory in physics. I think we don't have systems yet that can do that type of creativity. I think they're coming, and these types of paradigms might be helpful in that. Things like thinking, and then probably many other things. I think we need a lot of advances on the accuracy of the world models that we're building. I think you saw that with Veo, the potential of how it amazes me how it can intuit the physics of the light and the gravity. I used to work on computer games, not just the AI but also graphics engines in my early career, and remember having to do all of this by hand, program all of the lighting and the shaders. Incredibly complicated stuff we used to do in early games. And now it's just intuiting it within the model. It's pretty astounding.
I saw you shared an image of a frying pan with some onions and some oil. Hope you all like that. There was no subliminal messaging about that?
No, not really. Just maybe a subtle message.
Okay. So we've said the word AGI a couple times. There's a movement within the AI world right now to say let's not say AGI anymore. The term is so overused as to be meaningless. But Demis, it seems like you think it's important. Why?
Yeah, I think it's very important. But I think maybe I need to write something about this also with Shane Legg, who's our chief scientist, who was one of the people who invented the term 25 years back. I think there's sort of two things that are getting a little bit conflated. One is like what can a typical person do, an individual do? We're all very capable, but there's only a certain slice of things that one is expert in. Or you could say what can you do that 90% of humans can do? That's obviously going to be economically very important and from a product perspective also very important. So it's a very important milestone. Maybe we should say that's like typical human intelligence. But what I'm interested in and what I would call AGI is really a more theoretical construct, which is what is the human brain as an architecture able to do? The human brain is an important reference point because it's the only evidence we have, maybe in the universe, that general intelligence is possible. And there, you would have to show your system was capable of doing the range of things even the best humans in history were able to do with the same brain architecture. Not one brain, but the same brain architecture. So what Einstein did, what Mozart was able to do, what Marie Curie did. And it's clear to me today systems don't have that. And the other thing that I think is overblown in the hype today on AGI is that our systems are not consistent enough to be considered to be fully general yet. They're quite general, so they can do thousands of things. You've seen many impressive things today, but every one of us has experience with today's chatbots and assistants. You can easily within a few minutes find some obvious flaw with them, some high school math thing that it doesn't solve, some basic game it can't play. It's not very difficult to find those holes in the system. And for me, for something to be called AGI, it would need to be much more consistent across the board than it is today. It should take a couple of months for a team of experts to find an obvious hole in it, whereas today it takes an individual minutes.
Sergey, this is a good one for you. Do you think that AGI is going to be reached by one company and it's game over? Or could you see Google having AGI, OpenAI having AGI, Anthropic having AGI, China having AGI?
Wow. That's a great question. I guess I would suppose that one company or country or entity will reach AGI first. Now it is a little bit of a spectrum; it's not like a completely precise thing. So it's conceivable that there will be more than one roughly in that range at the same time. After that, what happens? I think it's very hard to foresee, but you could certainly imagine there's going to be multiple entities that come through. In our AI space, we've seen that when we make a certain kind of advance, other companies are quick to follow, and vice versa. It's a kind of constant leapfrog. So I do think there's an inspiration element that would probably encourage more and more entities to cross that threshold.
Demis, what do you think?
Well, I think we probably do. I think it is important for the field to agree on a definition of AGI. So maybe we should try and help that to coalesce. Assuming there is one, there probably will be some organizations that get there first. And I think it's important that those first systems are built reliably and safely. And after that, if that's the case, we can imagine using them to shard off many systems that have safe architectures built provably underneath them. And then you could have personal AGIs and all sorts of things happening. But it's quite difficult, as Sergey says, to predict beyond the event horizon to predict what that's going to be like.
Right. So we talked a little bit about the definition of AGI, and a lot of people have said AGI must be knowledge, the intelligence of the brain. What about the intelligence of the heart? Demis, briefly, does AI have to have emotion to be considered AGI? Can it have emotion?
I think it will need to understand emotion. I don't know if it will need to mimic emotions. I think it will be almost a design decision if we wanted to mimic emotions. I don't see any reason why it couldn't in theory, but it might be different, or it might be not necessary or in fact not desirable for them to have the sort of emotional reactions that we do as humans. So I think again, it's a bit of an open question as we get closer to this AGI timeframe, which I think is more on a 5 to 10 year time scale. So I think we have a bit of time, not much time, but some time to research those kinds of questions.
When I think about how the time frame might be shrunk, I wonder if it's going to be the creation of self-improving systems. And last week I almost fell out of my chair reading this headline about something called AlphaEvolve, which is an AI that helps design better algorithms and even improve the way LLMs train. So Demis, are you trying to cause an intelligence explosion?
No, not an uncontrolled one. Look, I think it's an interesting first experiment. It's an amazing system, a great team that's working on that. It's interesting now to start pairing other types of techniques, in this case evolutionary programming techniques, with the latest foundation models which are getting increasingly powerful. And I actually want to see in our exploratory work a lot more of these kind of combinatorial systems, pairing different approaches together. And you're right, that is one of the things: a self-improvement loop would be one way where things might accelerate further than they're even going today. And we've seen it before with our own work with things like AlphaZero, learning chess and Go from scratch within less than 24 hours starting from random with self-improving processes. So we know it's possible, but again, those are in quite limited game domains which are very well described. The real world is far messier and far more complex. So it remains to be seen if that type of approach can work in a more general way.
Sergey, we've talked about some very powerful systems and it's a race. It's a race to develop these systems. Is that why you came back to Google?
I mean, I think as a computer scientist, it's a very unique time in history. Honestly, anybody who's a computer scientist should not be retired right now. Should be working on AI. That's what I would just say. There's just never been a greater problem and opportunity, a greater cusp of technology. So I don't think it's because of the race, although we fully intend that Gemini will be the very first AGI. But to be immersed in this incredible technological revolution, it's unlike anything. I went through the web 1.0 thing, it was very exciting. We had mobile, we had this, we had that. But I think this is scientifically far more exciting, and I think ultimately the impact on the world is going to be even greater. As much as the web and mobile phones have had a lot of impact, I think AI is going to be vastly more transformative.
So what do you do day-to-day?
I think I torture people like Demis, who's amazing, by crashing this fireside. I'm across the street pretty much every day. I work with people who are working on the key Gemini text models, on the pre-training, on the post-training. Mostly those. I periodically delve into some of the multimodal work, Veo 3 as you've all seen. But I tend to be pretty deep in the technical details, and that's a luxury I really enjoy. Fortunately, guys like Demis are minding the shop. And that's just where my scientific interest is: deep in the algorithms and how they can evolve.
Okay, let's talk about the products a little bit, some that were introduced recently. I just want to ask you a broad question about agents, Demis. When I look at other tech companies building agents, what we see in the demos is usually something that's contextually aware, has a disembodied voice, often interacted with on a screen. When I see DeepMind and Google demos, often times it's through the camera. It's very visual. There was an announcement about smart glasses today. So talk a little bit about if that's the right read, why Google is so interested in having an assistant or companion that sees the world as you see it.
