Demis Hassabis: We're Three Quarters of the Way to AGI

Exciting to be here. Thanks everyone for coming. It's great to be here.

Yes, I just heard about that. Yeah, looking forward to the chocolate afterwards.

There is a common thread and maybe I made it into a common thread so it could be a post hoc sort of shaping, but I wanted to do AI for a long time. So I kind of decided it was the most important thing I could possibly and most interesting thing I could do in my teenage years. And then I picked things to study or do that I felt eventually would help me build a company like DeepMind. So I had that as a plan from about 15, 16 years old. I had a detour into games because actually that was in the 90s. That's where all the most cutting edge technology was being done. Obviously not just in AI but in graphics especially including hardware of course the GPUs we all use today. They were designed for graphics engines and we used the, I mean I was using the very first GPUs back in the day in the late 90s. So there was a lot of really cutting edge technology and then all the games I made, including all the games I did for Bullfrog, but also my own company, Elixir Studios, all involved AI as the main gameplay component. So probably the most, you know, well-known game I made was Theme Park when I was about 17 and that was a simulation of an amusement park and thousands of little people came into your theme park and played on your rides and decided what to buy from the shops. So there was a whole kind of economics AI model underneath and it was one of the first games of its type along with Sim City. And when I saw it sold, you know, 10 million plus copies and when I saw how delighted people were to interact with the AI, you know, that was one of the things that made me think about spending my whole career on it. And then of course neuroscience is to get inspiration from how the brain works and different algorithmic ideas from that and then just bringing all those different things together for the start of DeepMind when the timing we felt was right. And then of course we use games as the early proving ground for our AI ideas.

Well, look, we start, I started Elixir Studios straight out of college. And I was lucky enough to work at Bullfrog Productions, which those of you know games, it was a kind of legendary game studio in the early days of the game industry, probably the best one in the UK, in Europe. And I wanted to do something that combined, pushed AI. So effectively I was funding AI back in those days through the backdoor through games development. And then push the forefront of that and combine it with cutting edge creativity. And I think that's still relevant today with the way we do our blue sky research. But maybe the biggest lesson I learned was you want to be 5 years ahead of your time, not 50 years ahead. So we tried to do a game called Republic at Elixir Studios which simulated a whole country and then the idea of the game is you could sort of overthrow, I think there was a dictator in charge of the country in any number of different ways and we basically simulated living breathing cities and this is bearing in mind this is like the late 90s on a Pentium. So we had to get all the graphics and all the AI for a million people working on a home PC at the time. So, it was a little bit ambitious and maybe it was too ambitious and it caused some issues. And I took that lesson with me of like you want to be ahead of your time. You don't want to be, obviously when it's obvious to everyone it's too late. But if you're 50 years ahead then there's probably no way you can get it to be successful.

Mhm. Yes.

Well, there were some interesting threads that I think we picked up on. I think we thought we were five years ahead, but maybe we were more like 10. But it was, you know, deep learning had just been invented by Jeff Hinton and colleagues sort of in academia, but almost no one had really realized it was a big deal. We knew a lot about reinforcement learning and we felt there was huge progress to be made by combining those two techniques which almost had never been mixed together really certainly not any kind of anything other than toy problems in the academic subjects. There were two quite siloed parts of AI. Then we could see the compute, the GPUs at the time were going to be really useful. Of course we use TPUs now. But the accelerated computing industry was going to be very helpful. And then we also felt at the end of my PhD and postdoc and some of the other people I got together were computational neuroscientists that we had enough ideas and principles from the brain that could be useful including the idea that reinforcement learning could eventually scale to AGI. So we felt we had these ingredients and we almost felt like we were keepers of a secret because no one either in academia or industry really believed that any big progress was possible. In fact a lot of the people in academia used to roll out, you know, literally roll their eyes up at us when we were sort of suggested we would work on AGI or strong AI it was sometimes called at the time because it was like well we know this doesn't work. So, you know, everyone tried it in the 90s. You know, I did my postdoc at MIT, which was the sort of center point for expert systems and first all the logic language systems. I mean, it seems amazing to think that now, but I was already feeling like that was archaic then, but that's still how it was done both in Cambridge in the UK and also in MIT, these big centers of traditional AI. And it felt like, but actually that convinced me even more that we were on to something because at least if we were going to fail, we would fail in a different way than people had failed, you know, to get to AGI in the 90s. So, that felt like it was worth doing no matter what, even if obviously it was research, we didn't know for sure it would be successful, but at least we would fail in an original way if it didn't work.

Well, it was, I mean, put it this way. I would have spent my life on AI no matter what had happened. So, as it's turned out, it's gone, you know, it's sort of gone on the absolutely amazing side of the optimistic side of what we thought. Still actually within what we were predicting in 2010. We thought it would be a 20 year mission. And I think we're basically exactly on track as a field for that. And obviously we played our part in that. But even if it hadn't transpired that way and it was still now a niche subject, that's what I would still be doing because I felt it was the most important technology ever if it was obvious to me. You know our original mission statement at DeepMind was step one, solve intelligence, i.e. build AGI. Step two, use it to solve everything else. So it was always, I always thought it was the most important technology that could ever be invented but also the most interesting one. So as a tool for science, as an interesting artifact in itself and actually as one of the best ways to understand our own minds, you know, like the nature of consciousness, dreaming, creativity, all of these questions I had as a neuroscientist. I felt one of the things that was missing was an analysis tool like AI, but also a comparison for that you could do sort of a controlled experiment study and compare two different systems against each other.

Well, that was the ultimate goal at least for me, my personal passion. There's my own drive to build AI which was to advance science and medicine and our understanding of the world. My expression of that mission was to sort of do it in a meta way, right? Build the ultimate tool and then come back when that was ready and use it to make breakthroughs in science things like AlphaFold that we've done and I think many more things. So we've always had that at the heart of what we've been trying to do at DeepMind. So actually we've had an AI for science group, division, led by Pushmeet Kohli that has existed for nearly a decade now actually pretty much the day after we got back from Seoul and the AlphaGo match which is sort of 10 years to the month now is when we started formally the AI for science efforts because I was waiting for the algorithms to be powerful enough and the ideas to be general enough. And for me, you know, cracking Go was that point, that time that we thought okay now we're ready to really apply these ideas to important real world problems starting with these big scientific challenges. So we've always had that in mind as the most beneficial use of AI. Like what could be better than using it to cure diseases and give us healthier lifespans and to help with medicine followed obviously by other really important areas like material science and the environment and energy and these kinds of topics which I think AI is also going to have a huge part to play in the next few years.

Yeah. Well, I mean, I'd argue we've already had one of those moments with AlphaFold. So, you know, it's a 50-year grand challenge. Protein folding and the 3D structure of proteins is incredibly important thing to know about if you want to design medicines or if you want to understand biology. Of course, it's only one part of the drug discovery process. It's an important part, but it's only one part. So Isomorphic Labs which is our latest spin out, having a lot of fun running that as well, is to build adjacent technologies in more biochemistry and chemistry space that can actually design the compounds automatically to kind of fit and bind to the right part of the protein. So we now know the protein, the shape of the protein. We know what's on the surface of the protein and what we have to target. But now we got to build the right compound that of course binds strongly to where you want it to bind on the target of interest but doesn't bind to anything else ideally because that would be a toxic side effect. So the dream is to do almost all the exploration which is 99% of the work and the time in silico and then save the wet lab step just for the validation step. Right? So that would be, you know, I think if we can do that and I think we can get there in the next few years, I think we could reduce drug discovery times instead of taking like an average of 10 years down to months maybe even weeks and perhaps even days one day. And then I think then all disease could be in reach and I think things like personalized medicine will become possible, you know, like personalized variations off of base medicines. So I think the whole medical area, drug discovery is going to be revolutionized in the next few years.

Well, I think there's several things along those lines that I think is going to happen. So, first of all, the understanding and the analysis of AI systems themselves I think is going to become a whole science, a kind of engineering science. These are incredibly interesting artifacts that we are building and they're incredibly complex as well, eventually they'll be as complex as the human mind and the brain and so they'll need to be studied so we can understand fully way beyond where we are today how these systems work. So I think there's a whole kind of field, mech is part of that but there's a lot more I think that we can do to analyze these systems. So that will be a science, but I think also AI itself will maybe unlock new sciences, which is maybe what you're getting at. The one I'm particularly excited about is AI for simulations. So I love simulations. All the games I wrote not only had AI, but they were simulations. And I think simulations is the way we can address some of what we maybe think of as social sciences like economics and other more humanistic subjects because it's very difficult to do control studies in that. You know why aren't they just sciences like physics today because the problem is they're emergent systems just like biology actually and it's very hard to do repeated control experiments. You know if you raise interest rates by half a percent you have to do it in the real world and then see what happens. You can have theories but you can't run it thousands of times. But if you could simulate things really accurately then maybe there's sort of new sciences to be done where you can sort of rigorously sample from a very accurate simulator. And then I think that will allow us to make much better decisions in these today what are very uncertain domains.

Yeah. Well, look, I mean I'm thinking a lot about that and we do a ton of that work, like learning simulators basically. So you know these are in domains where we don't know the mathematics of it well enough or it's perhaps too complex. We can't just write a directly down a special case simulator. It's just not accurate enough, doesn't capture all the variables. We're doing that, we've done it with weather. We have the most accurate kind of weather simulator in the world, WeatherNext, and it's far faster than what the meteorologists use.

No, we can't, no, and I'm not sure that would be a good idea but the first step is to understand it better. And but then even biology, you know, we're working on a kind of what I call a virtual cell. So you know hugely dynamical emergent system and I think biology is perfect, machine learning is the perfect description language for biology in the same way maths is for physics because I think in biology and in lots of these natural systems you have loads of weak signals, weak correlations, tons of data far too much that any the human mind can analyze but there are connections and correlations and interesting causalities within that mass of data. So I think it's sort of, it's always struck me that machine learning is the perfect tool to describe those kinds of systems where until today, you know, mathematics hasn't been able to do that either because we can't manage it as top mathematicians because too complex or the expressive power of maths is not enough to understand these sort of highly emergent dynamical systems.