Welcome back to the Google DeepMind podcast. I'm Professor Hannah Fry.
Picture this scene. It's March 2016. Inside a hotel suite in Seoul, South Korea, two players are playing the ancient game of Go, a game of unimaginable complexity, long thought impossible for a machine to master. On one side is Lee Sedol, a legendary 18-time Go world champion. On the other, AlphaGo, a neural network-based AI system built on a powerful technique called reinforcement learning.
Welcome to the DeepMind Challenge Live in Seoul, Korea.
That's a very surprising move. Not a single human player would have chosen move 37.
What are you talking about?
After hours of intense gameplay spread over 17 days.
Yeah, that's an exciting move.
Lee Sedol placed two stones on the board to signal his final resignation.
And in the blink of an eye, the world changed.
The final result of 4-1. Congratulations to AlphaGo and to the entire team.
That was exactly one decade ago. And the field of AI has changed unimaginably since then. We have seen the rise of large language models, the growing sophistication of AI agents, and the solving of scientific grand challenges like protein folding. But in many ways, the modern AI revolution arguably began right there on that wooden board in South Korea. So in this episode, we wanted to look backwards and forwards to how a bold experiment in teaching machines to play games became the foundation stone for the AI breakthroughs of today. And with me are the perfect guests to tell that story. Tor Lattimore is a distinguished research scientist at Google DeepMind who was right there in Seoul as a key architect of the AlphaGo project, and Pushmeet Kohli, who leads Google DeepMind's science work and is the person to tell us how those early techniques pioneered in Go can tackle crucial problems today. Welcome to the podcast, both of you. Tor, I know you're an accomplished Go player yourself. Just explain to us why Go was seen as a good challenge for AI.
Yes, the game of Go seemed like the perfect challenge for AI because the game has such simple rules, yet it leads to such complex gameplay with tactics and strategies and complex patterns. And once the game of chess had been solved, as it were, or at least, you know, Deep Blue had won against the world champion, then Go was this open challenge. It's much more complex than chess by many orders of magnitude, and nobody was expecting it to be solved anytime soon. Yet it looks so elegant and simple for computer scientists. And so it was the perfect game to tackle at the time.
I mean, that idea of nobody thinking it would be solved anytime soon, that sort of hits the nail on the head, right, Push? I know you were working at Microsoft at the time, but just how complex was this problem considered to be?
I think it was considered extremely complex. And that is not only because of the breadth of the search space, of the number of moves you can make, but also the depth, how long you have to reason and how long the games are. In a game of chess, you might think about reasoning about 60 to 70 sort of moves. In a game of Go, it's much, much longer. And that leads to the challenge of the problem.
Tor, I know when you first started at DeepMind, being a Go player, didn't you play against AlphaGo on your first day?
Yeah, exactly. So imagine I come in on my first day of work at DeepMind. I know a couple of people, including David Silver, and he asks me, 'Tor, you're a Go player, right? Could you do us a favor and test our baby version of something that wasn't even called AlphaGo at the time?' Of course, you know, it was an internship project and they had just about taken a few thousand games from the internet and had trained a system, or a few hundred thousand games maybe. And I had the opportunity to be one of the first people to play against it. But you can imagine I was excited, but I was also nervous. It was my first day at work. And there I was being dragged to a centrally located table on the other side. I think it was Aja Huang, who would later be known as the hand of AlphaGo, with his poker face. And I got to play against this baby version of AlphaGo.
With people watching, presumably.
With a lot of people watching all around me. You know, there was no escape. Later, Demis showed up, and of course David was there the whole time.
And so what does one do? Play conservatively, right? So I just thought, just don't make a mistake. Surely this can't be so hard. But of course, that was exactly what that version of the program was good at. It was trained on human professional games. So it knew exactly what to do against conventional play. And so as this little test match proceeded, my position became worse and worse, and I ended up losing by a small margin. But I took the crown of the first person who officially lost to AlphaGo. It was quite the experience, and of course afterwards everyone knew me, and it was a wonderful way of introducing myself.
A humbling way. Exactly.
Absolutely. Pushmeet, just remind us, I mean, okay, so I know that the algorithm advanced quite substantially from that early point where it was an internship, but just broadly explain to us how it worked and this idea about cracking the kind of combinatorial spaces in particular.
Yeah. So I think if you look at the game of Go, the number of moves that you can make at any given time, there are a finite number of moves, but if you look and reason about the overall game state, it's exponential.
And that exponential growth in the number of states that you have to reason about is what makes the game extremely complicated.
So how did they crack it then? Just remind us of the solution that they discovered.
The beauty of AlphaGo was there is this element of thinking fast and thinking slow. And AlphaGo in some sense was the perfect combination of those thinking fast and thinking slow processes coming together to take on this extremely large search space.
And it matches quite well to how humans play the game. I think, you know, if you imagine how a human would play a game of chess or a game of Go.
We also have the capacity to look at a position and pretty quickly appreciate if that's good for black or good for white. And we can also look at a position and already see moves that seem promising. We never look at all the possible moves, which would be maybe 20 or 30 in chess or 200 or 300 in Go. We immediately draw on to certain, maybe even aesthetically pleasing moves that seem like just the right ones, guided by our intuition. And that element is complemented by planning, where we explicitly reason through the possibilities. If I make this move, my opponent might make that move, and then I have to counter with this move. And these two different ways of thinking come together in how humans play these games, and they also come together in how AlphaGo plays.
The intuition and the calculation, as it were.
So was that the inspiration then? Did you sort of think about how you were playing the game, how other Go players were playing the game, and draw that direct inspiration from neuroscience, effectively, as it were?
Yeah, I think that is definitely one direction because a lot of team members were actually game players who were able to introspect and see how we tackled the game. And then of course that comes together with deep learning that at the time, you know, since 2012 had grown as a direction and now for the first time gave us the tools to learn these approximate functions for that. For example, the value function that takes a board and tells us how good it is for either black or white, or the policy network that takes a board and effectively ranks the available moves according to how likely it would be that a professional player would take them. And so deep learning was just ripe at the time to tackle this problem and gave us the opportunity to implement the fast thinking. The slow thinking is not unlike what happened in Deep Blue. You know, it's the search of the game tree that was already known and that we might now call good old-fashioned AI.
Okay. Well, I mean, you lost to this thing quite early, but once it had gone through a lot of the people on the team, let's say, I know that you tested it with a professional Go player because you had Fan Hui come into the office.
How confident were you at that point that it was going to beat him?
Yeah, we had different levels of confidence, which was really interesting. So we had been really lucky to find him. You know, he was the European Go champion at the time. He lived in Bordeaux and came over. We lured him into playing this game with us. And the setup was that he would play 10 test games against the version of AlphaGo at that point. And I personally thought that AlphaGo cannot possibly be at the point already that it beats the European champion, a professional player.
And so I had a bet with David Silver. David Silver was confident. He said, 'I think AlphaGo is going to nail it 100%.' And I said, 'No, I think AlphaGo will lose at least one game.' And the bet was that whoever lost would have to show up at the office dressed as an ancient Japanese Go master and be in the office for one day with that. Well, who showed up like that? It was me, because it was in fact 10-0.
Yeah. But it did give us confidence and gave Demis confidence that we would be able to tackle even harder opponents in the near future.
Which you did, of course, on a plane you got in 2016 to Seoul in Korea to play against Lee Sedol. I mean, just tell us, give us a sense of how phenomenal a player he actually is.
Yeah. So, Lee Sedol was really one of the, or maybe the best players at the time, with an incredible track record of winning tournaments. He was compared to Roger Federer at the time for his success and intellectual brilliance. And so for us, it was a tremendous honor that he accepted our challenge to play against him. And it was a tremendous challenge because we had to set a date, right? You can't just say, you know, we'll tell you when we're ready. A date was set, and we had to work towards that date to actually make AlphaGo strong enough. And what added tension and excitement to it was that he was convinced that he would win. He thought it highly unlikely at the time that AlphaGo would win. And of course he was basing his assessment on the game records that he had seen against Fan Hui, and he assessed that he was better. But of course what he wasn't so aware of is that AlphaGo was constantly improving through the training and the algorithmic refinements and so on that we made. And so the entire team basically went to South Korea, and you wouldn't believe the excitement of people there. You know, the truth is in England, Go is a bit of a niche activity, right? Very few people would be able to play it or even know about it. But in South Korea, people were so excited. The best Go players are celebrities. And, you know, we came there and there were hordes of photographers that took pictures. We had a documentary film crew with us. And so imagine typical computer geeks, as it were, suddenly in the limelight of the world for this match. That was quite the adventure.
Yeah. I mean, were you nervous about the performance of AlphaGo?
Yes, we were definitely nervous. So of course we had a very sophisticated evaluation pipeline. You can test against players that you have access to, like Fan Hui. That was super helpful. You can also test against previous versions of the program. And you can calculate what we call the Elo score of the system, which basically takes the outcomes of all the games that you play against other versions, maybe earlier versions of your program, and calculates what the rating of the new version is. And you can calibrate these things quite well. But of course we didn't know where on that scale Lee Sedol would be. And of course we wanted a cushion as well. You know, it would have been nice to be quite a bit better to have some certainty, because this is the world stage, right? If you lose this, that's a bit of a hit to the reputation. And so yeah, we were nervous. We worked up to the last minute. We also needed to make sure that the system is really stable. You know, you don't want to make last-minute changes to make it that little bit better, but risk that it now becomes unstable.
But in the end, we were quite happy with it. And so we entered that now kind of famous hotel floor where all the action happened, where all the press was waiting and so on, and embarked on the match.
And people were watching from around the world, including Pushmeet, including you.
Yeah. So, I mean, where were you at this point? You were watching on...
Yeah, I was in Seattle. I mean, I really started getting into it in the middle of the first game. It became so clear that AlphaGo had reached that specific milestone, and you could even see the reaction from the press and the commentators and Lee Sedol himself.
Well, it's interesting that you said in the middle of that game, because in the early stages of that game, was it clear who had the upper hand?
I think from a person who was just watching it, I felt that in the early stages, everyone felt quite confident that Lee Sedol would win. In fact, only as the game progressed and it became closer to the final outcome that they realized that as you count the territory, AlphaGo had an advantage. In fact, it came as a surprise to people.
Yeah. So I had this interesting interaction on site with a professional Go player, an American professional Go player who was sitting next to me while we were watching. And there was some sequence unfolding in a corner, and he kind of approached me and said, you know, I always tell my students not to play that stupid move that AlphaGo just played. So I mean, it's pretty hopeless. And I was like, I'm not as much of an expert, but let's just wait and see, was my reaction. And then after that first game, this gentleman came to me and said, 'This is the most phenomenal thing I've ever experienced. I'm so grateful that I'm allowed to be here to witness that a machine can play Go at this level, and there's going to be so much we can learn from it.' And he was already embracing this. I mean, you have to imagine these people dedicate their lives to the study of this game, and they've often trained from being young children to their current age just to master this game. And so, of course, it comes as a shock to them that a machine might match or even exceed a human Go player.
Because if that was the first game when AlphaGo won, in the second game, AlphaGo did something that really surprised everybody.
That's a very surprising move.
Professional commentators almost unanimously said that not a single human player would have chosen move 37. AlphaGo said there was a 1 in 10,000 probability that move 37 would have been played by a human player.
Just explain to us what happened with the now famous move 37.
Yeah. So this was a remarkable scene, and I was sitting in the international English-speaking commentating room, and Michael Redmond, our American commentator, he had this big demo board on the wall, and he would put all the stones up there on the board to show people what was being played and comment on different variations. And so he took this stone corresponding to move 37 on the board, and then he stepped back and said, 'Ah, this must be wrong,' and he took it back. And then he looked at the screen again and said, 'No, no, that is actually what AlphaGo played,' and he put it back. He was puzzled. You could see that that was such a counterintuitive move for a human player. It was a shoulder move on the fifth line. And this is typically something that human Go players avoid. So often in Go there is some kind of pushing going on along the edges, and one of the players builds territory along the wall of the board, and the other side builds influence towards the center of the board. And if that happens on the third and fourth line, this is considered to be roughly equitable. You know, both sides get something out of it. But what AlphaGo was effectively suggesting is that it's still profitable if you do it on the fifth line, and you give that much more territory to the other party. And that's what was so surprising to people, that there would be situations in which that would be correct. And so not only was it a very special move, but in a way it represented a new way of weighing these two factors of immediate territory versus influence towards the center of the board against each other.
Something that went beyond what a human Go player would normally do.
Yeah, absolutely. I mean, there are moments like this where you see the true potential of an AI system expanding human knowledge, where people have regarded, in this particular case, the game of Go as a thing to be studied for many, many years, and there comes this particular point where that knowledge is expanded, and people are at first skeptical, which was the case in the game as well. When the move was played, it was considered a hallucination or a mistake, right, for quite a bit of time before its implications became clear.
Because it proved to be pivotal to the second win.
Yeah.
It was not just a moment in that game, but it was also a moment, I think, in the whole sort of history of AI, where that particular moment showed us that there will be times when these systems will produce insights which we might not even be able to discern whether they are the right things or amazing breakthroughs, but yet they will have a lot of influence in how we look at whole areas of study in a completely new light.
Well, I also want to talk about move 78. This is a move that was played by Lee Sedol that confused AlphaGo, causing it to resign the game.
What is Lee Sedol up to here? He's just burned like seven or eight minutes just on this move already.
Oh, look at that move. That's an exciting move.
You know, I'm not actually sure what AlphaGo is trying to do here.
So by this point, AlphaGo has won three games in a row, and now Lee Sedol does a move that confuses the system. Is that fair to say?
Yeah, that's absolutely fair to say. So move 78 was an unusual wedge move that Lee Sedol played. There had been a very interesting battle, as it were, at the center of the board, and Lee Sedol found this move, and it was also surprising to people, similar to move 37. And from then on, we observed that AlphaGo didn't have a good grasp of the position anymore. We saw that the moves that it made didn't really make sense to us in a bad way. You know, 37 also didn't make sense to us maybe, but these moves, even to amateurs like us, seemed strange. And so it had been confused by the move. And just to zoom out to give you a sense of why this still mattered so much. So you might say, okay, it's a match of five games, and AlphaGo has won the first three. What more is there to prove?
But then we were thinking, well, if Lee Sedol is to win the last two, what would you conclude?
He's got it figured out, right?
He's found the fragility.
So it would have been the human triumph. And so that's why that game and the last one were still very exciting to us. But it wasn't entirely the case that we were disappointed. We were certainly disappointed, but also we had so much admiration for Lee Sedol, you know, as a human to be able to find this move. You just have to imagine this master who has dedicated his life to playing this game in this battle that must have been so hard on him, right? To see this machine play so perfectly and him struggling to find a way, and then in game four he finds a way. And as he put it in the press conference, I think later he said that he was so happy and proud that he was able, maybe for the last time on behalf of humanity, to find a way to overcome the machine.
Because some people called it the divine move, didn't they?
Yeah. And I think given the tension at that point in time and him really outdoing himself at that moment and finding that move, I think it's a good name for it.
Well, the final score was 4-1 to AlphaGo in total.
What was the reaction from the Go community?
Yeah. So the Go community followed the match very closely, and of course the outcome was dramatic and for many people unexpected. And so people showed very different reactions. You know, some people were absolutely amazed and surprised about the outcome. Some people couldn't believe it. Others, of course, also thought that some era had come to an end because now maybe the strongest Go player was no longer a human but a machine. But overall, what we found amazing is that there was an uptick in interest in the game of Go. I think more people play Go now than did before, and the Go community really embraced the learning from AlphaGo. So there are now many programs that work essentially the same way that AlphaGo does, and people use it for teaching purposes. They analyze their games through it, and overall I think it has provided a lift to the whole Go community.
