DeepMind's Pushmeet Kohli on AI's Scientific Revolution

So I went to a biology conference, and after I gave my talk, a biologist approached me and he said, 'Push, I have been working on this protein for the last 10 years, and I had collected so much lab data to characterize this protein, to figure out its structure, but somehow this has eluded — this has evaded all kinds of investigation, and we still didn't know the structure. But we had all this data. If we knew the structure, we could sort of validate it very quickly. I ran AlphaFold 2. It gave me the structure. It perfectly fit the answer. I've been working on this for 10 years. Wow. What do I do next?'

Yeah, so it's certainly a key sort of milestone. What we have shown is that you have an AI model — a large language model — when coupled with a harness, is able to discover new algorithms, and not only that, it's basically able to prove new mathematical results which have been studied for many, many years.

Yeah. So if you now go back to AI models — the history of AI for science is very long. We have a number of different models that have tried to do scientific discovery. Like one of the key models in this category is AlphaFold, right, which is the prototypical example of what can be achieved by AI in science. We released AlphaFold 2 at the end of 2021, and it won the Nobel Prize last year. So the impact of AI in science is very well understood. Now the question is whether LLMs and foundational models — how can they impact science? In around two years back, we had an agent called FunSearch, in which we took an LLM and we coupled it with an evaluator, and the evaluator allowed the LLM to figure out when it was making new conjectures or coming up with new ideas to solve problems, whether they were hallucinations or whether they were brilliant insights. So essentially, in this particular case, hallucinations were great because some of those hallucinations were in fact brilliant new insights that nobody had thought about. So this is where the harness comes in — that you have this evaluation function and essentially a search protocol associated with the LLM that together is able to come up with completely new discoveries that are really impactful.

Yeah, so FunSearch was our first instantiation of taking a large language model and trying to see if it can discover new algorithms. The models at that time were weaker, right? And the type of search that we were trying to do — we had not sort of explored things much further. So what we asked the LLM to do was essentially try to complete a small function and see if it can do that much better. And surprisingly, it was able to discover completely new algorithms that mathematicians had been trying to study for a long time. But the limitation was that the mathematician or the researcher had to give a template in which the algorithm should be found. With AlphaEvolve, we have removed that restriction. AlphaEvolve is not just sort of searching for a few lines. It's basically looking at whole algorithms themselves, right? Very, very large pieces of code and sort of optimizing them over a long period of time. And secondly, FunSearch — our original sort of model — used a lot of function evaluations to make these new discoveries. AlphaEvolve can work with many fewer function calls, and it can basically, by looking at fewer proposals, discover new algorithms much more quickly.

Yeah. So I think, see, we have been evaluating — as Gemini improves with the various generations, it is becoming much, much better at its understanding of code. Now, if you have a proposal generator which can understand code much more effectively, then it generates proposals which are not only syntactically correct, they are also semantically trying to solve the task, and then you are sampling what are the different ways in which the task can be solved. So as the baseline model Gemini's abilities to perform coding improve, our sample effectiveness in searching for the right solution on these very hard math and computational problems becomes much better. So if you want to search in a large space, there are two sort of elements. One is the speed of how you can generate these proposals, and then the speed at which you can evaluate those proposals. Right? Like, so first, how quickly can you say, 'Can you give me a new sort of candidate algorithm?' And then secondly, how quickly can you evaluate whether the algorithm is any good or not? And both things are really important. And the fact that you have these variants of Gemini, like Gemini Flash, which can do that very efficiently and very quickly, this is really important.

Yeah. So AlphaEvolve essentially allows you to search not only in terms of the size of what you can search over, right? You can now discover whole new algorithms, but it also is extremely general in its ability of thinking about algorithms in various different languages. So not only can it sort of search in C++, but it can also do it in Python. It can also sort of do it in Verilog, which is the language used for describing chips in chip design. So the generality of AlphaEvolve is in its ability to search for these large algorithmic spaces, but also in different syntactic and semantic representations. Right? It is not restricted to a particular language like Python, but it can sort of do that search across many different types of languages and many different types of tasks. The only expectation it has is that you have a function evaluator that you can quickly evaluate whatever proposal there is and say how good it is.

Yeah. So I think there is also — if you think about it — there is another sort of agent that we released earlier this year, which was called Co-Scientist. And in Co-Scientist, essentially what you had was Gemini playing the role of the whole scientific academic process. So Gemini playing the role of a hypothesis generator, Gemini playing the role of the critique, Gemini playing the role of ranking different — of reviewing those ideas and ranking those ideas and then editing those ideas. So it was Gemini playing all these roles in a multi-agent setup, and these were all sort of Gemini models prompted differently to play different roles. And very sort of interestingly, this combined multi-agent system came up with behavior that went much beyond a single Gemini model's answer. So it was able to give much, much, much better proposals and new ideas compared to a single sort of model.

Yeah. So I think that it is something that is still being studied, but it is a fascinating sort of thing. The one thing that I actually sort of noticed is that, especially with regards to Co-Scientist, you would run Co-Scientist on a particular problem, and the very first answer that you might get might not be very different from the baseline Gemini model. But what happens is, even as you sort of increase the amount of computation — this is not talking about just a few minutes or a few hours, but even days — as the whole multi-agent system looks at the solutions and then refines them and tries to rank them, it just becomes much, much better. So why that might be happening? It might be that the proposal — that there are deep insights or there is some sort of intuitions that are buried in the tail of the distribution, right? And then somehow Gemini's ability to evaluate which proposal, which idea is better, is much better than its capability to come up with a new idea. It's the same sort of thing in computer science, right? Sometimes we know whether a particular solution is correct or not, but it's very difficult to come up with a solution, right? So it's the same sort of thing appearing again in this multi-agent setup, that somehow the agents working together are able to extract many more impactful results.

Yeah. So I think there is going to be more work in agents, right? What we are seeing is basically the very start of research on agents. Whether in AlphaEvolve you had a generator coupled with an evaluator — the generator was a neural network, a foundational model, an LLM, and the evaluator was even hand-coded, right? But together with an evolutionary sort of search scheme, you were able to get these much more effective results. In Co-Scientist, you didn't have just one agent; you had multiple agents working in a shared memory. Now, like, what is the optimal agent configuration? Like, this is still an open research problem.

So let's go back to the sort of original motivation for why we started even working on the first iteration of using LLMs for algorithmic discovery, which was FunSearch. So a few years back, like as you know, DeepMind has done a lot of work in using AI systems for searching over large spaces, right? We have done a lot of work in building agents which have been trained using reinforcement learning, which can deal with many complex challenges, from the game of Go to playing StarCraft, which are quite complex challenges. We set ourselves a challenge: can we take the same kinds of models, like the AlphaZero family of models, which were extensions of what we had done in Go and the development of AlphaGo — can we use the same types of models for discovering new algorithms? And we came up with a new sort of agent called AlphaTensor, which was particularly focused on finding solutions for the matrix multiplication problem. And we found that this agent was able to improve over the past known results which had stood for 50 years. But the key question sort of remained: can you do something better? And secondly, can you come up with a solution that is more interpretable? At the same time, when we were looking at practical problems in Google, like how do you schedule jobs in a data center — now there has been a lot of work on coming up with new algorithms, and these heuristics have been designed by some of the best researchers and engineers at Google, and because they have a huge amount of impact in terms of compute utilization. If you use a typical reinforcement learning agent on this kind of problem, you might get better results, but it might come at the cost of interpretability, because now you have a neural network deciding which workloads go to which computers, and if something breaks, then you don't know how do you debug this thing. So what engineers would really prefer is instead of giving them a neural network, you gave them a piece of code that they can interpret and they can run. And this was essentially the motivation: can we now use LLMs? So instead of searching in the space of specific algorithms like we had done matrix multiplication algorithms in AlphaTensor, or coming up with a neural network policy to directly solve the problem — can we come up with an agent which can search in the space of programs and come up with a program that solves this hard problem? And the benefit, of course, will be interpretability: that you can see the sort of code, you can see what its properties are, and so on. And that's what happened. We found sort of programs that not only were effective, but when the experts actually saw those programs, they could sort of recover insights. So for instance, one of the math problems that we had looked at for FunSearch was called the capset problem. This is a problem that Terence Tao, one of the famous mathematicians, is very interested in, and we collaborated with this mathematician, Jordan Ellenberg, at UW. And when we looked at the program that FunSearch had produced, he found that there were certain symmetries that were in the problem that had not been recognized before. And somehow the program — like FunSearch, the agent — had discovered those and was utilizing those to get a better solution.

Yeah. So math certainly has some properties which are very interesting, right? The fact that it's very precise — like you know whether the property that you're looking for, whether you have found it or not, right? You know, for matrix multiplication, how many multiplications you require — like for a 4x4 matrix, what was known was that you can do it with 49 sort of multiplications, that's by Strassen, and we showed that you can do it by 48. So that's a very precise result, right? There is no sort of arguing about that. So it gives you a very crisp way of evaluating how well you have done. And there is no sort of like RLHF needed in terms of human feedback whether this was a nice result or whether this was a nice output or not, and you don't need to rely on sort of an LLM score — you just know that you're better.

Yeah. So wherever you can find a good function evaluator, wherever you can find an evaluator where you can say, 'I really trust this evaluation scheme. If you give me a program, I can tell you very concretely how good it is.' If your problem satisfies that setup, then you can use AlphaEvolve, because unlike a human programmer who can try 10 things or 100 things or a thousand things, AlphaEvolve does not — it can go on and on and on and on, right? It can come up with very counterintuitive strategies to solve that problem, some things that you might not have ever imagined.

Humans can be the function evaluator. It's a question of sort of scale, right? Like how many can you sort of evaluate, and whether you can evaluate the property of the program effectively. So at scale and with the right level of accuracy.

Yeah. So I mean, like, so we haven't used a human in the loop for AlphaEvolve, right? Most of our evaluators were programmatic evaluators, right? But imagine a hypothetical scenario where AlphaEvolve was told that you have to solve this math problem and come up with a new algorithm to solve this problem, and suppose it came up with many different kinds of solutions which are all equivalent in performance, okay? But then which one is the best? The best is the one which is not only very effective on the problem, but is the most elegant according to a mathematician, or the most simple to understand, right? And that's a very subjective human thing — like simplicity or interpretability — like we don't have a sort of crisp definition of it. It depends on — it is grounded in the human observer.

Yeah, that's a very good question, and I think this goes back to how much do you trust the evaluator? If your evaluation was based on a computational method, and the computational method was perfect and you completely trusted it, then you don't have to — then you think, well, I believe the computational model. The computational model says that the solution that AlphaEvolve came up with satisfies these properties. Job is done, right? But if you don't believe that the computational model is the perfect characterization of reality, then you want to make sure that you sort of validate that result in the real world, right? And you see whether that assessment of the evaluator was indeed correct.