Hi listeners and welcome back to No Priors. Today we're joined by two of the key folks behind one of the most compelling developments in AI this year, Alpha Evolve. Pushmeet Kohli and Mate Balog worked on this autonomous coding agent that uses Gemini models and evolutionary search to discover new algorithms. It marks a major leap in AI's ability to contribute to core computer science and math, and perhaps sciences beyond that. It's not just a stochastic parrot or a boilerplate generator. It has shown what you might consider technical creativity in the way that Move 37 did with AlphaGo, something humans hadn't done before, even in thousands of years of play. It might even be a real step on the path to self-improving AI. Push, Mate, thank you so much for being here.
Thank you for having us. It's a pleasure.
Congratulations on the success and the launch of Alpha Evolve. Can you give me a brief description of what it is broadly?
Yeah. So in maybe one sentence, Alpha Evolve is an AI coding agent that is able to discover new algorithms that are able to make new discoveries on open scientific problems, and at the same time those algorithms can be so practical that they are already deployed in key parts of Google's own infrastructure.
What is the origin story of working on this particular form of coding agent or this problem statement?
So we are not new to this space of algorithm discovery. As you might know, the mission of all of DeepMind is to build AI responsibly to benefit humanity. And the way our particular team has been doing it for years now is to look for ways how AI can discover new algorithms. New algorithms are everywhere around us. So this is a very, very important question and can have very high impact when we can discover algorithms that solve important computational problems with higher efficiency than what we have been able to do so far. And kind of the first breakthrough we had in this space was in 2022 when we released a system called AlphaTensor. And so that was a system that was an AI system using reinforcement learning that for a very specific but fundamental computational task, so multiplying matrices, for the first time showed that AI agents can discover better algorithms than what humans had been able to do before them. So this was the first system that gave weight to this idea that indeed with AI we'll be able to go into this superhuman region of algorithms that we as humans have not been able to discover ourselves.
How do you differentiate Alpha Evolve from like AlphaTensor and FunSearch and some other projects in the sort of lineage of this?
One way to also describe what we have done is if you look back at the history of DeepMind and see a number of sort of projects that have come even before we started working on computer science, our earlier sort of work. And if you go back to the project on AlphaGo where the AlphaGo agent was able to beat the world Go champion in the game of Go. And what one was the remarkable sort of thing in that agent was that it was able to explore this amazingly large search space of all possible sort of Go positions in such an efficient manner that it can sort of come up with what is the optimal move at that time. Right? And that really surprised people, both Go professionals as well as scientists. Scientists believe that that event would come much, much later because it was a very hard problem. And so what that gave evidence for is that is the ability of these large-scale neural network-based systems to be able to reason and do very efficient exploration in these large search spaces and come up with amazing new insights about the particular domain. And then in the game of Go, I mean, there is this move called Move 37, which is a very creative new move that the agent discovered that was not in the Go literature, right, that really surprised the Go professionals. So in some sense, we asked ourselves the question that if you have an agent which can do very efficient search in the domain of Go, why can't you use the same kind of philosophy to search for algorithms in the space of algorithms? And in fact, that sort of was the underlying basis of the work on our first sort of attempt at that problem, which culminated in AlphaTensor. So how we structured the algorithmic discovery problem is we looked at first a very important problem, and that problem was matrix multiplication. It is a problem that is ubiquitous in computer science. It's one of the key fundamental operators that underlies not only computer science but also neural networks and machine learning and AI. We said, can we find a way to improve matrix multiplication algorithm? So there's a history of matrix multiplication which is very interesting for people who might be interested in it. Like it's a, even though it's such a fundamental operator, people thought that the complexity or the time it takes to multiply two matrices is order cube.
And around 50 years back, more than 50 years back now, a German mathematician, Strassen, came up with this very counterintuitive construction which showed that in fact the complexity was not n to the power three or what's not cubic, where n is the sort of dimensionality of the matrix, a bit lower. And so, and that was a very counterintuitive sort of result, and but it stayed for more than 50 years, and until sort of AlphaTensor came up, and we said, well, can we actually improve this result? And remarkably, we were able to show that that AlphaTensor, while by having this amazing ability to do search in this very large space, even much larger than the space of possible Go moves, was able to come up with this amazingly new algorithm which improved things. But then the question was, well, we now have proved the thesis that you have these super intelligent agents which can go beyond what human computer scientists have been able to do. But can we generalize them? This is sort of AlphaTensor was very smart, but was only sort of purposefully constructed for the matrix multiplication problem. Can we have build an agent that is more general? Both more general in the sense of it can handle more general problems, but can also search in the space more naturally in the space of programs rather than in the space of very specific sort of operations that were required for matrix multiplication. And that was the origin of sort of the first attempt of us with FunSearch, which was an LLM-based agent which for the first time by searching in the space of programs showed that you can come up with completely new solutions, and this made the first scientific discovery from an LLM. And Alpha Evolve is basically an extension of that.
I'm very inspired by the idea, I think as many people are, that AI will actually have creativity, does actually have like technical creativity as you are describing as one way to conceptualize this, where you're, you know, outside of the patterns that we already know as engineers. I want to go back to some of the mechanics here and the limits to generalization and how to think about automated evaluators and a lot of different topics. But, you know, when you think about these problems that are clearly like economically valuable and interesting, like matrix multiplication, the potential efficiency of it, what is your intuition for why, you know, those solutions have not been found before? Is it simply like the search space is too large, or people in this field were complacent in that they believed a certain solution was like the maximum efficiency, or because clearly there's value to be had here?
My sort of opinion on this is basically that if you look at the structure of the algorithm, what Strassen produced was quite sort of ingenious. It was not a natural thing that you would sort of think of, and that was for only 2x2 matrices. As you sort of go to larger sizes, the space is so huge. The constructions are not sort of something which is very natural. These are very involved and intricate sort of constructions that would be very hard to discover by chance. So it's quite interesting that it has this very special structure, but it's not something that comes naturally to a human computer scientist.
Just to add to that, so I definitely agree the search space is just unbelievably vast. The solutions are maybe non-intuitive. And the third thing I want to emphasize is that I really believe the people who worked on this in the past were definitely not complacent. And in fact, the problems we chose to apply Alpha Evolve to in the first instance, both on the scientific side and the practical side, we deliberately chose problems which have been worked on for a very long time by the very best people. So on the scientific side, since we're talking about matrix multiplication, this has been a known open problem for decades, and many people have been working on it. And similarly for the practical applications that we mentioned in our Alpha Evolve release in key parts of Google's infrastructure, again, like these are things that have been heavily optimized inside Google because they are so important. And so by having a system like Alpha Evolve or any other discover something new on these problems, I think that's as strong a demonstration as I can imagine of the fact that this is indeed something that is new because no one found it before, and also it is something that was not easy to discover because those results stood for such a long time and have been worked on by such strong people.
I noted that this is not a comment on the, you know, broad efforts of the computer science industry to date on matrix multiplication or data center optimization. I think this is a good moment to try to demystify what's happening under the hood for a broader set of people. Can you walk us through a concrete example of how Alpha Evolve actually evolves code? And say, let's take the example of trying to optimize data center scheduling, right? What does the step-by-step process look like from initial random code to final solution that saves millions of dollars of power?
I can walk you through that. So the user of a system like Alpha Evolve, they basically specify what is the problem that they are trying to solve. So that's the most important thing, and you specify by providing what is called an evaluation function. What this function does is whenever there is a proposed solution for solving the problem, you're able to tell how good this solution is. So you basically define what makes a good solution for discovering an algorithm for scheduling jobs on a data center. This evaluation function could be something like a simulator of jobs in a data center that, given an algorithm for doing the scheduling, it simulates how good this algorithm is. So that's what the user provides.
And this is a simulator you already had?
Yes. So that's a simulator that we already had. And I would say it's something that is quite natural to have in many domains because, you know, whenever you want to innovate on something, you need to have a way of telling, okay, is the innovation actually good or not. So it's a very natural object to have at least in principle. So you define the what by providing the evaluation function, and then Alpha Evolve fills in the how. So that's the job of our system, and you can do it in two fairly different ways. One is you tell Alpha Evolve, I have no idea how to solve this problem. Let's start completely from scratch and let's try to be creative and come up with something completely new. So that's one option you can take. Another option you can take is actually we have already worked on this problem for a really long time. Here is a very strong initial solution that we can provide to the system, and you can start from here. And that's what we did for the application to discovering new algorithms for scheduling jobs in a data center. So Alpha Evolve takes this initial solution, and then on a high level, it combines the creative power of large language models to propose like creative new ways how to improve that solution, the strictness of the evaluation function provided by the user that is able to actually filter out the things that work from the ones that don't. And then this is wrapped inside an evolutionary algorithm that makes sure that we kind of discover the whole space of algorithms in that region so that we don't commit to a very specific type of solution early on, but instead we maintain a diverse pool of potential solutions over time. Maybe we combine ideas from different solutions that are already strong until we actually have an algorithm that's so strong that we are happy to deploy to a critical part of Google's infrastructure.
Let's say, and intuitively, not in the machine learning sense, but in the evolution sense, you have different generations where you're getting closer to an optimal solution.
Yeah, that's right. Like you would expect that in each iteration of evolution, what you're doing is you're looking at the previous iteration, looking at maybe the strongest solutions you have, and then trying to be creative about how can I combine ideas from those solutions or maybe bring in completely new ideas to come up with something even better. And so yes, each generation gets stronger and stronger.
How much scaling are we talking about? Like, is there a way to predict how many generations it takes, or how do you, you know, constrain the number of iterations that the model can use?
So there are two parts to your question. One is about, okay, how does scaling work, and then how can you predict it. So for the first part, this is actually a really nice feature of Alpha Evolve that it can adapt to the difficulty of the problem. If you ask Alpha Evolve to find a solution to a problem that's actually unexpectedly easy, then it will just do it very, very quickly, like almost immediately you will have the solution. But if you ask it a problem that's really, really difficult, and by really, really difficult I mean like really difficult, maybe an open question that has stood for decades in the sciences, or like you want a practical algorithm for a really high-value application in Google, then you would of course expect this is not an easy problem. You might need to spend a longer time considering different solutions, exploring the space, combining ideas. But what's really nice about Alpha Evolve is that it is able to sustain this scaling in a way that it keeps improving over time, and it keeps improving for so long that you can make discoveries on this level of difficulty, like, you know, breaking decades-old scientific challenges or discovering high-value algorithms. Now, I know it maybe sounds trivial that if you, you know, wait longer, you get better results, but in practice that's actually like a really difficult thing to build automated agents that are able to sustain this continual improvement without plateauing quite early. This is, I think, a nice feature. There was a second part to the question about predicting how many iterations you will need. So that is something that is actually not so easy because it's like asking a priori, do you know how difficult this question is going to be? And especially in the sciences, that's something that often has a very surprising answer, like very trivial questions can turn out to be extremely, extremely difficult and vice versa. But the nice thing is that you have continual improvement if you run this system. And as long as you can run it, you can expect to get better and better results, and you just have to see where this gets you.
If you think about the coding agents that general developers have access to and are increasingly using today, one frustration with them is on relatively trivial problems, it is set out to do autonomously. It will get lost in and blow itself up or plateau, as you said, in frustrating ways. Like, can you talk about if you think there are implications from Alpha Evolve to these other general coding agents?
While large language models and coding agents are getting much better in their understanding of code, they're not perfect, right? So they do make mistakes. The other sort of element is to think about what is the task that these agents have been assigned. Mostly, if you are asking an agent to solve a particular task or write a particular program, you are providing a specification. You are specifying the task either in natural language or you're saying, well, I'm trying to do something, completed, right? So it's not a complete characterization of what you want, it's a partial specification of what you want, and the agents then try to solve the problem and might get lucky and might get the right result, or they might hallucinate and get the wrong result. And the issue is, how do you know whether the result is right or wrong? And that depends on having a good evaluator. That's how Alpha Evolve solves the problem. So in some sense, we are able to leverage the hallucinations for a beneficial purpose, right? So the creativity and the wrong answers that Alpha Evolve can somehow come up with, how do we know that they're wrong? They might be very good, we just don't see them in that way. And which is why the role of the evaluator is really important, and how do we even do the evaluation is very important, because when you come up with a new idea, should you try to explore that idea much further, right? Or how deep should you go into stress testing that idea? Should you try that idea out on a few different instances or sort of a thousand different instances, or really stress test that the idea actually works for the whole thing? This is one of the interesting parts of Alpha Evolve. Getting that balance right is really important so that you can look at where are the creative solutions, how can you sort of filter out the ones that are promising, and then use them later to refine the search process to get the final solution.
If evaluation functions, automated evaluators are really like such a limiting constraint here in terms of what we can get agents to do, any intuition from this project or others on how to overcome that? Like, can models get good at helping us create automated evaluators? Should we imagine simulators that are better for lots of different domains? If I, you know, lame product manager putting in incomplete natural language spec to coding agent, should I work with an assistant to like complete that spec? Do I use traces? How do you think that gets solved?
