Cofounder and CTO, Databricks
Matei Zaharia - Reflective Optimization of Agents with GEPA and DSPy
MATEI ZAHARIA: Hi, everyone. I'm Matei. I'm one of the speakers giving shorter talks here. I'll be talking about some research ...
About Matei Zaharia
Matei Zaharia, cofounder and CTO of Databricks, introduced Omnigent, an open-source meta-harness for AI agents, at the company's Data + AI Summit in June 2025. He described Omnigent as a new layer above existing agent harnesses that aims to make agents interoperable, allowing sessions, policies, and skills to follow users regardless of where the agent or model runs. Zaharia said the project was released as open source because the company believes the meta-harness layer will benefit from being open in the same way data formats and sharing protocols have. He also demonstrated features including contextual security policies that vary permissions based on prior agent actions, cost controls, and an OS sandbox. In a separate appearance, Zaharia discussed Databricks' broader strategy, stating that many traditional software products will be rewritten under a paradigm of "get the data to be there and then let's slap some AGI on top." He attributed Databricks' differentiation to its commitment to open formats and its focus on AI. He also commented on the company's LTAP (Lakehouse Transactional and Analytical Processing) offering, describing it as providing the benefits of the HTAP (Hybrid Transactional/Analytical Processing) holy grail by making data available immediately for both reasoning and analytics workloads.
Transcript (16 segments)
M
Matei Zaharia0:02
Hi, everyone. I'm Matei. I'm one of the speakers giving shorter talks here. I'll be talking about some research we've been doing very recently in our group at Berkeley with some collaborators from Databricks, Stanford, other places as well on what we call reflective optimization of agents. And this is part of the DSPy open-source project that my student Omar started and that we're actually contributing this work to.
So the big question we're looking at is, how can we teach AI new tasks? Everyone is here, like 12,000 people or whatever, because AI is promising, but it doesn't quite do what we want it to do. If it just did what we wanted to do, we wouldn't come out on a Saturday to a conference about it. So, obviously, it's important to be able to teach it to do the new things we're trying to build in our application.
Now, traditionally, the way this was done in machine learning is through weight updates. You've got these large models with lots of parameters. And you use calculus. You use gradient descent. And if you set up the right objective and loss function, you do weight updates and you train your model. And this is everything from pretraining, where you predict the next token, to supervised fine-tuning, to reinforcement learning with verified rewards, which is the latest things we've been using for math and coding.
So it works really well. We've been able to teach a lot of things, but it requires huge numbers of examples. And that's what makes it hard for, say, everyone in this room to pretrain their own model, for example. It's literally trillions of tokens you have to practice predicting to pretrain a model. It's hundreds of thousands of rollouts for RLVR if you look at any paper that does that. And for supervised fine-tuning, it's tens of thousands of high-quality examples where someone spelled out what you should say.
So one of the things happening is as we saw in the earlier session on hardware and infrastructure, actually, FLOPS are getting cheaper, training algorithms are getting better. So actually this kind of training is going to get cheaper. But the thing that's not getting cheaper is if your task is slow to get a reward from, to run an example and get something from. Like for example, scientific tasks where you have to run an experiment or something like that, you can't run 100,000 experiments to teach a model something, regardless how cheap the GPUs are and the data centers and all that.
So actually, I think that for many capabilities we care about, progress in them is going to be limited by the sample efficiency of our machine learning methods. So we need to reduce the number of samples required. So how can we teach models more effectively? What is there to do? Because we've seen gradient descent work so well.
Let's look at reinforcement learning as an example with these verifiable rewards. We got this big model. We give it some tasks to do like solve a coding problem. Maybe we give tools. It might write code. It might search the web. It might use a debugger, whatever it does. So it does all these sessions. Imagine an agent for something like [INAUDIBLE]. It's doing all these trajectories, trying to solve the problem. And then at the end, we just check, did it solve it? And we give it a 0 or 1 based on what it did. And then we pass this through our loss function. And we do gradient descent on all those weights.
So there's so much that happened in these trajectories that maybe is informative. Why did it fail? It got this compiler error message. Or it ran out of time or something like that. But all we get is the 0 and 1. And then we ask, oh, can we take a derivative with that, and see how to update the weights. So basically, we've lost a lot of information. And imagine—I mean, these rollouts can take minutes. But imagine you're using your model to where it has to do an experiment or something. In the physical world, these could take hours. And we just condense them down to a 0 or 1.
So the idea we're exploring is what we call reflective optimization. And we do it in prompt space. So there's two pieces to that. One is instead of only using this 0/1 signal, we actually ask a language model to look at the whole trajectory. Look at what you did. You run this command. You got this output. You got this message, blah, blah, blah. And reflect on what happened. So potentially, integrate all this information to figure out what went wrong. And then instead of using that to update some weights, which is hard to do with a natural language insight that the model might have had, we use it to update our prompt. Basically, we tell it, write down some notes for yourself for the next time you do this task so that you can do better.
And the great thing for that, in terms of sample efficiency, is even from one sample where you make a mistake, you might write an update to the prompt that completely avoids that mistake in the future, which is not really possible if you make a small change to your weights. So this is what we're doing in this project called GEPA, Genetic-Pareto. It's basically an evolutionary algorithm for prompts that combines these ideas. Also related to AlphaEvolve, if you've seen that, which is a genetic algorithm for writing programs.
So how does it do? So I'll show you some examples where we compared with other things. So this is an example where you teach a model to do multihop search and question answering on this benchmark called HotpotQA. And we compared it with GRPO, which is the premier way to do reinforcement learning with verified rewards. And you can see we ran GRPO up to 25,000 rollouts. Actually, people run it for hundreds of thousands for this. But it's getting better, but very slowly. And then we ran GEPA in blue and MIPRO, which is an older prompt optimizer. It's also part of DSPy. And these learned the task a lot faster. Even in a few tens of rollouts, they're actually higher than GRPO. It's going to get in 25,000 rollouts or experiments. So they learned quite a bit faster. They actually also outperformed. The numbers where we stopped running them are higher than what you get in papers that specialize models for this, like Search-R1 or ZeroSearch. So definitely, they learn a lot in a short time.
What are they actually learning? This is the cool part for me. So this is the original prompt. HotpotQA is something where you do multiple steps of questions. So for each step, you got your results from before and you generate a new question. So there's the original prompt, which is very simple. You got your first summary. And now you generate a new query. And then there's the new prompt. And this was completely rewritten. This one's from GPT-4.1 mini. But the current one is similar, just by the model reflecting on its past mistakes across a few hundred steps. And it's a very detailed prompt. It has some examples. It tells pitfalls. Unfortunately, I have a short talk, so I can't read the whole thing for you, but you can find it in our paper. And it's very quick to build this up. And it actually greatly improves performance on these tasks.
The same thing is seen across other tasks, this is a claim verification one. So MIPRO, the green, was our previous prompt optimizer. So we also had to beat that. It would have been underwhelming if we didn't. So we do better than that, but definitely both do better than GRPO on this. This is also multihop search for claim verification.
Some other fun ones. You can apply this to closed source models. So you can—even if you can't fine-tune them, you can actually apply this. So we tried this on math. There's this LiveBench for math, which is recent math problems with GPT-4.1. And GPT-4.1 is also the model that reflects and updates the prompts here. There's no cheating, no distilling by having a very powerful model to write your prompt. And you can see GEPA, I think this is about 2,000 steps, ends up making GPT-4.1 do better than the full GPT-4.1. And this is the prompted. And it's like—I mean, it's not the prompt I would have written. But basically, it's tailored to the kinds of mistakes that this specific model makes that were found through this genetic algorithm to do better across lots of samples.
We also did this—sorry, we also did this for coding. We have this new hardware accelerator from AMD that does pretty much no resource for on the web. And we were able to outperform human written agentic plus RAG pipelines for this. And we're doing it for other tasks as well.
So that's GEPA. I think it's a class of new optimizers that is going to be very powerful. Because as language models get better, they should get better at reflecting and at following instructions. So maybe the weight updates of the future are going to be prompt updates that use the full capability of these models. It's part of our DSPy open-source framework. It's a framework where you can just declare an AI workflow and will optimize the prompts and even tune models for you based on our training set to do as well as possible. And a bunch of other optimizers are also in there already if you want to try them out. Lots of users are doing that. So that's something that I'm very excited about. And it could be the way that we train AI in the future. And we'd love to try it on your task. If you have a task, reach out to us. These are all the folks on the paper. Thanks.