All right, so this is a kind of emergency episode, I think it's fair to say, about the state of mathematics and artificial intelligence. A kind of crisis moment in the history of mathematics. Over the last few weeks, machines claimed to have solved some of the hardest problems in the history of math: Navier-Stokes, the quasi-Riemann hypothesis, the unique games conjecture, and hundreds of other results. So here with me, we have Steven Brunton, who's a mathematician, fluid dynamics expert at University of Washington, who can speak to Navier-Stokes, turbulence, all things fluid, and many other topics. Also, here's Ankur Moitra, a mathematician and theoretical computer scientist at MIT, visiting Institute for Advanced Studies in Princeton this year, who can speak to many things: unique games conjecture, and complexity theory in general, and the mathematics of how the AI learns and all things theoretical computer science. And Alex, a mathematician, number theorist at Rutgers, who can speak to also very many topics, including Riemann hypothesis and Lean, the automated proof and the automated verifier and checking and so on, and so many other topics in mathematics. All three are brilliant. All are very kind to show up this early morning and talk to me, assemble this kinda multi-agent mini swarm of human brains to try to interpret what happened in this very moment. A lot of mathematicians, brilliant mathematicians, including Terence Tao and others, are freaking out. It's a mix of awe, terror, dread, and few inklings of light, of hope also. So I apologize to start with a question about feeling, but what have you been feeling about this historic moment? Maybe Alex, do you wanna go first?
Yeah, I don't know if I would call it a crisis. There's a lot of opportunity in that crisis. It's a chance to rethink what it is we're trying to accomplish, and it's the chaos of opportunity that some people are reacting negatively to, some people are reacting positively to, and sometimes those reactions are happening in the same person. And so I think we're entering, if we're not already in, a period of some kind of golden age of mathematical knowledge after millennia of having a candle that we are walking around the woods, and this is as far as we can see. Now we have a flashlight.
I think we should comment on the magnitude of this moment for people who are not aware. So maybe to quote you, something like, "Holy shit, if a human did this, it'd be an instant Fields Medal." Maybe you didn't say, "Holy shit," but basically saying, "This is an incredible result." So a lot of people wouldn't have expected this amount of progress in this amount of time, so this is truly shocking. So you're now speaking as a person who is dealing with the feelings you've felt over the last couple of days, having found this out. So maybe also it'd be useful to go back to the feeling you were having when you first saw the report that, in this case, OpenAI dropped. And the two of you happened to be on camera loosely on the same day, maybe a few drinks in because it was a conference or a get-together. Maybe, Ankur, can you speak to what you were feeling?
It was an interesting night when the results dropped. Now, it was not entirely unexpected because for weeks there'd been rumors about the two biggest problems in my areas that were in that collection of results, so the unique games conjecture and RL equals L. There were even rumors about matrix multiplication, so really my first reaction was, "Wow, it's all true. Thank goodness it's finally out there because now people can actually go through it and process what's happened and figure out what to do next." Now, as to your earlier question, you know, what are we feeling? Well, for me, it changes on a daily basis, but I think the biggest, most overarching feeling is really feeling like you're getting pulled in two very different directions. My work has always been about asking the right questions, and to that extent, these tools are amazing for me because, you know, they increase my ability to learn about other fields. It's like having a world expert who's there to teach you about things in your own language. Your ability to make connections is just magnified. But, you know, one of the parts of my job that I really love the most is the mentorship aspect. Not just training grad students and postdocs and seeing them do great things, but having them come with me on these journeys. So the part that pulls me in the other direction is the recognition that, you know, the playbook and the blueprint for success that we had for so long is now thrown out the window. So it becomes a lot harder to advise students and postdocs about, you know, what are things they should be working on that are gonna make an impact, are gonna make a name for themselves. So I feel very excited, but maybe more intellectually lonely than I felt before, in part because, you know, a lot of times great results come from asking very humble questions, at least for me. And now there's so much anxiety and dread and feeling that, wow, you would have to do something really momentous to stand out in the sea of AI results, that it feels like we've lost some of that patience in terms of planting the seeds from which the great oak trees grow.
Terence Tao, Fields Medal winner, wrote, "I am horrified by the many person-years of ongoing patient and deliberately slow research efforts, particularly by graduate students and postdocs, toward many motivating problems in mathematics being casually disrupted or destroyed by such a release. Much as one cannot unhear a movie spoiler or a crossword clue, one cannot explore a problem as profitably and richly once one is aware of an existing solution, and I mourn the path not taken and the opportunities lost in the frantic race to develop the technology." So I mean, if we're being realistic, it's more dramatic for a PhD student and a postdoc than maybe the way you've articulated. I mean, you're saying it quite calmly, but if I was a PhD student right now, I'd be in a state of, like, shock. Like, my whole life-- I was going through some of this. I know a lot of my friends are programmers. They're going through this state of shock in last December when basically most of the programming world switched to AI writing most of the programs. So I would approximate it to a feeling of kind of a loss of meaning, kind of terror, like dark stuff. Um, we'll keep exploring that. Uh, Steve, what do you think? What were you-- what are you feeling?
I would say most of my feelings are pretty hopeful. Um, I think I'm an engineer. I like solving, you know, problems that make technology more realistic for humans, artificial hearts and airplanes. And our ability to solve real applied problems is going to get much, much better. Our scope and reach is gonna increase, and so for that, I'm extremely hopeful. But there is a lot of uncertainty, and I think, you know, some of this fear of meaning, of losing control, of, you know, just being kind of cut out of the process that we all love being a part of it is a little scary. And I don't think any of us would be surprised at either of the kind of two paths that might take place over the next five or 10 years, where we really are just dwarfed by, you know, other intelligence, or it kind of levels off, and this becomes an incredible tool that we use for our curiosity, for our human problem-solving. And that uncertainty is massive. Those are two very different worlds. And so uncertainty is difficult for humans. I think it's also difficult for those of us who have chosen math. We chose math, you know, probably because we liked rules and formalism and, you know, uncertainty is uncomfortable. I think the humility aspect is really important, though. This is a really good opportunity for us to remember why we're doing this. Like, what is human purpose? It's not to be famous. It's not to have a Fields Medal. You know, solving interesting problems, helping other people, being curious, just wanting to understand the world-- those are really good things to kind of get back. Like, what do we need to be satisfied humans? And I think that's a really interesting moment for us to think about. Like, I've been thinking about it a lot lately. Um, it is more difficult for young people, right? Like figuring out how to find their way. But I actually have more faith in them than us in a sense.
Do you think there's something to what Terry says about it's harder to be motivated when you know a problem has been solved, to explore that space when a problem's been solved like this by an AI system?
Yes and no, in the sense that I do very much agree that the path is as important as the destination. Like, if you go on a road trip, the road trip is, you know, is usually just as fun as getting there. At the same time, if we found out that another country had invented teleportation or time travel, I guarantee we would reinvent it in the next five years. Like, if you knew it was possible and you could reverse engineer a little bit, we would be curious, we would dive into it, and we would figure it out. So knowing there's a solution also is an incredible motivation to get there much faster.
Let me ask the question this way. If this moment marks the moment of the trajectory where AI basically, quote-unquote, "solves math" or kills math, can you make the case that that is in fact a trajectory, and then can you make the case that that is not a trajectory? Can you steelman both paths? That this is a path towards which, like, mathematics as we know it is done completely, so that's gonna be a thing done by machines, and then what humans do is something fundamentally different, or there still will be something that looks similar to our current mathematics. Can you steelman both cases? Alex.
Yeah. So, you know, you said two things, and I'd like to separate them. One is solve math, and the other is destroy math. So, uh, let's make that precise. What does solve math mean? If you think back to when chess engines first came out and people were like, "Oh, this is gonna be amazing. We'll play centaur games." So this is a human being plus a computer system against another human being and a computer system, and that lasted like 10 seconds, and then the computers got so good that if you did anything but choose the top move that the computer recommended, you would lose. And so the human being's role in that game became, "Yes, accept the first move. Yes, accept the first move." That's no longer fun, and we don't see many games of centaur chess anymore, right? And so in fact, I just had a conversation with a colleague named Thomas Koberda where we axiomatized our argument for what the future might look like, and one of the axioms that I'm willing to even accept for the sake of argument is what I call the solved axiom, and the solved axiom says, let's assume at some future point, we're nowhere near there now, and I don't expect us to be there necessarily. We'll see what, you know, where the capabilities taper off if they do. But let's assume the solved axiom which says any mathematical problem that can be solved with human interacting with AI could also have been solved by AI alone. That's the solved axiom. So that would mean that really there's nothing new for a human being to contribute. You know, you can say to an AI system, "Okay, I have problem X, I wanna think about problem X," but the only way that I will make progress, any progress that it'll make on problem X, will just be press a button. The AI thinks about it. It doesn't necessarily solve it, doesn't mean that it'll solve every problem. The Riemann hypothesis is not solved, okay? That's what I really want the solution to. I just didn't think in my lifetime I would see any kind of progress towards it that we saw a few days ago. So it's not that AI systems are solving every problem. They're solving every problem that could be solved by humans interacting with AI. That's what that axiom would be. So I'm saying, if we are ever in that world, does that necessarily destroy mathematics? And my strong argument is no, I don't think it destroys mathematics.
If that axiom is true, what's the role of humans?
So I don't think that mathematics, pure mathematics, not engineering mathematics, not things that will cure cancer and solve, you know, jet-- everybody has a jet pack or whatever other technological advancement you might imagine. Pure mathematics, you know, we know that we're proving theorems just because we're curious about them. I don't think the solution to theorems living in ones and zeros in data centers is valuable to human beings. I think the value of pure mathematics is that it lives in the heads of human beings.
So just to, you know, add to this, I mean, I'm glad you came right out with the question that's on everyone's minds. So thank you for that. Um, I would caution against overfitting to one objective function for this, right? Because, you know, we used to think of ourselves as just the smartest things around. You know, human mathematicians can solve, you know, reach the highest treetops. But now we're faced with the fact that many of the problems that we thought were, you know, quite a ways out have been solved by AI. But maybe one of the consolations we can take is actually the fact that we're very computationally limited compared to the way that AI is solving it. You know, given the same finish line of proving the same theorem, I can't really hold that many papers or ideas in my head at one time. So in order for me to reach that same finish line, maybe I have to develop more generalizable and abstractable ways of getting there that then help you not just solve that one problem, but, you know, explain connections or other directions you want to get into, right? So it's always amazing when you look back on papers you've written, and you can see these threads, invisible sometimes, that connect different areas of work, because developing systems of ways of understanding problems helps you for surrounding problems, not just to get a Lean, formalized, you know, one theorem down.
But the interesting thing here is those threads, what you're saying is sometimes invisible, are quite visible to the AI systems now. It seems like that's perhaps the actual very reason why they're able to make so much progress, is they can extract the threads and pull on them because they have the view of so many papers.
I think it depends on what you mean by threads. You know, for me, the reason you prove results is so that you better understand the world around you. Mathematics, to me, is a vehicle for abstraction and understanding things. And you know, it's when you have a major result, what's important is not just the truth of the statement, but what you've learned en route to doing that. Usually, people's first question is, "Okay, you proved something fantastic, but how did you do it?" And then people, once they digest that new idea, that new definition, that new way of looking at things, there are lots of other problems nearby that fall. So somehow you're building structures that help you attack not just one problem, but a whole sea of problems. And, you know, it's not that we need fully formalized, you know, results to do that. Sometimes we reason in very sloppy metaphors that, you know, help us conceptualize the world around us.
And how much of this is something Alex mentioned offline, attention is all you need. How much of this is the human ability to shine a light on particular areas that, quote-unquote, matter? So the human ability to ask the right questions, the important questions.
That's always been the case. So within this-- before AI, there's a-- you can think of, like, here's the axioms, and just abstractly from the axioms, you can get to logical deductions, right? There are theorems that we could prove from the axioms. And there's a much, much, much larger space of theorems that we could prove than the space that human beings ever actually care about. And so, you know, so by, almost by definition, it's like a tautology that the theorems that we'll get excited about are the theorems that we care about as humans. So maybe AI can convince us that there's some interesting structure. Maybe it can say, "Hey, look over here. Let me give you this new definition or something." It hasn't done that yet.
And sometimes the mechanism by which we select the things that we care about, it could have an application. So Navier-Stokes is connected to fluids and all natural systems. But it could also be a thing that some kinda mixture of beauty and difficulty that just pulls at us. Maybe there's an intuition. You know, the prime numbers are full of these really complicated problems that maybe don't have an obvious application in the immediate sense, but it pulls at us somehow and maybe, as a symptom of the pursuit of the hard thing, great applied mathematics and engineering and solutions to real world problems comes out.
So just, you know, as an example, right? A lot of times when I'm working on a problem, I might have some, like, bigger goal that I want to accomplish, but usually what I do is I try to find the simplest, most embarrassing open problem that I still don't know how to solve. You know, that aspect of like digesting it down to something, boy, how can we not know this? It, you know, it feels like it's on the right track because then when we can solve that example, maybe we're assembling the set of tools we need to make attacks on other problems. So I do think we're gonna have to become a lot more deliberate about, you know, what do we care about? How do we use these tools from a metacognition standpoint? You know, what exactly are we delegating to them and what are we not delegating to them? And we're really at the beginning stages where we're just trying to piece it together as we go along. You know, on the one hand, looking for a proof is kind of like exploring this tree of possibilities. You might have some intuitions based on some other proofs you've seen about, you know, which directions to go in. Maybe I should go down this branch, maybe I should go down this branch. But sometimes we encounter, you know, points on the tree which have really high branching factor. We just don't know which way to go down. We don't even have a guess about if we went down them, what would we discover. And now in this centaur world of math, we suddenly, you know, have something else intelligent that we can interact with, right? And what is the right way to use that tool to search for the beautiful theories, I think will be challenging and evolving, but it's important to not lose sight of the fact that we don't just care about theorems. So I, you know, I'm a mathematician, I'm a theoretical computer scientist, but I've also been reading a little bit more broadly, you know, about social sciences just to sort of make my peace with all of the changes that have been happening. One of the things that really stuck in my mind that I liked is there's something called the doorman fallacy, which is very interesting. So the idea is that you take a complex job and you boil it down to its simplest, most visible task. So what does a doorman do? I guess they hold the door for people as they're coming and going, right? And when we lived in New York, we really enjoyed talking to our doorman. Uh, you know, they were wonderful people, but, you know, if you digest it down to just that simple job, you can think, "Oh, I can replace them just by a sliding door that detects when people are going or coming," and you miss out on all the aspects of their job that, you know, add a lot of value and are harder to articulate. And I think that's a good way to think about mathematics in the age of AI too. What does a mathematician do? You know, the simplest and most visible part of the job is we prove theorems. And, you know, theorems about conjectures that were open for hundreds of years, like Fermat's Last Theorem, these are very visible things that we've spent a lot of time popularizing. But I don't think that that's all that a mathematician does. I like to think about it as a mathematician is someone who thinks deeply, and a byproduct of thinking deeply is, you know, proving old conjectures, but it's also building new theories, building new ways of looking at old things, making new conceptual connections. And these are parts that, you know, maybe AI will swallow up too, but we're not there yet at this moment. So even though this is a momentous time, I don't think it's the right time to hit the panic button. I think it's the right time to really recenter in terms of what our values are and have these conversations as a field, and remember that what we do really has a plurality of different aesthetics and goals, and that when one thing becomes cheap, like problem-solving for well-defined things, it makes the other things that are orthogonal to it all that much more valuable.
That's actually really beautiful. The doorman fallacy, going to Perplexity here, is the mistake of reducing a person's job to its most obvious task and assuming that automating that task replaces the value of the whole role. A hotel decides, quote, "We're paying someone to open a door. An automatic door can do that more cheaply." But in fact, the doorman also welcomes guests and recognizes returning customers, handles luggage and hails taxis, provides security and discourages unwanted visitors, and signals prestige and makes guests feel cared for.
That was my favorite part. I looked up what a doorman does, and boy, we have a lot of commonalities with mathematicians. If I don't signal brand value, I don't know what I'm doing.
Exactly. That's funny. But I think the task for us humans in general, but also mathematicians, is to come up with that list, like actually make more explicit exactly what it is that humans do. And I haven't heard any of you mention beauty. I apologize if I've mentioned beauty excessively already. But there's something that mathematicians are, from my outsider sort of fan perspective, are kinda expositors, scholars of things beautiful, broadly defined.
That's exactly right, is until five seconds ago, there was a near perfect correlation between the ability to solve new deep theorems and what we actually want as human mathematicians. And what we want is exactly what you said, beauty and understanding and depth of thought, and the fact that you can press a button and get what used to be our proxy, without any effort, without any deep thought, and without any, necessarily knowledge. Like we have people saying, you know, "Do a breakthrough," and then putting whatever they find on Twitter, and now, you know, what are mathematicians supposed to do with that?
I would go one step behind answering questions and doing problem-solving is asking the right questions. So like that is how I think you measure good problem-solving is asking the right sub-questions, asking, you know, a question that is interesting and can be solved and can be interpreted. And so I feel like the value of asking good questions has always been something we value in our colleagues, and that's gonna become increasingly more valuable.
I agree completely, and just to, you know, emphasize a point that was sort of hidden in there, I think asking the right questions requires a lot of mathematical insight.
So, you know, you'll see people make that comment all the time, you know, "What are we gonna value going forward? It's asking the right question. It's giving great talks." But, you know, I wouldn't minimize, not that you were, but just to be explicit, I would not minimize the role of mathematical insight in both of those tasks, because it's when we say that, you know, talks will become more important, I'm not imagining that, you know, we're all gonna become Twitter influencers and TikTok whatevers.
I hope not. Because, you know, really what stands out is having ideas that get adopted by your community that end up becoming powerful over time. So, you know, I wanna tie this back to something that Lex brought up earlier on, which is, you know, he asked, am I being overly optimistic? Probably, right, compared to, you know, how grad students view these things. But I think one of the things that I'm most worried about in the near term is what's gonna happen to our systems and institutions. You know, it's not just hiring and tenure and all these things, but it's really that when a grad student starts off their PhD, you know, they're a new entrant to the field. They're trying to absorb what are the techniques in their field, what are the important problems, and they're trying to figure out how to make a name for themselves. And what we used to do is we would give them a blueprint that was very simple to follow. Take a problem people care about and improve it, and then people will know who you are. You know, you'll get papers in top venues. You know, doors will open for you. But now I think that blueprint has changed in a lot of ways because there are so many results that are happening right now that we're all just overloaded with it, and I think it's okay to take time to rethink that blueprint. We still care about the same things, like ideas that are gonna influence your community, but they're just harder to operationalize in terms of some simple rubric that we can give them. So I'm still optimistic. I just think it's gonna take, you know, reworking a lot of our systems and institutions to do that.
By the way, and so to, sorry, to go back to your point, I don't actually believe the solved axiom. I don't think we're there. I can prove a theorem, but I don't understand why the proof works. There's some magic that happens over here, and until I generalize somehow the phenomenon and understand it more deeply, then I'll see why the proof works the way it does. Maybe that's one of the reasons why I don't actually believe the solved axiom, because the things that we do are much more exploratory and what constitutes a good question beyond just, "Here's the Riemann hypothesis, make some progress on it."
So I think you also brought up a really important point, this, like, generality and generalization of our concepts. So related to beauty and mathematics, I think one of the gold standards has always been how simple can your result be and how much does it generalize. And, you know, out of this infinite space of theorems that could be proven, finding some structure that is simple and generalizes, like, that is, I think, still the gold standard. And unless I'm mistaken, humans are still asking the questions. So, like, we, you know, prove existence uniqueness of Navier-Stokes with AI, but humans are guiding this. Humans are still asking the questions. And, like, I would love to see the next breakthrough if you had a machine that can prove any theorem you pose. I want it to start telling me what questions I should be asking.
Do you think it's possible that that machine just does a for loop and finds all the simple and generalizable, basically all the E equals MC squared?
That's happening today, actually. So, like, there are absolutely research programs that are systematically, automatically discovering kind of minimal physical laws. So this is less mathematical theorems, more physics governing laws and equations.
So observing natural systems and trying to see in those natural systems if there's simple physical laws that model how they operate.
Yeah, distilling it down to, like, the basics. So you can kind of recapitulate known theories. If you observe balls dropping, you can, you know, deduce gravity and F equals MA. If you observe fluids, you can deduce Navier-Stokes. And then taking that to data, we haven't written down simple equations: weather, turbulence, solar flares, things like that. And I'm assuming similar things are happening in math where, you know, there are probably these, like, minimal pieces that can be deduced.
Do you think there's a large space of simple and generalizable things in math that could be discovered automatically?
So my version of the solved axiom is that anything that I do as a human mathematician, I've tried on several occasions to sit down and say, "Okay, you know, LLMs will never do this that I do." And I'm like, "Well, why not? I can ask a question. You know, I can ask it a question. I can ask it, 'What questions should I ask?'" Right? There's sort of it-- Because it's such a generalized system, uh, I'm not saying that it's good at all of those things now. But my version of the solved axiom is anything that I can contribute as a human being, it could also have contributed on its own. And so what I really mean by that is in terms of value, what do I bring to the community? It can't be I asked a great question. It can't necessarily be in this world of the solved axiom, which again, I don't believe. But even in that world, it should be something other than I asked a great question, I proved a great theorem. And even in that world, I think you need mathematicians, people with human mathematical expertise.
Ankur, do you believe in the solved axiom? Where on the spectrum of, like, how close we can get to it, and in that spectrum, what's the role of the human, do you think?
I guess I don't really believe in the solved axiom. I think-- You know, maybe it's half true in some ways, and figuring out which half of it is true will be very interesting and challenging. So I think there'll be major surprises and more to come in terms of what AI can solve. But, you know, for me, when you're facing a new technology that's really, really good, the danger is that when you try to conceptualize why it's working, you do so by anthropomorphizing it. So, you know, maybe one of the earlier lessons here is from deep learning. So there are these amazing examples. You know, you can take neural nets, and you can do human-level or better object recognition. But in general, the way that it succeeds is less robust than how a human does, because there are all these examples like adversarial examples, where you can add very slight patterns to natural images that can completely fool these object detection systems, but not a human. And, you know, I had a former colleague at MIT, Alex Madry, that had some really nice work, which I really enjoyed. It was, um-- I think the title was something like, uh, Adversarial Examples Are Features, Not Bugs. So they did the following cool experiment. So you take a bunch of natural data, maybe pictures of cats and dogs. You train a deep network to identify them very well. You add some adversarial perturbations to them that, you know, make to the AI system a cat look like a dog and vice versa. So it's kind of think like a small little pattern that changes how the object detector works. And then you create a totally fake data set. You know, it's got pictures of cats with dog patterns on top, and you label it as dog, and vice versa for cats. And amazingly, when you train a deep network on this with all the wrong labels, it's gonna learn something that does well on natural data. So somehow, when we recognize what a cat and dog is, we do it, you know, in a way that we can articulate in terms of, you know, its legs and its tail and its shape in general. But a lot of times when, you know, deep networks or AI can do things, they might be able to find trapdoors or backdoors to solving the problem that use a lot of statistical relationships in the data that a human doesn't. So to me, you know, it's important to not, you know, draw this boundary, you know, very sharply between what you can do and what AI can't do because you're gonna be wrong. But that doesn't mean that I don't believe there will emerge some kind of boundary, if only for the fact that I think we over-anthropomorphize the way that they solve problems. And when I'm looking through the OpenAI solutions, they just don't look like a human has solved those problems. So to me, there still seems to be a very big qualitative difference, and that's an opportunity really for collaboration and centaur math more than it is, you know, dread about the AI overlords. But, you know, who knows? Could be wrong next week.
What's your intuition about how much of this is AI just tying together a bunch of thousands of disparate papers and pulling at the threads, sorry for bringing back the metaphor, but the threads of the different papers and then just basically articulating the proofs that were already there in the different papers and you're making minor connections? Or is it actually developing new mathematical ideas? Have you seen glimmers of whatever that genius AlphaZero move in the game of Go?
So I haven't read all seven hundred of their papers. The ones that I have looked at, you know, some people speak about this dismissively like, "Oh, it was in the convex hull." You do the moves that we know as humans, and you just do them really well and do a really good tree search. I wouldn't call it the convex hull. I think Tim Gowers had a really good way of talking about it instead as the semigroup generated by all of these things. Yes, it's the same notion of putting together known moves, but the space of known moves, what it can accomplish is sort of much, much bigger than this kind of derogatory convex hull. So with-- in terms of the quasi-RH, the moves that it took are all things that were known to human beings and there's a very long cascade of them that is kind of shocking when you look at it and say, "Oh my God, that's so beautiful. How did you think to-- How did you..." I'm anthropomorphizing it, of course. "How did you think to do this move after that move and after that move?" And yeah, that does it.
Could you articulate to the more general public the intuition, the line between semigroup and convex hull, the derogatory convex hull, and quasi-RH, of course, is the Quasi-Riemann Hypothesis, that one of the big proofs.
Yeah. Let me try to do it like this. You have-- think of like a balloon. So the inside of the balloon is its convex hull. And so we have like all of these results that-- all these ideas that mathematicians have thought up, and then anything inside of that is some linear combination of it. But it feels compact. It's like there's this kind of closed, enclosed area, and within that area, like AI hasn't really made new progress 'cause it hasn't moved the balloon out. It hasn't given us a new idea yet.
And it's derogatory because the intuition there is we could have done it, mathematician could have done it because it's already quite obvious. It's already within the space of things that have already been thought about and developed.
Yeah. So it has read every paper. It's exactly what Ankur was just saying. It has read every paper. It has internalized all of that. That's more than any human brain not even could hold, but has time to read and learn and internalize. So in terms of breadth, yes, it beats any human being. And then in terms of depth, you know, has it really generated a genuinely new idea? And you can argue that at least in the case of quasi-RH, I would not say it's a genuinely new idea. It's a beautiful combination of a lot of things that were very well known. There are a lot of people who are kicking themselves that had some idea in that direction twenty years ago, and two years ago, there was a big breakthrough on cubic theta functions and cubic Gauss sums and, you know, that ended up being the path that the AI took to the solution. But so to contrast that with a semigroup, I think of a semigroup as like tiles. So you know, you're just playing tiles on the board, and what you're allowed to do is whenever you have a tile, you can put another tile to the left, to the right, to the front or to the back. And then you can just start tiling things. And that's like a semigroup action. And the question is, well, how far can you reach with that? You can go all-- I don't know. You know? You can go really, really far out with existing ideas might prove a whole lot more than we as humans have been able to explore.
So your intuition is more in the semigroup direction, meaning like there is at least glimmers of expansion versus linear combinations inside the convex hull.
You know, how we choose to interpret what it's doing I think is up to us. But throughout the history of mathematics, there have been very seemingly difficult and deep problems that then someone comes up with a three-page solution that is very elementary and could've been explained to an undergrad and could've been discovered twenty years before when the problem was first posed. And had it been discovered twenty years before, it would be an exercise in a textbook somewhere. And because that solution wasn't discovered, and really bright people worked on it for a long time and made incremental progress and made the problem really famous, then by the time it was actually solved, it was a really big deal. That happens from time to time in mathematics. And what I said a year ago when I wrote-- So there was this, um, International Congress of Mathematicians this summer, and I was writing my proceedings that they wanted ten months in advance. So I said, you know, "What am I gonna write about math and AI now that will still be relevant ten months from now?" And I think what's there is somewhat evergreen. I stand by what I wrote. But one of the things that I wrote there was exactly that we're going to discover that a whole lot more problems that we thought were unreachable by current technology actually are reachable by current technology if you can put it together in just the right way, and that's what AI is amazing at. Again, someone could interpret that as, "Oh, so it's inside, you know, these solutions were inside the convex hull." You wanna call it that, that's fine. I think that's, you know-- It's not invalid.
So I like all these metaphors, you know, also the balloon. So the important thing about the balloon is the boundary between what's inside and outside, and what makes it derogatory is the fact that, you know, you haven't seemingly changed the boundary. But I-- you know, definitely these problems have changed the boundary. And, you know, maybe a more concrete thing is just that the types of things that people are gonna work on after that drop of OpenAI problems is forever different. You know, there were at least a couple of results there that are like, you know, field-altering results that will change what types of problems you give to young researchers, what areas you want to explore, because your conception of the landscape is just that much different.
So not just because it expanded, but because the very expansion changed the direction of the interesting questions.
That's right. So in fact, you know, one of the problems which probably, you know, maybe it's the easiest to explain what kind of effect it had on people is not actually from the OpenAI drop. It was an Anthropic result that was written up by my colleague Virginia Vassilevska Williams and then Josh Alman. So this was about the 3SUM problem. And I'd say that there it has a major effect on what's called fine-grained complexity theory because there was a big surprise that, you know, a very basic problem that you give people in, like, coding interviews can actually be solved asymptotically much faster than the solution anyone knew, and that was counter to what the community was betting. In fact, there was a whole theory built up around the idea that if this problem really is that hard, then all these other problems inherit its hardness. So now, as a result of that problem being easier than we thought, you know, having a bit better of an algorithm, you know, yes, that gives you a lot new algorithms across the entire landscape. But it also changes the character of the field because the goal there was to put all those problems together and said that we should be studying them all together. But now we're sort of back in the state of affairs where maybe the right way to study them is separately after all. So I think these are examples where, you know, irrespective of how much, you know, deep new generalizable ideas we get out there, even just knowing the, you know, truth of the theorem is gonna change the way that people organize research in that area. And, you know, there's no putting the genie back in the bottle on some of these problems.
I mean, imagine one of the seven hundred was factoring in polynomial time. That would absolutely change-- Everybody would know. Like, you can't use your credit card on Amazon anymore. All of e-commerce is gone. All of secure communications is gone in-- like that.
Can you explain factoring in polynomial time?
Yeah. So if you give me two very large numbers, a million digits each, I can multiply them together and do that in a split second. If you ask me to go backwards, say you took two large prime numbers, you multiply them together, you give me this, the product, and you ask me, "Figure out which two numbers..." You know, just factor. Don't we, don't we ask our kids to learn how to factor in school? You know, it's a lie. We do not know how to factor. And the fact that we don't know how to factor is why we have the capacity, the ability to communicate with one another in all kinds of secure ways. So all of modern-- basically the modern world depends on the fact that we don't have an algorithm for taking a large number and pulling its prime factors out of it.
So a lot of e-commerce does because that's the basis for a lot of the schemes in public key encryption. But, you know, famously, there are quantum algorithms for factoring, so there's a lot of emphasis these days on, you know, finding post-quantum, you know, bases for cryptography too.
Yeah, lattice-based methods and-- Yeah, we're already concerned about what will happen when we have quantum computers large enough to, but we still have a little bit of time before quantum computers get large enough to make this an actual problem. But there's no theoretical reason. I mean, okay, factoring's probably not NP hard but You know, I mean, let's say, let's say prove that P doesn't equal NP. Maybe that doesn't-- That would be amazing, obviously, but that doesn't if it proves that P equals NP, that would be completely...
That would be very interesting.
That would be very interesting.
I'll definitely ask about more broadly in the context of complexity theory, but presumably OpenAI or any other company would not drop that result carelessly.
That's the kind of thing that should go through, yeah, the government and...
Sometimes certain mathematical results quite obviously lead to dramatic impact on the world. Other results might take maybe months to digest, but that one would just, yeah, would affect the economic systems of entire nations, so.
How many mathematicians does the NSA employ?
Thousands, like something like this. I've always done this thought experiment, you know. What, what might they have that they just wouldn't tell us about? And this was ten years ago, you know, that some evidence showed that collaborative teams of mathematicians could solve much, much harder problems than any one individual mathematician if they, like, broke it down in the right way. So I don't know. Do you mind if I just revisit the...
...the balloon for a second? 'Cause I actually like the semigroup analogy better than the convex hull, but I don't think it's as it should be interpreted as derogatory as it is even if it was just that these machines are interpolating human knowledge rather than extrapolating. Because even if there is this convex hull of ideas and things that could be pieced together from bits that humans have invented, we're not doing that fast enough or, you know, there's a lot of opaqueness, and it's-- this balloon is not see-through. And so even if you can make this convex hull much more transparent, kind of like lift the fog, give mathematicians radar so they can see what's, you know, over the horizon, that's pretty incredible, and I don't think we should be interpreting that as, you know, derogatory at all. I do think it is more, more extrapolatory than that, though.
Yeah, we should say probably the size of the convex hull. We don't know actually how big it is.
Well, that's part of the point. We don't know how big it is. We don't know what's next to each other. Like, there's so much fog of what we know. Like, and that's exciting to lift that.
But human beings do make new definitions, you know, bring new concepts in, and that fundamentally feels different from combining existing ideas and repackaging them. When human beings do that, and they do that from time to time, it really feels like, okay, there's a new boundary point, and now what's the convex hull of that, of what you could prove? So it's like you can take the balloon, and there are ways to pull on it and...
And we know when it expands. Like somehow we have an intuition for when it's getting bigger. Like, you can recognize that.
In many ways, the archetype of what the researcher looks like is gonna change because of this new technology too. You know, I'm always amazed that when you look back historically, how much very different fields of math really spawned from similar ideas. And a lot of those connections drove the development in the beginning, but over time, they become a bit more specialized. And I think certainly the way that we train graduate students, you know, we'll need to make sure that they're not just overspecialized in the same way. You know, I don't know what the future of hiring will look like. I don't know what the future of journals will look like. But I think people who think broadly about, you know, a bigger swath of mathematics will be in the best position to benefit from these technologies. You know, that would be my guess.
Can I ask an almost silly-sounding question? But so a large part for many decades of research has been the paper. If you were to guess in the years to come, how does that change how a human contributes their idea, maybe collaborate, how a team of humans contribute an idea into the public space?
So, I mean, I don't know about in pure math, but in engineering and some of the applied sciences, we already have a pretty rapidly shifting view of what an academic contribution is. So if you write code in part of a larger infrastructure that is reusable, that, you know, supports larger projects, so think of LIGO or some of the big, you know, billion-dollar astronomy projects or, you know, cancer research projects that require hundreds of people and huge resources already, it's not necessarily one 15-page PDF. You know, there's lots of different pieces, code, data, reusable bits that can be fit into a larger framework. And in my mind, that is kind of obviously even more relevant with AI, is having, you know, a little workable piece or widget or API or code base or data set that can be leveraged by the community or by AI is increasingly valuable, even more than just, like, one result or one paper. And I don't know how that looks in pure math.
Can you comment on your view how everything's changing? Even just observing the insane evolution of even arXiv, which is similar in some ways to what's happening on GitHub, for example. A lot of people in the open source movement are really struggling to know what to do. Some are predicting the death of GitHub.
Why? Just 'cause there's so much code being added to GitHub that it can't sustain the growth?
So one of the things that makes the open source movement successful is there's a kinda one or a few folks that really manage each repository, the human beings, and they are, um...
psychologically and on many levels are becoming overwhelmed by all the pull requests. So it becomes impossible to manage and they get exhausted and there's this whole feeling. The open source movement, maybe similar to the academic environment, requires the humans to be motivated. And if you get worn out by the just similar the ideas that Terry Tao mentioned, that you just quit. And so that's a failure mode that you want to kind of avoid.
So journals were already in math under very, very severe strain from the number of things before AI, having nothing to do with AI, just there was, you know, more people producing more knowledge, sending it to journals, and journals had to referee it, and referees are exhausted and being asked to work really hard to understand these theorems and check them. The only thing worse than that system, I think, is not having that system. Because the way that, like, our mathematical epistemology, the way that we arrive at truth, is deliberately slow and careful and precise. Go back to Perelman's solution of the Poincaré conjecture. He puts these papers on the archive, and then people around the world are, you know, shocked by this, and so they invite him to come give lectures. He gives-- he comes to Stony Brook, he comes to NYU, he comes to Princeton. He's going all around the country and the world, and he's giving three-hour long impromptu explanations of, of, you know, maybe an hour is him explaining what he's doing, and the next two hours are people saying, "On page two hundred and forty-seven, Lemma four hundred and twelve, I don't understand how you do this." And then he goes to the board and gives them a half-hour lecture on how he does that. So the entire thing lives in his mind. He's done the best he can to communicate it on paper, and then it's up to us, the community, to go back through it very carefully, very in a detailed and precise way and agree, yes, this is a real proof of the Poincaré conjecture. And that took years to do and a whole lot of human effort. So coming up with a solution is only the first step in the process, and that process has to be deliberate, and it has to be community-driven. And so that's what the journal system is trying to accomplish.
So you make a really interesting point and it kinda highlights this traditional view of a pure mathematician as this, like, lone genius who, you know, solves this singular problem and then goes explaining it to the world. In many other fields, we have moved past the point where we can't individually advance, you know, circuit design or aircraft design or protein folding or any, you know, astronomy, and we have to work together in these billion-dollar collaborative projects. And again, I'm an engineer, so I'm thinking leadership is obviously the thing that a community needs when it's having this kind of crisis of organization and individuality moving towards a more collective model. And that's interesting because I know so many mathematicians, and, like, by kind of organization, this is a very independent group of people who don't like being told what to do, who don't like being told what to study, who are very independent by their very nature. You know, we like solitude and working alone. That's why we, you know, spend so much time doing it.
I think that's a really good point. I mean, it's interesting just to see the differences in working styles, even between math and physics, you know, with things like the Large Hadron Collider. So I mean, let me take a contrarian view for these things, which is, you know, you should never let a good crisis go to waste. I agree the journals were overloaded, the conferences were overloaded. It always strikes me that we're doing so much work, you know, to review these things, to handle all these things, and now we're faced with the task that someone's gotta review all these AI-generated papers. And some of them are important, but, you know, it's really hard to get your community on board with doing that. You know, I don't know what's gonna happen, but I would really be in favor for finding new models, not to replace the old ones, but to relieve some of the pressures that have come about. So, you know, I'd love to see models where, you know, maybe we take a more collaborative effort to verifying some of these AI proofs, right? So some of the OpenAI results that have dropped, you know, improvements on Fast Fourier Transform. You know, there's been tweet streams with people improving the constant. I don't really know what to make of that. You know, I'm not even on Twitter. You know, and it seems to me that we need to figure out other modes of...
working, you know, for organizing what are the state of all the improvements, but also how do we organize and tame these ideas? So it's kind of striking that, you know, the way that we think about publications is you work hard on your draft, you post it on arXiv, and that's your draft. You know, you might update and create a new version at some point or a third version, but the draft belongs to you. It's not really up to the community to add notes. "Hey, I think you can do this argument differently. Hey, I think this is simplified this way." But maybe there are some kinds of things that we need other models for, you know, allowing crowdsourcing for our understanding to evolve.
I'd be all for that. You know, the other thing is I love giving research talks. You know, one of my favorite parts about giving research talks is you can take all of the messy complexities of the paper, the things where you just knew you had to dot the I's and cross the T's, but that wasn't really what excited you. There was some key idea, and there was some special case that really illustrated why that idea was powerful. And that's the part you really want to explain to people, because when asked, someone asks me about Lemma 57, I don't even remember it, right? And maybe, you know, this idea that a paper is just this monolithic thing designed to be, you know, written down and completely formalized. Well, now we can trust Lean formalizations to some extent for some of the technical building blocks, and that can allow us to rework the way that we do the writing in the first place, so we can put more of the emphasis on the work on, you know, the promulgation of those ideas.
The Mathlib model is exactly what you were describing, where instead of a paper, you have a PR, you have a pull request. The maintainers and the reviewers will go through the pull request. They'll make suggestions here and there. I mean, the referees make suggestions, you can improve this and that, but it's not a public process in a journal. In a GitHub open workplace, you can see exactly who contributed what idea that simplified an argument or you're, you know, you're reusing this... You're reinventing this idea that already exists in this part of the library. You should just quote that and refactor this part, whatever. So and Mathlib working formally allows one for the first time. Like, one of the problems with mathematics is that we don't have a perfectly well-written specification to our theorems, our technical lemmas. There's-- If you open up a random page in a random math book and read a lemma, there's so much context... that the author-- 'cause it's so annoying to write something that fills in every bit of context. So we just assume you've read the previous sixty pages. Now I can just tell you what the next step is in that lemma. But then when you try to-- That means you can't just take it off the shelf the way you can a part, a physical part when you're building a rocket. And so if all of mathematics is done in the future formally, then the specifications are built into the object. You know you're not using it in the wrong way, 'cause you can't use it in the wrong way 'cause Lean won't let you, and that is the sort of crowdsourced version of mathematics. And that was the dream of Mathlib... when it was started ten years ago, was to run mathematics more like a science.
As a quick aside, can you mention what Mathlib is and Lean, just 'cause we mentioned it a few times for folks?
Sure. So mathematics is, in the ideal, a formal process starting with an axiomatized system and then logical rules of inference from there. So if people think of -- I don't know, do people still learn Euclid's geometry in high school or something? That's supposed to be the kind of model for kids to start seeing proofs for the first time and seeing an axiomatic structure. And you can take that idea and write it in software and say, "Here are the actual axioms of mathematics. Here are the rules of logical inference." And Lean, there's a long history of this kind of software. It's called interactive theorem provers, where you're writing line by line in a proof, and the computer says, "Yes, this line does follow by logical inference from the goal state on the previous line." So Lean is software that allows you to check all kinds of things. It can check the specifications of your code align with what you, what you are trying to define. That's what it was -- what it was invented for was bug-free code. And hopefully, in the future, we will have bug-free code because AI systems are really good at finding bugs in code written by humans, so that's-- it's becoming ever more important for that application. But another application is to check mathematical results... and allow you to write a mathematical result in a very formal way. And so now Lean itself is just the hardware that checks the logical steps. You do not want to start every math project by defining what the natural numbers are, using zero and the successor, you know, the Peano axioms of the natural numbers. So it's nice to have a large general library that's extremely well maintained, extremely well organized, and so on. And that's what Mathlib is. Mathlib is the Lean library for formalizing mathematics in the Lean ecosystem. And again, Lean is just one interactive theorem prover. There are lots of others. It's the one that so far has been winning the math wars, if there are any math wars. It's a competition of two people or something.
And it's the technology used in part to show that the OpenAI release of all the different proofs, that the proofs are correct.
That's right. It has become now standard to, because autoformalization has gotten to the point where AI systems can not only prove a theorem in natural language. You can't trust their natural language proofs. These are stochastic processes. And so the fact that they're saying things doesn't mean that they necessarily have a model of mathematics in their heads, whatever that means, the same way that we do. But that's okay 'cause they'll write a Lean -- Lean code that builds on top of Mathlib, builds on top of the Prime Number Theorem Plus project or whatever, and checks that it has indeed solved this problem down to the axioms.
So this is where I'll just take the tangent, and the thought experiment may be a bit philosophical at this moment, but what if you have that black box that gives you proofs that you can't understand? Let's say the more interesting case where, not because it's so big, but because the ideas just don't make sense to you, to any of the world expert mathematicians in that field. What do you do with that?
So the only way, sort of almost by definition, because the logical rules of inference are written down, and there, there's only a few of them. There's a handful of them. The axioms, there's a handful of them. So the only way that you have mathematics that's not understandable is scale.
Let me add a contrarian take.
So one of the things I also wonder is what would unformalizing math look like? You know certainly formalization helps tremendously that we have confidence that the results are correct. But much of the way that we understand the results ourselves is really at some unformalized level. You know, what is an interesting special case? What, you know, where does this come from, you know, this technique you're using, right? And, you know, maybe that's what that world would look like. If we have these black boxes, we would try to tease apart and figure out how to ask them easier questions to develop some kind of system of understanding of how they actually got to the answer.
Yeah, so there are really two different things that we conflate when we write a paper. One is the engineering of the proof, and the other is the communication of that to someone else, and they're almost at odds with one another. I had this experience-- So I taught real analysis last fall, and you open up any real analysis textbook, and this was my experience when I took real analysis. The first couple of lectures are what are the real numbers? You know, we have to define them or construct them as equivalence classes of Cauchy sequences of rationals, whatever the hell all that means. It takes, like, two weeks for a first semester course in real analysis to get there, and then we have the real number line, and now we can start doing calculus, you know, with formal epsilon and delta definitions, yada yada. And it always struck me as bizarre that we do this because before the class, I knew what the real numbers were. It's this number line, and then we spend these two weeks talking about Cauchy sequences of rationals, and then on week three, they're, they're a number line again. What the hell was the purpose of that? And so it's because the way things are written or had been written were very engineering. You need to construct the real numbers before you go start using them. But the real numbers, human beings used the real numbers long before we understood that they needed to be constructed formally. So what I did in my class is I jumped straight into epsilon delta calculus. We did, you know, six weeks of calculations until we got to a point where wait a second, you know, if here's a theorem, it's true in the real numbers. There's some convergence, something called Bolzano-Weierstrass theorem. You have a sequence of real numbers that are jumping around, but in a fixed range. There has to be some subsequence that converges to a real number. Well, that's not true in the rationals. Why is it true in the reals? What are the reals? So you don't start asking that question until you hit a problem that needs to be solved. So the way I taught the course was sort of backwards from how it needs to be taught if it's an engineering project, 'cause you can't just start talking about the real numbers until you've constructed them, until you know what they are. Well, that's correct if what you're doing is building up the theory. But if you're telling a story, then you absolutely can just start with the real numbers. You have an intuition for what that is, and until your intuition breaks, you don't need the resolution of how you actually construct them. So I think the way that we write papers in the future is gonna be very different, where we spend a lot more time just talking about the story of what have I learned in this process, what are the interesting-- Uh, here, here's, you do the standard thing here, here, and here, but then here, you're sort of stuck, and where do you go? And here's an idea for why you go left and not right and not up and down or whatever. And there's a Lean formalization that accompanies this story that lets you know that the engineering actually works out. 'Cause you can't just tell stories. If you just tell stories, you know, math is not music where you can tell a beautiful story, and it's okay that you have parallel fifths moving somewhere, and, you know, you've broken some rule, but Bach does it all the time, so.
So I think that's a very important point, just to sort of emphasize the point that Alex is making. You know, there's this view of mathematics which I don't believe in, where it's just this rising tide of theorems. You know, how can AI be bad? It's contributing to this rising tide. But, you know, really, mathematics is not just about the number of theorems we have, far from it. It's also about that story of fighting with problems, because when you fight with a good problem, it fights back at you, and you learn something about it. It's really about the system of beliefs and abstractions you come up with en route to, you know, when you're fighting this problem, you discover some more general meaning that applies more broadly. And I think it's a great opportunity to sort of disabuse ourselves of this idea that we're just, you know, axiomatizing things until we get to the next theorem. You know, telling that story of how we actually got to these problems, what the parallels are to other problems, is now, at this pivotal moment, more important than it ever has been.
Yeah, so to get all the way back to your question about, you know, is it a matter of scale, well, yes, it's just a matter of scale, but exactly to Ankur's point, the way that we think about mathematics is about stories, and it might be that we don't understand the story of this. You can follow a proof line by line and not understand what's really going on. You might say, and you just ask some other LLM to explain to you the story if it can come up with such an explanation. But that's what we want. We want stories. We want understanding.
When you fight with a problem, it fights back. I think there's a deep truth to that about all the great pursuits that humanity has taken on. And on that topic, I would like to talk to you about some of the greatest fights of all time with some of the hardest problems in mathematics that have fallen. Um, but first, we'll have to break if it's okay. So Alex, you're a number theorist. Um, maybe it's fair to say the central problem in your field is the Riemann hypothesis, and you showed a lot of excitement on the aforementioned Twitter/X about the quasi-RH, the quasi-Riemann hypothesis proof. Can you explain that? Can you explain what the Riemann hypothesis is, and can you explain the nature of your excitement about it?
Sure. There's sort of a one-minute version of the Riemann hypothesis... or a five-minute version or a thirty-hour version. I don't know how far you wanna, you wanna go.
Let's start pulling at that string.
So we know what the prime numbers are. People are interested in the distribution of the prime numbers, and what Riemann discovered is that there's a function, this is totally crazy, that there's a function that takes complex numbers as inputs and has complex numbers as outputs, and that function governs the behavior of the prime numbers. It's called the Riemann zeta function, and by the zeros of the zeta function, we mean complex inputs where the output is the number zero in the complex plane, so the, the origin in the complex plane. And the question is, where do these, where are these complex zeros? What inputs, what complex number inputs give you a complex number output of zero when you apply this zeta function? Why in the world should that-- What, complex numbers? Are you-- What are you talking about? We have these discrete numbers that aren't divisible by anything else sitting in the natural numbers on the number line, and now you wanna talk about functions from a complex space to a, to a complex space. Um, that was the great breakthrough. The great breakthrough was to figure out how to do signal processing on the prime numbers. If you think about, you know, the reason people can hear you, your podcast is that my voice box is producing pressure waves of sound. That's the natural space. Then that goes into the microphone. The microphone analyzes it, does the Fourier analysis on it, converts it into amplitudes and frequencies, and it's just a bunch of numbers, stores those numbers or sends those numbers, whatever. The speaker then takes those numbers and reconstructs back out what sounds to people like my voice. The same thing is going on-- That's what the zeta function does for the prime numbers. It takes the prime numbers, you can think of the prime numbers as particles of air molecules wiggling around, and it does a signal processing analysis on it that can recover the same prime numbers back. And the frequencies and the amplitudes correspond to the zeros of the zeta function. The zeta function has that information in it. And the Riemann hypothesis, so the amplitudes of all of these, you can call them Riemann waves, and the superposition of those waves recovers the distribution of the primes. And all of those amplitudes are supposed to be exactly the same. Like, the orchestra that plays the, the music of the prime numbers has every single instrument sounding at exactly the same amplitude. That's what the Riemann hypothesis predicts.
So can you-- Actually, we know what the prime numbers are, but maybe going into the space of philosophy and the romantic notions of aesthetics, why are they so interesting and important and central and beautiful at the core of mathematics? And, connected to why the Riemann hypothesis is exciting, is why is it so difficult to figure out if and what the pattern is in the prime numbers?
Let me start with your second question first. Why something is difficult I only know how to answer once I know what the answer is.
Right? Maybe it's not difficult.
Maybe it's not as difficult as we thought it was. I did not think I would see a proof of the quasi-Riemann hypothesis in my lifetime. So the fact that I saw it on Tuesday was extraordinarily shocking.
What did you do the moment you saw it, like, in 30 seconds after? Like, what was your-- Do you remember, like, what-
Yeah, I remember where I was... the second I saw the drop. So we were at this National Museum of Mathematics Gala... in New York City. And so I had, you know, a whiskey in one hand, and, I kept refreshing. I was talking to people, socializing, but I also kept refreshing 'cause I knew it was dropping around six o'clock Eastern Time. And then the drop hit and, in fact, I was talking to there was a Quanta reporter there and we were just chatting, "Oh, yeah, they're gonna drop these, you know, 300, 400," we didn't know how many, 700 it turns out problems. And I was like, "Oh, wow, that's really cool. It dropped." So I'm looking at my phone, and, like, the second thing I see is quasi-RH. And I'm like, "What do you mean, quasi-RH? You don't mean, like... For what?" And then I look, and it's, it's the zeta function, and it's all Dirichlet functions, so they've solved no zero, zeros. And I'm like, that can't, that can't be what's in this drop. That just-- I, I could not compute... in that moment what I was seeing.
That speed of progress in that little time, yeah.
It was so far outside my model of what was possible. And this is-- It's been a month of things happening way outside of my model of what was possible. The Fermat's Last Theorem formalization, I was sure we were three years away, at least, from being able to do something like that.
So, maybe you can elaborate on that. It was in Lean, it was formalized as... How big was the Fermat's Last Theorem in Lean?
So I just-- Two days before, I gave a colloquium in the Princeton Math Department where I explained in painstaking detail what Lean is, what Mathlib is, but also that AI systems today, you know, there's this Mathlib library where everything is kind of perfectly done, exactly the right levels of generality, enough-- There's no-- There's not supposed to be any code that's reusing code that's reproving theorems somewhere else. Everything is the most general, the most abstract, and what AI systems are capable of doing is building, like, one or two layers on top of that. But they're, what they're doing is jerry-rigging. They have not been trained yet, I'm not saying they can't do it, but they're not good at it right now, at what I call canonization, which is taking kind of sloppy code, refactoring it, figuring out exactly where in the library it belongs, what's the canonical version of this statement, of this lemma, and then everything, every proof should just be a combination of you apply this lemma from over here, you apply that lemma from over there, put all these things together, there's your theorem. And then it's really clean, it's really beautiful, very elegant. That's what Mathlib is going for. And what I said in my colloquium is, where we are today is that you can get entire textbooks formalized as long as they're in the Mathlib halo. As long as they're, like, one, maybe two steps removed from things that have already been done in Mathlib. But when you start, and this had been my practice, my experience had been, when you start asking it to do things that are, you know, three or four steps removed from the library, the way that it came up with a definition of this object and the way that it came up with some other definition of the same object, 'cause it was doing it for two different purposes, then they clash. You can't stitch these things back together. Like, you really need the library, Mathlib, to grow... in order to get to a place where you can do research manuscripts, you know, that are, that are really using, synthesizing tons of different areas of mathematics.
Which is why you would expect Fermat's last theorem to take-- because it's so far outside of the halo of Mathlib, it would take quite a bit of time.
Exactly right. So Kevin Buzzard had a million dollar grant to spend the next five years, or I th- maybe he's already a year or two into that grant, to reduce the formalization of Fermat's last theorem to things that were known in the 1980s. So he wanted to do that last step, the, those seven years of Wiles' work, and Wiles Taylor, to finally finish, you know, Fermat's last theorem, but built on top of things that we knew would be extraordinarily difficult. He wasn't gonna do Langlands-Tunnell, he wasn't gonna do Mazur. There were, there were all kinds of things that were expected to be even much more difficult than what these five years would take. And everything I said in that colloquium was wrong. And I think my colloquium was on a Wednesday. On Friday, there was a drop of thirteen million lines of code from the axioms, Fermat's last theorem is proved. The statement of Fermat's Last Theorem is completely elementary. There's no semantic alignment to check. You just glance at it. Yes, that's the statement. It's-- Lean says it's sorry-free.
Can you explain how it was done?
How it was done. So it was Anthropic, and how it was done was AI systems just got really, really good at-- It's still not a nice proof. It's not the clean proof. It's not-- There's still a ton of work that needs to be done to make that proof a Mathlib quality proof, and I think that's important work and work that we want to do. But in terms of just getting that sorry-free theorem, they just blasted through all of these. They jerry-rigged this, they jerry-rigged that, and, you know, thirteen million lines later, they had a sorry-free formal Lean proof of Fermat's Last Theorem. It's just absolutely shocking.
By the way, it's beautiful to always, um-- Because of how rapid everything is, there's going to be a lot of statements that everybody makes, the world experts make, that are wrong. That's the whole point of it. And that's the fun thing about human discussion, is we have disagreements and I would venture to say somewhere between forty to eighty percent of everything said today will be wrong in five years because everything's changing so fast. But that's the whole point of human discussion and deliberation and discourse and the, even the scientific process is saying shit and seeing what sticks, over time. Over ten years, over a hundred years, and so on.
I was describing my experience. I had a lot of experience with Lean and, and with using AI. I started doing Lean before AI existed in the, in the sense that we understand it now, and it was very painstaking. And the second, uh, AI systems came on, you know, the first LLMs were coming online, they were, they were crap at Lean, but then they got a little bit better, a little bit better, and I was like, "Oh my God, I can do library search." I don't have to sit there looking, I know there's a lemma that does this, but I don't know what it's called, and I don't know which exactly. Is it in the manifolds library? Is it in the algebraic geometry library? Like, give me this, give me this lemmas. It was great at that, and I was like, "Oh, this is amazing. I'm gonna keep using these systems." And every bit of my experience told me that they can do, like, X, but not ten X, and this was a million X. It was just completely outside of the distribution in my mind of what systems were capable of.
Do you have a ballpark estimate of how many humans were involved in that, and, like, what is the level of human involvement in generating those thirteen million lines of code?
Uh, I believe it was two or three humans.
And I believe most of their involvement was, "Keep going," all caps.
Okay. All right, so you think it's really just, like gentle guidance, not like, you know, pathway creating and-
As far as I understand, there was no domain expertise... on the parts of the people who were doing this.
As the meme goes with OpenAI or Anthropic, you know, the prompt is solve math... or something. And then-
Yeah, yeah. Yeah, do a breakthrough.
Do a breakthrough. That's the prompt. And it goes and does it.
Of course, you never know from a company perspective, there's a lot of incentive to sort of sell it as that, even though there might have been a lot of intermediate steps that led whe- at any stages of the training or maybe some things left unsaid, even though the final prompt is do a breakthrough, there could be a lot of groundwork before that, that led to, we don't know.
So I remember that was a Friday because I just was completely, like my mind was completely exploded, and it was Labor Day, and we just took the kids to the beach, and I needed a few days of, you know, I'm not checking my phone, I'm not on Twitter. I'm just throwing, you know, throwing my kids into the ocean and, and-
Did you have that moment where you're just sitting and looking at the ocean and contemplating?
Yeah. Like, what does it all mean? So I had already gone through that a month before. And I was like, I-- So now what that says is that autoformalization is basically solved. It's not solved in the way that I would want it to be solved. I want-- And people are still gonna work very, very hard on this. There's all kinds of other things that happen, just like there's all kinds of other things that happen when you prove theorems that are byproducts of that process for human understanding. There's all kinds of other things when you work on making a beautiful library of mathematics, and that process will continue. That process will be very difficult. It's gonna be the, you know, thousands of man-hours of people around the world contributing in this open source way. But in terms of can you get a sorry-free proof of something that human beings have written down on paper, I think that problem is done.
And so you were standing there with a whiskey, having gone through this a month before, and now again.
Right. This was round three.
'Cause when I got back from Labor Day, it was Navier-Stokes.
So I was like, "I was just mourning FLT. You're gonna do this to me? Come on."
But now, I mean, this really hit home, right? With, with the quasi-RH. Um, so maybe can you speak to the, the importance and the beauty of the primes, and also how big and hard the leap is from quasi-RH to full RH?
Yeah, so the primes, the reason the primes are so valuable is that they are the fundamental particles in the theory of multiplication. Every whole number is broken down into a product of primes. You know, we know about the elementary particles in physics, the, and all of matter is made up of that. In the same way, the primes are the particles that make up the multiplicative structure of whole numbers. And their distribution had been a mystery, and it remains a mystery. Even the Riemann hypothesis doesn't solve everything we wanna know about prime numbers. But the Riemann hypothesis was so beautiful because it pointed the way to what the distribution should be, and it has a thousand applications in very faraway fields. If only the Riemann hypothesis were true, then we would know XYZ.
So let me interject a little story that came up last week, which was pretty entertaining. So we had Grant Sanderson was visiting IAS, and we had a long discussion about what is AI doing to the math community. And I really enjoyed this comment that Camillo De Lellis made, which was, I'll do my best impression. I can't do the accent. But there was a play written in the 1950s by a mathematician. So who knows that mathematicians are writing plays, but apparently this play, the premise of it was that Von Neumann died, and he went to heaven. And he met God, and he got to ask him a question. So he asked him, "Is the Riemann hypothesis true?" And of course, everyone holds their breath as they're waiting to hear the answer. And God says, "Yes." So finally, everyone can exhale. And then Von Neumann asks, "But how do you know?" And God thinks and says, "Well, I know where all the zeros are, and they're all in the R equals a half line." So my understanding is the premise of the play is that, that's not enough for Von Neumann. He gets a little bit agitated to try to find out more about why it's true, and I think he ends up at the end in the other place.
So I don't know what to draw as a moral from it, but, um-... it seems to speak to the mathematical experience.
Math is not the fact, it's the why. It's always been the why. It's our deepest quest for understanding. And if you just tell me a fact is true, but I don't understand why it's true, that has contributed zero to my knowledge.
So primes, I think, are also interesting because they relate to things that a three-year-old can understand. Like fundamentally, we know how to add one to numbers, and you know, there's this breakthrough moment when you're a kid, I can add one forever. That's a big idea. I can keep adding one forever. Then you learn how to multiply and group, and that's also a big idea. And primes are interesting to me, and have always been interesting, because we don't understand how adding and multiplying work together. The two most fundamental, basic, simple ideas in all of mathematics, we don't know how they work together. I can add one to a number that I can factor, and I have no idea what happens. I know if it's even, it becomes odd and vice versa, but I don't know if a number that has many, many composites becomes prime. We just fundamentally don't know that simple, you know, adding one, how that changes multiplication. And primes are basically, you know, this like, the distillation of that three-year-old mystery, and it's a mystery that all of us still wrestle with. You know, that would be the interesting bit, is if the proof told me something fundamental about that structure that I could actually use.
Yeah. To the three-year-old, I think my older one was something like two or three. He was playing-- We had these little blocks, you know, cubes that kids like to play with. And uh, there was like 20 of them in the pack, and so he would make like a little four by five grid, and he liked that, and he would make a 10 by two grid, and that was kinda cool. And then one day, I look over and he's like really frustrated. I'm like, "What's going on?" He's like, "You know, it's not, it's not working." And one of them had rolled under the couch. And he only had 19. And he couldn't figure out a way to make a nice-looking rectangle out of the 19.
Yep. It still bothers me today.
So... Yeah. So absolutely, kids love primes. Kids can-... discover primes. It's the non-re-
And so the solution is to lose another, right?
Do you think so the leap between quasi and RH, I presume you think humans will not, in your lifetime, be able to solve without AI's help, the full Riemann Hypothesis?
Can AI, I guess, is the follow-on question. Like, how hard is it?
I don't know. I don't know. I did not think I would see a proof of quasi-RH. Now that I see what the proof is, and there are a lot of technical details that I still have to go through for my own satisfaction. There's a Lean formalization. The Lean statement is, was already in the library, was already in Mathlib, so the fact that they have a sorry-free proof gives me 100% confidence that the proof is right. And moreover, having read the proof, you know, at a high level, the ideas that it's using are all completely valid. They're brilliant. You know, to embed the Möbius function, twist it by sextic characters. This is all stuff that's the bread and butter of analytic number theorists that was chained in just the right way and was absolutely incredible. And so what Riemann showed is that remember we were talking about this function that has complex inputs and has these outputs, and we're worried about when the output is zero. So he showed that there's what's called the critical strip, which is a vertical infinite strip in the complex plane, where the real part is-- So you can think about the X coordinate lies between zero and one, and the imaginary part, the Y coordinate, can go arbitrarily high or arbitrarily low. And the hypothesis is that the zeros all lie right at the dead center of that strip, where the real part is a half. That's exactly what Ankur was just saying. And what we knew until Tuesday was region where there are no zeros of the zeta function, that's like getting thinner and thinner the higher up you go. It goes towards the one line. The one line is something called the prime number theorem, which tells you the basic distribution of the prime numbers is equivalent to the fact that there are no zeros of this function on the line real part of S equals one. So if the X coordinate is one- As you go up and down that line, you will never find a value where when you apply the zeta function, it hits zero. Okay? So we have this zero-free region that was getting thinner and thinner the higher up you go, and that gave us the best known distribution on the prime numbers, the best error term in the prime number theorem, which I can go into if you-- You tell me how much math you wanna talk about.
But what I could have anticipated is that it would thicken that line a little bit as you, you know, it would still be going to zero. The thickness would still be going to zero as you go up the imaginary axis, but it would be better than what we currently know. That thickness hasn't moved in seventy years, but I could sort of imagine that it could maybe make some progress. What it did is it gave us a vertical strip, so there are no zeros with real part between seven-eighths and one, an entire infinite vertical strip. That's a quasi-Riemann hypothesis, and if you can get that seven-eighths down to a half, that's the full Riemann hypothesis. Now, the methods that it used in the seven-eighths proof, and there's a second proof that's about eleven-twelfths, never mind these technical details, that method sort of fundamentally can't get down to one half.
You need a new idea, but quasi-RH needed lots and lots of new ideas, and it came up with that.
Is your-- Having gone through this three times already, is your intuition, this is for all three of you, about remeasuring in your mind the hardness of the various problems? 'Cause you mentioned, like, picking easy things. 'Cause that presumably you have a good hardness radar, like, this problem is probably solvable, and this is not solvable today, solvable this week, solvable this month, you know. Are you readjusting to this new world and your-
I'm readjusting my inability to predict it.
Yes. Exactly. Well, you had the ability, no? It's noisy.
Sure. It's a larger amount of noise than I thought. I mean, I think we've always known that there are problems we care about that are within reach of the tools that are out there right now, but it's just surprising the scale at which, you know, how many different problems really fall into that category. So we also don't know a lot about, you know, apparently, they prompted thousands of problems, and what they dropped were only the positive solutions, that they only had about a ten percent success rate. So we're sort of left guessing, you know, is there a very jagged boundary that we could squint at and say we can make sense out of that distinguishes, you know, what did fall versus what didn't fall? I think one of the other dangers is that maybe the problems which are the safest are ones which you look at and it's just painfully obvious you need a really creative new idea. You know, because you can sort of tell that maybe there's a barrier and the things that are out there in the literature just don't seem to fit into that hole. But now I think one of the things that's changed the landscape of math forever is, you know, maybe people become a little bit more secretive with those ideas than they were in the past. You know, because once those ideas are out there, then, you know, it might be a lot more straightforward for AI to be able to balance that one new idea, that missing ingredient, that new tile it needs, and be able to fully exhaustively explore what it can reach out of that. So you might be a lot more protective of your ideas than you were in the past. I think that's also a bit scary because traditionally, the mathematics community has been really open with sharing ideas, and that would be something bad to lose.
But that's also because we had this first through the gate mentality, and I think that's also gonna change.
I would prefer that the openness stay and the first through the gate is what goes.
That's right, but that'll take time.
Yeah. So that could be an effect, you know, that's more transient in nature, that if we can learn how to value other things, then maybe that'll go away. There are other communities that have models for how to value contribution, even when it's not lionizing one individual's contribution. So it's not that there's not a way forward, but, you know, that is, in the short term, a real roadblock to doing mathematics the way that we used to and enjoying it the same way that we used to.
So speaking of which, Steve, you told us a story about soup... that made you maybe fall in love with the bigger problem of fluid dynamics. So going from math to applied math to engineering, the full thing. So maybe can you talk about Navier-Stokes, these set of equations that explain fluids, why it's such a big deal, why it's beautiful? Similar as we talked about primes, why is it so hard, and why is it such a big deal that Navier-Stokes was proven?
So fundamentally, the Navier-Stokes equations is just a way of stating the conservation of momentum for fluids, for objects that can move kind of freely, like gases and liquids. So we know F equals ma is conservation of momentum, you know, classical Newtonian physics. The same basic principle applies to the gas in this room, to water flowing, and it's true in any volume. So it's true in little volumes, big volumes, kind of imaginary volumes. And if you write down that conservation of momentum in a volume, and then you kind of take the limit as that volume becomes arbitrary and small, you get these Navier-Stokes equations. It's a differential equation, a partial differential equation that describes how the velocity field of a fluid changes in space and time, and also the pressure related to the forces acting on that fluid.
Can you give intuition about why fluid, given that F equals MA is a pretty simple equation, why fluids are such complicated systems?
F equals MA is simple to write down, but it can get complicated really, really quickly. So if I have, like, a single pendulum, it's really easy to write down the equations for that single pendulum. If I put, you know, another pendulum on the end of it, now I have a more complicated equation, and it gives chaos. Fluids is, you know, in a sense, if I try to model the air in this room, I could chop the room up into a billion little boxes and write down a little differential equation for how the momentum works in all of those billion little boxes, but now I have a really complicated system of equations. And so it's simple to write down, it's simple to conceptually understand what the equation is saying and representing, but actually solving it and understanding how the fluid actually moves in any kind of practical sense is very, very complicated.
I read something like, you know, like one space where fluid dynamics is important is reducing drag for when you're flying, for airplanes. Uh, I read something like airlines could save like, something like tens of billions of dollars by reducing the drag by one percent. So why is it so hard? So that's one. And drag is maybe easier than something like turbulence. So like why is, why is these things that are worth so much money so difficult?
So the good news is, is that if you can improve our ability to predict turbulence, to simulate it even a little bit better, faster more accurately, there are, you know, dozens of fields where you can have a huge impact, both economical, ecological, human health and safety, with a relatively small improvement. So drag reduction on airplanes, better artificial hearts, you know, more efficient wind turbines and gas turbines and, you know, you name it. The world is-- We live and work in a fluid and so do all of our machines. The inverse design problem, I know it's actually very closely related to this kind of like math problem. I know what I want to prove. I don't know how to get there. Uh, I know I want a wing that has high lift and low drag, but I don't know how changing the geometry, changing the shape of that wing will affect this whole cascade of smaller and smaller and smaller vortices that, you know, this turbulent fluid that generates over the wing that eventually results in its lift and drag. I don't know how to inverse design efficiently. Part of it's because the equations are expensive to simulate or to test. You know, wind tunnels are expensive. You need supercomputers to simulate these things. This is actually kind of related to pure math. We'll, we'll get back to this, I think. But, there's this really nice Feynman quote that, you know, we know how to write down these very general equations, Maxwell's equations, Navier-Stokes equations, Schrödinger's equations, but we currently don't know from the equation what phenomena is possible. You know, if you'd never visited Earth, could you have possibly predicted the sunsets and the tides and all the phenomena just from like a partial differential equation? Like, we're not even close to there. And there's some very fundamental math questions about, you know, if I can-- I can write down a differential equation. I can't tell you what's inside of it. We still have to simulate, we still have to test, we still have to do this very elaborate engineering process to understand how these equations manifest drag on an airplane wing or turbulence inside of an artificial heart.
Sorry, can I just ask a question on that? I've always wanted... So like, in the case of Einstein's theory of general relativity, he writes down these equations. A few years later, Schwarzschild finds a blowup, and they're like, "Oh, come on. So what's, what's wrong with these equations that there's a blowup? You know, if you get enough matter into a small enough space, then like light can't escape it. This-- We don't have any physical experience for this. This has got to be a mistake in the equations." Until we discover black holes. And then it turns out that actually the theory perfectly explains, you know, and predicts what, what's supposed to happen. In the case of blowup of Navier-Stokes, are we going to discover something new about water, or is the fact that at some scale you get to particles, you know, you get to an atomic structure that does not follow the liquid equations anymore? Why would we have predicted that there should be solutions to a continuous process that's allowed to be pushed to its limits?
Yeah, it's a great question. Um, I'm gonna give you the engineering answer, which is these equations are an exceptionally good model of how fluids work at a certain scale. And you're absolutely right. Like you need some number of individual molecules to be averaged over in a volume for these to be constitutive equations that are valid. And so if you zoom in small enough, you know, now we're talking about individual molecules, and Navier-Stokes breaks down. So, the mathematical problem of these equations blowing up is interesting from a mathematical standpoint. By the time they blow up, it stops being a fluid. It's like-
There's no black holes. Like other physics takes over.
But it's poss-- Wait, I think it's interesting from a philosophy of science perspective that where equations break down could, and maybe statistically high likelihood, is a window, a door in a breakthrough of needing some other discovery. Like Newtonian physics breaking down needs Einstein.
Right. And the grand unification is that... we don't have gravity at the same scales as we have, you know, interactions of subatomic particles.
But this idea that we don't know something so fundamental about these equations, there are so many people on Earth right now simulating and testing Navier-Stokes in every industry imaginable. I mean, it's probably one of the most studied and analyzed and used equations in the entire engineering world. And we don't know this fundamental property of the PDE. Does it blow up in finite time or not? That's like the zeroth order version of the bigger question is, you know, what other kinds of solutions does it admit? Does it, you know-- Are there sunsets and hurricanes and tornadoes and all the phenomenology that could exist? You know, existence and uniqueness is like the-- The simplest baby step. And that's where it gets interesting to me is even if there's not a direct tangible use of this mathematical result in an engineering sense, it gets us that one step closer to understanding the nature of these equations and what is possible, which is very fundamentally interesting to us.
Could there be-- could this solution that has the blow up be used somehow in a engineering application and, you know, you wanna drive it into this state and get some funny behavior that... helps airplanes fly with less drag or something?
Yeah, I mean, I think in principle, a huge industry, a huge part of what we try to do is manipulate fluid flows, like bend them into our will, right? So if you watch a bird fly or an insect fly, they are shaping vortices and carefully funneling, you know, energy into structures for their, you know, objectives of their daily life to, like, perch on a rock or, you know, hunt. And we try to do the same thing. So flow control is one of the highest challenges. It's very, very challenging. Um, but all of our science fiction technology would be kind of accelerated if we had better ability to control these flows. And this proof is one very interesting construction of this kind of, you know, very specialized flow control in a way.
I'm reminded that Feynman was asked about intelligence, artificial intelligence in the '70s. And he said something like, "The way that we solve problems with technology is radically different from how nature solves these problems. Look at birds, how birds fly and how airplanes fly." He was exactly pointing to this. And it's kind of interesting to think back to that comment now when we see what LLMs can do with theorem proving versus what human brains can do with theorem proving and all kinds of other applications. But it's-... it's interesting that he pointed exactly to this issue of how, how we've solved flight using technology and how different that is from nature's solution to it. Our airplanes don't flap.
Now I wonder if birds have existential crises when they see planes flying.
Yeah. Yeah. To get the worm.
