MIT, for example, has the luxury of having the world expert on just about anything physically within a mile of like, your dorms.
Yeah. And so we're designing all kinds of programs that, for example, in the School of Engineering, try to bring the students in for research experience, for hands-on learning, for going into industry and seeing what things look like, though. And trying to build the undergraduate experience more out of that and a little bit less out of exams and problem sets. Now there remain a lot of questions. For example, assessment is a big challenge for us.
What we observed in the last year or two is that our students are getting nearly 100% on all their homeworks. Maybe it's because MIT students are super creative.
Can I just say so? I took the train from New York to New Haven. Quite a lot. And there are a lot of Yale students, watching them on their laptops right now. They're all on mostly Claude actually writing essays or like, at least doing the basic research via chatbot.
Hello, and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal...
...and I'm Tracy Alloway.
So Tracy, you know, obviously one of the big concerns is like job loss. What are we going to do in the future? What are humans going to be better at? And then what are we going to do all day? What are we gonna do all day, etc. And we really don't know the answer, I think, to any of these questions yet. But I am aware that, you know, obviously the models are getting very good at math and at least some mathematicians perceive, you know, a state of existential angst about this and certain, like, crises, perhaps some feel about like, well, what are we doing here? These problems that I devoted my life to solving and, you know, the model solves it in a weekend, you know, like, is this one's potentially coming for many of us.
Yeah. Math is an interesting one because you can tell that a lot of the AI companies themselves have an interest in math, right? Like they see math as a sort of holy grail for, I guess, proving that the models are able to reason.
Like it's the thing they need to do in order to say that they've achieved AGI.
Yeah, I think that's the way it's couched. And it also makes good headlines. So even though I have no idea what the Navier-Stokes problem actually is, I don't know, even if I'm pronouncing that correctly, is it Navier or Navier?
I see the headline and I'm like, oh, a model has solved the Navier-Stokes problem, as if I know what that actually means and the significance for the mathematics profession. And it's also funny because like I downloaded the paper and it's like, you know, one paragraph and I have no idea, you know, like, I can't even begin to parse the output. You know, another fascinating aspect of this is just like it would not have been intuitive to me, say two years ago or three years ago, that a language model would be so good at mathematics. In fact, at one point they weren't very good at math at all. Like, very, you know, go back to like, I don't know, GPT-3 or GPT-3.5, if you ask them to count, like the number of letters in a word or something, they would get it wrong.
Yeah. And weirdly, they still like, struggle with certain things like that.
But it's not obvious to me, like, okay, you scan all of the text on the internet, you know, language, etc. and yet somehow that generalizes into this extraordinary math capability that to me is very surprising in this conversation.
Can I just say before we start, I would also just like to learn what advanced mathematics actually is, because everything I know about it, which is nothing, comes from like watching Good Will Hunting and just seeing, you know, a bunch of formulas on a chalkboard. And people get really excited when they start crossing off certain parentheses and stuff like that. I have no idea what it is. I have no idea how it differs from pure maths of like two plus two equals four. And so I would just like to come away in this conversation with a better understanding of what it is overall and how it actually fits into it.
Absolutely. So like there's two... I was thinking along the same lines like, okay, for someone who devotes their life to like advanced theoretical math and, you know, also things like advanced theoretical physics that don't necessarily have like obvious industrial applications, etc., who is the type of person, what are they hoping to achieve? Why is there still interest in, you know, some of these things that perhaps this sort of applicability is not necessarily so obvious, but then like obviously, you know, like what is the role of the mathematician at this point? And then actually specifically, it's like, okay, across the universe, this is actually the thing I'm very curious about, which is that like, you know, like my son studies math, right? Or my kids study math and like, hopefully not theoretical math.
Not just yet. No. Yeah. How old is he?
My daughter is ten. My son is seven. You know, they take math, etc., but like, setting aside like what the future holds, I'm very interested in just the present because it's like you have all these math departments at universities all around the country. And like maybe some are excited about AI, etc. but like, what do you do today in terms of like rethinking pedagogy? Because it must be, or I would assume it's pretty different in 2026 than like those same departments in 2021 or something like that.
And like how are professors, teachers, etc. thinking about just the sort of the process of teaching math right now? It's just like, just sort of like very straightforwardly, how is it affecting their jobs?
I used to be really, really good at sort of mental math like times tables.
All right. This is a question I have too. If you are an advanced mathematician doing theoretical maths, how often do you actually use a calculator?
I'm curious, it's going to be my first question, actually.
You know, there was Jensen Huang. I don't know if you read the transcript of his interview with Ezra Klein a couple of weeks ago. But he said something like, oh, maybe it's just not... He is a little kind of a weird interview. He said something like maybe, like, knowing your times tables isn't going to be an important skill in the future or your handwriting.
Which is like kind of weird, like, like I still want, you know, my kids to know.
It raises a very important question, which is you can go to a model and have them verify a proof of a theorem or something like that. But if you don't understand how you actually got there and how the model is getting to its verification, then what are we actually learning here?
Right. And then there's the question of like, you know, people learn a lot in the process, right? So it's like even presumably in just the, you know, people work on some of these problems for years and years and maybe they never arrive at any proof. But intuitively, you would think there is something. Maybe it's exercising a muscle or something or sort of other benefits to just the process of learning. How does that change anyway? I have a lot of questions about this, and we really do have the perfect guest. We're going to be speaking with Justin Solomon. He's associate dean of engineering education at MIT and very interested and involved in math education and thinking about the intersection of math and AI. So, Justin, thank you so much for coming on Odd Lots.
Tracy sort of already suggested it, but set aside AI for a second. What is it that people who end up going into sort of like advanced math or advanced theoretical math, like who are these people? And like, why? Like what are they hoping to do?
Yeah. That's true. I feel like the typical caricature that you see is maybe this character from, what, Good Will Hunting. Incidentally, they're always writing math on windows, which is very impractical.
Yeah. That's true. It's great for a camera.
But that aside, I think it's an intensely personal thing. So, to be clear, my personal career, like, I'm a professor of engineering, is in an area of math called applied math, which sometimes is applied and sometimes not. That's a discussion for another day. But essentially some people go into math because they want to see it translate into applications or to bring insight into the kinds of things we do every day, whether it's predicting markets or physics, what have you. Other people in areas of math, sometimes you call them pure math. I don't love that terminology like pure versus applied. But in other areas of math, it's almost just for the joy of the game or understanding the universe around you. All of these are totally legitimate reasons and true intellectual pursuit for the sake of intellectual pursuit.
But along the way, I think you develop a lot of really important reasoning skills and skills that support work in all kinds of different areas.
And how often do advanced mathematicians, whether they're in, like, applied maths or purely theoretical maths, how often do they use a calculator?
Okay. Yeah. No, my impression again is everyone is doing this by hand on chalkboards and windows.
I mean, there's like lots of Greek symbols and letters and variables and things. But no, I'm terrible at arithmetic. I've got a calculator out for real time.
Oh, yeah. This gives me hope, actually, that maybe I could understand some of this, even though I'm pretty bad at it.
Oh, that's my take as well.
Okay, so what's the difference between a math problem then and a math proof or a theorem? These sort of big picture things that we're talking about AI models actually doing.
Now that's a great question. And it's really important to dig into exactly what the models are doing and not doing.
And who deserves credit, by the way.
Yeah. But essentially in mathematics, it's all about verifying. In some sense, verifying true facts is one part of the puzzle, but it's also all about figuring out what are the interesting problems to study and what insight they bring to the broader world.
So in some areas of mathematics, there are all kinds of different questions you could ask. And some of them are super uninteresting, but you could ask a computer to prove them. And others maybe tell you something about a physical system or about a system of equations or something like that, that bring broader understanding. And so this exercise of proof writing is all about sort of writing a mathematically convincing argument that tells mathematicians, so there's a human aspect, which is an important one, that a certain fact is actually maybe reality, right? It aligns with logic. There's not some flaw hiding in there somewhere.
And where do your ideas for a particular mathematical proof or theory come from? Because again, I'm going to come at this from, I guess, the movie angle, but I think about A Beautiful Mind, right, in the famous bar scene. And was John Nash, right? That was his name. And he has this, like, big realization while he's at the bar with these women, and it ends up turning into a famous mathematical theory. Is it like that? Like, where do the ideas actually come from for the rules that you are trying to prove in maths or these big picture ideas?
This is a great question. So sometimes in mathematics, there's some really natural questions out there in the universe that people have been asking for a long time. So, for example, these days in the news, we hear about some of these really high-profile math problems that have been around forever. There are other types of mathematicians that are more theory builders, where they're the ones asking the questions rather than answering someone else's questions. Yeah, there are definitely different approaches that are out there now. When it comes to where these questions are coming from, some of them are inspired purely mathematically. And other ones maybe come from a very practical setting. Maybe somebody observed some phenomenon out there in the real world, and they're trying to explain it with the smallest model possible. Now, one thing I should clarify is that math is a really social exercise. I think that's something that people don't necessarily understand, especially because these movies really play into this almost great man theory of mathematics. But that's just not how it works. There's this huge community of people that are working on important problems. They're communicating with one another, collaborating. And I would say, for example, from my own research, almost everything that our team has worked on comes as a team effort. We're all sitting in the same room, we're at the blackboard writing computer models, what have you, and that's where the insight is coming from. So it's not just sitting in your ivory tower. It's being out there in the world observing things happening and talking to your colleagues.
Before we get more into the sort of AI question, what is... I've actually been curious, what's the difference between applied math and pure math?
Yeah, it's a great question. So there's sort of a divide in the math world, but I think it's fuzzier than maybe it sounds when I describe it here. So roughly, there's an area of mathematics that you might call applied math, where the goal is to either derive theoretical results or even come up with computer algorithms that explain phenomena out there in the real world. So, for example, in applied math, maybe you work on algorithms that simulate the weather, and behind the scenes is a really complex mathematical model. And it's all about making sure that your model is correct and efficient and well-behaved, all that good stuff. In pure mathematics, sometimes the motivation is a little bit less attached to a particular application, and more just attached to the math problem itself.
So something like Fermat's Last Theorem or something right there where he's like, okay, I forget what it is exactly, because that would not be applied math. Right?
So Fermat's Last Theorem and many things I think in number theory fall into that category. But also many times these things bleed into one another. So for example, a lot of modern developments in cryptography came from areas of mathematics that we used to think of as extremely pure and abstract.
Oh, interesting. This is also Good Will Hunting, by the way.
But it all goes back to... That's right, that's right.
You worked at Pixar before, many years ago. Was that applied maths, working at Pixar?
Oh, I love that question. The answer is absolutely yes.
So, movie studios are a great example of a place where I would say state of the art in certain types of mathematics are taking place. So when you watch a typical movie with different movie effects, there's cloth, there's smoke, there's all kinds of fluids, you know, rigid bodies flying around. All the dynamics of those are governed by things called partial differential equations. And coming up with the right algorithms for simulating those, and by the way, making them so well-behaved that an artist doesn't have to think about math, is a really challenging mathematical problem. I typically tell my students that if you're looking for really challenging math problems, you don't have to make them up. They're out there in the real world. And that's a nice example.
What's the Navier-Stokes?
Okay. So Navier-Stokes has been in the news a lot lately, and it's been great for those of us with a little bit of background in differential equations.
Okay. We're getting lots of media inquiries.
That's exactly right. Yeah. We have our 15 minutes of fame here.
It'll die fast, but enjoy it for now.
Right. So the Navier-Stokes problem is one that comes from the world of fluid dynamics. So the Navier-Stokes equations were this model, they go back quite a long time, that explain the motion of, for example, liquid in a cup. So like both of you guys have coffee cups here on the table. And if you like shake the coffee cup up, you know the initial velocity, you know what's inside of the cup. Then the Navier-Stokes equations are trying to predict like where the fluid moves over time, right? Like where is the velocity, vorticity, all these different things. Now the Navier-Stokes equations are really complicated. I mean, when you look at your latte, when you start in the morning and you see, you know, the milk making these cool shapes and so on, there's all kinds of complex behavior that can happen. And so the Navier-Stokes equations, although they're easy to write down, they're very hard to say a whole lot about. And so there is this big open math problem, which I think is what you guys are referring to because it's been in the news, which involves existence and uniqueness of solutions. So the basic idea here, I think actually my undergrad differential equations professor years ago explained it to me in kind of a funny way. He sort of said, you know, all you got to do to make $1 million is prove that water doesn't explode. And that's roughly right. I mean, with a lot of asterisks, essentially, what we didn't know is if you know the initial configuration of your fluid and you let it kind of go under its dynamics over time, then whether it exists for all time and doesn't have some really crazy turbulent behavior, or if, for example, maybe it blows up or it develops a singularity or something like that. So this was kind of a yes or no question, like prove or disprove. And I guess the recent news was actually a counterexample. So in other words, it showed in some finite amount of time, with an asterisk, which is with a very crazy spoon, if you start your fluid just the right way, you really can create this totally singular and frankly nonphysical behavior.
So proving in math is like one of these things. Like I think I, you know, like the formal proof, the two plus two equals four. It's like a nine-step thing, isn't it? Right? Like if you or like sometimes eight, sometimes eight, but like, can you explain like what a proof is like if I know two plus two equals four, it seems obvious, but like what is it explaining like this. What is a proof such that that would say that to prove two plus two equals four is something that requires eight steps, right?
This idea of proof writing I think is actually, you can think of it as almost the result of an identity crisis among mathematicians, maybe 100, 200 years ago, where it used to be a lot of math was drawing pictures and kind of working in the same fashion that you're talking about, kind of intuitively.
But at some point, people realized that they wanted to make sure that the field had really strong foundations.
And so mathematicians will agree on a certain set of maybe logical principles or rules. I would say that's more the domain of philosophy, precisely what those should be. And then the idea of proof writing is to build up from these kind of basically like we agree that these five facts are true...
...to a much more complicated fact by combining them in different and creative ways.
And so theoretical mathematics oftentimes is all about, I don't want to just say mixing and matching different facts, but coming up with smart ways to justify mathematical ideas built on very simple principles and steps from logic.
So when it comes to AI currently, is the role more in, I guess, proof checking versus proof generation or idea generation? Like what are the models actually doing at the moment?
Well, they're sort of doing it all and they sort of do nothing.
So right, the current AI models, in terms of the types of mathematics they're doing, I don't think it's quite right to say that it's just proof checking. And in fact, actually, they're quite bad at proof checking. And one of the revolutions in, I would say, the last year or so is the realization that actually what you have to do is to provide your model, like your large language model, with a second module that can check the AI's proof. And so for example, there's a programming language called Lean, which is basically just a sort of codified version of these mathematical rules. And these days what people will do is they'll put in their conjecture, so like the thing that they want to prove or disprove, and then ask the AI system to generate a proof, but then they'll say, rather than just writing it in human-readable language, which is the substrate that mathematicians use to communicate with each other, maybe in addition to that, they'll actually produce this Lean code so that there's a separate piece of software out there that can check that this proof truly is correct. So actually the correctness part of it is usually deferred to another piece of software, which isn't AI. But somehow because of the interaction between the AI tool and this proof checker, that's what's created a lot of progress that we can believe in a little bit more than we did in the past, if that makes sense.
This I have heard of only because, you know, I've been reading the news. I've heard of this thing called Lean. Yes. And it sort of seems extraordinary to me. If you explain it a little bit, it seems extraordinary to me that there is this, I don't know, is it software or this framework such that these extraordinarily complex problems or solutions, I guess, can be dropped into this, and it's capable of like, yes, this is a valid... but it's also kind of crazy that you need basically a translator for it. Right? Yeah. Can you explain a little bit like how this works, how Lean works.
Perhaps I can explain a little bit. To be clear, I'm not an expert in this particular corner, but so roughly, essentially, as we all know, the things that come out of AI are unreliable.
All of your listeners should know.
And that goes for mathematics as well. And also we think of it as this area that is extremely reliable.
Everything should be true or false. Now there are gray areas, but certainly we try to avoid that. So this programming language Lean, what it does is it has a few simple maybe axioms that are built in. So math things that we sort of agree are the starting points for writing proofs, like the steps of basic logic. And then what happens is that you write a theorem in this Lean language. So rather than in English, you write it in this programming language. And then it's kind of cleverly designed so that once the software checks that that theorem is true, it can get added to the library of sort of true facts. And then you can build it up.
So you make this more and more complex edifice out of these simple steps. And it's been really impressive what they can do. I mean, some of the modern proofs that you're seeing, especially the AI-generated ones, are like hundreds of thousands of lines of code. I don't think a human could check it. That also, by the way, means that we're trusting that someone wrote that software correctly.
Yeah, which is the philosophical question.
But, you know, if you believe that Lean is good at checking proofs, then it provides a lot of certainty that the things that the AIs are writing down are actually correct.
Devil's advocate question. But why is it important that we need to understand the actual problem and the proof? Because if I'm an engineer and I don't know, some model somewhere comes up with this really new, interesting mathematical theory and I can apply it to my day-to-day work...
...is that not enough that, you know, like I can use it? It seems to work for the most part. Maybe it's been rigorously tested in a physical environment or whatever. Why do I need to understand everything that's gone into it?
Oh, that's a fantastic question. So let's see. My research group works a lot on simulation algorithms. So like I have some physical system and I develop a technique for predicting what it would look like in the future. And we spend a lot of time writing theoretical results to verify that our algorithms are correct, they behave the way we expect and so on. And that's absolutely critical because without that, you really can't trust what your software is producing. So, for example, if I'm trying to say yes or no, is this building going to collapse during an earthquake?
That's right. And all of that is built on mathematical models, techniques like finite element modeling and so on. They're not too exciting outside of my universe. But you're also trusting these things to do really important engineering calculations and things that, sure, like one way to justify your software is that your building hasn't collapsed yet. But that's not a whole lot of, you know... Yeah, I hope I can trust a little bit better than that.
You know, my dad was a physics professor, and I have memories of watching a video of the famous Tacoma Narrows Bridge collapse. You know that video?
You know, the... Really? Yeah.
And like, that was, like, for him like this, like... Yeah. Yeah, I watched that several times in my household.
Our engineering students have to take a lot of ethics courses that are pretty fraught that way.
Yeah. Let's start with like... I mean, there's a couple of avenues I want to explore, including just the process of teaching and how it's different in 2026 versus 2016, given the new technology. But maybe give us a lay of the land for like the math community.
Yeah, because I know there are some people in despair, I believe, very anxious about what this means for the future of like math as an intellectual discipline, etc.
And then others seem to be very enthusiastic about the opportunity of human and AI exploring new ground. But what are the sort of the conversations that you're in these days with fellow mathematicians?
Oh my gosh, things are changing by the day. And really, it's a huge collection of individuals. Every single person has their own opinion here. So I can tell you my view of the world.
If you talk to somebody else, I'm sure they'll have a very different one. Math is an academic discipline. It also shows up in a lot of applications, as we mentioned. But every aspect of the work is changing from how people produce mathematics and choose problems to work on, to how mathematics is sort of measured in terms of productivity.
When we talk about hiring faculty, for example, like what are the standards to evaluate a person's body of work and what did the human contribute? I think are really critical questions to like why I do this exercise at all, maybe leave it to a computer. There is all kinds of identity crises that are happening.
Now among mathematicians, there are people that are maybe theory builders, and there are people that are like theorem provers. The theorem provers, I think are especially under threat at the current moment where like, maybe you've been working on one problem and it's well-defined and you've known about it for, you know, 50 years and it's just this crowning, you know, a question that you want to answer about the universe.
And these AI tools are sort of at least good at sort of cherry-picking the ones that are somehow close to being proved and taking that last mile in some cases. Or another, you know, other people disagree with that characterization.
What do you mean by that when you say they disagree with that? You know, like exactly how interesting.
For example, a mathematical proof is, is a very human judgment.
As is what problems you should or should not actually go about working on in the real world. So, there are a lot of true facts. It doesn't mean we have to write them all down, essentially. And because of that, this is definitely a human exercise.
To a large extent, mathematical research is about humans communicating to other humans, right? It's not just about verifying true things about the universe, but bringing insight and helping people understand systems and so on. So I think you mentioned an identity crisis. I think what's happened is, and this is reflected, there's a really nice write-up by Terence Tao, who's a famous mathematician.
