Hi everyone and welcome to another episode of AI Skeptics. This week we're really excited to have a mathematician join us, Daniel Litt. He's going to introduce himself in a second, but it's really exciting to talk to you about AI and mathematics in a less than expected skeptical way, I think, today. We'll see. Daniel, do you want to introduce yourself?
Yeah, sure. I'm Daniel Litt. I'm a professor at the University of Toronto. I work mostly in algebraic geometry and number theory. And I guess the relevance for this conversation is that I've been experimenting with AI, like trying to get it to do math, since maybe 2022 or so. So about four years.
Oh, I didn't know you had such extensive practice with AI.
Yeah. So, well, the original way I got into it is when GPT-3 was announced, you couldn't use it originally. Like, it wasn't publicly available. The only way you could use it was through a video game called AI Dungeon, which would get GPT-3 to be a dungeon master and walk you through a Dungeons and Dragons game. So I was curious about it and I bought a subscription to AI Dungeon, and inside of the D&D game, I would try to get it to do math. So I would be like, create my character as a math professor and go into my office and then, oh, take out a calculator and add eight plus two. And it must have been prompted or something not to do math because it was quite bad at it. And so it would always be like, 'Oh, the calculator broke,' and didn't output anything. So that was kind of my original experience with AI for math.
I like the idea that you're like, 'Look, just for the sake of science, I'm going to play D&D. It's just for science.'
Well, for the first section today, we want to set up the context for the real reason you are here today. And it's about Erdős and it's about mathematics and the culture of mathematics. I, of course, was a mathematician myself. In fact, my parents were both mathematicians. And I'm going to introduce this notion of the great man of mathematics. He was actually a pretty frequent visitor to my parents and my home when I was a baby.
And my father wrote a paper with him, and my father was a combinatorialist. He was like the student of John Paul Roth at MIT. Actually, he was at Rockefeller when my father was working with him. Anyway, so Roth would come by, and he would first of all, like, he was addicted to speed, and somehow my father became his supplier for speed. I'm allowed to say this. Everybody's dead, so I can say all this stuff. He also was not particularly adored by my mom. She was also a mathematician. She was in fact working full-time as a professor. But he would be like, 'Oh, my button fell off. You're a woman.' Like, he was famously misogynistic and just assumed that women were there to feed and clothe him. And then I eventually met him when I was at the Budapest Semesters in Mathematics, and he didn't know I spoke Hungarian, and I heard him say something like, because it was me and my friend Aaron, and he was like, 'Oh, there's a mathematician here and also a girl.' And I was like, 'Okay, this guy, am I right?' But he was pretty amazing, too.
I just want to point out that behind every mythological male figure, there's always a story about women somewhere, you know. But he was also a mythological male. I'm going to let you say the good stuff about Erdős.
Well, I remember he had a nickname — he called children epsilons — and I remember he had a bunch of other names for various categories of people. Do you remember any of the others? I think there was an excellent thing.
I think there was a nickname for women too, which I can't pull to mind. I read some bio of him and remember him scratching...
I want to give him a little bit of slack because his mom was nuts.
Nuts. I mean, nuts. We won't even go there.
I've learned a tiny bit about Erdős besides your stories, Kathy, from the book 'The Man Who Loved Only Numbers' about him, which talks about how he was crazy proficient, obviously, in many areas, particularly math, but just had these incredible gaps. I think couldn't tie his shoes is one of the things in that book. Caught pneumonia because he was sleeping next to a window that was cracked open and he couldn't close it himself without help. So things where it's like it actually reminded me a little bit of AI in a way, where you're like, wow, great at some things, not great at all at other things. Interesting guy, I guess.
Do you have a number?
Are our guests or our listeners familiar with the idea of an Erdős number?
He's such a boss in math that people characterize their own mathematical careers with reference to him. So will you guys talk about Erdős numbers?
Yeah. Well, my dad had an Erdős number of one because he wrote a paper with Erdős, and I had an Erdős number of three because I wrote a paper with somebody who wrote a paper with somebody who wrote a paper with Erdős. You know, like it's how it works. Daniel, do you know your number?
I think I have an Erdős number of three or four. But yeah, I mean, at some point, you know, now that he's no longer writing papers, it starts to correlate more with what career stage you're at than, you know, how mathematically close you are to Erdős, right? And he was pretty strictly a combinatorialist and number theorist, right? He wasn't an algebraic geometer like yourself, Daniel.
That's right. Yeah. Yeah.
I guess I do some number theory too, but I'm a very different flavor from Erdős. But you know, he was so prolific that I think probably most active mathematicians have some Erdős number, I would think.
And I mean, I think, look, I really did want to draw a picture a little bit of mathematics as a culture. You know, like the fact that he was beloved in the community because of his idea generation and his generosity of time. You know, he was a traveler. He would stay at people's houses, do math with them, then go off to the next house. I don't think he owned a house. And he just spread math everywhere he went, and it was pretty incredible.
Yeah. I mean, I have some sense that mathematics as a profession has kind of professionalized in the last, you know, 50 years or something. And maybe there's now fewer people like Erdős like that. And I don't know. I think that's in general a positive thing. But maybe we lost some kind of weird characters by doing that.
We definitely have. But what do you mean by professionalized?
Just I think this, like, you know, universities have become more corporate, there's a little bit more bureaucracy. Like I think it's harder to hire someone like Erdős. I know, I think my sense is it was probably already very hard to hire Erdős when he was active. But you know, yeah, I mean, I think presumably if Erdős went around these days doing the kind of things that you're describing, like he would actually face some professional consequences, and arguably he should have. But that's the identity. I mean, I don't have a lot of memory of him actually teaching classes or anything. I feel like he was mostly a visitor to departments and stuff.
But yeah, I think you're right. But I mean, having said that, like the spirit of mathematics, you know, as let's just compare it, for example, to lab science, which is a very, very different endeavor, right? Like lab science, you know, like if you're a graduate student in a lab science field, you should be expecting to work like an underling for your professor's lab for years, and then someday maybe you would get a lab if you play all your cards right and know how to write grant proposals and blah blah blah. It's like extremely complex social dynamics with huge groups of people doing stuff together, and that's the nature of the work. Mathematics isn't like that, and it still isn't like that, right? Would you agree?
Yeah, I think certainly like when I advise a graduate student, like they're basically doing their own work, and I'm maybe giving a little advice on taste, or if they get stuck, trying to get them unstuck or something. But it's everyone's kind of doing their own thing, and it's so much more like an individualized research atmosphere than I think a lot of the other sciences.
Do you think that has to do with the equipment required? Like this story is coming to mind from my analysis professor in undergrad, Pat Gallagher, who was maybe Erdős number one. And he would say, 'Oh, I would be going to a conference, flying, and I get on the plane, and Erdős is somewhere on the plane, and he saw me come in, and he would come take the seat next to me,' displacing whatever traveler was actually meant to be in that seat, and would just sit down and without saying hello or anything would just say, 'Okay, Patrick, p is a prime, not two,' you know, and just start with a problem they had discussed years ago. So it's just like you don't need anything. You need to sit down with another mathematician, maybe a piece of paper, and just start talking. All the equipment is right there.
I wonder if that plays into mathematics... community of math.
Yeah. And I would add, like, okay, so I guess you took a different thing from it, Daniel, and I'm not disagreeing with you. It might be more individualistic, although we're talking about collaborations. Like all of Erdős's work was collaborations. I don't think he wrote papers. He basically helped people come up with ideas. He often played a part in the proof, but they often proved most of it themselves and then they wrote it up but gave him credit, like co-authorship credit. I'm really talking about the generosity of mathematicians, you know, like yes, most people write papers by themselves or with a few other people or as a combination of those two things, but in particular, graduate students are not the slaves of their professors, you know, right? I'm sorry to say, Daniel, it's kind of the opposite, right? You said you're going to have a student defend a thesis today, and I'm sure that's like a culmination of a lot of work on your part. Not that you wrote their thesis at all, but that you helped them for the last five years without getting any papers out of it.
That's right. Yeah. And I should say, I think the culture of, like, you're right that it's kind of uncommon, for example, for a PhD supervisor to write a paper with their graduate student. My sense is that that differs a little bit from place to place. Like it seems like it's a little more common in Europe, maybe. But yeah, I think at least within American math academia, that's basically right. Like you kind of advise a student, and that's part of your job, and you don't get any immediate benefit except for like a warm glow in your heart.
All right, so let's, we're going to move to the second topic and explain why we're talking about Erdős and we're talking with Daniel on the AI Skeptics podcast. It's because AI has been doing some math, math that was teed up by Erdős, and Daniel was involved to some extent in this. There's a Scientific American article, which we'll put in the comments or the caption to this episode, that talks about a famous Erdős problem that ChatGPT helped to solve, and then Daniel took a look at it and at ChatGPT's work and has commented on it. So that's what we'll talk about now. Can you tell us a little bit about this problem that ChatGPT helped solve?
Sure. Yeah. So the question was the following. So you put n points in the plane, and you want to arrange them so that as many pairs as possible are exactly a distance one apart. And so Erdős found a construction that was pretty good. So maybe your first guess as to a construction would be you put a grid of points in the plane where the grid length was one. And that would give you on the order of n pairs, up to some constant, which are your distance one apart if you put n points in the plane. So Erdős realized that you could do a bit better just by rescaling the grid. And so he put some kind of sequence of rescaled grids. If you do this, you get about n to the 1 plus some constant over log log n pairs which are unit distance apart. So all that means is you do a little bit better than the grid. And then he conjectured that was the best possible. So that's some lower bound, which he conjectured was also an upper bound asymptotically. The best proven upper bound was n to the 4/3. So what happened recently was an internal model at OpenAI disproved Erdős's conjecture, found a new construction of n to the 1.03 pairs of points in the plane where some fixed constant of them, which is bigger than one, are unit distance apart. The original proof gave some very small improvement, so some constants that's just very slightly bigger than one. Now we know it's about 1.03 or so, or n to the 1.03 pairs of the unit distance apart. So, and one comment I want to make is you commented that it solved the unit distance problem, and that's not really true. So what it did is it disproved our conjecture, but there's still a huge gap between the new lower bound we have, so n to the 1.03 or so, and the best upper bound, which is n to the 4/3. And my sense is that actually most previous effort went into trying to improve that upper bound, and that's maybe still going to be what's happening. So there's somehow still a lot left to do on this question. Though maybe I should also say like this problem is kind of far from my expertise. Like I was asked to check the proof because there was some kind of involved algebraic number theory involved, which is a bit closer to me. But you know, I'm not an expert on the history of the unit distance problem, for example.
I love how you just dove straight into the actual best known lower and upper bounds here. That's super nerdy of you. Like one of the things, I didn't even read the Scientific American article. I just looked at the paper itself. So we will also link to that.
Could you talk about what was actually given to what large language model and like what happened, like how many efforts did it take before this counterexample was furnished by which is it? Yeah. Go ahead. Say a little bit more about...
Not all of this information is public. I can tell you what we do know. So, right. So what happened was this was produced by an internal model at OpenAI. So it's not something publicly available. It's probably some version of what will be GPT-5.6 when that comes out, but we don't really know. What seems likely is that they were giving it thousands and thousands of problems, among which was this one. So in particular, there's like a list of Erdős problems on a website called erdosproblems.com maintained by the amazing mathematician Thomas Bloom, and I imagine they gave all of those to their internal model, and then they had a grader, which is also an LLM, reading the solutions, and then the grader flagged this solution as being correct. And there are a bunch of quite serious mathematicians working at OpenAI, including people who work in this general area of combinatorial geometry, combinatorics, probably speaking, and they were like, 'No way, like there's no way it solved the unit distance problem. This is like a central problem in combinatorial geometry.' But you know, they read the proof, and to the extent that they were able to check, it seemed to be correct. And then there was some ingredient using something called class field towers, which is coming from algebraic number theory, where they didn't have the expertise to check it. And so they asked — I'm friendly with some of these people — so they asked me and another mathematician, Victor Wang, to take a look at this part of the argument to check it. So I think I was one of the first people outside of OpenAI to see the proof. And you know, to be honest, like class field towers, they're part of algebraic number theory. It's a little bit specialized, but this really is math from the '60s. It's stuff we understand very well. It was very easy to check, and it was pretty clearly correct. And so after that, you know, people became confident that the argument was right. And then Victor and I and a number of other mathematicians were kind of invited to comment on the proof and kind of digest it. One sort of funny aspect of this is that, well, the best available models and also internal models seem to be much worse at writing math than they are at doing math. So it's very — even if there's a sort of more or less autonomous proof of an interesting open question, it's still very useful to have humans kind of digest it and write it in a coherent and understandable way. And so that was one reason that I was involved in this kind of remarks compendium.
I mean, let's save that to the third topic because that's a really fascinating notion of like even just digging down into what constitutes a proof in the first place. But just in terms of the process, I mean, I just want to back up here and just mention, I'm glad you explained that this was not the only problem that this ChatGPT version was asked to solve. And it might not have been the only way of asking this particular question. Do you know anything about that? Do you know how many ways this was asked?
Right. So, I mean, we've seen the prompt. The prompt was basically just a statement of the problem and 'solve this.' So I think we can be confident it was pretty fully autonomous. We also know it can't have been too expensive. So we've seen a summarized chain of thought which is about 125 pages. So probably the actual thought was some constant multiple of that, maybe a thousand pages or something like that. So you can estimate the cost of that individual run was probably well under $1,000, probably on the order of $100, I would guess. And we also know that they attempted this question like something like a hundred times because they made a graph of how often it succeeded at different levels of compute. And so for low compute, it would succeed — it would have come up with a proof like 10% of the time. And then as one let it think for longer and longer, it would get up to like a 60% success rate. And so if you're running the same problem a hundred times or a thousand times, that also gives you some kind of upper bound on the cost.
Do you think it was going back and forth with that grader that you mentioned? Like is it getting feedback after the various questions?
So I think we don't know exactly what kind of scaffolding it had. Presumably it had access to some kind of tools, but yeah, that's a little less clear. But I think we presumably don't know if it was fine-tuned. I mean, it must have been fine-tuned on mathematical proofs. So what they've claimed is that that's not the case, that it's just a general-purpose reasoning model with no fine-tuning. I, to be honest, I believe that. I think that the improvements on model capabilities in the last year or so have been really remarkable. Like having looked at the proof, it doesn't seem to me to be much, like, post hoc, much beyond what you can get just from the best publicly available models, which we — I mean, of course they do some reinforcement learning on mathematical reasoning, but you know, it doesn't seem like this was like a specially designed model aimed for this kind of problem. Yeah, but of course I'm speculating to some extent here.
There's one other thing I want... Do you have any estimate of how many open Erdős problems there are? I mean, just focusing on Erdős.
Sure. So erdosproblems.com has around 1,200 — something like between 1,200 and 1,300 problems on it. I think a bit more than half of those have been solved. So there's on the order of 500 open ones.
I mean, that's if you add — if you think about 500, 100 runs of 500 problems, $1,000 each, that the money adds up after a while.
So probably they're not doing each of the problems 100 times. I would guess probably just once. They, you know, they wanted to make that graph, so probably they just did it on one. But yeah, I would guess that they, you know, it's conceivable they spent a billion dollars on this whole activity, not counting training. That's a lot of money. If you compare it to like the marginal cost of producing high-quality math research, it's maybe not that much, right? Like I would estimate the cost of producing like the best paper each year is like the marginal cost is like $100,000 or something. Like if you just think of like what the salaries of mathematicians are. So it seems roughly comparable. And of course, like the kinds of results the models are proving are like quite different from like the best results you see each year. So it's, you know, right now the models are getting very good at like some relatively small proportion of mathematical activities.
Okay, this whole conversation is making me think of a monkeys at typewriters, publication bias kind of thing, like, oh, they try 10,000 problems and, you know, and then it gets one right. But then it occurred to me like you could view the actual area of math research in a similar way if you wanted to. Like there are a lot of mathematicians spending a lot of time and not getting an astounding result. Like are they monkeys at typewriters?
Yeah, of course. But I do think like it's worth thinking there's like a very different way to use AI productively than from how humans do mathematics productively. Like the way I do math is like I think about two or three problems and I try to really deeply understand them, and then historically like that actually works. Like sometimes I get somewhere. Whereas like I think the productive way to use the tools right now at least is like throw a thousand problems at them and see what — you know, in some relatively small proportion of cases you get a nice result out. But because it's relatively cheap to throw a problem at them, that's like something you can do productively and pick some low-hanging fruit that way. So they seem to be kind of complementary approaches right now.
That's a perfect transition to the third topic, which is what is the future of math with AI as a participant? And I'll just start by asking, like going off the point that you were making earlier, what is AI good at? What kind of math? And look, what kind of mathematical progress do you expect to see coming from AI in either this approach or a different approach that you're thinking about?
Yeah. Yeah, that's a good question. So right now, I think the models are at the point where like if you have some sort of relatively well-understood technique, they can probably apply it pretty reliably. If a problem can be solved by kind of putting together puzzle pieces in the literature, you have a good chance. Like my general rule of thumb is like if there is someone or maybe two people for which some problem is relatively easy, probably they can do it. And to be clear, like this could be a person who has never heard of the problem that you're trying to approach. So for example, with this unit distance thing, I think there are like a lot of algebraic number theorists who are pretty comfortable with this idea of class field towers, which were used in the solution. Probably the vast majority of them had not spent any time thinking of the unit distance problem. Okay, so this is like super useful. It's like actually not that small a portion of the historical activity of math research, like putting things together or applying known techniques. There are definitely like a lot of people who have had very good careers because of their computational fortitude or ability to apply known techniques, you know, maybe in creative ways or maybe just like because they were, you know, really good calculators. You know, I know 150 years ago a lot of the activity of mathematics was like making tables of integrals and that kind of thing. And obviously that's changed.
