Great. Yeah, please go ahead.
So, my goal is just to talk about some experiments using LLMs to produce what I hope you'll agree is relatively high quality mathematics. I think there's been a lot of concern about LLMs producing slop mathematics, meaning incorrect or unchecked work, and I think there's a real danger with that. And I think there's also been a lot of work producing autonomous mathematics, some of which is relatively interesting, some of which are arguably less interesting. I think definitely there's an important leading indicator that the strengths of LLMs doing informal mathematics are rapidly increasing. That said, I do think it's hard to evaluate capabilities meaningfully — there's fairly bad incentives in the space and often the extent to which a solution is autonomous is unclear. So I think there's also been a lot of exciting and promising work around formalization to check informal LLM mathematics, and I'm not going to talk about that in this talk. Okay. And the other thing I'm not going to talk about, beyond what I just said, is the future.
Like, I think LLM capabilities are very rapidly increasing. Probably everything I'm going to say will be more or less out of date in a year, if not sooner. What I do want to talk about is how right now I think LLMs can be used non-autonomously to produce high-quality informal mathematics. So I'm sure all of you have experimented with LLMs at this point. If you're at a talk on AI and number theory, it's sort of hard to believe. If not, I encourage you to do so. I've been playing around with them since GPT-3 came out. So when GPT-3 originally came out, it was not accessible to the public and you could only access it by paying $20 a month to something called AI Dungeon, which was a game for having an AI, you know, more or less play Dungeons and Dragons with you. And I spent a lot of time trying to get it to do math at the time, without much success. But you know, I think in the last five months, maybe since November or December, the tools have really improved to the point where I think they can be genuinely useful for doing quite high level mathematics. So the goal of this talk is to talk about, since maybe November 2025, three or four little projects where LLMs have played a non-trivial role in sort of producing the final project, and talk about some of the experiments that went into producing those products, and maybe some best practices or, you know, what I learned from trying to use the tools. Okay. So before I get started, are there any questions about, like, the goal of the talk?
Okay, great. So I want to explain just a bunch of experiments in recent work and work in progress. So let me just start with some mathematics. So I want to introduce some ongoing work and explain the role that LLMs played in it. So here's a question of Serre that he characterizes as hazardous. So he asks: is there a motive with a Galois group given by the exceptional group G2 or E8? So I think the motive — well, okay, maybe let me just step back and say approximately what the Galois group of a motive is. So if you have a variety, you can take its homology; that's a vector space with some extra structure, and using the theory of Tannakian duality you can associate to that object a group. So the vector spaces with that extra structure are representations of some certain group called the motivic Galois group, and he asks just which groups can show up this way. So they're always semi-simple groups, and there's a classification of semi-simple algebraic groups into An, Bn, Cn, Dn, and he wants to know — and then there's five exceptional types, which are G2, E6, E7, E8, and F4. And he wants to know: can you get the exceptional groups? I think the reason for this question is that it's pretty easy to get A, B, C, and D from, like, constructions using curves in abelian varieties. So as you can see from the screenshot here, Serre specifically asked about G2 and E8, but other authors have, I think understandably, understood this question to be about sort of arbitrary exceptional groups. So G2, F4, E6, E7, E8. So despite Serre classifying this question as hazardous, it sparked off a lot of really interesting mathematics, and here you can see the answer is yes. So a huge number of people worked, maybe for the last 20 years or so, to construct motives with the Galois groups. So maybe just to briefly go through the attributions. So Deligne answered the question for G2. Jannsen independently answered it for G2, and also his method worked for E7 and E8. Then Patriat didn't quite answer the question but answered a variant. So he produced Galois representations which conjecturally come from geometry and have image in Zariski dense in F4 or E6. And then Boxer–Calegari — okay, this long list of names I won't keep reading — gave an unconditional answer for E6. And then, okay, for F4 an unconditional answer is not quite in the literature, but it's extractable from work of Gallistel, Lubicz, and Fargues. So for the experts in the room, Gallistel, Lubicz, and Yaffe found a rigid F4 local system, and Fargues showed that all rigid local systems come from geometry, and so you can use this to construct F4 motives, but I think it's not quite written down anywhere. Okay.
So if you analyze the proofs of this question, you'll see that they actually, in all but one case, answer a slightly stronger question, which is a question in complex geometry. So they answer the following: can you find a smooth projective morphism of, like, complex varieties so that the associated monodromy representation on the cohomology has some given monodromy group? So you're given an exceptional Lie group. So, meaning you take some representation showing up in the cohomology of a family of algebraic varieties, you take the Zariski closure of its image, you want to know: is that your exceptional group? This is actually a stronger question than Serre's. It's not totally obvious, but there's some specialization argument that lets you deduce Serre's original question from it. If you're familiar with the inverse Galois problem, you can think of kind of Serre's question as being about some analog of the inverse Galois problem, and this is some analog of the parameterized inverse Galois problem. So they keep kind of the same relationship. This is a stronger question. And if you analyze the proofs, you see that the existing constructions answer this for every group except for E6. So in fact they produce families of motives with the given Galois group, except in the case of E6, and in those cases you don't get families, you just get an individual sporadic E6 example. Okay, so the theorem — it's work in progress, I guess the paper will probably be out next week — with Thomas Kramer and Mark Mulan is that the answer is also yes for E6. So we found some interesting family of varieties whose monodromy has E6. I'll tell you what it is in a second. And from this you can produce lots and lots of new E6 motives, so giving a different answer to Serre's question for E6. Okay, so let me tell you what they are, and then tell you the role that LLM played in this result. So here's the construction. So you take a family of cubic three-folds in which every cubic three-fold shows up. So if you'd like, you can take the universal family over the moduli space, or maybe pass some cover. So inside of a cubic three-fold there's a lot of lines, and you can take the moduli space of such lines. That's a smooth projective surface, it turns out, and you can make this construction in families. So if you have a family of cubic three-folds, you get a family of surfaces parameterizing lines in the fibers of your family of cubic three-folds. Now you take a cyclic étale cover of that family, or if you'd like, a rank one local system of finite order, and you take the middle homology of that thing. So this is a family of surfaces, so middle homology is in degree 2, and that has connected monodromy E6. So that's the theorem. And we prove a lot more. So of course, it turns out there's lots and lots of choices of cyclic cover or rank one local system, if you'd like. And so you get lots and lots of E6 local systems, and they're all distinct and have different invariants, and it's a very fun story.
Okay, so now I want to tell you how did we use AI. And oh, sorry about that. The answer is — can you still see this?
Sorry about that. Let me reshare. I don't know why that's happening.
All right. So yeah. So why am I talking about this? Well, so in fact, no AI was used in the proof of this theorem. So okay, so why is this showing up in a talk about AI? So I want to talk about what happened when we wrote up the proof of the theorem. So we proved this completely, you know, in a good old-fashioned human way. But so here were a couple lemmas that showed up in the proof. You don't have to know a lot about cubic three-folds. Let me just kind of give you an overview of what showed up. So maybe the first one says that you take a general cubic three-fold, you look at this smooth projective surface associated to it, so it parameterizes lines in F, and then there's a certain cup product map in the cohomology of the surface. So it's a map from Sym² of H¹ of the tangent bundle to H² of Λ² of the tangent bundle. So you can compute the dimensions of these vector spaces. The source is 55-dimensional and the target is 51-dimensional. And we needed to compute the rank of this map. And what we showed is that it has rank at least 50. Okay, so that was what we needed to prove the theorem on the previous slide. So this is some 55 by 51 matrix. If it has rank at least 50, it has rank either 50 or 51. Okay. And then another thing that showed up in the course of the proof was we need to analyze some line bundles on this smooth projective surface F, and we need to show that they were generated by global sections on a certain curve in this surface. Okay, so the proof was some kind of complicated Hodge theory argument, and these were relevant to analyzing the Hodge theory of the situation. Okay, and you'll see I've bolded some statements in these lemmas. So in the first lemma we showed that there's a lower bound of 50 on the rank, and in the second we showed that some line bundle is generated by global sections on some curve, and this was — we sort of proved the exact statement we needed. But it was sort of natural to ask, like, what's the sharp statement? Okay, so here are the sharp statements. So, and you'll notice that I've added attribution to some LLM in the statement. So the first lemma, in fact, this rank — this 55 by 51 matrix has rank exactly 50. And in the second case, this line bundle, which we showed earlier was generated by global sections along some curve, is actually just generated by global sections everywhere.
Okay, so the first one we produced with some help from DeepThink, and I'll kind of spend some time discussing the process that went into that, and then the second one was with some help with an internal scaffold at Google created by Tony Feng called Althia. Okay, so yeah, I want to kind of discuss this first lemma as a case study. Before I do that, are there any questions about the statements or what I'm trying to explain here?
Okay, so this is some linear map. It's a map from a 55-dimensional vector space to a 51-dimensional vector space. We want to compute the rank. So here's the status of what happens if you just type this into ChatGPT. Well, at the time it was 5.2 Pro. Now you can try this with 5.4 Pro and with more or less the same result. And no LLM out of the box can, at least in my experience, produce a correct lower bound or upper bound. I mean, they can guess the answer sometimes. So they'll often guess 50 or 51, but I've never seen them produce a correct proof.
Okay, so the lower bound was what was relevant for us, and we did some computation with the Jacobian ring of a form cubic. So to prove the lower bound for a general F, or a general cubic three-fold Y, you just need to prove it for one. So we picked a convenient one and figured out how to compute enough of this map by hand to get a lower bound. Computing the map, by the way, seems to be quite challenging in practice. So I put it into Magma 2; I could not get it to run in finite time. That could of course be a failing on my part, but I think it's sort of challenging. So you do need some trickery to kind of compute enough of this map to understand the rank.
So I said that these models cannot run this computation out of the box. If you give them the general idea in a few lines — so ChatGPT 5.2, sorry, anytime I refer to ChatGPT I always mean Pro, by the way — Pro can do it. If you kind of give it the idea, it'll give a more or less correct computation, and Opus can also — Opus 4.5 or 4.6 can also do this computation if you kind of hold their hand a little bit. So you have to kind of give them some hints, and then if they don't succeed, tell them where they failed. So the upper bound — so we computed in some examples, and we were able to show that for certain examples of cubic three-folds, this lower bound of 50 seemed to be sharp, which was a bit surprising to us. And so for a while this was just sitting in the draft of our paper as a question, and I, you know, maybe tried for an hour or two to answer it on my own. And also, you know, of course I asked GPT Pro and Gemini DeepThink and so on, and none of them made any real progress on the upper bound. So they produced a lot of BS. Okay, but so I computed some examples, and it turned out that there seemed to be an explanation for why this map didn't have full rank. So here was the conjecture that my co-authors and I reached after kind of computing some examples, namely that there's some reason for why this map is not full rank. So let me kind of say a word about what it is. So you can consider the Serre dual map. So that has source H⁰ of twice the canonical bundle, and there's a natural section to twice the canonical bundle on the Fano surface of lines of a cubic, which cuts out the divisor of lines of the second type. So I'm sorry for this inside baseball. So there's certain special lines in a cubic three-fold whose normal bundle is not trivial. It's this bundle O(1) plus O(-1). That's a natural divisor in the Fano surface of lines. And it seemed — the equation for it seemed to be in the kernel of the dual of this map in the examples we could compute. So once we found this conjecture, the models — so Gemini DeepThink in particular — was just able to give a correct proof. So I really like this as an example of kind of human–LLM collaboration. So the proof was not particularly deep. It used some kind of obscure facts about cubic three-folds which I wasn't aware of, but you know, it was maybe half a page once we came up with the right conjecture. But the models couldn't kind of prove this without, you know, a human in the loop conjecturing the sort of reason for why this map did not have full rank.
Okay, so I just want to make some observations about this story. So first of all, the main one is the results were not necessary to the paper, right? So we had proven what we wanted to prove with no LLM help, but we had some questions. And so what I think this suggests is that we have — you know, when you write a paper, at some point you stop writing, right? Like even if there are ways you could try to improve the paper, you don't keep going because it's too hard, right? Like at some point you hit the point of diminishing returns. And this is kind of a shame, right? Like it means that there are natural open questions in your paper or whatever that you've asked, and you kind of just got too tired to answer. And I think we have the opportunity for that to happen less. Like we can produce better final products now just because the costs of sort of investigating a sort of natural open question that's maybe not the main thrust of our work have really, really declined. So that said, they haven't declined completely. So in this particular example, there was a fair amount of labor on my part and the part of my co-authors required to prove this. We didn't have to find the proof; we had to kind of find the key reason for why the phenomenon we were observing was true. And existing tools seem not to be able to do this, at least in this example and in many other examples in my experience, autonomously.
And so the other thing is I've reported a success here. So I would say in all of my work in progress, including in this paper, the models produced an enormous amount of garbage. So of course there were lots of other questions that we asked in the course of writing this paper that the models and we were not able to answer, and I spent a lot of time reading BS produced by a machine, which is, you know, I think it's very difficult to check mathematics produced by another being, be it human or LLM, and it's not very pleasant if a lot of it is wrong. So maybe I want to give some sense. I mean, I don't want to claim — I'm sure all of you have also been experimenting with these tools. I don't want to claim expertise above and beyond what you have. But at least some things I learned here were — so like there's a question of how to use these tools in a way that is a net benefit, like so it actually improves the paper you're working on or improves the science you're doing, and doesn't like waste a huge amount of your time. So at least one thing I learned is just to be very brutal with the LLM. So don't argue with it. If it produces the first — you know, when you're reading a proof and checking correctness, the first time you hit something you don't understand, just throw it away. Like almost everything the models produce right now on a difficult problem is just wrong. I think that may slowly change, but that seems to be my experience with like lemmas in my papers right now.
No, no, it's very difficult. So this is what I'm saying. I think it creates an enormous amount of garbage. And what I would suggest, and I'll talk about this a bit more, is just like not even trying to check correctness. Like if it's at all hard to check correctness, just throw it away. Like I think that there's a real danger here, which is that the models produce something that fools you. And so like the way I check correctness is like I read it, if I don't understand a single line of it, I just throw it away. If I understand it, I then put it away and then reconstruct the whole thing from memory or like, you know, by understanding it and reproducing the argument. So in particular, a necessary thing is that like when I'm going to write up an argument produced by an LLM, I do it without looking at what the LLM wrote, and like I make sure I'm going to fully understand the argument myself. I think it's very, very — the LLMs have been trained in a way to, you know, to fool a verifier. That's part of how the models are trained to get better and better. So they're very good at fooling you. And I think it's quite dangerous. I mean, there's a real danger of repeating an argument with a subtle error that one doesn't understand, and I think you should go to great lengths to avoid that. So in this case I'm quite confident the arguments are correct, but it's because, you know, I produced the arguments on my own after having, you know, gotten the hint from the LLM. And in fact if you look at the literal text produced by the LLM post hoc, there are various minor errors. The main idea was correct, but, you know, it did require a human to go in there and produce a correct argument. I mean, I think these are arguments that would pass — the LLM wrote an argument that would like pass a referee. But really the models are not truth-seeking enough right now to trust them. I mean, I think they can make subtle errors in a way a human would not make because the goal is to produce something that passes muster, not something that's literally true. Okay, so I'm going to move on to talking about another project in a minute. Maybe are there any questions about this one?
Okay, so I want to talk about a totally different question. So this is one of my favorite questions, and a lot of what I'm going to talk about is work of my PhD student Charlie Woo, who's graduating in a couple of months. So here's the setup. So pick a real semi-simple group. So think like SU(p,q). So the important thing is that I don't want it to necessarily be a compact group. So here SU(p,q) is the group of matrices of determinant one which preserve a Hermitian form of signature (p,q). Okay. And now I'm going to fix a bunch of conjugacy classes C1 through CN in this group G. And I just want to look at the following set. So it's n-tuples of matrices whose product is the identity and where the i-th matrix is in the i-th conjugacy class. And then I want to look at these up to simultaneous conjugation. Okay. So this is some set. There's actually the real points of an algebraic variety that's been studied for probably 150 years in various special cases. So in particular, if you take SU(1,1), that's SL2R, and there the study goes back to maybe Markoff or something, and it has a geometric interpretation. So it parameterizes representations of the two-sphere, like CP1 with n points removed, where the loop around the nth puncture is sent into this i-th conjugacy class Ci. So that's called a character variety.
So here's a picture. So this is a picture produced by Benedetto and Goldman in 1999. So you can really tell it's from 1999. And I think this is a great example of kind of the impact computers have had on mathematical research. So the point of this paper is literally just to graph some surfaces. So surfaces that arise this way when you take p and q to be one and n to be four. So here you're looking at four 2x2 matrices in SL2R with determinant one, of which multiply to the identity, up to simultaneous conjugation, and maybe you fix their traces. So the conjugacy classes in SL2R, up to, you know, maybe with one exception, namely the unipotent conjugacy class, are determined by the trace. Okay, so Benedetto and Goldman graphed these spaces, and you can see a picture on the right, and they noticed something weird, which is this little bean in the center. So you can see there's a compact component of this variety. And they at least — I mean, I think a posteriori it's not exactly clear why this should be surprising. Of course if you take some real affine real algebraic variety, like maybe it has some compact components, maybe it doesn't. But here maybe it's a bit surprising because the group SL2R is not compact. So if you took SU2, that's a compact group, you'd get a compact set of real points, but the fact that you get a compact component for SL2R was surprising to Benedetto and Goldman, and they kind of wanted to understand this. So maybe just let me slightly generalize this setup. So on the previous slide we had the two-sphere minus n points. So that's a surface of genus zero minus n points. And of course you could just replace this with a higher genus surface, and the corresponding objects are still very well studied. So let me take Sigma(g,n) to be an orientable surface of genus g with n punctures. Fix n conjugacy classes in G, so one for each puncture, and then I want to look at representations of pi1 of my surface which send the loop around the i-th puncture into the i-th conjugacy class up to simultaneous conjugation. Okay. And then a natural question, which I think has been of interest to people who study surface group representations for some time, has been to understand the compact components. I think this question goes back to the sort of original graphing-calculator result of Benedetto and Goldman. So they want to understand the compact components of these real algebraic varieties. And so this is fully understood for SL2R now by really beautiful work of Deroin and Fakin, and Arnaud Malot, who kind of gave a really beautiful explanation of the relevant compact components in terms of non-abelian Hodge theory. And then in genus zero there are a bunch of examples, not for SL2R, so for other non-compact groups, which were found by Thaddeus and Tolman. Again, you can interpret their construction in terms of non-abelian Hodge theory. And so these are now called — I think maybe originally by my student Charlie — minimal energy local systems. So there's some source of these compact components coming from non-abelian Hodge theory. And so the thesis problem I suggested to Charlie was just to answer the question: is this all there is? So these were all the known examples. Is this the classification of compact components of character varieties of surface groups? Oh, so I guess there's one more source of such, which is if your group is compact. So maybe yeah, so if you have a compact group, or you have these sort of special examples coming from non-abelian Hodge theory, are those the only ways to get compact components? And at the time they were the only ones known, but Charlie, about two weeks later, came back with a new construction. So he found a new source of compact components, again classified in terms of some non-abelian Hodge theory data, so certain sort of special complex variations of Hodge structures, which Charlie calls crest variations, and I'll try to give you a sense of what the definition is on the next slide. And this is a really useful classification. So for example, one of the main results of Charlie's thesis is that if the genus of your surface is at least two, so if you have a compact component, it's explained by the fact that the group itself is compact. So in the case of SU(p,q), only compact components happen if either p or q is zero in the case of high-genus surfaces. So this was known — I think it was due to Mandel if your group is SL2R, but it was not known for other groups, and Charlie really proved it for arbitrary real semi-simple groups.
Yeah. Okay. So let me give you a little bit more detail and explain the relevance of AI to this story. So here's, by the way, a picture of a new compact component of a character variety that Charlie found. I believe it was not generated with the help of AI, but Charlie can actually use Python without a crutch, unlike me. In this case, it's kind of a cool example where the whole character variety is compact. Okay. So once Charlie kind of found this classification of compact components in terms of crest variations, we started studying them, and they turned out to have a lot of interesting properties. And so Charlie had like found a few examples, like maybe he was finding like one a week. It was really an artisanal process. So here's a crest variation of Hodge structure. So I've written down the definition. If it's meaningless to you, don't worry. It doesn't matter. The point I want to make is that these are kind of complicated objects. So to, you know, find one, you have to do some kind of fairly involved computation in Hodge theory, and finding them is like — there's not an obvious algorithm to find them. It's sort of an artisanal process. You have to — there's a huge number of parameters, and you have to rig them in the right way. So a bunch of homology groups vanish, and it's quite hard for all those relevant groups to vanish simultaneously. So it really is kind of an artisanal process. Maybe to give you a sense that this is difficult. So an example of a compact component of a character variety is just a single point. So that's what's called a rigid local system, and I'll talk more about those later in the talk. But classifying those was a huge, like a very major result of Katz from 1996, and it took a whole Inventiones Mathematicae book. So this classification is some kind of generalization arguably of that one, and it's quite involved. Okay. So the question is, like, this is some artisanal process. We would like to find more examples and study them and understand their properties. And so I think a natural question to ask is: can you find them via computer search? And maybe just to be clear, like six months ago, I never would have even considered this. Like it would have been a massive project to try to explain enough Hodge theory to a computer to get it to do this computation. I'm not saying it would be impossible. Someone who's better at coding than me, you know, maybe it would only take them a couple weeks. For me, I think this would probably take me months to do what I had. Okay. So then, well, over winter break at the end of 2025, I got kind of bored, and I decided I would try it anyway. And I think this is around the time like Claude Code and Codex came out, and, you know, the software engineering world discovered that suddenly you could do a lot just by telling your computer to do it. And so I decided to do the same thing. So I vibe-coded some command line tools to compute with parabolic Higgs bundles, which are the relevant objects from non-abelian Hodge theory one needs to work with to find these crest variations of Hodge structure. So this took me like still a couple of days. It was not a trivial amount of work to explain to Codex the relevant objects and how to compute with them, but I think much less work than it would have taken to do this by myself unaided. And we found lots and lots of examples. So I mean, where before, you know, Charlie and I were finding an example a week — I mean, it was quite a lot of work to do the relevant computations — we now have many, many thousands of examples and examples of related objects. And it's — this has been very — I mean, Charlie's thesis will hopefully be out in — well, he's defending at the end of June, so hopefully soon. But it's been very useful in formulating relevant conjectures and sort of understanding the general situation and trying to come up with a finer classification of these objects and understanding their...
behavior. So a couple of comments about this. So first of all, the code is unbelievably low quality. So [clears throat] maybe let me give you a sense of what the relevant folder on my hard drive looks like now. So you can see there's like 10 Python files in this little screenshot I took. And in fact there's dozens more doing various sort of special case computations and answering related questions I asked in other folders to actually use this command line tool. You can see in the second screenshot there there's, you know, dozens of flags you can use. I provided a particularly nasty example. You don't need to set all of these flags to run the search. But you know I was trying to implement lots and lots of features and of course there was an unbelievable amount of code bloat and kind of low-quality code. And then also, you know, I generated hundreds and hundreds of files during this search. You know, I have a folder on my hard drive now with like 10,000 files which contain descriptions of various parabolic Higgs bundles, some of which are crepant variations and some are not. And so the sort of ease of use of these tools makes it very easy to generate bloat. Okay, so I want to just describe a little bit of what I learned from this. So I mean I think probably for anyone with more software engineering background than me this is obvious, but so first of all this code is literally impossible to maintain. Like I think at some point in the next few weeks I'm going to have to more or less rewrite it from scratch. And there were lots of errors, which is very problematic if you have code that's impossible to maintain. So one thing I learned is that in order to have reliable results, rather than trying to read the code and detect the errors, which I think is very unreliable and difficult in any case, it was useful just to have a huge test bed of examples that I'd worked out by hand. And so Charlie and I had done that, we'd found probably a couple dozen examples of crepant variations or near misses or various things, and we could use those to test the existing code and make sure that it was making sense. And then when it would produce an example that was relevant to us we would just check that example specifically. And so after a couple weeks of this, in fact, the code became extremely reliable. So I think, you know, while the code itself is a mess, the computations I'm quite confident are correct, and anything we're using of course we check and recompute by hand. But I think it's possible to sort of have a pipeline that lets you verify the correctness of the results without literally reading and checking the code line by line. That was a very different way of working than I was used to. Are there other questions about this example?
Okay. So, and what I really want you to take away here is I think there are things that would not have been plausible to implement on a computer, or most people would not have found it worth it to implement on a computer, that can now absolutely be put on a computer. I mean, this was maybe 2 days of work plus some intermittent work over a couple weeks to get something reliable and functional. I think it would not have been worthwhile doing this by hand. It would have been just much too much work. Like it would have been probably easier to continue computing examples by hand rather than coding up this pipeline by hand. And so I mean I think somehow this is like a huge increase in our power as mathematicians to be able to work out examples in situations where like previously you would have had to have, you know, a talented grad student work through hundreds of hundreds of examples to generate relevant data. Like here we can really do data science with these objects and that's not something that would have been possible six months ago. Okay. So I want to talk about one more mathematical project and then some kind of less mathematical uses of these tools within the profession of mathematics. Okay. So I want to talk about a sort of related question. So I alluded to this earlier. So it's natural to... so Charlie, my student, was studying these compact components of character varieties. You can ask a sort of more special question which is just what are the isolated points of these varieties. So can you find conjugacy classes so that the space of n-tuples of matrices which multiply to the identity where the i-th matrix is in the i-th conjugacy class is up to conjugation just a finite set. So these are called rigid local systems. And I mentioned earlier in the talk that if your group here now is just the general linear group, so not necessarily over the real numbers, like go over the complex numbers is fine, these are completely classified by work of Katz. And the classification is amazing. So let me just give you a brief outline of it. So he found some kind of operation called middle convolution that takes a rigid local system or a rigid tuple of conjugacy classes in the general linear group of r by r invertible matrices, and if r is bigger than one it produces a new tuple, so a new rigid local system or rigid tuple in GL r prime where r prime is less than r. And then the amazing, the magical thing about this operation is that it's actually invertible. So what that means is that every rigid tuple or rigid local system is generated from rigid tuples of one by one matrices, so just numbers, which are very easy to classify by iterating this operation. And so this is some kind of complete classification of these isolated points of character varieties when your group is the general linear group. So it's very, very useful. I mean, I wouldn't say that we completely understand these objects. So the classification is in some sense algorithmic. So there's still lots of open questions about rigid local systems, but at least you can try to... we have enough of a classification that you can kind of get your hands on them and try to answer some questions. So the main, at least one main question in this area is, well, what happens if instead of the general linear group you take some other group. And I think one of the main cases where this has been studied is where that group is the unitary group. So the reason for this is that one kind of primary interest in these rigid tuples is in finding examples where the corresponding matrices generate a finite subgroup of the general linear group. And that's the same as asking... so it turns out that these matrices are always in the ring of matrices over the ring of integers of a number field. So it's the same as asking when all the conjugates are unitary. So this is related to the inverse Galois problem. If you can find such a tuple, you've solved an instance of the inverse Galois problem. Okay. So many people have thought about trying to classify these rigid tuples, so isolated points of character varieties when your group is the unitary group. And there's a natural question you can ask which is just like, does Katz's classification also work in this case. So this was, I think, I mean, I think it's been considered by many people, but at least it appears in print in a paper from 2022. Basically he asks just, can you... does Katz's algorithm work in this case. So the answer literally interpreted is no. So there are some choices in Katz's algorithm and there exist choices which don't work. That's quite easy to see. But the question is, can you always make choices? So given a rigid tuple in the unitary group, can you always make choices in this algorithm to find a rigid tuple in a smaller unitary group? So this is something I think that people have wondered about for some time. I don't think it's like a... it's not a huge open question, but it's something that, you know, has appeared in print and was apparently of interest and certainly was of interest to me. So here's the question again. And the theorem is no. So this is joint work with Josh Lamb. It's probably at some point it will appear on arXiv. I think you should think of this as kind of a minor weekend project, but I want to give it as an example of maybe a more substantial use of AI tools. So here's an example. I've written down some conjugacy classes in the 16... the unitary group of 16x16 matrices. So maybe let me explain the notation. So you take zeta to be a primitive 100th root of unity. Take your first conjugacy class to be zeta to the minus 15 appearing with multiplicity one and zeta appearing with multiplicity 15. Okay, and then in general I'm describing some diagonal matrices or conjugacy class of a diagonal matrix and the number in parentheses indicates the multiplicity of the value. Okay, and there's a lot of... you have a lot of... and we found this... I just wrote down an example but of course we have a way of generating examples. And so for example there you have a lot of freedom, you can replace the number 100 in the definition of zeta with any sufficiently large positive integer and the same example works, but lots of other examples too. Okay, so I want to kind of indicate how this example was found. So again, this is not something where if you just ask ChatGPT 5.4 Pro this question, it will not succeed. At least I've never seen it succeed. You know, who knows, maybe in a year or two, but not right now. So it required some ideas. So let me kind of briefly describe the ideas and then the process by which an AI tool was relevant to the solution. So you can reinterpret this as a kind of question in pure algebraic geometry. So it turns out that understanding these unitary rigid tuples, so tuples of conjugacy classes in the unitary group where there's a unique up to conjugation solution to, you know, the product of matrices is the identity subject to the constraint that the i-th matrix is in the i-th conjugacy class. So you can understand this in terms of group actions on flag varieties. So here it turns out understanding these tuples is more or less the same as understanding products of flag varieties. So G mod a parabolic, so that G acts with a dense open orbit with finite stabilizer. So this is not obvious. I think it's actually... there's a large community of people that work on understanding the equivariant geometry of these objects and understanding where there's a dense open orbit. And I think it's maybe not well known in that community that this problem has another guise in the world of unitary rigid local systems. Okay. So in this world it turns out you can understand Katz's middle convolution operation too. So it's called... it's related closely related to something called a Gale duality, which is something that takes a... well, it takes a tuple of flags, you can understand it as something that takes a tuple of flags in a vector space and produces a new tuple of flags in another vector space. Okay. And so if you... if you just sort of... just once you discover this dictionary, if you... if you unwind it, what you want to find when you... when you... to find a counterexample to this question of Belkala is, well, a product of flag varieties where G acts with a dense open orbit with finite stabilizer where the Gale duality operation, which again, okay, has some parameters, can't reduce rank. Okay. And what's very easy to do, it's a little tricky to check if a group acts with a dense open orbit on something, but what's much easier to do is check that two varieties have the same dimension. So, if G acts with a dense open orbit on G mod P, then they have the same dimension. It turns out that's not enough. So, okay, so I've described something that's very easy to write a computer search to do. So there's just a little bit of monodromy here, but we need something a bit stronger. So maybe let me give you an example to show this equality of dimensions is not quite enough. So here SL3 is 8-dimensional and it acts transitively on P2 cross P2 cross P... sorry, it acts on P2 cross P2 cross P2 dual cross P2 dual, which is also 8-dimensional. So you might naively expect there to be a dense open orbit. But let me show you there's not. So a point in P2 cross P2 cross P2 dual cross P2 dual is two points in P2 and two lines. So I claim this has moduli, that SL3 or PGL3 doesn't act on it transitively. So here's why. Well, you can draw another line through two points and then look at where it intersects the other two lines. And look, this is four points on P1, and we all know hopefully that four points on P1 have moduli, they have a cross ratio. So the cross ratio of these four points is preserved by the SL3 action, and so it doesn't act densely. Okay, so once you do this computer search here, sorry, this computer search here, you still have to answer a question. You have to check, is there a dense open orbit? You could write code to do this. I mean, there's sort of an obvious randomized algorithm that would work. And you have to check a few other things where maybe there's some slightly less obvious algorithms. I think it probably would have taken me, you know, a few days, maybe even with Codex, to write code to do this. But on the other hand, you can also just ask, you know, if someone gives you a bunch of flags in a vector space, it's not that hard to check by hand. You know, if you're good at linear algebra, is there a dense open orbit? You can just fiddle around and try to move them into some kind of canonical position. And ChatGPT 5.4 Pro can also do this. So instead of writing, you know, a few days' worth, at least for me, of code to check the long list of conditions you need to check once you've found a sort of numerical candidate for a counterexample, you can just ask ChatGPT to verify them. And well, so it'll go through your list, and most of the examples you find will not actually be counterexamples, and then every once in a while it'll say, 'Yeah, this looks like a counterexample,' and then you can go and, once this is identified, take a look at it by hand and try to check that it is indeed a counterexample. So what I want to stress here is like the existence of this tool is like a huge amount of increased attention that you have access to, and it can do things that previously, you know, would have required someone who knows a bit of math. So, you know, at least a sharp undergrad or something to do this computation, or, you know, some somewhat involved work with a computer. And I should say also, I am not a good coder. So somewhat involved work for a computer is in general or has historically been very challenging for me. And so it's very pleasant to have just an expert attention that one can throw at this problem. And, you know, once you get it, once you kind of reduce your problem to something that the models can handle, it's very easy to throw a huge amount of data at them and just like have them search for good examples. So I think this is a really powerful technique, and I actually think like right now with existing capabilities, this is where I'm most excited about LLM work to produce. Like I think it's kind of common in mathematics to have a large collection of examples one wants to understand, and understanding each individual example requires some amount of expertise, maybe not like a huge amount, but like expertise that is time-constrained. And I think the existence of tools that we can now throw at such examples will let us really understand sort of big data in mathematics in places where it would have been very challenging to understand it previously. So this is a little example of that. I mean, this was kind of, you know, a few-day project, but I think it's a nice demonstration of the power of these tools to help you do kind of like somewhat involved computations.
Okay. So for the last 10 minutes I want to talk about using LLMs for writing. So, you know, mathematics isn't just about discovery. We also have to explain our work to other people. Okay. So let me try to advance the slide. Okay. Yeah. So it's kind of dangerous to use LLMs for writing. I think hopefully some of your eyebrows raised when I put this slide up. But nonetheless I think it's possible to do it productively. So I want to discuss that a little bit. So before I go on, let me just stress right now I think it is not a good idea to put LLM-generated text in one's papers. I think inevitably that will probably happen, you know, in the future, but I think the LLMs are not reliable enough to do this in a safe way. And in particular they're very good at producing arguments that can fool the reader. And I think copy and pasting text from an LLM to your paper is a great way to get errors in your paper, like fundamental errors. Like I think if you don't write down the argument yourself, you have not really checked it, in my view at least, that's my experience with these tools. Okay, but how can you use them? So first of all, I think they're quite good editors. So one thing I have started doing is asking Codex to give me line-by-line comments on sections of my paper. So here's an example, and you'll see on the left this is from a paper called p-curvature and nonabelian cohomology, which is joint work with Josh Lamb. It found in line 1143 of the TeX, it found a potential typo, and in fact this was a typo. So it really is able to read the math and find a typo in the math. I mean, this is something where you'll never... I don't think you'll ever get a human referee to look at a paper at this level of detail. One kind of funny aspect of this is that I asked the model to go line by line through my paper, and it really interpreted that quite literally. So you can see on the right, it went through literally every line and said fine. Like this line is good. I guess some of them in fact were even empty lines that I picked out, like a line break that it told me was great. Okay. So can you use them literally for writing as opposed to editing? So again, I think this is unwise, but I want to describe an experiment. So this is right when ChatGPT 5.2 Pro came out, which I think was at least for me the moment the models became like genuinely net useful. So I was visiting my collaborator Josh. He's at VANTAGE right now, and we decided we were going to finish this paper. Sorry, is there a message in the chat? Oh, there's a question about journals. Okay, maybe I'll talk about this a little later. Well, we can talk about it in the question and answer period. So we almost finished writing this paper, and ChatGPT 5.2 Pro came out, and I decided to try an experiment, which is... so I wrote... there were two lemmas left in the paper whose proofs I had written a sketch of and hadn't written down details, and so I decided to give the full TeX of the paper to ChatGPT 5.2 Pro and asked it to write proofs of the lemmas, and then I raced it. So ChatGPT 5.2 Pro is quite slow. It can take, you know, an hour, hour and a half sometimes even to answer a question. And yeah, so I won the race, first of all, which was very gratifying. And here's an example of one of the lemmas I gave ChatGPT. You can see it's some kind of somewhat complicated-looking statement about harmonic maps and monodromy and blah blah, some kind of fancy nonabelian Hodge theory and differential equations in the picture. But okay, so first of all, I produced correct proofs. So these were kind of technical lemmas. If you give the lemma statements to the model without the rest of the paper, it will not be able to produce proofs. But if it's scaffolded appropriately, at least ChatGPT 5.2 Pro was able to take the sort of precursor results in the paper and understand how, you know, given the sketch I'd written, understand how to put them together to give a correct proof. And more than that, I actually liked... sorry, I wrote Codex here, but it's actually... it was actually ChatGPT Pro. I liked one of the arguments. So it made a slight twist on the argument I had in mind, and I liked its presentation more than my own. So I didn't use literal text it wrote, but I used the idea it generated in the final presentation. So I mean, of course, it was essentially, you know, the same mathematics, but like a better way of describing it and explaining it. And so I think this is really like a... it suggests to me it's a leading indicator that these things are really going to, you know, affect the way we write. So yeah, so I guess I've talked a little bit about best practices. Let me just mention there are a number of other tools, and I think at this point like I can just unambiguously recommend you use them. So not necessarily Prism, but so Prism is an Overleaf competitor that's AI-enabled. I just want to mention it's not clear to me that it's actually a benefit, but I want to mention two other tools, refine.inc and course.inc, which are tools that give you essentially referee reports on your paper. They are very good. So I would say, you know, in terms of catching low-level errors, much, much better than you'll see in most human referee reports. I think that like basically they just dramatically improve the local correctness of a paper. They're not useful for sort of overall global problems in an argument. They just won't catch that. They're not useful for sort of big picture questions. A human referee will be much, much better than that. But like if you have a missing case in the statement of a lemma or some typo somewhere that is mathematically meaningful but, you know, wouldn't be caught by a spell checker or something, it's very likely they'll catch it, and it really dramatically increases the quality and correctness of one's writing. The comments are often wrong, so you do have to be careful. That said, they're also very nitpicky, which I find very positive. Like they really find very minor typos and errors that I think a human would maybe even not bother mentioning in a referee report. Yeah, so let me just say the upshot here since I'm out of time. I think these tools are rapidly growing more capable. They're producing a lot of nonsense, and I think when you use them you have to be very, very careful. Like it's very easy to fool yourself or let the tools fool you into thinking they have produced something correct when they haven't. But on the other hand, I think it's clear now, at least to me, that these tools have the capacity to really increase the quality of our work, and I'm enthusiastic about that. I think that at least in the near future, like we should be using them to just, you know, produce good science and actually produce better exposition in mathematics that would have been much more challenging to produce with the same amount of effort, you know, even a few months ago. All right, let me stop there.
