Stephen Wolfram and Matt Mullenweg | dev/ai/nyc

I have to have my laptop safety blanket here to use it. We might pull it up, you know, in case we need to. So Stephen, you have had the privilege of working with like some great people like Richard Feynman. What do you think he would think of what's happening right now? This LLM stuff.

So that was my... that's a small kind of thing that happened recently that was kind of a bringing together sort of ancient history for me and modern LLM stuff. And you know, these Feynman diagrams, I don't know if people here know about them, but they're this way that's used to calculate things in particle physics and they're really difficult to calculate. And Dick Feynman was always saying, you know, they're really a silly way to calculate these things. There's got to be a better way. Now LLMs can help a little bit in doing these things.

Yes.

Yeah. Not quite. 1991 I...

Yeah, I mean, I'm just a person who likes data so I've collected lots of it about myself. Occasionally it's useful. Certainly having an email archive that goes back 40 years is really useful because there, you know, lots of things... a lot of projects that I do are things that I actually started decades ago and for some reason or another I couldn't make progress on them and then, you know, the world changes and progress is possible. I mean, and sometimes these projects are things like, you know, I was interested in neural nets back in the beginning of the 1980s and didn't manage to get them to do anything very interesting and then eventually that turns out to be something where exciting things happen. That is just working on things about biological evolution which turns out to be closely related to the foundations of how neural nets work. I tried to do it in the mid-1980s, it didn't work. I basically got back out the same things I worked out at that time and now managed to make them work recently. So, it's really useful to have this sort of long archive of what one's tried to do.

Yes. Well...

Oh, I don't know. I mean, a lot of stuff we've done. Okay. So, in science, okay, I do two kinds of things. I do a bunch of technology and I do a bunch of basic science. And right now happens to be a time when both a bunch of technology things that I've done for a long time are suddenly of significant interest and also a bunch of things have happened in basic science that suddenly have been big breakthroughs. So it kind of keeps me really busy. But let's see, in terms of science, gosh, things about fundamental physics that I've been sort of working on for a long time. I think we now can claim that we understand kind of what at least the machine code level of how the universe works, what that is, which is kind of exciting and it's sort of the result of things that I did starting in the beginning of the 1980s and so on. Finally in 2019 that was sort of a big breakthrough in that kind of the formalization of things. Right, you can think about things just talking about them in ordinary natural language for example. But sort of the big kind of the ability to have a more formal statement about what's going on is what's led to a lot of the developments of technology and science and so on. You know, the big example of that is mathematics, where you can kind of formalize things about the world and build this kind of big tower of consequences. And the thing that I think is the thing of our time is kind of using computation as a broader sort of way of formalizing things than mathematics has been and building this kind of big tower of consequences of thinking about things in computational terms. And sort of a big effort of my life has been providing in a sense the notation for thinking about things computationally. I mean, back in the day, you know, people went from talking about mathematics in terms of words to having a notation like plus signs and equal signs and things like this. And that's what sort of launched algebra and calculus and eventually all the mathematical sciences. And kind of my effort has been to sort of build a computational language that provides a notation for thinking about things computationally. And that's a thing that has been, I would say, increasingly valuable. And now we can see, you know, LLMs do the thing that we humans had been able to do before of just sort of talking about things in terms of words and so on. That they too benefit from this kind of formalization and this ability to kind of make a sort of structured tower of consequences. And so that's been an important sort of complement to what's happening with LLMs is providing this kind of... I mean, I view there are foundation models, I view what we've tried to do with our computational language as providing sort of a foundation tool for foundation models, the sort of formalized version of what can happen of how you talk about the world.

66.

The meta thing is that I've spent years building tools to let me do stuff. So building technology tools, building organizations and so on which allow me to sort of be as efficient as possible in going from an idea I have to sort of real... you know, actually implementing it. And I'm, you know, I guess I do work pretty hard too. I work all the time so that helps. And I think...

I sleep eight hours. I'm a very... I lead a very boring life.

No, not yet.

Ah, that might have been a good thing. Yeah, it's... No, I think the, you know, I think the thing that has been really useful to me is, you know, figuring out the strategy for what to work on. And that's something people think, you know, oh, you do intellectual kinds of work and you do running companies kind of work and those things seem very different. To me they're actually very much aligned. That is the things one needs to do. For example, in science a lot of doing science well is not just knowing how to answer a question. It's knowing what questions are worth asking which is really a strategy issue. And that's very similar to the kinds of things that one's confronted with all the time, you know, in technology and companies and so on is okay what's the... you know, what strategy should we adopt? And I think, you know, in my efforts CEOing, which I've been doing now for 40 years, it's, you know, you're always having to make decisions about what to do. It's great practice for doing intellectual work and science and so on.

Yes.

Right. I mean, you know, about eight years ago, I thought these software design reviews that we do within our company are fairly interesting. I think designing software, particularly designing a kind of computational language software that we do, it's a very intense intellectual activity. It's sort of the most difficult, the most concentrated intellectual effort that I know because what you're trying to do is take all of these ideas and sort of drill down to the essence of those ideas to be able to know what are the, you know, what are the primitives that we should implement in our language that capture all of these ideas. So I thought this is fairly interesting and I thought maybe people out there in the world would find it interesting to see this stuff and so we just started live streaming our design reviews and that had many interesting features. I mean, you know, first of all, when we actually had our act together to announce what topics we'd be talking about, we ended up getting a bunch of sort of world experts in those things, you know, logging in and giving us kind of real-time feedback on...

Yes. Yes. And I, you know, I think and then we, you know, it's also a place where we find, you know, people who are giving all kinds of interesting comments and eventually we said, who are you? You know, would you like to come work for us? And that was one thing. Then when the pandemic started I thought all these kids are out of school and so on. I should do a public service and do some kind of science and technology Q&A for kids and others. So I started doing that and I found it really fun. I mean I hope people find it interesting to watch. They seem to. But I found it really interesting. It's kind of a great, you know, people ask this whole stream of bizarre questions and I'm kind of... it's a great exercise just look at the camera and try and without stopping answer a bunch of questions. I found it's made me a significantly better explainer of scientific kinds of things. Plus, in my day where I'm doing a bunch of meetings, reviewing projects and things like this, you know, it always improves my mood to do this, you know, live stream Q&A thing, even when I've been kind of grinding trying to get projects to happen and frustrated because things haven't gone as quickly as I would like and so on. You ask about productivity. I find that, you know, for me kind of, you know, I spend most evenings doing science for example and I'm kind of, you know, it's always open, you know, open from notebook and start sort of interacting with our computational language that's my kind of medium for working out my thoughts. And I mean I think for me right now it's like people say well, you know, LLMs will write Wolfram Language code fairly well at this point. It's not particularly useful for me to use them for that purpose because I'm a fluent writer of our language and I can typically, you know, I can start typing code before I could explain to anybody including an LLM what I want to do so it's kind of a... you know, that's a faster process. And I guess our language is sort of unique in that it's intended as a language for humans to think in as well as a language that computers are supposed to implement. So that's... it's become an interesting thing because it's kind of like it's sort of the unique thing if you want to understand what you are computing then our language is a good place to go. You know, even if you're using an LLM to sort of develop what you want to do, it's like you could just have the LLM write code that you're never going to look at and that's fine if what you're trying to do is, you know, lay out a dialogue box or something and you can tell whether it got the right answer. But if you're trying to do some piece of science or something you kind of want to know what computation was actually done. And so, you know, we have this kind of unique situation of having a language which was intended for humans to read as well as humans to write and computers to interpret and so on.

Well, they're missing the most... I don't know that they... it's been a... it's funny because, you know, I know the leaders of most of them and there's a sort of a slightly different set of understanding among the leaders and among the kind of the people in the middle so to speak. I think, you know, there's been a certain sense among the people in the middle if we just train a bigger neural net eventually everything will just, you know, work inside the neural net. That isn't... it's not... that's not I think correct.

Yeah. I mean the issue is there are things where, you know, computation... there's a certain irreducibility to at least some kinds of computation that one wants to do.

The what's...

Which bitter lesson?

I haven't been following... I've known Geoff a long time but I don't know the... I can't claim to know...

Right, well...

Well, okay, I mean look, the thing to understand is, okay, first of all why do neural nets work? Why are they doing things which are useful to us? You know, it's the... I think the number one reason is because neural nets are a bit similar to brains. In other words, if you just say, you know, distinguish two images, which one is a cat? Which one is a dog? You say, okay, there's a neural net that does a pretty good job at that. You say, well, what is it really doing? What is the mathematical definition of what a cat image should be versus a dog image? Nobody knows. The point is that in practice neural nets do a job that we think is a good job because they more or less agree about what's a cat image and what's a dog image. And why is it that the way that they, you know, make these distinctions is similar to the way we make these distinctions? It's basically because their architecture seems to capture sort of essential features of how brains work. So things that brains are good at, neural nets, I think, have a fighting chance of being good at. Things that brains aren't really good at, like working out these sort of irreducible computations, there's no reason to expect that neural nets will be good at. And I think that that's an example. But you're asking what are these... I mean just to say more quickly, what... you know, I think one thing is neural nets don't do everything, you know, they have to interact with tools. People are understanding that more and more. I think the big transition that's happened recently is going from kind of next token prediction to kind of more like next task prediction where you're kind of doing these larger scale sort of, you know, what's the next tool I should call type things rather than what's the next token I should write. And that's a... you know, so I think that that's one thing. The other thing that's happened is the question of sort of the scientific foundations of LLMs and how neural nets work, that's surprisingly unresearched.

Doing some cool stuff there.

Anthropic, they are working on it, yes. They have serious effort there. I mean I'm skeptical that the mechanistic interpretability story will end well.

Because I don't think there's... okay what do we mean by mechanistic interpretability? The thing that has happened, you know, we've been on a run for the last 10 and something years post-industrial revolution that we think we can sort of understand the machines we're making. I don't think that is a continuing story. It wasn't a story, you know, back in the day when people used donkeys to move things around, they kind of knew how donkeys worked operationally, but they didn't know sort of what was inside the donkey, so to speak. And, you know, we've gone through a period when yes, we understand the gears and levers and so on...

Not quite. I mean, my... I think that it's... okay. So the thing to realize is there's a question of does... you know, are you building something which in the end after all that neural net training ends up with a bunch of cogs and levers that we can look at and say we know how the cogs and levers work. Okay, I don't think that's what's going on. I think a better analogy is imagine that your objective is to build a wall. Let's say one method you can use to build a wall is you make bricks and they're all very nicely shaped and so on. You build the wall up in this very engineering way. I think what's actually happening in machine learning is it's like you're building a stone wall and there are a bunch of rocks lying around on the ground and you notice, oh, this rock more or less fits in here and this one fits in there and pretty soon you've built a stone wall. But if you say, well, why is the stone wall, you know, why exactly does the stone wall have that shape in that place? There's no good answer to that question. I think that that's... so I think, you know, that will be a lot of what one will find. Now, you know, the question is are there kind of laws of LLMs just like in, you know, in physics you might say there's lots of microscopic things going on, they're very complicated but there's some emergent law that's fairly simple to describe. You know, what are the kind of laws of LLMs? And I think... you know, the bad case, okay, you can ask the same question for neuroscience. In neuroscience you can say what are sort of the general things we can say about brains and you might be able to like derive brain waves. Okay, that would be a thing that is a sort of a feature of brains, a general feature of how brains work. It's not particularly useful if you want to understand how brains think. It's just a... and I think there are things like that that we can find about LLMs. I don't think we're going to find the... this is the machine that the LLM is using inside. I mean, it's the same story with neuroscience, by the way. People look for kind of machines in brains and beyond a few very simple things they don't find very much.

Yes.

Post-LLM? I don't know what post-LLM is. The... well let's see, the... Matt, we're going to take a few breaks here. So the... it's... how do LLMs affect discovery and science? You know, I think there are a few themes to that. One of them is LLMs are pretty good at doing sort of thematic searching of the scientific literature. People have studied lots of things in science, but the question is can you put the pieces together to, you know, make a new discovery not by searching keywords, but by looking at sort of themes that have emerged. You know, can LLMs discover sort of grand analogies between one field and another? I'll give you an example of something I'm trying to do along those lines. So I mentioned I think we've made great progress in understanding of the machine code of physics. The question is what experimental implications are there to understanding that machine code of physics. So one of the features is we think that now that space is ultimately discrete. So we would like to find evidence for the discreteness of space. Well people back in the 19th century were interested in is matter discrete? Is it made of molecules or not? And people didn't know that until the end of the 19th century. Turns out there was an experiment that had been done in the 1820s by botanists who discovered Brownian motion, a chap called Robert Brown. But that wasn't known to the people who were studying kind of studying physics. And the question is now in the situation that we have trying to understand whether there's discreteness in spacetime, can we do better now that there are LLMs? Could we discover that weird experiment that somebody did in some different field and realize that that's actually the thing that gives us the clue to show that, you know, spacetime is discrete or whatever else. You know, the cautionary tale is that for 80 years of the 19th century nobody knew that there was really good evidence that molecules exist. And I think this is a potential application for LLMs is sort of this thematic searching of the scientific literature. Another example. Okay. One thing I've been chasing for decades is can one make a universal constructor? Can one make something which out of molecules will assemble any molecule, any molecular structure one wants. Okay, that would be a big deal if one could find that. That would be kind of, you know, the ultimate, you know, forget chemical synthesis and having to have things in test tubes and so on. Just have a little device that's just, you know, 3D printing molecules so to speak. Is that possible? Well, could be that all the components for that exist scattered across the literature of material science and chemistry and so on. You know, could one find that with LLMs? It's not impossible. So that's, you know, another example. So there's this kind of thematic searching thing. I think the... there are sort of... well there are a bunch of places where people make a lot out of the idea that, you know, AI can do more searches for things than one was able to do before. Fact is there's an awful lot of sort of computer experiments that can be done where you're just saying let's just enumerate all the possible machines of this type. All the possible very simple machines of this type. Let's look at a trillion of them and see whether we can find ones that have properties we want. Let's mine the computational universe for things that we want. Okay. How do LLMs help with that? The way they help is if some of that mining is using the scientific literature basically that already exists to determine where the better places to mine are is potentially useful. So while you're enumerating all these things or randomly sampling things, it's kind of informed by what exists in the kind of scientific literature. I mean that's a... I'll give you an example of that. So...

I roll my eyes slightly because we've been... it's... okay there's a... you have to understand, okay this relates to sort of what is math, what's the point of math. What's... okay so let's... okay the thing that... one of the things that's become interesting right now is the idea of using proof assistants with LLMs to think about math. Okay. What is a proof assistant? A proof assistant is something where you're saying mathematical proofs have to fit together in just this particular way. If you as a human say, I'm going to have this piece and this piece and this piece. The proof assistant will tell you, yes, you successfully, you know, put the puzzle together, so to speak. But the traditional way of using these things is you as a human assemble the pieces and the proof assistant tells you whether it's right. There's an alternative kind of thing which you can do which is automated theorem proving where you say I have this theorem I want to prove. Can you find this sort of path that uses a sequence of axioms or moves or whatever to actually get from the sort of axioms we're assuming to the theorem you want to prove. A shocking fact perhaps about automated theorem proving. Automated theorem proving is an old idea, 50 years old. Pretty good algorithms have existed for automated theorem proving for a long time. People kind of use automated theorem proving, but they use it mostly to be very proud of the fact that they validated a theorem that was already known to be true. So, and that's... in fact to my knowledge there remains just a single example of a somewhat interesting theorem that was proved by automated theorem proving but was not already thought to be true which is actually something I did 26 years ago now, finding the simplest axiom for Boolean algebra for Boolean logic. And so this is something where, you know, I found it using automated theorem proving, it was kind of cool that one could do that. You say well what's the proof? Like the proof is 100 steps long and nobody has ever understood it and a bunch of people have tried including me. It is incomprehensible.

So in other words it's not a proof from which you learn anything. So what might you be able to do with LLMs? The one thing you might be able to do is, okay. So first point is that it's easy to enumerate mathematical theorems. It's easy to find mathematical theorems that are true. The issue is most sort of random theorems out there in sort of metamathematical space just aren't very interesting to anybody.

Because they haven't been part of the dialogue of mathematics so to speak. They're just, you know, this is a random fact. And, you know, mathematics, it's kind of like you have this whole universe of possibilities and it's like some places are going to get colonized by people saying we care about these. It's kind of like what happens in natural language. There are lots of things out there in the world and, you know, in a typical language there are 50,000 of them which we've given, you know, which we describe with words and, you know, we choose words that are things that are useful to us and that we're then going to build things on and it's the same kind of in mathematics that all these possible theorems that are out there, you know, there are particular ones that, you know, the history of mathematics has caused us to think are interesting and we're kind of building out these kind of, you know, these paths of things that we think are interesting. So I mean that's... this is another place where I think LLMs can be helpful is in saying, okay those humans they've written down three or four million theorems in the history of mathematics and we now have some idea of what the humans find interesting. So when you're searching all these possible theorems, you know, these are ones that the humans might find interesting. I've tried to make that work and I haven't really managed to make that work yet. Another thing that people seem sort of excited about is taking mathematical literature and formalizing it, you know, in some proof assistance system. I have to say mostly I don't see the point. I mean, in other words, there's a certain degree of interest in kind of lawyering the mathematics, so to speak, to be absolutely sure we really nailed Fermat's last theorem or whatever there wasn't a mistake. But the thing of most interest to working pure mathematicians is kind of the story of Fermat's last theorem. What are the ideas? What are the big ideas that happen in that proof? Proof is a kind of form of exposition for many mathematicians. It's not like this proof that I found with automated theorem proving 100 steps. We know it's right, but it's absolutely incomprehensible. That's not the useful thing.

I have one more thing to say about that which is that I actually do have one case which is there's a particular class of problems that have to do with, okay so undecidability was this thing, you know, Gödel introduced in 1931, big thing in mathematics. There are sort of given a set of axioms there are things that... there are theorems that can take sort of arbitrarily long to prove or aren't reachable from those axioms and so on. Undecidability has not been particularly important in everyday mathematics. There are interesting reasons for that. But the question is how far away is undecidability? Is undecidability something very special and weird or is it something actually quite close at hand? A bunch of science I've done suggests it's actually much closer at hand and just mathematics has been set up to kind of avoid it. But there's a question of how do you prove that it's closer at hand? Well, historically I back in the 90s I had some research assistant who spent a couple of years doing one proof of one thing that turned out to be, you know, undecidable and so on. Then in early 2000s I put up some prize for another thing like that. And so these two proofs that exist, you know, they're each long, you know, 5000-page documents of very detailed kinds of things. I think this is an area where some of this sort of auto-formalization type stuff with LLMs may be helpful. We may be able to churn out more of these, but I think it's a very... it's not a big market that's of interest. That particular proof is of high interest to me and but I don't think it's... it's not something... I mean to just say that, you know, the work that's going on in sort of trying to set up mathematical proofs and so on, the place where it's really likely to be potentially relevant is in software verification and in computer security verification and so on.

I've not seen that.

But okay. No, I mean the thing there that's the big question is software verification has been, you know, there's some stuff you can verify fairly easily. The question is how quickly do you run into undecidability and things which you fundamentally can't verify. And nobody really knows and that's, you know, that will be a thing that will probably get figured out over the next few years.

Academic physics, well gosh, I mean, you know, there's a certain rhythm in science, you know, typically what happens in science is...

Yes. But...

Well the thing to understand is that there's sort of a general rhythm in science. Some methodology gets invented then there's 5, 10, 20 years of low-hanging fruit to be picked and then the field goes into this kind of, you know, this cruise state for the next hundred years or something until some new methodology gets invented. That's one thing. The other thing that tends to happen is when fields are small and young, they are very receptive to kind of new ideas and everybody's like we're going in lots of different directions. Then fields are often the victims of their own success. You know, things go well, they get big, there are big institutional structures, there's big funding channels and so on. And then it's like you have something new. We don't want to hear about anything new. You know, we have a thing and that's the institutional thing that we're pushing forward with. Physics has, you know, its really big break came about 100 years ago. And where a bunch of methodology got invented that came out of sort of the effort to formalize things that have been started at the end of the 19th century. And, you know, that led to, well the big theories in physics, there are basically three big theories in physics. Theory of gravity, general relativity, quantum mechanics, quantum field theory and statistical mechanics and things like the second law of thermodynamics. Those are the big achievements of 20th century physics and they all sort of came together roughly 100 years ago.

Yeah. Right. And basically it's been, you know, there was another sort of puff of excitement in the late 1970s when I was a kid and I happened to get involved in that which was really fun because there was a certain amount of low-hanging fruit to be picked at that time but, you know, it's been kind of... it's not been in a high growth phase for a long time. I think, you know, I've been lucky enough to have ended up being in a situation where actually we have a bunch of ideas that allow one to sort of reactivate kind of the things that you can do in physics. I mean, I think we've got... here's the big picture as I see it. You know, end of the 19th century, people realized that formalization of things like mathematics was possible. And that effort of formalization eventually led to the idea of computation. But formalization, it was like you're formalizing this one thing, you're making this one conclusion from it, etc. What we now have 100 years later is what one might think of as sort of computation in bulk. We can make statements about sort of all possible computations, things like this. And I think that type of thinking opens up sort of a new direction for a bunch of these fields and that's what I... I must say I didn't expect this particularly but that's what particularly starting in 2019 has been sort of my big adventure in science is realizing that we can now make sort of the next step in terms of making progress in these areas. And it's been, you know, in physics there was sort of a certain degree of pessimism because it seemed like every time one got to, you know, that physics was never finished. People had believed back in the 1600s that one would just nail physics, that when we just figure out this is how the universe works, we're done. That didn't happen. And sort of it seemed like every time, you know, there was another step in physics, everything was more complicated and it looked like there was even more down there. I think the surprise of recent times has been that I'm pretty confident that we kind of know what the bottom level of physics looks like. Going from that bottom level to say what happens in the actual world that we experience, that's a heavy lift. It's a lot of work. You know, there are great indications that it's all going to come together, but it's going to be, you know, it's a lot to do to get there.