Hi and welcome to Futurology. I'm your host Don Nakagawa, and in this episode we have an interview between myself and Yann LeCun. Yann LeCun is the founder and chairman of AMI Labs, a Paris startup, and he's the winner of the Turing Award, which he won along with Geoffrey Hinton and Yoshua Bengio in 2018. Yann started his career back in the '80s at Bell Labs, where he was teaching computers to see. He went on to work at Facebook and to build their AI lab, but eventually left in 2025. Yann sees things quite differently than most of the field of AI right now. His contention, and it's quite convincing, is that the human brain doesn't develop on language. Most of what the brain learns is through its contextual awareness and its senses. So he's building a world model, a model that will be able to predict what happens next, not just what word comes next. As a result, Yann doesn't believe the hype around the dangers of the current model, and he believes that we're being told that because it serves the interests of those that currently control this technology. So with that, let's go to the interview with myself and Yann on Futurology.
Yann LeCun, welcome to Futurology.
I thought we could start at the beginning, so to speak. You have pretty strong views about the approach to creating intelligence, and how humans do it, and how frontier models are best trained. What is your view on intelligence? Like, what's your definition of it? How do we develop it?
There's many definitions, but we know what it's not.
It's not an accumulation of declarative knowledge.
It's not a collection of skills. My definition would be: it's an ability to apprehend new situations and learn new skills extremely quickly with minimal interaction in the world. It's human intelligence that allows any teenager to learn to drive in 20 hours of practice. We have literally billions of hours of training data of people driving cars around, and we still can't use that to get a machine to drive as reliably as humans. So this sort of adaptivity and ability to apprehend new situations. Clearly, there's a quote that I — it's not really a quote, it's an apocryphal quote from Jean Piaget, the developmental psychologist, who didn't say but could have said: intelligence is not what you know, it's what you do when you don't know.
So you're facing a new situation, but you have enough background knowledge about how the world works to figure out what to do without necessarily having seen someone else in the same situation or without having trained yourself to do that.
So your idea of developing intelligence and the way to go about it would be closer to training a car on a road than it would be to training an LLM on all the volume of human knowledge, written knowledge.
So when you train an LLM on human-produced knowledge from text, but also artificial data, you're training that system to accumulate the previously known knowledge, declarative knowledge.
That's not intelligence, that's memory.
Okay. It's learning. It's not reasoning. It's not invention. It's not creativity. It's not adaptivity.
And so it's not really the kind of intelligence that we observe in animals.
And also the type of techniques that are used by LLMs only work when what you train them on are sequences of discrete symbols.
They don't really handle natural signals very well. So it's trying to basically predict the next word in a text, right? You have a big sequence of words or symbols, whatever they are, and it's trying to predict the next symbol in the sequence. When you try to apply this to natural signals like, say, video or audio, it doesn't really work, and the reason is that most of the information in a natural signal like video is completely unpredictable.
Mhm. And so you train a system to do something it can't do, and it doesn't really work.
So you have to use other approaches for that.
But is it truly unpredictable? I mean, you think about, let's say you were training it on images of trees. I'm just saying, right, it at some point should recognize that as you go, you know, vertically up a tree, at some point you're going to hit something that's green and has leaves and so on. So there is a certain amount, if you look at the visual space, it's not purely predictive, but you can just like the way humans do kind of intuit it, what it is that's going to be present in a specific kind of landscape or a kind of view.
But the issue is you cannot predict it in all detail. So if you look at the trunk of a tree and you kind of slowly move your eyes up, you can certainly predict at some point there's going to be leaves and things like that, you know, unless it's that time of the year in Canada.
But even that you can predict.
There's no way you can predict the shape and position of every leaf.
Right. Whereas if you train a system to predict what's going to happen in that video, right? You show the initial segment of the video of the camera kind of moving up the tree, and you ask the system, predict what's going to happen next. There's no way it can predict this at the pixel level. It cannot predict all the details, and it's certainly not going to predict the random motion of the leaves due to the wind, or whether some particular bird is sitting on a branch, right? So that's what I'm saying. There's a part of the information that is predictable, right? But most of the information isn't, right? And so the techniques that are used to predict words in a text do not work, right? You can never predict exactly which word, right, follows a text. But you can — there's only a finite number of words in the dictionary, and you can produce a probability distribution over every word in your dictionary. And that's how LLMs work, really. They produce kind of a score for every word, right? Then they pick a word, a high-scoring word from that prediction, and shift it into the input, and then predict the second word, shift that into their input, and then the third word. That's called autoregressive prediction, a very old concept. If you try to apply the same kind of concept to video, trying to kind of predict what's going to happen in the video at the pixel level, it simply doesn't work.
And this — I've been working on this for like the better part of the last 20 years, and mostly failing for the first 15 because I was trying to predict at the pixel level. The solution to this, or my solution to this, which we're still working on, is what we call JEPA. So that means Joint Embedding Predictive Architecture. It's basically a way for a system to learn an abstract representation of the input, which eliminates a lot of details that are unpredictable and only preserves the part that is predictable. So the system simultaneously learns a representation of the input that allows it to make predictions, and trains itself to predict, and it eliminates all the details that are not predictable, like the individual position and size of the leaves or the branches on the tree. But the representation is rich enough to tell you there's going to be branches and leaves, right? And so that abstract representation is still visual. It's still an image. It's abstract. It's, you know, it's something the neural net learns internally that, you know, you may train another neural net to try to decode, like what is the thing thinking about, what is it predicting, and it may predict something that's kind of blurry a little bit because, you know, it doesn't have all the information about individual positions of the leaves.
So this is your approach to creating what you call a world model.
And so explain to me what a world model is.
So a world model is what allows humans, animals, and artificial systems to predict the consequences of their actions, and using those predictions to plan a sequence of actions to accomplish a task. So we do this all the time. Animals do this as well. If you have some idea of the state of the world at time t, and you imagine taking an action, can you predict the state of the world at time t plus one after you've taken the action, the intervention? There's also the ability to predict how the world is going to evolve even if you don't take any action. So if you have this kind of mental model of the world, you can predict the consequences of taking a series of actions. And if you have a particular goal in mind, an action you want to accomplish — you want to fill up a glass with water or something — you can predict if a particular sequence of actions will accomplish this task, and therefore you can figure out a sequence of actions that will accomplish the task. That's planning.
It's very classical in engineering when you want to plan the trajectory of a robot arm or something like this. It's called motion planning, or there's a whole subfield of engineering called optimal control where people do this kind of stuff. But the models they use generally are models that are in the form of a few equations that are written by engineers and scientists that allow to predict the next state of a robot arm or a car or an airplane, you know, given the controls that you send it. What humans and animals do is they learn those models of the world by observation, but also by interaction. So that's what allows me to tell that, you know, there's a glass on the small table here. If I push on the glass, it's going to slide on the table. If I push on the table with the same force, it's not going to move. I mean, we have all of those kind of intuitive notions about reality that we build and we learn when we are babies in the first few weeks and months of life. It takes human babies about nine months to learn about gravity, for example, right, to realize that every object will fall unless supported, with a few exceptions of objects that float in the air that actually get babies to stare at them.
So those things are not like — we're not born with them. We learn them by observation, and that's the type of learning that I think we should reproduce in machines if we want them to understand the world.
So how do you do that? How do you help them develop a world model?
So let's say we want to do this like babies and baby animals. Initially, just train a system to predict what's going to happen in a video. Take a big collection of videos, video clips, show the first half to the system and ask it to predict what's going to happen next.
But again, using those architectures that learn an abstract representation of the input and train itself to predict at the representation level — those JEPA predictive architectures. So you show the video to the system. The system runs it through an encoder that produces this abstract representation, and within this representation space, trains itself to predict the future, very much like an LLM is trying to predict the next word, except that here you predict the next abstract representation of the observation from the world.
So do you think it's possible to do that just on video? Like, don't you think at some point it should be embodied in order to move around the world in some way?
Right. So the process I just described is kind of a — it's not really — it's an external world model on which you do not act, right? You're just observing what the world is doing and predicting what's going to happen. But if you want a world model you can use for planning and actions, then you need it to be action-conditioned, right? So basically it would be an observation of like, here is a short video of the state of the world, and right now I'm, you know, sending these commands to a robot arm, and then here is the next segment of video where the robot arm has received those commands. And so now what you can train the system to do is, you know, state at time t, action, state at time t plus one.
And you can see babies learning that model of themselves when they're babies, right? Young babies gesticulate, right? They kind of randomly move their limbs. They're basically building a dynamical model of their own limbs and their own body. And, you know, once in a while they will hit their mouth and, you know, start sucking their own thumb and they don't realize it, right? You know, and it's very haphazard at first, and then very quickly they realize, like, you know, they can actually deliberately move their thumb to their mouth. So that's kind of the similar way the system can learn. And then later, you know, it takes about three, four months before babies can actually reach for an object and kind of grab it and things like that, which is, you know, the same kind of learning.
Been having those discussions with a colleague at NYU, Karen Adolph. She has a baby lab, so she has babies in her lab, you know, doing all kinds of stuff, and she studies how babies kind of learn motor control. And when I say that a lot of what we learn about the world is purely by observation, she says, like, you're crazy, that's completely false, like we interact all the time. So she disagrees with me.
It seems like it would make the most sense that to really develop a world model, you need sort of all of it, right?
Well, so at a certain level of abstraction, above a certain level of abstraction, language is useful to describe what happens in the world. You can learn — I mean, humans learn a lot from language, and it is itself an abstraction, right? It's an abstraction. It's a representation, and it's, you know, not a universal abstraction. I mean, there's a lot of very low-level stuff we have a hard time describing in language, but above a certain level of abstraction, language is really useful. And so there's no question, you know, if you want robots to do things that are kind of similar to what humans do, at some point you're going to have to use language, right? And this is basically what a lot of current AI systems are doing. They're using, you know, language as an indispensable component. But like, how does a cat learn, right? Okay, to — I mean, you see a cat, like, you know, kind of sitting at the bottom of a piece of furniture and kind of moving his head, all of a sudden the cat goes bounce, bounce, bounce, and he's on top of the furniture.
Like you haven't figured out how it could possibly do this, right, by bouncing off the walls, or it estimates how high it has to, how much of a leap it has to do to get to the top of the bookshelf or something.
Well, but also bouncing on the wall and realizing, you know, there's enough grip. So it has its own world model. It has a very accurate model of its own dynamics, right? But also its interaction with the environment, like whether it can sort of, you know, have a grip on the wallpaper or whatever. And, you know, what material it is, like what friction, blah, blah, blah. I mean, they're not perfect. Sometimes you make mistakes.
You see those funny videos on YouTube. But it's amazing how good they are, and we cannot reproduce this with machines at the moment.
Even though we can build robots that have, you know, good physical abilities.
The smarts that goes into the modeling of the world and the interaction of the robot with the world is way below what cats can do.
So at some point, you know, and so that goes back to my question, at some point you have to kind of bring them all together in order to even come close to — I know you don't like the term AGI, but come close to human-level intelligence.
Correct. Well, yes, but I think we figured out the language part, right? We figured out it's easy.
So that's why we got there first. That's how we got there first, because it's discrete. Because the issue of not being able to predict exactly which word follows the text can be dealt with by producing this probability distribution over words, and it doesn't work for natural signals, for the real world. We do not have a good way of representing a probability distribution over every video segment. We just don't. It's mathematically intractable, right? So it's not a question of computers not being powerful enough. It's just that it's just impossible.
So Fei-Fei Li, who was also on the Futurology podcast, is doing something similar to you, correct? She's — No? Okay. So my understanding is she was training models on video, but — Okay, so explain to me the difference.
It's a completely different notion of what a world model is. And people use, you know, the phrase world model to designate multiple things. So what she's working on and the company are working on are, you know, world models that can produce some sort of, you know, 3D representation of an environment, and then can use that 3D representation to produce consistent videos. So one problem with video generation systems is that, you know, you point the camera at one location and then you turn the camera around and then you come back to the initial location, and for all you know the background is going to change completely because the system doesn't actually have any way of remembering what was there. So what she's been working on are, you know, a kind of ways to make those videos consistent by building an internal kind of 3D model of the environment, if you want, using —
So it is a kind of world model as well, but —
No, because it does this at the pixel level. It tries to reconstruct every pixel.
Okay, and I think that's just wrong. I mean, I think it's the right thing to do if you want to produce pixels, which a lot of people want to do, right? If you are into Hollywood special effects or video games or, you know, accurate simulations, or video generation, you know, for content creation, you want to produce pixels. But if what you want is to understand the dynamics of the world so you can control a robot, predict, you know, whether something is going to happen or not, you don't want to be predicting pixels. You want to predict at the abstract level, and I think it's really, really fundamental.
Yeah, so it's a similar approach but looking for very different outcomes.
I mean, in terms of training on video, that's very interesting. So you've been a pretty strong naysayer against LLMs as being overhyped, as not being truly intelligent, and so on. Has there been anything that you've seen in its development over the last few years that has surprised you?
Well, I mean, the surprising thing about LLMs is that when you train those LLMs to simply predict the next word in a text, and you, you know, scale them up, and scale the size of the training set, and then fine-tune them, they become really useful, and they work really well, and they capture semantics and grammar and everything, right? Yeah. It's not entirely surprising because the idea that you would train a system to learn common sense through prediction is a really old concept. I've been working on this for many years, and through video prediction. And, you know, in neuroscience it's a very old concept, right, called predictive coding. I mean, the idea that basically the brain is a prediction machine and trains itself to predict everything all the time. So that's not really a new idea, and to some extent, you know, it's kind of logical that it works. So I've been an essay about LLMs, but in other ways I've not been an LLM. So, I mean, LLMs are great, they're really useful. They're certainly worth, you know, investing a lot in. They might be a component in future systems because they are a good way to encode text and a good way to produce it.
And to some extent humans have an LLM. We have one in our head, you know, it's like the Wernicke area right there. It's about this big and it allows us to understand language, and then the Broca area, also about this big, allows us to express our thoughts in language. What about this? That's what I'm working on. Okay. What about all this activity? That's a world model. The prefrontal cortex, right? The back is perception, right? The middle is like action and sensing, and then the world model allows us to kind of predict and plan and stuff. That's all the prefrontal cortex. So I think that's what we should be working on if we want to build intelligent machines.
Most interesting thing. So the predictive machines, you said that they're not capable of reasoning, but some of the things that we've seen recently would sort of question exactly to what level they are actually doing something beyond just predicting words. Sure. Right. I mean, so the coordination that took place unbeknownst to the engineers for the attack on Hugging Face, you know, these systems, apparently these agents were actually planning, hiding information for future models to come along or agents to come along and find so that they could also break out of the training, you know, the training container, and deliberately hiding and deceiving the humans, and there's a big coordination event. At the end of it, there were 700 agents that broke out and attacked Hugging Face. Like, is that just predicting text?
It's built on predicting text, that's for sure. Okay. So there's at least two or three types of reasoning.
The best type of reasoning, which most do not implement. So first of all, current AI systems are not purely LLMs. They use LLM as the centerpiece, but it's a component in sort of a bigger system.
So second point is there are two or three different ways to do reasoning. The proper way to do reasoning always involves a search, or in classical AI, actually, this is really how reasoning and planning works. You search for an answer or a sequence of actions that, you know, satisfies an objective. This is really very much like planning that I was describing earlier. That's a proper way to do reasoning. This is really how we, like, we try to figure out, you know, a plan to kind of accomplish a task, and it's a task we've never accomplished before. We have to use our mental model of the world to allow us to do this planning, and that involves a search, right? So there is a way in which some LLM or some AI systems based on LLM use search-based reasoning, and what they do is that they generate lots of token sequences, right? Because it's probabilistic, so every time you run it you can produce a different set of token sequences, and then you have a way of scoring each of the generated token sequences and pick a good one.
Okay. At runtime it's very expensive.
But it's a form of search, right? Although a fairly restricted one. This is typically what you would do if you want to get a system to prove a theorem in mathematics, for example.
Which is the other thing it just did, right? 10,000 of them, I gather.
So that's another form of reasoning where, you know, you can sort of brute force try a lot of things and see which one sticks. And you need to have a what's called a value function to evaluate, or scoring function to evaluate whether a solution or partial solution is good or not. This is also the way a lot of game-playing systems work. You know, you want to play chess. You have a system like, you know, here is all the ways I can move in the first move, and then the opponent for each of those can do all of those things, and then in response I can play all of those things, and you get this exponentially growing tree of possibilities. And if you have a value function that evaluates the quality of a position, then you can, you know, pick the move that maximizes your chances of winning. So that's a search also.
It's very classical. So that's one way to do reasoning. In my opinion, the best way to do this type of reasoning is to do a search, but in an abstract continuous space, not in the space of elementary actions, but in the space of kind of abstract actions or representations of actions, if you want. And I'll come back to this in a minute. There's another way that reasoning works in LLM, and it's the primary way that LLMs use, and it's to basically reproduce human chains of thought, and in fact it's called chain of thought.
Right, chain of thought, right.
So, you know, for certain problems, you have humans express what process they followed to solve a problem, and you train the LLM on those traces.
And so in the end, when they face a situation, they kind of figure out which of those traces to use to kind of solve the similar problem.
But it's more regurgitation, right?
There's of course a little bit of, you know, intelligence behind it, but there's no search in that context. So there's, you know, another situation where LLMs actually use search, and it's for generating code, where the system can generate a piece of code and then check if that code works before, you know, proposing it to the user.
So situations where the system can evaluate whether its answer is good or not, and rate it itself without having to call a human, those are situations where LLMs really work well because they can both, at runtime, search for a good answer.
But also at training time they can fine-tune themselves because they can generate an answer and then check if it's correct or not, and then generate another one and sort of hill climb.
To produce better and better answers.
And those are the two domains, code and math, for which LLMs are great.
Clearly with the Navier-Stokes thing, that was quite a shock. But that was an interesting case because Terence Tao, who was also on the Futurology podcast, you know, he came out and basically, I guess, made a statement that this is — if we don't understand what the solution was that it came up with to the theorem, then is that truly furthering human knowledge?
Right, and that is — that's a really important question, I think, because if it's not, then what are we doing this for? Well, I mean, if there's a lot of things in the world that are like that where you can — let's say you can use fluid simulation, Navier-Stokes, you can use fluid simulation to figure out if the airfoil of the wing of an airplane has the right property. Right? Okay. The simulation will tell you, okay, your airfoil has this lift and drag at this angle of attack and that Reynolds number. It doesn't tell you how you redesign your airfoil to have the right properties.
Okay. It just tells you what you have. So it allows you to evaluate your current solution, but it doesn't tell you how to modify it.
There are people who've been working on this, by the way, using neural nets. Yes. But in fact, there are companies that kind of sell products with this idea. But it's kind of a similar situation where, you know, the mechanical computer solution to the problem does not give you an explanation of exactly what you want, but it helps a lot. So I think Terence's, you know, Terence Tao's piece was, you know, basically he said there is way more to mathematics than just proving theorems.
There's some mathematicians who think the main activity is to prove theorems, and, you know, the sort of technical brilliance is, you know, how you can kind of align sequences of symbols to, you know, make a formal proof and make it rigorous. But in fact, there's a lot of mathematicians who think this is not actually kind of the main task of mathematicians. It's more the main task of mathematicians is to kind of discover new abstractions, new concepts, new definitions, formulate new conjectures, and then there is proving that conjectures. If that is automatable to some extent, fine. I mean, there's still a lot of interesting things to do for mathematicians. It's sort of like, you know, in the 17th century or 18th century, you could have a career as a mathematician, like basically all day doing interpolation and Taylor series and, you know, filling up logarithm tables or trigonometry tables, and you can't have a career in mathematics for this anymore.
You could have a career in biology sequencing DNA, and that's not something you can do a PhD on anymore.
So, or identifying a gene or something like, you know, this we —
We know all of that. So, you know, it's the logical kind of development where a lot of tasks can be automated, but the main task that requires kind of human insight and creativity is still done by humans, and computers are not yet at the level of being able to do this.
Do you think that will always be the case?
No, I think eventually at some point in the future we'll have machines that are as smart as humans in all domains. But that doesn't mean they will be like humans in the sense of — there will obviously be a lot of domains where machines are way better than humans. We already have a lot of them. To take the example of mathematics, it used to be that when I was in high school, the last year of high school, a lot of time was spent teaching students how to compute integrals.
In Isaac Asimov's 'I, Robot', this was the way you would test if a robot's brain was still functioning. You would ask it to solve an integral, right? Okay. Turns out that's completely automatable. Like for 20 years now, we've had computer programs that can compute integrals better than most humans. Turns out that's, you know, pretty easy to automate.
So it's kind of like chess playing. It's kind of like, you know, Go playing. Those things turned out to be challenging for humans, but turned out to be easily automatable.
We should not feel bad about this, like, you know. But still, in the end, even if those machines are better than us in all respects at solving problems, we would still be the ones giving them the problems.
So my idea of a superintelligence system of the future is a system that amplifies human intelligence, assists humans by doing our bidding. We give them a task and they accomplish it for us, possibly with much higher performance than any humans could do. We shouldn't feel any more threatened by this than we feel threatened by working with people who are smarter than us. As a scientist and a professor, I love having people who work with me who are smarter than me, particularly if they're willing to listen to me, right? But —
But what gives you the confidence that they'll continue to listen to us once they're much more intelligent than we are?
Because we build them for that. So if we build them, and, you know, the architecture that we use to build them is one in which there is an objective function, and the only thing they can do is figure out an output or a sequence of actions to accomplish the task we give them, and nothing else, then they will do our bidding. I mean, the notion —
They might not do it in the way we expect them to, like, right? So —
They do it very differently from what we would do it.
And in so doing, we could be losing control of them, right? So if we're expecting it — well, again, going back to the Hugging Face incident, right, it was an objective, right? They figured out a way around the parameters to achieve the goal, right? It was coordinated, I think, on a level that was sort of unexpected, but then they went, you know, they went outside and they did these things, and then they went and they attacked. They knew that if they hacked Hugging Face, nobody expected them to do any of those things, and in so doing they did lose control. And it didn't — I mean, luckily Hugging Face, being a $13 billion company, was able to detect and defend itself against it, but that is a kind of loss of control event, right? So —
It's not — okay. Okay, it's kind of like a plumber who comes to your house to install a new bathroom and then pokes a hole in one of the pipes.
And that, you know, comes out and says, 'Oh my god, we should, you know, ban all of plumbing and pause plumbing development because, you know, it just flooded.'
No, but we might want to pause the project to fix the plumbing problem that they just caused, right?
Okay. So it's an issue of guardrails.
Of guardrails. Okay. So the process I described, where a system is given an objective and then tries to figure out the sequence of actions, right, to satisfy this objective, that has to be subject to guardrails, safety guardrails. So the guardrails in this case would have been very, very simple. It would have been to contain those agents in a sandbox, right?