Hi everyone. It's really great to have you all here. I'm very much looking forward to the conversation today. By way of quick introduction, I'm a partner at Radical Ventures. I lead our San Francisco office. And today we have the great privilege of speaking with Yann LeCun, who is a legend in the field of AI. He's the co-founder and executive chairman of AMI, Advanced Machine Intelligence, a new startup, and is also a professor at NYU.
Yann really needs no introduction, but I will faithfully do my duty as the moderator and give one anyway. Yann is one of the founding figures of deep learning. He received the 2018 Turing Award, which is known broadly as the Nobel Prize for computing, together with Geoff Hinton and Yoshua Bengio, for his foundational work over the years on neural networks and AI.
Yann spent over a decade at Meta, most recently as chief AI scientist of Meta, and was also the founding director of Meta FAIR before leaving last year to launch Advanced Machine Intelligence, his new startup. AMI is a research lab built on the contrarian bet that scaling today's large language models won't reach human-level intelligence, and that world models that learn from video and physical interaction are the real path forward. Earlier this year, in March, AMI Labs announced it had raised over $1 billion, $1.03 billion, which is the largest pre-launch AI funding round in European history, at a pre-money valuation of $3.5 billion. Yann is also the founding director of the NYU Center for Data Science, a member of the US National Academy of Engineering and the National Academy of Sciences, as well as a member of the French Academy of Sciences. Yann earned his PhD from Sorbonne University, and earlier in his career held research positions at AT&T Bell Labs as well as at NEC. Yann, thank you so much, and welcome to the Radical AI Founders Masterclass.
A pleasure. Thanks for having me.
So first question, I'm curious to hear your thoughts on this. You were, you know, an early deep learning pioneer. You believed in neural networks before pretty much anyone else did, and got them to work before pretty much anyone else did. You know, the convolutional neural networks that you worked on at Bell Labs in the 1980s were some of the first examples of neural networks really being able to work in a real-world use case, you know, for check reading and beyond. Through the 1990s and 2000s, much of the AI field moved away from neural networks. And yet you and a small group of colleagues continued working on them. I'm curious if you can share a bit about what gave you the conviction to keep going when so much of the field didn't agree in the future of neural networks.
Well, we knew that it worked, and we had firsthand experience with it. We knew that the reason why most people had been moving away from it is that it's very difficult to implement, and you can easily make mistakes when you do this. And at the time there were basically no way of exchanging basic software like PyTorch and things like that that we have today. We at Bell Labs basically built our own kind of deep learning framework, if you want. But we could not share it at the time. It was very difficult to distribute software in open source because people used different platforms, and the big companies like AT&T were not really used to the whole concept. And so that gave us superpowers that allowed us to try things that other people weren't willing to invest the time and effort to do, or the few who did maybe did not succeed. So open source software is a major factor in the dissemination of techniques like this that require a lot of kind of black art, if you want, to get them to work.
There was another reason, which is that theoreticians of machine learning and statistics could not understand how neural nets could possibly work. You know, according to the textbooks it was impossible; it could not possibly work. But it did, and we had empirical data to show it. So yeah, we were convinced — Geoff, me, and a few other people — and sort of kept pushing for it. We kind of started really pushing for the whole idea of neural nets and deep learning in the early to mid 2000s, and by the end of the 2000s and early 2010s the whole thing took off beyond our expectations, really.
Yeah, it's incredible. It's incredible how farsighted and prescient you and Geoff and Yoshua were. And yeah, it's amazing when you have something like that that inverts the typical phrase and works in practice even if it doesn't work in theory according to some theories.
That's really inspiring. So more recently you founded FAIR at Meta. You spent more than a decade building one of the most influential AI research labs in the world at Meta. Many of the folks in the audience are researchers who are thinking about pursuing an entrepreneurial path, becoming founders, and also considering joining larger organizations like Meta or Alphabet or whatever it is. I'm curious to hear from you: what made you decide to start AMI, and why did you feel like it made the most sense for it to be a startup as opposed to it being something that you built within Meta?
Right. So, by the way, it's pronounced Ami.
Okay, that's good to know.
In French, right? It's actually a French company. It's headquartered in Paris.
Even though I'm in New York, and we have a big office in New York. So it's a combination of things. The first thing is I had been working on the concept around which AMI Labs is based for, you know, 10 or 20 years depending on how you count. So the whole idea of using self-supervised learning from video, like training the system to understand how the world works by watching videos, basically. I've been sort of trying to get this to work for about 20 years, and essentially failing for the first 15. I've been sort of more publicly advocating the whole idea, and the idea of using that to build world models, for the last 10 years. I gave a keynote at the NeurIPS conference in 2016 where the end of my talk basically was to say, like, the future of AI is in world models, in planning, and things like that. And then spent the next five years trying to do this with generative architectures, mostly failing.
About five, six years ago, having a bit of an epiphany due to the fact that there was quite a bit of work in self-supervised learning for image recognition and speech and various other things, and the best methods worked using non-generative models, basically trying to learn a representation of images or video, or natural signals in general, without attempting to reconstruct, right? And I thought it should have been pretty obvious to me, because I'd worked on models of this type many years before, in the early 90s — something called Siamese networks — and the whole concept of not using generative models, of course, and to use that to build world models was not particularly popular. You know, this is just about when LLMs were kind of taking off, right? And so in the last five, six years at FAIR, a group, a colleague and I sort of pushed this technology, and we were starting to get really good results by 2024, 25 for images, video, and things like this. And it was clear that it was time to kind of scale this up and accelerate progress in that direction.
And there a combination of things happened. The first one is the main applications of things like world models — the low-hanging fruits, if you want — for things like industrial process control and things of that type. All kinds of applications that Meta basically had no interest in. Meta is really a consumer company focused on connecting people with each other. There were some applications, but the bulk of the applications probably was someplace else, including robotics, which Meta was going back and forth as to whether they should get involved. So that's one factor.
The second one was, you know, the whole AI enterprise at Meta was being refocused on catching up with the rest of the industry with LLMs, right? The company was pretty ahead of the pack in 2023 with Llama 1, Llama 2, Llama 3, and then Llama 4 was a disappointment, and Mark Zuckerberg decided to kind of reboot the entire effort. I was, by the way, not involved in Llama at all; I was not working on Llama. I was working on this other approach. So some people on social networks blame me for the failure — I had zero involvement in it, other than pushing for the open sourcing of it. That was my non-technical involvement, if you want.
Because I was not director of FAIR at the time. I was chief scientist, but not actually having any management duty. So it was clear that the company was going to refocus its efforts on LLMs, and despite the fact that I had the support of Mark Zuckerberg and Drew Bosworth, the CTO, the rest of the AI organization was really going to be focusing on LLMs and nothing else. So politically it was not the ideal environment anymore. And then I realized I could probably raise enough money to continue the project and make it real in a startup environment. And that also kind of succeeded beyond our dreams to some extent. So it's a real combination of things, but I left Meta on good terms with Mark Zuckerberg, Alex Wong, etc. It's just that our interests diverged. I mean, that happens. Totally.
Yeah, that makes a lot of sense, and it's really helpful to hear. And yeah, there's a long and storied history of leading folks leaving the big organizations to start new organizations that go on to become, you know, the next generation's large organizations. So I think it's...
So I'm a young entrepreneur at 66 years old.
That's right. Yeah, young at heart. Yeah, that's really exciting. And core to the work that you were focused on while at Meta, and then certainly now at AMI, is building world models. And this is a phrase that has become extremely popular in the year 2026 and also extremely overloaded. I think it's used to mean a lot of different things by different groups, different companies. And so I'm curious to hear you help the audience parse a little bit what world model means, maybe the different ways in which people use it to mean different things, what the right way to think about what a world model really is, and what, I guess, ultimately why AMI's approach to world models is the right one to take.
Right. So let's start from the beginning. So if you want to build an agentic system, you need that system, in my opinion, to be able to anticipate the consequences of its actions. Because if you have a system that can predict the consequences of its own actions, then it can plan a sequence of actions to accomplish a task that you give it. And this is not a new concept. This has been the standard way to handle control systems in optimal control theory, in motion planning for robotics, things like that. You have a model of the system you want to control, and the definition of world model in that context is: given the state of the world at time t, and given an action that you imagine taking, can you predict the state of the world at time t+1 after you've taken the action or done the intervention?
So the classical way to build a world model in optimal control is: you have a team of engineers and scientists modeling a process and writing a bunch of equations, and now you have a simulator, sometimes called a digital twin for the system, and you can use it to plan a sequence of actions, or simulate a scenario, things like that, and perhaps by optimization figure out an optimal sequence of actions to accomplish the task. Now here is the thing, though: most of the things we want to control in the world are much too complicated to be reducible to a small number of equations. And so there the obvious solution is you use a machine learning approach, where you collect lots of data of measurements on the system, actions, interventions that have been accomplished on the system, and then the next measurement, and with that you train a phenomenological model of the system. Once you have that, you can use it to optimally control that system. If that model is a model of the natural world, regardless of what the situation is, we call this a world model — which is a bit of a misnomer if your world is basically a small environment, a chemical plant or something like that. But okay, that's the term.
So that leads to a completely new architecture for AI systems, which instead of being trained to imitate humans, to basically apply recipes without having any idea what the effect of their actions are going to be, now you can build agents that can anticipate the outcome of their actions, and then plan actions that not only accomplish a task but can be subject to guardrails. And so, in my opinion, those systems based on world models and planning would be intrinsically safer than systems based on LLMs. So that's the first thing. Yeah.
So a lot of people are claiming to work with world models, and what they often mean by world model is something different. It's basically video generation, particularly controllable video generation, right? So a typical thing is you would record a video game together with controls by a player — or not a video game, but some virtual environment of some type — and then you train basically what amounts to a video prediction system to predict what's going to happen next in a video, given the previous state and an action taken by a player, or automatically, whatever.
In my mind this is not entirely appropriate. This is not a world model. But most of the people who talk about world models talk about generative world models. Okay. Why is that not a good idea? It's not a good idea because most of the information in a video... if I shoot a video and I stop the video at some point and I ask, predict what's going to happen next in the video, you can predict part of the information about what's going to happen next. So if I take an object — which you don't see because of the background substitution — and I let it fall, I let it go, right? You know it's going to fall. And so if I start the video just when I start to let it fall, you can predict what's going to happen next to a pretty good accuracy in that case.
There's a lot of situations where you can predict to some extent what happens, but there is no way you can predict all the details. So let's imagine we are actually physically in the same room right now, and I point the camera at the left side of the room, and I slowly pan, and I stop, and I ask the system: continue the video. Well, I mean, the system will predict we are, you know, in a conference room, there's a bunch of people listening. There's absolutely no way it can predict what all of you look like, okay? That information is just simply not available. So if you train a system to do video generation at the pixel level, you force it to predict things that it simply cannot predict, you basically kill it. So of course, you know, you can get around it if you want to do video generation, which is what a lot of the world model companies are... you can always get it to produce a cute video that looks nice, but it's not necessarily a good model of the underlying dynamics of the world.
So the idea that we've been developing over the last five, six years is a concept we call JEPA. That means Joint Embedding Predictive Architecture, and the basic idea of this is that you don't force the system to predict everything at a pixel level. You train the system to learn an abstract representation of what's in the video, and then make predictions but in this abstract representation space, which does not contain all the details. It basically eliminates all the details that are simply not predictable. And that's a much more interesting way of doing things, and in fact all of modeling is about ignoring details you cannot predict, eliminating noise and things that are irrelevant. So we think it's a fundamental concept that people using generative models will never get to, essentially. And for LLMs it will never get to this, because LLMs make generative prediction in time, but they basically only work with discrete sequences of symbols, not with high-dimensional continuous noisy data like video or sensor data etc.
That's really helpful, and yeah, super interesting. Thank you for sharing that. Maybe as a quick reminder to the audience: we will have a Q&A portion of the session. So Yann and I will continue our fireside chat for another 20, 25 minutes, but then we will take questions from the audience. So folks in the audience, please feel free to drop your questions into the Zoom chat. I will try to get to as many of them as I can, but yeah, feel free to drop those in throughout the course of the conversation.
That is really, really interesting to hear, and helpful to hear how you think about what makes something a world model, and the ways in which video generation models, while they create things that look like worlds, are lacking in certain pretty fundamental regards. Maybe to take a concrete application area to help make this more real for folks: one end market that AMI talks about as a potential focus area is robotics, and basically deploying AI in the physical world. And one of the major discussion points and open issues in the world of robotics today, of course, is training data and the right sources of data that are most valuable to build effective robotics models. I'm curious to hear how you all at AMI think about the types of data that are necessary to build robotic systems that are as intelligent from a physical perspective as humans. And maybe in particular to hear your thoughts on what people often refer to as the sim-to-real gap — the challenges in going from a totally simulated environment, or a model that's trained purely on video, to taking action in the real world. Is real-world data, like from sensors and so forth, necessary to build high-functioning robotics world models? Curious your thoughts there.
So the answer is obviously no, because humans learn to act in the world extremely quickly without requiring all those large-scale simulations. Yeah, so the way robotics — robot AI — is pursued these days is basically trying to take advantage of the machinery of LLMs and apply it to robots. They're not pure LLMs, because to appropriately encode images and video you basically have to have some special type of encoder that is trained separately from the LLM itself. So people call them VAS or VLMs, and basically what those things are trained with is examples of videos of people doing stuff, and they're trained by imitation. So you train an LLM-like thing to predict the actions, the sequence of actions to accomplish a task — but those systems unfortunately don't have the ability to predict in advance what the consequences of their actions are doing.
So basically they are executing a recipe, sort of blindly, more or less. It's like, similar to the situation of an apprentice cook in a top restaurant kitchen, and the cook is asked to execute a recipe for a complicated sauce, and the cook fails, does not actually accomplish the task, and probably has no way of knowing where the misstep occurred, because he most likely does not understand the underlying chemistry that experienced chefs have, right?
So it's a bit of the same thing. And so what happens is that the AI companies that want to deploy AI systems for robotics, they have to collect enormous amounts of training data to make sure that the system can be trained by imitation with sufficiently diverse data, so that when the system faces a slightly different situation it's not going to be completely lost. And that doesn't really work. The best example of that not working is autonomous driving. If you go back 10 years ago, there were a whole bunch of demo videos on the web of people who had put a dash cam in their car and collected a bunch of data, then trained a network to map the video to the steering angle. And it works sufficiently well to shoot a video of the car driving itself for a few minutes on a country road, certainly on a highway. We're 10 years later, we still don't have level 5 self-driving cars despite the fact that we have literally billions of hours of training data, right? So imitation learning is simply not sufficient, and certainly does not explain how 17-year-olds can learn to drive in 20 hours of practice or so.
What allows a 17-year-old to learn so quickly is a world model: the ability to anticipate consequences, in particular bad consequences, and not actually ever trying those, or rarely trying the actions that could result in a bad outcome. So another kind of interesting example there is: take what a 4-year-old is capable of doing. 4-year-olds are way smarter than the smartest robots we have in terms of their understanding of the physical world. I will go further: actually your house cat is way smarter about understanding the physical world than the biggest AI systems today, despite the fact that the AI systems can demonstrate proofs and stuff, right? If the task is to empty the dishwasher or clean the dinner table, they're completely incapable of doing it, even if they had a body, right? So that creates a huge issue for the robotics industry, which is that they can build the robots, but nobody can make them smart enough to be useful. And so they're kind of crossing their fingers that this will happen fast enough that they don't go bankrupt before that. So the real world is messy, much more complicated than language, to some extent more complicated than mathematics or code for that matter.
Yeah. And so the alternate way in which robotic companies train AI systems is through reinforcement learning in simulation. So if you can accurately simulate the situation, you can fire up thousands of simultaneous simulations, and then use reinforcement learning or something similar to kind of get the system to fine-tune itself. The problem is simulation works for certain types of scenarios, but not the majority of scenarios. So like fine-grained manipulation of objects with dexterous hands, simulation doesn't work very well for that. But for doing, like, kung fu moves — yeah, it works great.
Yep, yep, yeah, makes a lot of sense. And so to your point: in order to have robots that can really act as intelligently as humans in the real world, they have to have a sophisticated world model. It can't just be this adapted LLM thing. Do you think that building a world model for robotics requires grounding in some real-world data, or do you think just learning from video, for instance, is enough?
Okay, so there are several modes of learning. One of them is just understanding the dynamics of the world independently of what you can do about it, or what a robot can do about it. So just understanding that objects fall, that there are animate and inanimate objects, objects in front of others, the world is three-dimensional — like very basic stuff that you and I learned in the first few months of life, basically. And that can be mostly learned by video prediction, right? So you train a system to predict what's going to happen next in a video — as I said before, not at a pixel level, but basically learning a good abstract representation of the video that allows you to make predictions, short-term or long-term predictions, depending on the level of details you keep in the description.
Then from that you can build world models. So if you have data which either is natural data with video and sensor input from a real robot, and you have data that basically is state of the world at time t, action, state of the world at t+1, then you can train an action-conditioned world model which would allow you to control the robot. Okay.
Generally, there's a much smaller amount of data of this type than there is of the first type. So think of this as pre-training, post-training if you want, or mid-training, whatever. And then when you want to get the system to get better at accomplishing a particular task, perhaps it can fine-tune this world model or its objective functions to accomplish the task better, with higher reliability as it accomplishes it. And so that's a third phase, similar to post-training in LLMs. But the point of all this is to maximize the amount of learning that takes place in the pre-training phase, because it's cheap. You can basically get as much video data — way more video data than you can ever handle. Yeah, and then minimize the amount of data you need for mid-training and post-training.
Yeah, yeah, makes a lot of sense. That's super helpful. Another topic I'm very curious to hear your thoughts on: you often hear JEPA described as being brain-inspired or more biological than conventional neural networks. This is one of the more interesting topics in the world of AI today. I'm curious to hear your thoughts and reflections today on the extent to which it's valuable and additive to study and understand the human brain, to basically introduce takeaways from neuroscience and how human intelligence is produced in creating artificial intelligence. What some of the interesting connection points are, what the limits of that are, because obviously we're not going to necessarily fully simulate a brain, and the right way to think about that versus just being okay with the fact that AI is a different paradigm and works in a different way than human intelligence does.
So the first thing is that I personally get a lot of inspiration from either neuroscience, computational neuroscience, theoretical neuroscience, or cognitive science. Inspiration doesn't mean copying. Yeah, because you can't copy blindly without understanding the underlying principles. But clearly there are things — I mean, the whole idea of convnets is inspired by the architecture of the visual cortex, but it's also justified by mathematical arguments: local patches of pixels are highly correlated, so it makes sense to extract local features, and the statistics of images are the same all over, so it makes sense to extract the same feature all over. You get convnets, and you also get the architecture of the primary visual cortex. So yeah, to the same extent that airplanes have wings and so do birds. But the wings of birds are incredibly more complex than airplane wings. There is inspiration of one on the other. And certainly if you understand how an airplane wing works, it helps you understand how birds can fly, because you get some idea of what aerodynamics is.
But again, there's nothing particularly magical about the precise way in which the brain does computation. It just has to do it with biochemistry and has to power itself with sugar. So that poses a very interesting question: are all the characteristics that we observe in the brain fundamental, or are they just a consequence of the particular way in which computation in the brain is constrained by the hardware? So neurons in the brain are binary, they spike, they communicate through trains of spikes. Virtual neurons in an artificial neural net don't do this; they communicate through numbers, which may represent the density or frequency of a train of spikes or not. And so some neuroscientists say spikes are actually essential, and perhaps if we had fabrication technology and electronic circuits that could use spikes, perhaps it would be more efficient than the current way of doing things. And I used to be a hardware guy; I used to design chips many years ago before I was doing my PhD, and a little bit at labs also. So I know a little bit about this. I'm not optimistic about the potential impact in the medium term of exotic electronic design or exotic new technologies. I think we're stuck with digital CMOS for quite a while.
But if you look at things like transformers, transformers are basically gigantic associative memories melded with computation circuits. And in the human brain there is an explicit circuit for associative memory. It's called the hippocampus. If you don't have a hippocampus, you can't remember things for more than about 90 seconds. And so you have parallel evolution sometimes: people in AI or machine learning discover something, and then you talk to your friend, a scientist, and they say, 'Well, that looks very much like the hippocampus or the cortex or whatever it is.'
