Take your pick of chairs, Yan.
Here we go. Here we go. Yan. Godfather of AI.
You know, Godfather is I live in New Jersey. Godfather in New Jersey means you work for the mafia.
Yes. Okay. Well, maybe we use a different terminology than that. I should start by wishing you a happy birthday. It was your birthday yesterday.
Okay. So, there we go. Happy birthday, Yan.
I turned 26 for those of you who are older.
Yes, 26-year-old Yan has achieved quite a lot. Formerly chief AI scientist at Meta for 12 years and so consequential with the world we live in right now is the convolutional neural network CNN which you pioneered in the late 1980s that underpins how we use our phones, image recognition, hospital scans, autonomous driving. And seven months ago you launched Ammy Labs, co-founded Ammy Labs. So give us an update. How is that going? The focus of AML is to build world models, physical AI that can interact in the world, learn, react, and take in physical cues. How is all of that going?
Right, it's AI for the real world for real signals. So LLMs handle discrete sequences of symbols quite well.
But when it comes to real signals, video sensors from industry or whatever, LLMs are completely useless essentially. And so it's a new type of AI that we're developing. Our ambition is to really bring about the next revolution in AI. We have LLMs and derived systems that can pass the bar exam, prove mathematical theorems, write code. All of this is great, very useful, improves productivity, can write long emails which the recipient will feed to an LLM to produce a shorter version.
It's great for that but not for understanding the physical world. That's why we don't have level five self-driving cars, domestic robots, robots that can do what a 10-year-old can do or even what a cat can do. So there's a huge gap in performance between intellectual tasks and tasks we take for granted that are really difficult for machines.
So just explain for us why scaling LLMs at the frontier the anthropics and the open AIs cannot do this.
Okay. So there's a lot of complexity about the physical world that LLMs really cannot capture because the representation of the world that they have is through text descriptions. Of course, modern LLMs have vision pipelines, so they can feed an image or video and produce a tokenized version, but there is a lot more to the world than just a textual description. There's a lot of intuitions we develop about the real world. If I drop this bottle, you can predict what's going to happen. You ask an LLM, it will tell you it's going to fall. But most of the details in the physical world are not predictable. The very idea of a generative model does not work in the real world. It's relatively easy to predict the word that follows a sequence of words, which is what LLMs do. They train to predict the next word, produce a probability distribution, pick a word, shift it into the input, produce the next word. That's called autoregressive prediction. It's a very old idea and that's what LLMs are based on. If you want LLMs to reason a little bit, you have it produce lots of sequences and then another neural net selects the best answer, which is ridiculously expensive. But this is how coding mathematics kind of systems work more or less.
Now, if you want a system to understand the real world, why not use the same technique? Take a piece of video and train the system to predict what's going to happen next in the video. And if you train the system to predict at the pixel level, it simply doesn't work. Most of the details in video are not predictable. Most of the information contained in video is entirely unpredictable. If I take a video of this room, point the camera, slowly rotate, stop, and ask a system to fill in the blanks, it can't predict what every one of you looks like. There's no information to make that prediction. So if you train a system to make that prediction at a pixel level, you kill it because you are asking it to solve a test that is completely unsolvable. Now, of course, you're going to say, 'But wait, we have systems that can do video generation.' Yes, they can, but they're going to make a prediction that is false. They're not going to predict what you look like. They're just going to put random people there. And those systems don't really construct a good understanding of the underlying physics or reality.
So the technique we've been working on for a number of years when I was at FAIR and since we created AM Labs is called JEA. It consists in training a system not to generate and predict all the details of a signal but to construct or learn an abstract representation of that signal of a video and then to make predictions in that abstract representation space. And what the system does is that it eliminates all the information from that representation that is unpredictable. And that's the way we actually can apprehend the world as humans or as scientists. We always have a mental abstract representation of reality that allows us to make predictions and ignores all the underlying details.
All the underlying details.
When do you prove that out? When do you release your first model from AMI?
Okay, so we know this idea works because we've been working on this for the better part of the last 15 years. I had a keynote at the NIPS conference in 2016 where I said the future of AI is world models. Things that can predict given the state of the world at time t, an abstract representation of the state of the world, given an action that you imagine taking, a world model will predict the next state of the world that results from this action.
But you made that forecast in 2016.
2016. Yeah. I had this idea for a long time and it's not particularly new, but I said this is the way we should build intelligent systems. Have a system that can predict the consequences of its actions. Because if you have a system that can predict the consequences of its actions, it can plan a sequence of actions to accomplish a task. You can say, I want to move this bottle from here to here. You can have an objective and then figure out what sequence of actions should I accomplish so that it puts the bottle here. Of course, there's an infinite number of possibilities. I could do this in my left hand or right hand, turn the bottle around, follow a funny trajectory, throw it. Probably not a good idea. So can you plan a sequence of actions that will result in the outcome you want? If you have such a mental model that predicts the consequences of your actions, you can do this planning. This is very classical in optimal control. It's called MPC, model predictive control.
What's new is can we train this world model from data, from observational data? And that's basically how humans and animals train themselves. You gesticulate when you're a baby, you randomly hit objects, and you realize you can hit an object. You realize also that if an object is not supported, it falls. In humans, it takes about nine months for infants to learn about gravity and inertia. So can we do the same with machines? We have had models for the last two years that already do this. Models called VJA, VJA 2.1, VJ 2, VJA 2.1. They understand video, they can tell you if something impossible occurs in a video, so they have a little bit of common sense. The representation of video that they extract can be used for all kinds of downstream tasks with very minimal fine tuning. So we know it works.
How much of a bottleneck is visual data?
No, it's not a bottleneck. Not nearly as much as text. When you take the totality of all the text available on the internet publicly, it's about 20 trillion words. Turn that into tokens, that's 30 trillion tokens. A token is two or three bytes, so that's 10^14 bytes. That's the total amount of publicly available text on the internet. Of course, big AI companies also train with synthetic data and licensed non-public data, but it's not a lot compared to all the internet. So 10^14 bytes is just about the amount of bytes that gets to your visual cortex in the first four years of your life. Whereas the publicly available text on the internet would take any of us at least 400,000 years to read through. So a 4-year-old has seen as much training data as the biggest LLMs, except it's video. Video is very redundant, not as rich and dense as text, but you learn a huge amount of basic knowledge about how the world works from this. You learn that the world is three-dimensional, objects cannot disappear from one place and appear in another, trajectories are continuous, inertia, gravity, all kinds of stuff. You learn that if you push this bottle from the bottom it's going to slide, if you push it from the top maybe it's going to flip. All those things we take for granted, we think we don't need to be particularly smart for this. A rat can figure this out, a cat too. And a cat only has 800 million neurons, about 100 times less than the human brain. But it's still a challenge for computers.
So if you succeed, let's pick one industry and see how it changes. If this succeeds, let's take defense because we're seeing companies like Anduril and Helsing in Germany, Stark in Germany, they are already embedding AI and intelligence into hardware being deployed in theater. What does success at world models mean for an industry like defense?
Okay. So what they're doing at the moment, a lot of defense or industry more generally, not just defense, are examples similar to what you cited at the beginning. Things like image understanding for autonomous driving or driving assistance or collision detection for a car. So a lot of it is just perception. The next step is more than perception, it's action. Can we plan a sequence of actions to achieve a particular goal? Plan a trajectory for a robot or a sequence of motions for a robot to fry eggs or whatever. That's a bit harder. For defense, there is something relatively easy which is planning a trajectory for a drone avoiding obstacles that already exist. But there are a lot more complex decision-making tasks to accomplish in that context where world models really will play a role.
You were at Meta for 12 years. Why were you not able to do this with Mark?
I totally was. I could have stayed at Meta and do this at Meta. The project which we called internally AMI, Advanced Machine Intelligence, it's the name of the company, we kept the same name. This had the support of Mark Zuckerberg and Andrew Bosworth the CTO and a bunch of other people in the leadership. They really believed in the idea that eventually AI systems of this type with that kind of blueprint. But what happened in the last year, 2025, was that a lot of the AI organization within Meta kind of focused itself on catching up with the rest of the industry for LLMs and there were a bunch of people who really believe that AGI will emerge from scaling up LLMs and training them on more data, which I don't believe and I was publicly very vocal about this. So there's a bit of a disconnect. So despite the fact that the project had the support of Mark, this probably was not the optimal place to do this. Also we were starting to get really good results with those systems, with VJA etc. And we knew there were applications. Most of those applications would be for industry, industry process control, controlling a complex system like a power plant, a jet engine, pharmaceutical manufacturing plant, or a patient, modeling and predicting the behavior of complex systems. Most of those applications are B2B and Meta is not a B2B company. It's entirely focused on connecting people with each other. So it made sense to leave, start a new company, and go into high gear for that project and make it real.
I know you think about AI sovereignty a lot as well and you're a leading member of the AI Alliance and there's a program called Tapestry. Certainly for the industry in Europe there was a kind of gut punch moment with the export controls around Mythos and Fable. Talk to us about what sovereignty around AI should actually mean in practice for Europe, where the focus should be.