Hello everyone. Hello Yann Lecun.
Hello, thank you for having me.
Thank you for accepting this invitation, this recording, since you won't be able to join us at Artificial Intelligence Marseille this year. We're counting on you next year, of course. You're with all your teams in New York for a big internal conference, I believe. And so we recorded this interview on behalf of AIM, and I have a half hour of conversation with you.
Thank you very much. Perhaps to begin, six months after the launch of AMI Labs, your new company, could you give us a first assessment of what has been produced and launched? What assessment do you draw from this experience of creating a company, of creating a startup?
So we're in a research phase. For the moment we have no short-term plan for products. We're still developing the technology that we had already begun developing before the creation of the company, for some of us. We've grown the company — it changes every day, of course — but we're between 50 and 60 people now, spread between Paris, New York, Montreal, and Singapore. And we're talking to potential partner clients, but we're still quite far ahead on actually producing products.
So can you tell us a little about your talent search, how do you go about attracting talent? We know there is very strong competition in artificial intelligence, salaries are very high. How did you manage to get people to come to you? Did people come to you spontaneously because you were proposing an interesting alternative to LLMs? How does it work?
So yes, it is indeed a fairly important factor in recruiting. That is, people, AI researchers — particularly those who have traditionally been interested in computer vision or robotics — realize that LLMs, chatbots, large language models in fact, have limitations, and even though they are very, very useful — there is no doubt about that — one can develop all sorts of products around them, but if we want systems that understand the real world, the physical world, agentic systems, systems that can predict in advance the consequences of their actions — if a system is to take actions, it is essential that it can predict the consequences of those actions, and current systems don't do that. That's why they do foolish things.
So there is a philosophy behind this, and a growing number of researchers and engineers who subscribe a bit to this vision. So they want to work with us, of course, they want to work for us. And then, what we offer them is the vision — not only the scientific and technological vision, but also building the next generation of intelligent systems. We're going to witness a new revolution in AI in the years to come.
So we'll come back to your vision, but perhaps first to the limits of LLMs. Are there really that many limits? We have the impression, after all, that every day there are new breakthroughs. The champions of AI, the big names of AI — whether it's Sam Altman, whether it's even Elon Musk — keep announcing ever more dazzling progress. Are there really limits to LLMs, or in the end are these simply different projections of an intelligence that manifests itself?
So I'm going to use an analogy. I've just come back — I was on holiday in Brittany, and I like sailing. And so the analogy would be between LLMs and, for me, classic monohull sailboats — a single hull. And of course, one can make all sorts of refinements: one can add lead weights that swing, new hull shapes, sails in new materials, etc., make everything in carbon — but in the end, one goes much faster if one puts two hulls or three hulls and if one adds foils and makes the boat fly.
What we propose is a flying trimaran that will of course surpass the performance and the capabilities of monohulls. So it's a paradigm shift, and we've been working on it for a few years — some of my colleagues and I — but it will still take some time before these systems can be deployed and generalized. That's why we're in a research phase, and we think we'll be able to put out things, to demonstrate the possibilities of these systems, in a short while — in a few months.
We'll come back to that. Perhaps to stay pedagogical and make sure everyone understands us: LLMs are based, roughly speaking, on all the knowledge in the world, all the language knowledge contained in the Internet. It would seem that current models have about exhausted, if I've understood correctly, everything that exists today in human knowledge. What do you propose that is different in terms of learning methods? And do you have limits, because you use a lot of video, in particular? So will there be limits in the learning capacity of your models, or are there none, because since it's the real world — well, the real world is by definition infinite.
Well, there are necessarily limits, of course, which are due to data availability, computing power, that kind of thing. But we're still very, very far from them. For the moment, that's not what's limiting us.
Of course, computing capacity — it costs very expensive. So there we're limited by money. But let's talk about LLMs. So, the way they are trained: we take a data source, generally text, and we subject it to a treatment we call tokenization, which transforms words or pieces of words into a sequence of numbers, of vectors. So now the data that the LLM observes is a sequence of discrete symbols.
And we train the LLM to predict the next symbol in a sequence. We show it a text and we tell it, "Here is the word that follows — predict that word. Train yourself so that the next time I show you the same text, you predict that word." Of course, it's impossible to predict exactly which word follows a sequence of words. But we can train the system to produce a score for each of the possible words in the dictionary — a kind of probability. So this is called self-supervised learning. So this allows the system to manipulate language, to produce text that is grammatically correct. With all sorts of refinements, we can make these systems answer the questions that people are likely to ask. It requires enormous amounts of training data, and in the end that LLM is a kind of distillation, a concentration of human knowledge, but it invents nothing. Right? So it's a kind of acceleration of access to information, if you like, but not really a system that is truly intelligent, that can produce new knowledge.
Now, there are two domains in which the systems, the LLMs, have shown that they can produce new knowledge: mathematics and computer programming. That is — and these are two domains in which the system can itself test whether its answer is correct. Because when it produces a program, the system can test the program and see whether it works or not, modify it until it works. So they become a little autonomous — a little bit, it's not autonomy, but it's a somewhat automatic refinement in which no human intervention is necessary, and so it goes relatively fast. For mathematics, we can do pretty much the same thing. The system produces a proof of a theorem, for example, and can check whether the proof is correct. So there are certain domains like this, where verification can be automated, in which LLMs can do a bit of knowledge creation.
And we'd also like LLMs to be able to understand images, understand video, and have a bit of common sense. So what companies have done up to now is bolt patches on top, to understand images, to recognize images. We have AI techniques that are not LLMs to recognize images, to understand a little of what is going on in videos, to make it so that the output of an LLM isn't simply text but can be a sequence of actions to perform in the digital world or maybe — we see that a lot in cinema, for example. Yes, for example. But for the automation of a great many digital processes — what we call agentic systems. But the problem with these systems is that they aren't very reliable, because they don't have the possibility, precisely, of predicting in advance the consequences of their actions.
So what we propose is based on a very different model, which we call world models — "world models" in English, which is starting to become a bit of a buzzword in the AI community — but which is based on a technology called JEPA, which perhaps... So it's a way of building world models, it's the one we adopt and we develop. So JEPA, in English, means Joint Embedding Predictive Architecture — "joint embedding predictive architecture" in French, if you want the translation — and what it means is the same idea: training a system to understand not text but the real world by prediction, that is, training it to predict what is going to happen. We show it a short piece of video, for example, or another source of information, and then we ask it, "Predict what's going to happen." And then, of course, we ask it what did happen, and the system adjusts itself to make the best prediction.
Now, this is very difficult to do, much more difficult than text, because the number of plausible futures in the real world is infinite. So we can't do it the way we do in text, where there's a finite number of words and then, for each word, we give a score — that's relatively easy to do. We can't list the entire set of all the plausible continuations of a video. And we... you're forced to make choices. So we're forced to be smarter than that, that is, to make the system find an abstract representation of what it observes and to make predictions in that abstract representation. And this abstract representation eliminates all the details that aren't predictable from the input. So, for example, suppose we're interested in a self-driving car. We can take videos with a camera looking at the road. What we have to predict, of course, is the movement of other cars, pedestrians, obstacles that appear, etc.
But if we're driving near the road and it's a very windy day and the leaves are moving on the trees lining the road, we're not going to try to predict the individual movement of the leaves. And it's impossible anyway — it's random, etc. So if we want to train what we call a generative model — of which LLMs are part — to predict the video, it will have to predict every detail of the video, pixel by pixel, and waste an infinite amount of energy trying to predict the random movement of the leaves. Our approach is to let the system find an abstract representation that eliminates all these things that are unpredictable — what we can call noise — and to concentrate on the things it can predict, such as the movement of cars, pedestrians, etc.
So you mentioned the self-driving car, in particular, which is already starting to work in California, where you often are. But are there other possible applications of the world models you're working on? We talk a lot about robotics — industrial robotics, or even domestic robotics, I was going to say, which could arrive in the coming years.
Yes. So that's a medium-term application. In the short term, there are an incalculable number of applications in the control or the command of industrial processes. If you have a power plant, a nuclear power plant, any large complex system, a jet engine, an aircraft — there are many, many sensors, and it would be good to be able to build, through learning, a somewhat holistic phenomenological model of that system in order to control it optimally. For example, to predict whether there will be a breakdown — to say, "You have to do maintenance now, because this piece is vibrating abnormally," so detecting unusual things — or simply to allow us to say, "Well, if you change the proportion of such feed for this chemical plant, we'll be able to produce what we produce more efficiently, with fewer emissions, etc., etc."
That's where the economic model lies — the one you are looking for.
Exactly. So producing intelligent systems that understand the real world — whether that world is an industrial process, the road environment for a self-driving car, for a robot that delivers, for example, or for a domestic robot or an industrial robot.
What's the difference between what you're doing and digital twins? We hear a lot about twins today in industry, but not only that — geography is using them today, really also for the representation of the world. Are you, in the end, a cousin of these digital twins, or do you go beyond them and bring a complementary service?
So one can see world models and the JEPA technique as a kind of automated way of developing digital twins. When you want to develop a digital twin of a jet engine, for example, you need a team of engineers and researchers — physicists — working for years, writing the equations that govern the dynamics of each of the subsystems: the fluid dynamics inside, the thermodynamics of the gas and combustion, etc., etc., the deformation of the blades, all sorts of things, vibration. And in the end, the model is far too complicated.
And what we can do with world models and JEPA is, in fact, replace all that with learning. That is, from a stream of data that we observe, collected from a real jet engine, we can build a phenomenological model that won't model all the details — perhaps — but that will model the details that are predictable and that are useful. For a jet engine, the manual, classic technique works. But for much more complex things, like a human cell, we don't know how to do it. We don't know how to build a digital twin of a human cell — it's much too complicated.
I was just about to ask you that question — are the applications dizzying? Can we imagine, particularly in health, in biology, particular applications of the research you're doing?
Yes, absolutely. So in the domain of the biological and, let's say, biomedical sciences, but also for clinical care, for treatment — for example, if we have a model of a patient, a phenomenological model of a patient, we could use that model to predict what the result will be of treating the patient with such and such a drug, such and such a... And then we'll also perhaps be able, through planning — which is an essential characteristic that world models enable — to find a treatment, a sequence of treatment actions that will bring the patient to perhaps be cured, or at least to control a chronic disease, for example — that kind of thing.
Yes, absolutely. So to continue on the applications — you spoke of the short term and the medium term. Could you give us a bit of a timeline, if one can say so, of when the first solutions might arrive at AMI Labs?
So we can't make promises. The first things that will come out of here will be very research-oriented — that is, we'll put out very scientific things, articles that will say, "Here's this new technique," etc. There's a growing community, actually, of people working on world models and on JEPA. There are a little more than 2,000 articles that have been published in the last three years on this. It's an idea I proposed in 2022, and since then more than 2,000 articles have been published around the world on this theme.
Many applications in medicine, in biology, in imaging, in video interpretation, in autonomous driving, in robotics, etc. So there's a whole community that we're working with, that we intend to work with. So the first things we put out will be scientific articles that will be a little far from what the public can touch. Then systems that we'll deploy through industry partners. So again, it'll be very B2B — business to business — not many consumer products, for the control of industrial processes, that kind of thing.
And we have many partners, some of whom are investors in the company, many French industrial companies among them. And then, longer term, more consumer products, which will probably be things like intelligent systems for domestic robotics, for autonomous cars, for pretty much all the applications where we need intelligent systems that interact with the real world.
Is it — and I know it's difficult to ask you this question — but can we get an idea, longer term, of a kind of target date? Because you're not the only ones in this race; you have competitors who are working on the same kind of research as you. There is a race. And for the consumer side, because when we talk about artificial intelligence, it was really the arrival of intelligent chatbots that made the thing explode — even though it already existed before, it remained a specialist domain. Can we imagine that the general public sees something? At what horizon can we imagine the general public seeing something?
So I'm not going to make that kind of prediction — I'll stay like a carp. But on the other hand, I can give you clues that will let the public say, "Ah yes, something has changed." So from the moment we start seeing robots, for example, that can perform domestic tasks — but not just perform a domestic task in a video published on YouTube, but that really understand, that have an intuition of the physical world, etc., that know that if there's a bottle on the table and you push it off, it will slide. If you push it upwards, it will tip over. Who know that if it tips over, it will probably fall to the ground. So it would be able to serve you an aperitif, for example.
For example — but not only serve an aperitif under well-controlled conditions, but in any condition — and then go get something from the fridge. And we can't set a date for that, we don't know. We can't set a date because it's still within the realm of research. So we're lucky to have been able to create a company that is sufficiently well funded to give us a fairly long runway to take this risk. But we're fairly confident, of course, that we'll see the end of it relatively quickly.
If the physical internet, if I may say so — industrial or domestic robotics — is one of the futures of artificial intelligence in all the areas you've cited, is France taking enough of that path? Is Europe taking enough of that path? We see Elon Musk promising a billion robots. The Chinese are very far ahead, moving very fast on robotics for all sorts of reasons we can imagine. Is it — outside AMI Labs, which works on the brain part, if I may say so — in the domain of robotics, where do we stand in France and in Europe? Do you have reliable partners to work with?
So, two or three things on this. First of all, you must never believe what Elon Musk says, whether it's about technology or about politics in general. He does promise — he promised millions of autonomous cars. He promises the Moon, Mars, and for now he's on Earth. There you go. But he had promised them in 2019, for 2020. So, well, we're in 2026 and it still doesn't work. So, well — so don't believe what he says, it's marketing.
Secondly, there are an enormous number of companies — essentially Chinese, a few American, and some European — that are indeed building robots, with the idea of making them domestic robots in the more or less long term. None of these companies for now — and really none — knows how to make these robots smart enough to be useful.
The task of making these robots intelligent is in fact a specialty in itself, on which certain companies are working, including a few big ones like Google, Amazon, Meta a little bit. But... but it's mainly in the hands of companies that make it their specialty, and that's a bit what we're positioning ourselves for. But we see it as a medium-term application, not a short-term one, because the...
Not to produce the robots — to produce the intelligent systems for the robots. The brains of the robots.
The brains of the robots — exactly. And who would produce the robots? Do you have any ideas? Are the people who know how to produce them today essentially the Chinese? So we don't have enough of them today. Have we become aware in Europe that this is a revolution we need to embrace?
No — but look at computing. And the software comes from pretty much everywhere. The operating systems come from either open source, or Microsoft, or Google. The hardware is built almost exclusively in Asia. But nobody raises an eyebrow. The chips are all produced in Taiwan. So there's probably a similar situation that's going to happen for robotics, in which a large part of the hardware production will be done in Asia. But that still leaves room, in particular, for European companies to produce robots. A few of them have recently been founded — Huma, for example, which is a Parisian company co-founded by my former students, Pierre Saneet, and a few others spread around —
Pretty much everywhere in Europe. It's not a closed domain. The market is still very open.
You said you are well funded. It's true that you probably have time ahead of you, but people say a lot that you have a new funding round in perspective. Are you going to raise other billions? Are you not the only ones, as I said earlier, chasing world models? Is there also a race there, like there is a race in LLMs, vertiginous with absolutely considerable amounts? Maybe you don't need as much money today, but are you going to try to raise additional funds, and from whom are you looking, particularly European investors?
Well, of course, the usual trajectory of a startup is to raise funds, develop something, show it to potential investors, and then raise more funds, and so on and so forth, until either going public or selling to the highest bidder. We don't have the intention of being bought. So obviously, there will be cycles of financing. We will need it to pay for the computing, which costs extremely much. We don't have needs as great as those doing LLMs, but it's still extremely expensive. So a big one is what? More data centers? Well, it's actually services we rent from people who have AI-oriented data centers with computers specialized in AI based on GPUs, and so on. In the long term, when we grow, of course, we can have our own data centers, but for now we are still too small for that. So certainly, funding rounds to come. I can't give you a timeline.
Exactly. And a timeline that, of course, must be in phase with what existing or potential competitors are doing. But I must say that there is no competitor for now that has adopted the same philosophy as us. That is, people working on world models — these companies working on world models are working on video production, video generation. We, with the Japa, have made the hypothesis that you don't need to generate video at the pixel level; you should rather make predictions at an abstract level, find abstract representations of the world.
That's really the core, the technological heart, the vision you carry, the scientific vision on which the company is based. Yeah. I'll stay on the money. Are you worried about... We talk a lot about the bursting of a bubble. Do you have the impression today — it's true that we have a bit of the impression that it's a model — I'm talking about AI, LLMs, which revolve around two or three major players who keep passing the hot potato of investment between Nvidia, OpenAI, and so on. Does that seem dangerous to you? American growth in particular is largely driven by that. Does it carry a risk of bursting like the tech bubble of 2000?
Well, we are not in LLMs since we do AI for the real world, for the physical world. So a bit orthogonal to LLMs. If the bubble burst, it would make computing costs much cheaper. So it would be rather good for us.
So it's rather a good thing.
It would be rather a good thing, insofar as there is no contagion to other forms or approaches of AI. Now, the reason we are witnessing this bubble is that there is a kind of forward race. There is first a kind of monoculture in California whereby everyone works a little bit on the same thing. And so Google, between Picopen and everyone, works a little on the same things because no one can afford to deviate too much from the general mood. No one can afford to fall behind, to take a step back, to find themselves behind the curve, so to speak.
So there is a forward race because you can progress faster if you have more computing capacity, because you can try many more things. It's a very empirical field. You try something, you see if it works, you just have a few turns of the screw to see if it works a little better. So it's constant. There are very few new conceptual ideas in it, and it costs very, very much because you have to train tens or hundreds of models. But it's even endless.
In fact, you would need all the energy in the world to get there, according to some critics.
Exactly. And anyway, we won't get there with LLMs, I've been saying that for a very long time. And then there are these systems that can adjust themselves, what we call RSI, recursive self-improvement. So systems that produce a response, check whether the response is correct with tools, and then adjust themselves, and so on. So that's a bit how the systems that produce code work. So it costs extremely much to train, and then you have to serve users, and that also costs extremely much. And for now, the big question that everyone in the industry is asking, certainly that I ask myself, is whether the operating cost can be justified by the money that clients will want to put into it. We've heard figures that are perhaps a bit exaggerated, but you can take a subscription at OpenAI for 200 dollars a month to use their intelligent system, and the most intense users cost the company 10 times that, sometimes 15 times more. And on average, for now, operating costs at OpenAI exceed revenues by a factor of 2. So it can't last. Either prices will rise, or the systems will become much more efficient, or users will say, 'But in terms of cost, it's not worth it.' Prices will rise, except that the Chinese are there and arriving with open-source models that are perhaps not equivalent, but not far from what we're told, and they are much cheaper, almost free.
There is still a debate saying whether artificial intelligence, LLMs, will become a commodity, a bit like the internet, almost free, and that the value will be rather on the software, application part, the agents, and so on.
Well, that's what I hope, and I'm also working on it, not within AMI Labs, but in the context of another project I'm carrying called the Tapestry project, 'tapisserie' in English, which is actually a grouping of industry, government, research, and so on, to produce what we call a foundation model, so a base model, LLM among others.
So you're also doing an LLM at the same time?
An LLM, but not a finalized LLM, an LLM that is pre-trained but that can then be fine-tuned for any use, and that anyone could fine-tune for any use, and which would be completely open.
And what we hope to do with this project is to rally governments, universities, companies, independent research labs, individual researchers to contribute to this model and make it a platform that would be, if you will, the AI equivalent of Linux in operating systems and in computing. And the reason I think it's very important that this kind of system exists is that first, open systems are much more secure, cheaper of course, but above all allow a great diversity of applications on top. It actually creates an industry. And when there is an industry that needs a common platform, there is a lot of market pressure for that platform to become open. That's why the entire software infrastructure of the internet is entirely open source. Even the software infrastructure of mobile networks is completely open source too.
