Extremely excited for this conversation that we have coming up. Unfortunately, Dr. Shashua could not be here in person. A man of his responsibilities, it's very difficult to make it happen unfortunately, but still very excited to have him here joining us remotely. Are you there, Dr. Shashua?
Yes, I'm here. Nice to see you as well.
Thank you so much for taking the time to be here. You know, I think as a sponsor, you really were important in making this whole event happen. And of course, we're in a room full of industry leaders here. And yet you stand out, I think, in this crowd as having a really unique history and perspective on not just driving automation technology, which of course Mobileye is one of the biggest names in the business, but really AI kind of more broadly. And I wanted to start by asking you to share a little bit of your historical perspective on all of these technologies because you were there as an academic many years ago. You started multiple companies and multiple versions of this sort of technology and I wanted you to sort of just start by reflecting on your own journey and sort of where you see us now in this exciting journey towards the future.
Okay. Well, I think modern AI took the world by a storm. So I see two reasons for it. One is called tokenize everything. So many different machine learning domains, they all use the same single AI architecture called the transformer. So as a student, there were very different disciplines of machine learning, could be computer vision, could be language processing, could be acoustic processing, voice to text. Each one had its own community. Each one had its own tools. Today they're all together. Today there's one architecture. So this is the first thing that is exciting. The second is the scale. By scaling data, scaling model size, scaling compute, the performance gets better and better. So those two reasons took the world by storm. But I think there are two open areas that call for some breakthroughs. I think those two areas, if we don't find good solutions for them, I don't think that AI will be truly transformative. So the first is precision. Precision is about providing guarantees that your answer is correct. So anyone who uses AI notices that sometimes the AI will provide a wrong answer. Sometimes it will invent false facts. Sometimes it will hallucinate. So for example, in the area of autonomous driving, guarantees means that you don't cause an accident, that you can tell what is the probability of making a perception mistake, what is the probability of making a lapse of judgment, and you want to reach a very high bar of precision. So this is one thing. Take for example agents, this new trend of trying to automate tasks using AI, that also requires precision because if you have an agent that is 90% of the time brilliant and 10% of the time a complete idiot, it's very difficult to work with such an agent. So precision is something that you see all across AI today is still lacking. And when you talk about autonomous driving, you really need to think about what to do there. For example, create redundancies, how to infuse correctness into the models in order to reach the very high bar. The second open area is lack of data. So when you think about language models, it has exhausted all the possible data there are today. When you pre-train these models, tens of trillions of tokens are used to train a today model and that's more or less exhausting all the data there is. So if you want to continue to increase performance and you're talking about a data-driven paradigm, data-driven engine, where is the data coming from? So take for example in autonomous driving and also in robotics, one technique of generating data is to use simulators to create an environment in which you have multiple agents. Each agent sees the world in a different manner. Each agent has a different weighted reward function or kinematic state and they all interact and in that way you generate infinite amount of data. So this is again another open area that they call it the need for a new scaling law, right, because data has been exhausted.
Yeah, I want to dig into the precision point because I think it's really interesting that we've had all this progress in driving automation technology where precision is not something you can just laugh off a hallucination the way you can with large language models sometimes. And then you've seen large language models come out and kind of be less obviously precise, right, and have these sorts of issues. Is driving automation as a use case for AI, is there a forcing function there because it's so safety critical that really pushes forward on this precision problem that you lay out?
Look, I'll say it's unprecedented to see a data-driven model to reach very high levels of precision. Take for example, the latest eyes-on autonomous driving, the Tesla FSD, there's public data, the FSD tracker. The meantime between critical interventions is about 10 hours. The bar is many tens of thousands of hours. So we are really far away from truly becoming eyes-off when you look at precision. So the question is where would the precision come from, by cranking more data, I said it's unprecedented, or by looking at aviation as an example where you create redundancies. By creating redundancies and those redundant systems don't share the errors or the error profile is different and find a way to fuse them together. We published a paper a few months ago, we call it PGF, primary guardian fallback, like a system how to fuse the results of different subsystems. And we do another thing which we call artificial community intelligence, as I said before, in a way to create a game in which you can generate a lot of synthetic data in order to train driving policy. So instead of relying on millions of hours of real data, we'll be relying on billions of hours of simulated data and in that way crank up the precision. So precision in autonomous driving is a big thing. It's a big issue.
Yeah. And so on the other piece of this, on the data one, it's very interesting to hear you talk about this simulation piece of it because again, Mobileye, one of the key advantages is the scale that you have, right, the number of vehicles you have with your technology out there, the data you're able to get there. You've talked in the past about some of the misperceptions about how that data is useful. Bring that into the context now of this new era of, as you say, generative AI and how does the value of that scale play into this new world that we're entering.
So we generate data through driving assist. So driving assist generates data and we see the data coming in three categories. One is video data that we use to train the perception system. So video data, we work with about 50 car makers. A production program generates about 10,000 hours of video data. So we have hundreds of petabytes of video data to train the perception system. So this is one source of data. Another source of data is to build enriched maps. Now there are different names for it. Sometimes it's called high definition maps. Sometimes it's called enriched SD maps. But you really want to collect details about the roadway, about the priorities, about where the lane marks are, about the connection with the traffic lights. It provides redundancy. It provides foresight. There's a lot of use of map data. So here we have a system where cars with a front-facing camera generates low bandwidth data sent to the cloud and we have millions of cars sending data and we build these high definition maps all over the world. So we have all US covered, Europe, Asia. So this is the second category of data. The third category of data is a bit controversial. The third category of data is data used to imitate humans in the way they merge into traffic. So if I have probes in cars with cameras and I see how the human has driven a particular section of the road, I would like to use this data to train my driving policy. We believe that the amount of real world data is not sufficient. And this is where we built this community of agents using simulated data creating a game where multiple agents, some agents see only the world through cameras. Some agents see the world with the maps. Each agent has a different weighting, differently the components of a reward function and generate billions of miles of data rather than millions of miles of data. So those are the three categories of data that we use.
