How can enterprises trust their data to AI labs and how can AI labs trust their model weights to enterprises? Let's talk about it with Renen Hallak, the CEO of Vast Data in a conversation brought to you by Vast Data. And Renen is here with us in studio today. Renen, great to see you.
Do you think OpenAI's caught up?
It looks like it. They really solved Navier-Stokes.
You're like a mathematician by training. So what happened there?
Okay. I don't know what happened there. But I can tell you that this is the most exciting advancement in AI for me since we started on this journey.
When I was young and foolish 20, 22 years ago, I tried to solve a different one of the millennium problems, the P versus NP problem. What the P versus NP problem is, it basically asks, is it easier to verify a problem to a solution than to actually solve that problem? And intuitively we all know that it is easier if you give me a solution to a problem. I'm sorry, not a problem to a solution, a solution to a problem. If you give me a solution to a problem, it's relatively easy for me to say, yeah, that works or that doesn't work. Whereas coming up with that solution in many cases is very difficult.
But it's unclear mathematically if it really is harder to solve a problem than to just verify its solution. And we still don't know the answer. That's the P versus NP problem. And in the year 2000, the Clay Institute came out with these seven millennium problems where they offered a million dollars for anyone who solves one of them. And I thought if I can prove that P equals NP, that it is easy to solve problems, as easy to automate the solution of problems as it is to verify them, then we can solve all the other problems by automating the solution. And of course most people think that P does not equal NP and in that case I get a consolation prize of a million dollars. So, I spent six months trying to figure this out. And of course, what I figured out is that I'm not smart enough to do it. But it bugged me that we don't know the answer to this problem and that we don't know how to automate problem solving. And so 10 years later, a little more than 10 years later, 2016, 2015, when this AI stuff began, I realized that for the first time, maybe there's a new technology that will enable us to automate problem solving. And when I was in school, historically, it was clear to everybody that the things that computers were very very good at, the easiest things thing for computers to do was arithmetic, to do 2 plus 2. And the hardest thing for computers to do was to solve mathematical proofs or to prove things mathematically.
And so I think they've done it. I think if they really did solve one of the millennium problems, that is proof to us that these AI systems can generate new IP, that they can understand things that no human was able to understand. And I think that's a very very important inflection point because until today, you could have said, 'Yeah, this AI it's just paring back what we already knew. It's summarizing things for us. It's making it nice. It's creating videos that we knew how to create ourselves.' For the first time, it's solved something that no human was able to solve and people were trying for the last 90 years.
So, it's an inflection point. I think it's a really really interesting inflection point. It is if we can again continue on this path and it can now solve the other five unsolved millennium problems and then after that it solves all of the other hard problems that we've been struggling with. Then it's done. It's what we've been trying for.
Mhm. Wait, sorry. What's done?
The game. The game is done. We don't ever need to worry about any of these hard problems. We don't need to worry about the environment. We don't need to worry about energy. We don't need to worry about disease. If AI, if you can set loose 10,000 agents and give them a hard problem to solve, and within a few days that problem is solved, we're done. Everything is good.
It's like both exciting and scary.
And it opens up a whole new set of problems because what you said about alignment, we want we don't want two companies to own the world in the old communist sense of them owning everything, all of the IP. We want it to be distributed. We want everybody to compete. And so you want the classic liberal principles and values to sustain. So you want life, liberty, and property.
Life, you don't want the robots to kill us. Liberty, you don't want them controlling us and telling us what to do.
And property, you want this IP to be owned by everybody. Every organization, every person should be able to own this intellectual property which it appears all intellectual property will come in the form of models and weights. So whether we build our own model or whether we fine-tune an existing model, we need a way to understand who owns it, who owns the data. And that's a big part of what we're working on at Vast.
Right. Right. Vast. For those unfamiliar, it's a company valued at $30 billion, raised a billion dollars earlier this year, and is trying to solve the problem of how enterprises can bring their data closer to models without losing that IP and models can bring their weights closer to enterprise without risking the leakage of the weights.
Correct. Very interesting moment right now I would say because we are seeing great enterprise AI adoption. I think OpenAI and Anthropic are on track probably this year to see something close to or exceeding hundred billion in ARR. Which is absurd given like last year it was something in the $10 billion neighborhood. And a lot of that adoption or a lot of that success has come on the back of enterprise adoption. Basically companies saying I find what you do valuable for my company. I'm going to put it into action and have it solve problems for me. But there's an issue here which is that companies have a lot of proprietary data. And there's been a big concern recently that if you trust your data to the AI companies, they might go upstream because they have to find some way to make so much money to repay all this financing that they're going to come into your domain. So companies have to trust the models with their data and the model companies are like the only thing that is valuable for me is the model weights maybe something a little bit more but we can't lose the model weights if we're going to be competitive cost billions of dollars to train these models so how do we bring it closer to companies and that's what you're trying to tackle so just talk a little bit I want to go a little deeper with you on both of these issues talk a little bit about for a company when you have an AI initiative and you want to bring your data into the picture. Where does it stand right now and what are some of the obstacles that companies have doing this?
I think right now it's the wild west. Nobody really understands what the rules are, what is allowed, what is difficult, what is not supposed to happen. We were just talking about this breakthrough that OpenAI just did with Math Proof and then immediately a question arises did they use information that mathematicians brought into OpenAI? And as far as I can tell right now they don't know the answer of whether they used it or not because the guidelines haven't been set properly. But as we progress and as organizations realize that everything that they use one of these big models for could potentially be leaked to them and be used to train the next generation then they will be very very careful in what they supply to these models. On the other hand nobody is going to stay in the old world of not using AI and so we have to find a solution for this and today it's a very clear force against force of enterprises with their data, model builders with their weights I think over time enterprises will start having weights that they want to protect as well.
Right, with fine-tuning the models for purposes.
If we have today an enterprise with a thousand people or 10,000 people. 5 years from now, 10 years from now, that enterprise will have a million agents and the only thing that those agents are worth are the models that they've fine-tuned throughout their experience. And so that becomes the IP of the enterprise.
And so we really really need to solve this. And the beginning of solving it is through confidential computing through the ability of us to encrypt memory in a way where if I run inference on prem in my environment doesn't have to be in my data center but within my virtual private cloud if you will, I can on the one hand be sure that I am the only one that can see my data but I can also give a guarantee key to whoever owns the model that I cannot see their weights because it's encrypted.
Right? And that's the solution that you're working on. We're going to get into it a little bit deeper in a moment. I find this stuff fascinating. I think it's something that Sat Nadella has referred to as breadcrumbs like you leave breadcrumbs in a way. It's very interesting the way that we talk about internet phenomenon, cookies and breadcrumbs and it's always some food.
Left behind. But basically when companies use a model there is a chance that maybe if not the data itself like some of the metadata or some of the back and forth is something that the model can see and then get smarter from. So that's the main concern. It is and what we're seeing is a few things. Jensen famously came out with his five layer cake earlier this year, which I really like because it crystallizes the fact that there's a new stack and an old stack and it shows us that all of the parts of the stack need to be redone.
And in that stack he has power and hardware infrastructure and software infrastructure and models and applications.
I think the models are becoming a resource in the same way that GPUs are a resource or SSDs are a resource or network infrastructure is a resource and historically the software infrastructure layer what we usually call the operating system is what manages those resources and so in the same way I think we're realizing now that the models are not above the operating system but rather the operating system needs to manage these models for this next era to really be successful.
Right? And then talk a little bit about Okay, so we talked about the sort of worry from the client side. We've also talked a little bit about the model weights and protecting the model weights. Just before we get into the solution here, I mean, how serious of an issue is that for AI model companies in terms of protecting their weights? I guess the big worry recently has been distillation, right? That if you, you know, make your model available through an API, some company that's competing with you, comes in and says, 'All right, I'm just going to run enough queries on this thing that I can distill the model and create my own based off of some of your answers.' So, how big of a problem is that in practicality right now? And is it you know some people have said if the model was so smart it could prevent distillation attacks like is it something that you know is it should be able to sense talk a little bit about that.
Well I always like to give analogies back to humans because that's something we know. And so if I have something in my brain and you ask me questions how easy is it for you to replicate my brain? If I'm being truthful and answering all of your questions, it could take you a while, but you could probably do it. But after you ask me a few questions, I should be able to understand something strange is happening. Why are you asking me all of these questions? And so, I should be able to stop answering at some point and stop you from distilling my brain into yours. And yeah, I think the model builders will at some point find ways to prohibit others from distilling their models. But if the weights are exposed then again there's nothing they can do.
It's like you getting a MRI machine and being able to see the actual connections between the neurons in my brain.
Right. Which is effectively what some of these distillers are doing today. No, the distillers I think are doing it through the API. So the distillers in my analogy are doing it by asking questions and talking to the person.
Okay. Getting hold of the weights is just grabbing the actual brain and understanding how it's built.
Okay. So two different approaches to effectively stealing the IP.
Okay. So, we're at this moment where for AI to continue its adoption, you know, you're going to need enterprises to be willing and it's not just like, you know, the I'm trying to think of a firm that's not highly regulated. Not just a not highly regulated firms, it's these highly regulated firms that are trying to get AI put AI to work, but they have data they just they cannot sort of expose to, you know, a third party.
And then you have the third parties that are like, we have this model, but we can't expose it to, you know, too many people or we could lose our IP. And it seems to me that if we're going to see the next level of adoption of AI, there has to be a way where both companies or both entities are comfortable playing ball with each other and a world where an enterprise can bring their data and an AI company can expose their weights and work together without worry that either side is going to steal from the other.
Correct. And I think that's going to be a much bigger problem in the immediate term as people realize how much access they actually have to our data. We saw this in the early days with the internet companies, companies like Google where at the beginning you just played and searched and enjoyed their product and then at some point people realized everything we do is being tracked and everything that we expose they have access to and then people started to worry about it and the whole field of cyber security and data protection became a thing as a consequence. And the same exact thing is happening now except on a much larger scale.
Right. And this is something that you're working on day-to-day at Vast Data.
Yeah, we think that again that operating system layer is the place to put the protection because especially if you don't want to put trust in the application, you need the infrastructure layer to protect you. And so what started as a way to access a lot of information very quickly which is the storage solution that we came up with and then a way to understand all of this information at very large scale at very high performance which is our database morphed into that full software infrastructure layer and because we are now that operating system layer I think it's our responsibility to not just enable this new revolution but also make it safe, make it secure in ways that again enable it to be more widespread.
Yeah. So talk a little bit about you know the solution itself because I think it's very interesting technically I mean to prevent to present something to enterprises and AI builders where they can feel comfortable doing business with each other so to speak without losing their IP. So how does it work?
So as we morphed past just a data system and started to present things like the data engine where we manage compute, it started at a very low level of running functions and triggers and not really understanding what the compute is that is being run. Over time it's become clear that all of the compute that will be run eventually is model serving. It's inference. And so we want the better understanding of the models themselves, and we want the ability to manage those models, especially as you have millions of agents fine-tuning models, you're going to have billions of models over time as they get fine-tuned. And so we need an understanding of what they are, how they were built, etc. And part of that is securing the model. Securing the model means that we are in control of who can see it and who cannot see it. Now this goes back to the old world. Many many years ago there was a thing called DLP, data leakage protection. And it was always a very hard problem to solve because if I give you access to something, then how can I prevent you from seeing that something? And the best way to do it is through encrypted memory. And so at the hardware layer we are guaranteed that certain applications cannot have access to certain memory even though it's in the same machine. And so we leverage underlying constructs from companies like Nvidia that enable us to make this very very secure. And then at our layer we manage the control plane. And so we know who is allowed to see what. We manage the keys. We understand where it can be exposed and where it cannot be exposed such that this full solution comes together.
So this is really interesting. So let's talk about from the model side first. So up until recently, a company like let's say OpenAI, I'll just use it as an example, would have their API and companies would in the cloud work with that API. And so only OpenAI really had exposure or had possession of its weights, correct? And you would have to kind of work with it through the cloud to be able to run any AI computing basically. So is what you're saying that like you know foundational model companies like OpenAI are now going to effectively take their weights which is that's the IP and give it to you to manage and effectively trust that when it's with you it can't leak elsewhere.
So they'll give it to their customers to manage and the underlying system that will ensure that their customers can take those weights will be our system.
And so we don't have any access to their weights in any way, shape or form.
And the customers do not either.
And the customers do not either.
So nobody has access to it.
