How Vast Data is Surpassing Traditional Storage Leaders in AI

The other unsexy decision that we made was to build a system for on-premises deployment. The reason that we didn't start in cloud was because a lot of the concepts and a lot of the technologies that we bet on, like containers and NVMe over fabrics and new NVMe very, very low-cost storage devices, they just weren't available in cloud. And so, you know, try as you'd like to get Amazon to ship a new hardware SKU just because you have a cool concept, doesn't really work like that over there. I would bet probably around 90% of the on-premises GPU deployments that are or NeoCloud GPU deployments powered by VAST. We've got customers that are now eclipsing a million agents within their environment going to, you know, our biggest is going to 100 million right now. And imagine the computational pressure when you have agents and agents on agents and agent swarms all just working on a common data set. Typically, when you see companies that are growing as fast as VAST is come into the market, they're often burning like mountains of venture capital on an annualized basis. The company now generates like hundreds of millions of dollars in free cash. The other thing that happened last year is we actually became like, you know, from an accounting perspective, profitable.

Thank you for having me.

It was super unsexy. There was an investor at Andreessen Horowitz, I won't name his name, but like a year or two prior, he ended up taking a bath on a bunch of investments that Andreessen had made and just he traveled around the world saying that this market is totally dead. Now, what he didn't appreciate was that until that time, most of the world was kind of building new derivatives of an architecture that came out actually in 2003 when Google introduced a new concept called the Google File System. That concept where you just take large data sets and spray them across a bunch of commodity servers was very popular, created, you know, probably what today is a hundred billion dollar industry of all sorts of different types of distributed systems, storage, database, hyper-converged systems, backup systems, everything in between. You know, today you even have companies where they kind of, their claim to fame is that the founders came from the Google File System team. So really, really impactful. And we saw, I would say, probably three opportunities, well maybe one challenge and two opportunities that we wanted to kind of take a stab at solving. First is, when we looked at the compromises that were introduced by what Google made, what we realized is that a lot of them could be solved through the implementation of a new distributed systems architecture. And, you know, best example that I could provide is from a scalability perspective, transaction performance never really scaled as you built these really large clusters. The most obvious representation of that is if you take like, you know, kind of a classic Hadoop cluster, you can't really write into it any real-time data. That's why you have to go build other systems for let's say Kafka-based processing. And so these things weren't really transactional. The bigger, and that also kind of the question is why, and the answer was well there's a lot of internal communication across the nodes in these clusters and as the clusters get bigger and bigger, they start to slow down and there's other, you know, kind of knock-on effects that happen there. And I think for the most part, outside of Google, most organizations really couldn't figure out a way to scale that architecture to exabyte scale. Number two is that the world was still very much convinced that the only way to build capacity was to use hard drive-based media. And what we realized is that the only value that hard drives had in an enterprise data center was low cost for capacity. And if you could solve that one problem and you could build a very, very low-cost all-flash system, two things happen. One is nobody needs hard drives anymore. So the world is better and processors benefit from the fact that all their data is available in real time. But two is once your archive is built from flash, it turns out you don't need a faster tier of storage. And so from a first principles perspective you look at it and you say okay well capacity equals performance when you get to flash and if I can afford to put all my data there then I don't need a classic storage hierarchy that I have today with either, you know, classic storage applications or database applications and so it's just better for processing. And then the challenge was, well if you have this like highly parallel all-flash system that could be, you know, presented in a lot of different interesting ways, what's the thing that's going to kind of unlock this new concept and take the market to adopt it? And we really, really spent a lot of time even before I joined the company looking at the knock-on effects of deep learning. And what would happen if that became something that was like commercially viable across the market and how deep learning would change the relationship between computing and data. And it wasn't that nothing was happening at the time, right? You know, that year Google just won Go with AlphaGo. That year OpenAI was founded. That year Jensen introduced the DGX server which was basically, you know, regarded as one of the first AI computers. When you put those three things together, kind of look at it and say from a market perspective the world's starting to get its act together with how deep learning could scale and we realized that there was just a huge opportunity over the horizon to solve the data problem and get ready for that level of scale with a new systems architecture. And so, you know, when we started, there's probably like 20 other new flash companies that came out into the market specializing in NVMe. All of them were shipping, you know, kind of new spins on old architectures. And what we realize is that whereas storage and database companies come in and out of the market all the time, it's almost never that a new systems architecture gets invented. And we feel like we're picking up the work that Google laid down back in 2003 and carrying it forward. And that has just given us a ton of advantage as we've come into the market and helped people really start to scale up their AI initiatives.

Yeah, that's a good one. So, you know, we were looking at what was happening at places like Meta and Google and Tesla and Uber and Byju's. The reality is we weren't selling any of them right at the time. And we needed to go and get the first product out into the market. The first product was, the first version of it was introduced as a storage product. We called it Universal Storage. And so here, what do you do with a very low-cost parallel all-flash system? We realized the high-performance computing market looked pretty prime for modernization and so we started reaching out to two types of organizations. The first one, and they largely do the same type of processing, first one was the life sciences market. So people that are doing like high-scale genomics is a good example. And, you know, first customers were organizations like the National Institutes of Health. And then the other market was arguably like quantitative and high-frequency trading based hedge funds. And these are organizations that would spend any amount of money for competitive advantage. And while we were kind of waiting for this thesis to manifest, which is okay we can get flash to the cost of disk, those customers were paying a little bit of a premium for flash for reasons that, you know, it just made their compute a lot more efficient and so people were justifying the expenditure of VAST on the back of like, okay well if this makes my pipeline two to three times faster, that's essentially like me getting back half or, you know, 65% of my servers back that I can use for other applications.

Back then, well today we kind of view the whole world as our competitor. I will get to that in a moment but okay so back then the other unsexy decision that we made was to build a system for on-premises deployment. And actually I took a lot of time to, you know, decide to come to VAST just because I thought the whole world was going cloud. And the reason that we didn't start in cloud was because a lot of the concepts and a lot of the technologies that we bet on like containers and NVMe over fabrics and new NVMe very, very low-cost storage devices they just weren't available at cloud. And so, you know, try as you'd like to get Amazon to ship a new hardware SKU just because you have a cool concept doesn't really work like that over there. So we built these, we kind of like with a few partners we co-engineered these reference architectures that we brought to market and we started to go to battle with two styles of companies and this is kind of one of the interesting attributes of, on one hand it was like, you know, highly parallel and designed for crazy levels of performance and so that had us competing with a lot of the parallel file system companies that were used commonly in high-performance computing. And then on the flip side, we were very quickly building in all sorts of enterprise data features that would also allow us to compete with more legacy enterprise products that nobody ever thought of from a high-performance perspective. And so, you know, the same product that today we build the world's like top five supercomputers with from a storage perspective is also being used for a backup appliance in, let's say, like a Fortune 100 insurance company. And so, you know, the idea is if you look a little bit farther out and when deep learning becomes like commercially viable, turns out that all the world's data will need to be processed by GPUs. And so that's why we spend a lot of our time building in these enterprise features just so people could be ready when that moment hit.

All of the legacy storage players for sure as well as some of the kind of the older incumbent high-performance computing storage players.

So in 2022 ChatGPT hit. Right on the back of that what started happening in 2023 is that all of these new AI clouds were emerging. Some of them came from Bitcoin miners that were evolving their business and they were popping up, you know, as you got to the back half of the year very frequently with new requirements for large-scale GPU deployments that needed things like multi-tenancy and enterprise data management features because they didn't just want to go after business from AI labs. They also wanted some of the bigger enterprises to also use their platform. So if you think about CoreWeave, they sell to companies like Jane Street which is a very serious trading business. They sell to let's say ServiceNow which is a very serious enterprise software business and once you get to that level you can't just use old high-performance computing research technologies to service those customers. So we sat there and said no, we built all those features and we have the scale and the performance and oh by the way it's super cost-efficient from a capacity perspective and that got us into the NeoCloud business where we just started selling to a lot of the biggest players in the space. CoreWeave made an early bet on VAST and it's been a great partnership. We sell to companies today like Lambda and Crusoe and Nscale and, you know, around the world we probably sell to about 60 of these AI clouds. I would bet probably around 90% of the on-premises GPU deployments that are or NeoCloud GPU deployments if you lump them together are powered by VAST. And so most of that is either NeoCloud-based in terms of where VAST is selling, on-premises based in terms of where enterprises are still building their own infrastructure, or we're now starting to work with the biggest cloud players to deploy our software in hyperscale platforms. So we made announcements with Amazon, we made announcements with Google, we made announcements with Microsoft and we're slowly rolling customers into those environments as the platform gets more and more mature for the hyperscalers. And the cool thing is customers don't need to choose, like we can even federate everything together into one unified data management system. And a great example of that is, you know, we have customers like Jane Street who spoke at GTC two years ago, is that they have VAST systems in their data center, they have VAST systems at CoreWeave and they have kind of a unified environment that is stitched together by VAST that helps them manage their data across those two different types of destinations.

I agree. Now you can, there's a whole spectrum of different types of partnerships that you can make with them. We've announced a very strategic relationship with Microsoft that we're working on now ramping up. There'll be another one that gets announced over the next couple of weeks that we think is equally strategic. So I think the cool thing is when you're VAST and you talked a little bit about valuation, I think of that as a reflection of kind of market traction and product-market fit. We've done a great job showing people that we're the right platform to scale up your AI training and AI inference investments and that memo has been received by the bigger cloud players who are now starting to take us very seriously as they build out the next generation of their infrastructures.

Sure. So first of all as I mentioned we got into the market with a very strong focus on multi-tenancy and enterprise features. And some of those other companies that you mentioned, you know, for example, DDN, I used to work there. Their pedigree was really enabling very high performance with a lot of feature compromise and complexity that typically and historically has come along with those platforms. So, you know, great for research computing, not necessarily great if that's the back end of your business. And so what you get from VAST is a system that has all of the enterprise features that you'd expect to get from a traditional enterprise IT platform, with the added benefit of the scale and the performance that you need for these AI workloads. Now, that's kind of like table stakes differentiation. I think the more interesting part of our story is that in 2023 we made a move that nobody in our industry has ever attempted to make which is now paying a lot of dividends for our customers which is we endeavored into the tabular data space. And so now the competitive landscape really gets expanded because you can use VAST for everything ranging from event brokerage to large-scale SQL-based analytics to vector database search all from that same platform that you started using originally as a file, object, and block storage system. And so when we add that to the table then the question becomes well what's the relevance to VAST AI customers? It turns out that, you know, for reinforcement learning I need to capture a bunch of feedback from my models using let's say Kafka streams. As I contextualize data I need very, very scalable and low-cost vector search services that help me, you know, run things like RAG routines on my data. From an analytics perspective you then need to go and take all that information and analyze it either for vector search or for, let's say, model evaluation. All of that's already in the platform. And so if you think about the larger players in the AI data platform space or let's just call it the data platform space, you know, today Snowflake is used for data preparation, oh sorry Databricks is used for data preparation for some of the larger AI model builders. Snowflake is used for model evaluation with a lot of the larger AI model builders. And now VAST has a platform that couples both the training and the inference part of those two problems in the same environments that you can train and infer in the NeoClouds that you run in. And so from this perspective, we're in a different zip code than where those parallel file systems are because now you've got a product that is competitive with the best streaming platforms, the best vector databases, and the best analytics systems in the market. And then earlier last year we took the covers off of a compute engine that also runs in the system. And so basically think of this as a serverless compute platform coupled with workflow orchestration tools and eventing infrastructure that basically triggers pipelines when something happens in the system. So, that looks like a little bit of a combination of like Kafka and Apache Airflow and AWS's Lambda Services in the same system that also has all of your database infrastructure, all your streaming infrastructure, your unstructured data storage, that takes you all the way down to bare metal. So, when we have thought, okay, well, how do you characterize this? How do you explain to customers what it is? It's like basically a full computing stack that you can deploy wherever you want to for full-stack AI inference. And here, like I would argue that there is no natural competitor to what we do if you look at the totality of what we're bringing to market and the only other place that you can find the equivalence of this capability is in cloud, right? But in a large cloud you'd have like 10 or 20 different services that you'd have to put together to get to the same thing that VAST can give you with one codebase. And if you're building a new cloud then you have to reinvent the whole thing for yourself. And time to market is absolutely critical. So these guys, they tend to really like our value proposition because, you know, I'll tell you we had a physical AI end customer just yesterday raise his hand and say I need a vector database that supports like half a trillion vectors and it's just sitting there already in the same space that he's doing his training. And it makes it really easy.

No, we try to avoid integration as much as possible and we build wherever we find that there's big opportunities to save customers money and to scale up applications. And so the whole vector database market is built on that kind of legacy application. When we started, we said A, we want to build something that doesn't require memory. And I think a lot of our focus in the early days on using flash as kind of like network-attached persistent memory took us to a place where we could build much more cost-efficient vector database server infrastructure than ever was thought possible. And B, the way that VAST systems work is you have basically cores that handle all of the processing. Think about that as like the logic of the system. And then over a network, what you have is a shared set of SSDs that all the cores can see in parallel. So the cool thing about this is if you go back yesterday, the way that people typically would build systems is you have a bunch of commodity servers that own a portion of the system. They all have to talk to each other in order to coordinate a transaction. In our case, all the cores see the same global volume and none have to talk to each other because they can see all the transactional data structures in parallel. So for vector databases, not only can I build something that I can search from in constant time regardless of the size of the vector space, so you can get through like billions to trillions of vectors in less than a second. And that is very, very unique without having like memory-based indices. But number two is because the system doesn't have any east-west traffic, I can write into all the vector database servers in parallel which makes it really nice A, from the perspective of, well if I'm going to put this like RAG pipeline at the front of my business so I use new unstructured data to drive business decisions, VAST can keep up with that. But B is well what am I vectorizing? Probably my data that lives in my object store, my file system. And if I can combine these two into one unified data management platform, it means I don't have any inconsistencies between my data and my vector presentation because all updates are atomic. It means that when permissions change on my, let's say, on my objects, those permissions immediately get populated into the vector database where you never have like a synchronization problem which is common in particular with a lot of like SaaS-based document search platforms that are in the market that are trying to just like scan the data that you have in maybe some other environment and just keep pace with the changes that are happening. That doesn't work for a CISO and for VAST it's all completely atomic.