Welcome back to the stage, Brian. People are coming back in from the coffee break. If you're here already, you are in for a very special discussion. So today we have Ion Stoica, who is a big deal. He's not someone who pushes his name out on the internet constantly to get attention. He's rather someone who's driving profoundly important research into AI and machine learning at UC Berkeley, all the way to the point of being truly the sort of intellectual lead and guide around Apache Spark, which then led to Databricks, which is a business that Ion was a co-founder of, and he's the executive chairperson of Ray, which then led to Anyscale, where he's also a co-founder and executive chairperson. I was trying to understand Ray better and started having a conversation with Aaron Schrader, who's going to be interviewing Ion. Aaron is my best friend in the entire world and also the person who I go to to try to learn about technologies that I don't understand. From there, he started explaining to me not just Ray but also Ion, and he's got an unbelievably inquisitive mind. So we're in for a true treat to hear his interview of Ion and all the ways that they're contributing to and thinking about what's happening with AI today. So welcome to the stage, guys.
Hello everybody, it's great to be here. Ion, welcome to New York City.
Thanks for having me. It's great to be here.
Ion and I have had the pleasure of working super closely together over the last eight months at Anyscale, and it's been a total pleasure. Ion is a luminary in the data space, and I think everybody's in for a really awesome discussion today. So thanks for coming. All right, let's start with company building, what you know a lot about. You helped found Conviva, Databricks, Anyscale. You've played operational roles at each of these. At Databricks, you were the CEO, and I'd say Databricks is arguably the most impressive software company to be built in the last decade. I think it's 11 years now, but we'll give you the decade marker. Just talk to us about what you think the fundamental lessons you've learned about how to build truly important companies.
So, one thing is that many things have to happen to be successful. You need to... it's like the famous quote, actually I think Peter Thiel used this, he's using this quote in a different context from Anna Karenina: all the happy families are happy the same way, the unhappy families are unhappy in their own way. The same with companies. There are many reasons a company can fail, but in order to be successful, it needs to check a lot of boxes: right time, right team, everything right, a little bit of luck, take advantage of that luck. But at the same time, you try to look for some patterns, some conditions which you believe are necessary, not sufficient. They don't guarantee your success, but help you. And I think that, at least in my mind, the pattern of many successful, I'm talking here about iconic companies, is basically almost all of them start with identifying a niche, a market niche. Then they develop a product to dominate that niche. And if they made the bet correctly on the niche, the niche is going to grow. So what this helps you is because you have a product dominating the niche, the niche is growing, you grow with the niche. It's very easy to grow. Then of course you can go into adjacent markets and so forth. But if you think about almost every company you know about, I think it fits this pattern. Microsoft: the personal computer was a niche, actually, or hobbyists, they're not even making money. Other companies like doing the software for personal computers. And then of course, even recently, the most valuable company, Apple, is the same. They built this computer again when it was a do-it-yourself hobby. Your company like the first personal computer, Altair, was a kit they were selling, and people put the pieces together to make it work. And almost everyone, obviously, Facebook: social networks, a new market. Google: a new market, search. Even a company like Oracle, which you may think is an enterprise company, databases, well it was a niche in the sense that they were the first to develop software database software to run on mini computers. Before them, you buy the mainframe and it comes with a database. So you want a database, you go to IBM, you buy the machine, while they were doing the software. So I think that's kind of the pattern I believe you have in many cases. And the one important thing I said again: you need to identify the niche, but that's about betting. You have to bet right because it's a niche, it means it's not mainstream. And you fundamentally have to bet because if you don't bet, you do whatever everyone is doing. But obviously you need to be right because if you bet and you are not right, you are a fool. Everyone will tell you, 'I told you so, it was a smart thing to do, you didn't listen.' And Databricks' bet in some ways was cloud. Databricks' bet was big data in the cloud. Everyone was accumulating more and more data. It was evidence from already back then, from Google and Facebook, that data is extremely important for their businesses. And everyone kind of started to accumulate more data in the hope that they are going to leverage their data to improve their business, to get more competitive. All the enterprises, and then there was Hadoop. So Hadoop was there already, creating the big data market. So in some sense, the pattern there, but still it was a niche. I remember like 700 people, that was the huge event. Now Databricks is like 15,000. So basically it was a niche created by Hadoop. And the other angle we had was it was in the cloud, because up to that time everyone was on-prem, and going to the cloud also helped us to have a much more lean and better, more scalable business model. But that's the things: data science at scale in the cloud.
But you're saying in a way, like you're picking a niche and it in some sense should feel fundamentally uncomfortable because it's a bet. Did it feel that way at Databricks? Like absolutely everybody's telling you...
Absolutely, absolutely. They were telling us, because it seems kind of obvious now, but no, it was back then. We talked with a lot of VCs and so forth, and everyone said, 'Okay, Spark is good because Spark was better than Hadoop, like Google was better than previous search engines like Lycos and many others.' And the problem was that people were telling you, 'Yeah, well, this is great, Spark is great, but you have to go on-prem because that's where the business is. Yes, it's smart, and yes, maybe in a few years cloud, but now you have to go on-prem.' And the problem was that they had a proof point, and the proof point was Cloudera. Cloudera, as the name implies, is 'cloud era,' so their first thing was to go to the cloud, but then it didn't work and they pivoted and went on-prem. And on-prem at that time they had huge success. They raised almost one billion from Intel, it was the largest at that time of any startup probably ever. So that's the thing. But the thing about going to the cloud, there are many reasons, some people recognize certain advantages to go to the cloud. The one thing you need to focus on is: is the market large enough for you? Because you build a company, and if you give me a 10 billion market, how long will it take you to make a dent in that market? It's not possible to build a strong company. So that's why, and by the way, if you ask everyone, 'Hey, in the future, what do you think is going to grow faster, cloud or on-prem?' everyone will tell you cloud. But still you need to go on-prem because it's a bigger business. But you don't need that. It's a niche. And the nice thing about the niche is you have fewer competitors because the incumbents will not come to the niche, because even if they get the entire niche, it's not going to move the needle for their revenue. Oracle coming to big data, it's a distraction. It's exactly the innovator's dilemma at some level. So if you are convinced that the niche is enough to sustain you as a company, to have a very good revenue to start with, and if the niche is growing very quickly and you are dominating the niche, that's the only thing you need.
So much of that sounds like the founder leader having the nature to take the right bet, to take the risk, to say no to all these people. You were CEO, and then you hired Ali. He was a founder too, and I think he's clearly done an unbelievable job. I'm sure that's been gratifying to see and to be a part of. What are the qualities in him that you've seen that have made him so successful in building and running the company?
So, by the way, I've been working with Ali closely since 2009. We started Databricks in 2013, he became CEO in 2016. So we knew each other for a long time. He came from an academic background, he came to work in our lab at Berkeley, it was peer-to-peer, that's another era. So we knew each other for a long time. And to answer your question, it was a pretty comfortable transition, and it was natural because I wanted to go back to Berkeley, that was the right time to engage. But what kind of person is Ali? One thing is, if the company is growing, you have to grow with it. Everyone tells you a 10-person company is different from 50, different from 150, different from 500, different from 1,000, and so forth. So Ali is extremely good, it's unbelievable talent. He's always learning, always up-leveling himself, getting one step ahead of where the company is going, reading books, meeting people from larger companies, absorbing the information. The second quality he has is a great mix between strategic thinking and low-level attention to detail. You can see this in the acquisitions of Mosaic and Temporal. The performance of the startups doesn't justify the prices Databricks paid, but if you think from a strategic perspective, it makes a ton of sense. And the last thing is he works very hard. I've never seen anyone being very successful, not only entrepreneurs but academic researchers or whatever, without learning and without working very hard. And he's very charismatic in the sense that he is a good connoisseur of human nature, and he knows what drives people. He's done a really amazing job shifting the story of Databricks, repositioning it as an AI company.
Anyscale is obviously the company behind Ray, which powers AI workloads at OpenAI, Spotify, Uber, Netflix. So let's turn to AI. From minute one, it's a little bit of a misperception. Actually, Databricks started as an AI company because we had on top of Spark, when we started, we had this library called MLlib, machine learning library. It was not deep learning, it was the previous version, standard machine learning like random forest, logistic regression, things like that. So it was a library that was scalable, running on multiple machines, because before then, the libraries were running on a single machine. So this is what we were selling to our main user, the data scientist. But what happened? We saw they were buying it, they were playing with it, but when you go back to them after a few months, what were they doing? Because the consumption looks good, that's a nice thing about cloud, you see their consumption, you see how many people use it. ETL. They were saying, 'Hey, you know, what great application?' Because we at that time had some marketing saying we want you to use cases, let's go and see the use cases. Almost none of them were doing AI or ML. But what were you doing? 'Well, we realize that we don't have all the data we need. We need to curate the data, process the data, and Spark was very good at that.' So for us, it's coming full circle. And then obviously you listen to what your customer is saying, and you focus on whether they are successful. I'm still not sure the data is ready.
So on the AI front, what's your take right now on the big models? How's it going to play out? Are you a winner-take-all, an oligopoly of the clouds, open source? Where does the LLM game at the model level end up?
Before answering the question, just to give you a little bit of context: at Berkeley right now, we run this Chatbot Arena. How many people know about Chatbot Arena? So basically, you can go there, it's one of the best ways to evaluate large language models in the wild. We rank the models for different tasks, and we have 130-something models now. It's very popular. Many companies, including Google, OpenAI, and so forth, test on Chatbot Arena before they release their models. The reason I'm telling you this is because we have pretty good visibility about how these models evolve, individually and in comparison with each other. Okay, and what I think happens, even after the new release of o1, OpenAI's o1 today, what we are seeing is a little bit of commoditization at the model level. Commoditization here not from the point of view of pricing, but from the point of view that a lot of these models become more interchangeable. You can use Anthropic, you can use OpenAI for your application. Just to give you a sense: historically, first of all, ChatGPT almost two years ago, November 2022, was so much better than everything else. Then two, three months after that, it was Anthropic's Claude, which is pretty good. Today, before this release, you have five players: OpenAI, Anthropic, Google, which is very competitive, xAI, and also Meta with Llama. So you have something like that. Because eventually, the training procedures, recipes are kind of the same. People know, and it's hard to keep secrets. People move between these companies. California has no non-compete, so people move very fluidly. And they train on the same kind of data, so it's natural to converge. So that's one thing. I think Meta is going to continue to invest because they see that as strategic. They do not want to depend on one of these big model providers, like they depend on Android or Apple's iOS. So just because of that alone, I think open source still has a bright future. But fundamentally, the models do become, even after this huge investment, it's not one to dominate all. And we'll see now, OpenAI is focusing on reasoning and problem solving, which is much better than anything else. But I would say that in a few months, you'll see similar results from other players.
