GAEA Talks Live from HumanX - The Open Source Engine Powering AI with Anyscale CEO Robert Nishihara

Thanks. Yeah, great to be here.

Yeah, so I'm one of the co-founders of Anyscale. We got our start building actually as grad students back at UC Berkeley doing AI research. At the time we were doing research on algorithms, reinforcement learning algorithms, deep learning training, and optimization algorithms. One of the challenges we ran into over and over was that in order to really try out the next algorithm, you had to scale it. You had to run it. It wasn't enough to just run it on your laptop. To really test if it worked well, you had to try it out on a cluster of machines, a cluster of GPUs. And doing that, we were spending all of our time on the systems, on the distributed systems, and software engineering aspects of scaling our experiments. How do you actually run this at scale? How do you make it reliable? How do you make it fast, cost efficient, all of these things? And so, even though we wanted to spend our time on algorithms, we were spending all of our time on systems. And we thought, from our vantage points in doing AI research, it was very clear to us that the need for scale was only going to grow. Models were getting bigger, the more compute and the more data that you put into training these models, you could get better results. And so, we thought the challenges were only going to get harder. And the tools at the time felt inadequate. So, we were excited to build open source tools. We wanted to build useful open source tools that people like ourselves could use to do distributed computing, to run AI workloads across large clusters of machines. And we started Ray, which is an open source project that we continue to develop today. And that was how we got our start. At the time it was just adopted by early adopters. Very few people were really serious about scaling AI. Now, it's commonplace. You have many companies with tons of data training custom models using GPUs. And so, after graduating from the PhD program, we started Anyscale to commercialize Ray and continue developing it and really try to provide better software infrastructure for doing AI. So, that's how we got started.

Yeah, so the bulk of my time before Anyscale was as a grad student at Berkeley doing research. Some of the internships I had before then were either software engineering internships or research internships, really trying to analyze optimization algorithms and prove things about them. So, I came from more of a theoretical background early on, before getting into systems trying to analyze algorithms. And over time that really shifted toward building tools to really enable, because this was very clearly the bottleneck for us, for our own research. It was doing the software engineering to run our experiments in a performant way.

There's far more complexity that goes into building and deploying a product compared with just prototyping something, building a model, and building a demo. One of the most obvious places where you see this is in robotics. There's often a 30-year lag from demo to production, which is extreme. But if you think about autonomous driving, the first demos of self-driving cars that looked impressive were at least 30 years ago. And it's really today that that's making its way to production. That's an extreme example, but this comes up all across the board because reliability... All of the work is really in making something good. Doing the demo is often... And this was true from our experience with building Ray, building the software. You can build a prototype of anything, like a web browser, an operating system, some complex piece of software in not that many lines of code in maybe a couple weeks. But then you're not done. If you actually want it to be used by people, if you actually want companies to rely on it, all of the work is in hardening it, making it really good, making it reliable, handling all sorts of edge cases. And that's something that we continue to see with what we're building today. Now, as coding tools have gotten so much better, you can vibe code many things. You can use AI tools to do many parts of the job much faster. Build your UI, experiment with different ways of doing things, refactor things. But there's a long tail of reliability issues that you'll run into in production that you will only be able to solve if you experience them in production and have earned the scars of really dealing with that. And that is something you're very unlikely to catch or to be aware of just through vibe coding something.

Yeah. So, this is what a lot of customers don't see. Customers, whether it's business enterprise or just general consumers, they don't see these challenges. So, it's almost an invisible thing. But if something doesn't work for them, they get very upset.

You know, one counterpoint is that in research or in model building, there's actually the line between development and production really gets blurred or can get blurred. And the reason for that is that with a typical software business, the artifact you're producing is software. Now, if you're a research team or a model building company, often times the artifact you're producing is not the code, but the model. And so, there's a whole process you need to follow to get that model that involves writing code to process your data and prepare your training data. And once you have your training data, then you have code to train the model and produce the model, but you might only run that code once. You might throw it away as soon as you're done. The code is not the artifact. It might be single use. So, we see this all the time where maybe it's okay if some of the code is unreliable. Like it crashes while you're processing your data. As long as you can save your work and restart it, and maybe you have to have somebody wake up in the middle of the night to restart it, and just keep going, and eventually you get the data processed, then the fact that the code is unreliable doesn't really matter because you processed your data and you're done with that stage now. Now, that's not completely true because in reality, you're running tons of experiments. You need to process your data in many different ways, see what works, see what doesn't work. So, having reliable code really does matter. But the important mindset shift is that the code is not the artifact.

The subject matter experts, it's a lot of the way we make models smarter today is by distilling human expert knowledge into the models. And that is why these companies spend so much money on data. And the type of data they need has continues to shift. It's a constantly moving target, where the early LLMs were trained primarily on tons of data scraped from the internet. And now, if you look at companies that sell labeled data, a lot of the data they sold early on was low quality, low skill data labeling. And now they're hiring the top experts in every field to actually generate the data, supervise the model. But, of course, that's a moving target, and it will reach its limits. There are limits to how much smarter you can make the model solely by paying more and more expert humans to generate higher and higher quality data. Because once you're getting data from the best subject matter experts, there's not a lot of room to go from there.

I think another direction is to basically use AI to generate higher quality data. And this is actually something we've seen a huge shift over the last, say, decade. If you don't know about the ImageNet computer vision challenge, this is one of the early benchmarks that drove a lot of progress in deep learning, where there was a data set of, it was an object recognition benchmark, where you've millions of images, and the question is who can build the best model to recognize what, this is a car, this is a dog, there are like a thousand different breeds of dogs. And the mindset at the time was, okay, you've got this data set, it's split into two parts, your training set and your test set or validation set. And all the researchers would compete to come up with the best model architecture and optimization algorithm, and train it on the training set, and then evaluate it on the test set. And whoever got the highest accuracy was doing the best. And so, there was a lot of innovation on model architectures, you got ResNets and things like that come out of it. But the mindset has completely shifted. At the time, the data set was considered a static thing. It was your training set. And all of the innovation was going into the architecture and the optimization. Now, it's almost the reverse. Architectures are more converged. Obviously, there's still innovation there. Optimization algorithms are more converged. And where all of the innovation and effort and money is going is, how do I get the highest quality data? I treat that as a variable to build my model. And there's endless room there. If you look at the type of data processing and preprocessing that people did with ImageNet, it was very limited. You would take your data set, each image, and you might try to take each data point and generate more data points. So, you'd randomly scale and crop the image to generate more data. But now there's this idea that we can really curate our data. We can take tons and tons of potentially low quality data, and filter it and enrich it down to a very high quality subset. And that process uses some standard processing and filtering, but tons of AI to actually think about identifying or correcting mistakes if you're working with text documents, judging how interesting a video is from the imagery or from the audio. Really narrowing it down. Maybe you need your model to get better at identifying pedestrians, and so you search and filter down for more data that has pedestrians in it if you're working with video data. And this is becoming, you run tons of classifiers and other models to annotate your data and sort of enrich it or maybe generate text descriptions or captions. So, this is something no one was doing in a meaningful way before, but now it is a fundamental part of the training data preparation and preprocessing is where a lot of the experimentation and innovation and really the quality comes from in your model. It's not just how do we improve the training efficiency. The part that's growing really quickly in terms of complexity and research and innovation is this data preparation component.

And there are many different dimensions of scale, right?

Yeah. And that's why it's so important, that understanding of what your data is is why it's so important to actually look at the data. And like inspect individual data points, because if you're not doing that, you're probably missing basic insights or understanding.

And the point about synthetic data, the same thing holds true there. And I think of synthetic data as similar to data curation, in that they are both methods for turning computational power and more compute into higher quality data. Right? As opposed to just paying experts to generate more and more high-quality data, this is a way of instead of putting in more high-quality data as an input, to put in more compute. Because we're going to run out of experts as the target moves. But we're not going to run out of compute. We're going to be able to. So, if we can have techniques for putting in more compute through synthetic data, through curation, through simulation, reinforcement learning is an example of this where it's relatively data efficient and mostly focused on compute. I think all of these types of methods are going to let us get to the next stage of scaling.

I'm sure there'll be many surprises. I think there are also trends that are clearer. One is what I was saying about finding ways to put in more compute to create higher-quality data and generate more intelligence. Whether that is synthetic data, reinforcement learning, all of these techniques will be used. I actually think a lot of, so, that's one trend that I think will be super important. Another that we're seeing more and more now is a shift toward really, and it's related to this idea of putting in more compute. But a lot of learning happening outside of the model. So, outside of, perhaps there are two ways to make the model smarter or get information into the model. One is by updating the model weights through training. Another is to feed information into the context of the model. And so, we've got these sort of two knobs. Now, a lot of what we do is the first one. It's like, how do we up train the model and update the model weights. But I think perhaps even greater amount of energy is going to go into how do we refine and curate the context and engineer the context so that the models can make the best decisions. And if you think, here, I think you can draw a lot of inspiration from how humans learn. So, for example, imagine you read a news article or you read some scientific research paper. It's not that many tokens, right? In terms of length, it might be like this long. But you might spend hours reading a research paper, and most of that time is spent thinking. It's spent trying to understand like, how did they get this line from the previous line? Or how does this relate to other knowledge I have? Like, fact-checking, is it even true? Like, maybe it sounds contradictory with something I know, and I'm trying to resolve that contradiction. And at the end of that, what has happened? After doing tons and tons of reasoning, the output might be some updated mental model of how the world works or some insight, some lesson. And that actual lesson might be quite compact, or some compact mental model. And what do I do with that mental model? Well, one thing I can do with it is I can take that and prompt myself with it, right? I can include it in my context to retrieve at later dates. I think about playing a board game. You might come up with some strategy. And the next time you play the board game, you might remember that strategy.

Now, can you get more clever and ultimately distill that, instead of having it in the context, can you distill that into the model weights? You probably can, and that probably makes sense in a lot of ways. But I think the core loop that needs to be closed here is going from doing tons of reasoning and inference as you make decisions and encounter new data, and then feeding that back into learning. Today, when you use your models, you ask them hard questions, they do a ton of reasoning, and they solve the problem. And then typically whatever insights or lessons they learned along the way get dropped. They're thrown away. And you need to close that loop. When you do a lot of reasoning, that should translate into actually getting smarter. So, that's another thing that I think we're starting to see more of, and that will be hugely important. Like, figuring out how to do learning outside of the model.

The challenge is a recognition of turning data into intelligence. Intelligence is in many ways, and well, maybe the true view of artificial intelligence is to recognize all that personalized data, but in sync with the global data as well. And maybe that's the basis of a world model. I think that there will always be, you know, at least for many situations, uncertainty about the ground truth. You'll have many different sources of information. I think one of the beautiful things about the AI systems we'll be able to build is that you'll be able to store pointers to all of the previous sources of information that formed your impression about some topic. And when you encounter new information, you'll be able to retrieve those and reason about all of them together with the new information. And that's something that is difficult for humans to do today. Like, you may have some opinion about some topic or some impression you formed over years of experience, but you don't remember the exact things, all the information that fed into forming that impression. And so, it can be hard to know exactly how you should update your opinion in the presence of new information.