Who’s Ready for Anything at GTC26 Inside the AI Cloud Shift with Chen Goldberg

Thank you so much for having me.

Yes.

So this is for me the second time as part of CoreWeave that we are presenting at GTC, and the big change, if you think about last year, there's a lot of uncertainty around how inference will be, what will be the size of the models, can you create applications with AI? And what we've seen over the past 12 months has been mind-blowing, and everybody here noticed that. I think that's the most exciting thing for us: instead of just talking about basic education, we are talking about how can you move from experimentation to production. What do you need to do? How do you bring folks on board as an engineering leader? There's also a lot of conversation about productivity and the tools you are using. There are so many things that have changed over these 12 months, and that's really amazing.

Yes.

That was definitely a very special moment for us as well, and we thank Jensen for that because it was a recognition of the hard work that the entire CoreWeave team, our customers, and the partners and everybody that trusted in us went through, because starting from everybody thinks of us as like a GPU reseller and other things, and then the recognition that we are leading in this space and being the AI cloud was amazing. Really, NVIDIA has been an amazing partner and customer. We are their customer as well, and there are a couple of things that work well between our two companies. One, we are really leaning in. We believe in AI. We have a very similar vision, and that really helps. And we are really focused on customers together. The second thing that is really critical, we talk about how our customers want to experiment. We like to experiment. I think all of us as people need to be humble and know that we're not great at predicting the future. But if we experiment and we see signals and we move fast, we get to great results. And NVIDIA also has that kind of culture. And we are both really wanting to build the best products. And that actually led to a big announcement last month. The media definitely cut the investment part, the billion dollar amount, but there were more things there that made me really excited.

One thing was, of course, us using CPUs now from NVIDIA. In his keynote, Jensen talked about expanding to new platforms, so expanding from just GPUs to also CPU. The other thing we were really excited about is two things: one is the collaboration. It was actually telling the world how our teams have been collaborating on producing reference architecture and improving products. That was one part. The second part was really about us getting into the world and offering our services and our software to other people in the industry outside of even CoreWeave Cloud. We started that with the acquisition we made with Weights and Biases, which is already multicloud, but having more and more cross-cloud solutions is something we plan to invest more in, and that's what customers are demanding.

Back again, this year what Jensen was talking about on stage and also what we hear from people around is that in most companies, because the tools have gotten so great, they start seeing value. So yesterday on stage we had, for example, a person from Mercado Libre. Sebastian from Mercado Libre joined us and he was telling the audience how they are planning to reimagine their search capabilities in their e-commerce platform. But even before doing that, he was talking about how even with small experimentation, they've been getting amazing results already. And what I love about it is, downstairs, we have a physical AI demo. So you see in different industries, whether it's health, finance, e-commerce, media, a lot of areas where we see starting from small experimentation to bigger opportunities. And that's really what NVIDIA is talking about. And I think NVIDIA again was really highlighting, Jensen was highlighting yesterday that it's not just NVIDIA on its own. Jensen has been really investing in building an ecosystem.

And I really appreciate that. I was actually part of the cloud transformation in the industry, and the ecosystem was key to that. I think Jensen is definitely recognizing, and we are participating across the board from developer tools to researcher tools to applications to infrastructure, and just building that momentum, that flywheel that creates that innovation.

I think that a year ago, people were not sure that we need an AI cloud. But now folks, once they are trying to experiment more, they see there is a need. And one of the things, maybe I can tell you why I joined CoreWeave. Before CoreWeave, I was part of Google Cloud and working on building that cloud native ecosystem. Back in the day when we talked about the value of the cloud, it wasn't just enough to move to the cloud, a lift and shift. There was a change of ecosystem. You have to change the way you do tooling, how you develop, how you deploy, how you manage. And when I started experimenting and experiencing more of these new type of workloads, it felt similar but actually much bigger, a bigger opportunity to reimagine how cloud should look like. And I think that's the most unique thing about CoreWeave, that from the get-go, if you talk with the founders, that's what they were trying to do. It was not like a me too. Peter, our CTO, they took a step back and said, 'Okay, what are the hard problems we need to solve?' They didn't just look at the other reference architecture. They actually built from scratch. There are some things that we are doing very differently than others, and that's what delivered the results. I think it's easy to understand: there are probably three categories that we're doing differently. The first one is that we started with a vertically integrated stack. You now see some of the folks announcing it, but they're trying to retrofit that vertical integration. Huge difference retro versus what we are doing. We understand that the complexity of the stack is huge, and in order to manage that, you need to allow flexibility and quick decisions across the stack and knowing how to react to events lower in the stack. So, from metal to model, from metal to job, bringing all of that information and really creating a new way to operate a cloud stack in the AI era. We differentiate with capabilities like Mission Control that are both reactive and proactive in solving customer problems. So that would be one. The second part is that even though we were thinking about an integrated stack, we are still looking for opportunities to optimize specific problems or bottlenecks, and again we don't try to solve for everything. We are really, really focused. So if you think about AI workloads, one of the challenges is you want to make sure that GPUs are at high utilization. They're very expensive, a very important resource. So we are building, for example, specific solutions that will bring more data into the GPU with our own distributed caching mechanism, or building a new solution for orchestration of workloads. For example, for the technical audience, folks that are familiar with Slurm, Slurm is a known industry standard for HPC workloads. But what we've done is we've created a solution that brings those workloads into the cloud native era. And the last point, which maybe brings us back to where we started: we know when we talk with our customers that speed of experimentation is the most important thing. I'm a believer that we cannot predict the future.

So how can we increase the signals that people see? How quickly can I experiment, iterate, and see what I need to do? So we have a lot of investment on that, both from an infrastructure perspective but also on toolings like with the W&B models and Weave, just giving those signals that allow our customers to make decisions faster and with confidence.

Yeah.

Thank you so much for asking that. I think we are using a lot of jargon all the time, and people don't always understand what it actually means to build a cloud, how it looks like. So first of all, there is the physical infrastructure. On that layer, there are a lot of new challenges that appear in the AI era: a lot of power consumption, cooling needs. There's a lot of work that our team is doing around power, efficiency, space, liquid cooling, and making sure we build and innovate in that space, including security across the stack. But then on top of that, there are different layers of the stack. When we say infrastructure, we should think about starting with what you call infrastructure as a service: very simple compute, storage, and network. What's been very interesting when you move from cloud 1.0 to cloud 2.0 is that in the cloud 1.0 era, the goal was to make infrastructure—specifically compute, storage, and network—boring.

As a developer, as a business owner, you should not care about that. And that's been my journey in the industry, making that a reality with technologies like Kubernetes.

Make it invisible. Make the infrastructure boring. You can Google that and you'll see those kind of quotes. I think we've done a great job in the industry of creating technologies that have done that. However, what is happening right now, and that's exactly what Jensen was talking about yesterday, is that the workloads, the applications, they care about the compute, storage, and network. Because the amount of data that has to go on the network is impacting the latency. And now, let's say I have an agent. This is a mission critical workload that needs someone to immediately respond. Or if we have video generation, it cannot be lagging. And you think about storage: how quickly can I scale? How do I get the data? How quickly can I load things? From a compute perspective, not only do I want to get the best utilization, we actually have lots of workloads, a lot of models that are now running not on one node or 10 nodes, but sometimes 10,000 nodes. That's huge, and that creates a lot of complexity in that stack. We have been innovating in that area. And then on top of that, we have new inference services and training services, because as a developer I have new tasks that I need to perform, so I need new tools and a new ecosystem. And maybe on top of that, the thing that excites me the most is our entire serverless layer. It's not about making the infrastructure boring, but helping our customers rely on us in making some of those decisions. So one of the things I'm really excited about is the tools we are providing at that serverless layer, where we are not trying to make the infrastructure disappear but we're trying to help customers rely on us, to let us do the heavy lifting in making those tough decisions. We've been announcing this week a new tool in our serverless RL, which is reinforcement learning. In that space, before, a lot of this experimentation required simulation data. Now we're allowing customers to use production tracing in order to train the agents to get better automatically. So that would be one example of a tool.

In production. For example, we have a new customer client that they are using our inference service. They are a customer and a partner as well, and they are actually building a coding solution that allows them to leverage our platform. And maybe you asked about differentiation, I should have led with that. We offer all this new stuff without compromising security and resiliency and production readiness. We take a lot of pride in that, and that's probably the number one reason why customers come to us.

There are many challenges. We were just in a customer meeting today, and they said every time we solve a hard problem, a new hard problem pops up. So it feels like that's the business we are in. But the thing that I feel we are most helping customers with is speed of innovation. The idea that our customers are telling us that on day one they can be productive, that they have the signals to make the right decisions, that we are not wasting their time, that their team is productive, is really, really important.

Yes. For sure. The second thing is that we do see our customers as partners. I think it's a privilege for us to partner with them and enable that next gen innovation.

And it looks differently. Our customers, we are not just talking about the platform or what services. We are talking with them about what challenges they have.

How can we help? And if they need us, we'll be there. As part of that state of mind, it also impacts how we support our customers. We're not really like a traditional support process. The way we work, we believe in an engineering-to-engineering relationship because our customers are sophisticated. They have the hardest problems. They want to know that they can quickly find solutions. So we've automated and managed that process. We call it 'direct to expert.' We have engineers working with our customers day in and day out on the hardest problems, speaking the same language. That makes such a huge difference from a differentiation perspective, but really getting in there and allowing those customers to deliver to their customers what they expect.

I keep thinking about it. If someone had told me that 2025 would look the way it is, I would not have believed it. There have been so many things that happened that surprised all of us from an advancement perspective. However, I do expect that we will see an explosion of use cases, and more and more people will speak the AI language. It's so powerful, and the tools are becoming so accessible.

That I think we will see less people be worried about it, because what we are seeing today is that the people that lean in find that as their superpower.

Yes.

Thank you so much for having me.