Especially in technology business, we collectively as an industry have to just mature and mature at a very fast rate because the impact of digital technology in our lives and in our society is so deep. Now the first thing that we have been doing is building the system all the way from power to the DC to the rack to the accelerators to the network. You have to create magic.
Welcome to the stage Satya Nadella. Hey Satya, thank you so much for doing this. I know last minute schedule changes. You can be here in person.
No, not at all. It's my pleasure Anel. It's great to be with you.
Yeah. Right before this, we had an opportunity to hear from Bob Mechaf about buildouts throughout the 80s and 90s. So you've got a tall order. I know you admire Bob as much as I do. So maybe one place to start is you know there's so much buildout happening now. What do you think will make or break the next decade of this infrastructure buildout that's happening now?
Yeah. I mean the way I think, Anel, each of these buildouts of infrastructure always comes down to two sets of things, right? How efficient is the buildout for what is the new platform, and then what's the way it gets to ubiquity? In some sense it's the app server. I've always thought about at least from a platform perspective, what's the core primitives of the platform and then how does the platform have a developer interface so that developers can use it, right? I mean at the end of the day you could say that's what operating systems are, right? So the operating system book at least I read in school was that Tanenbaum thing which is you abstract the hardware and expose an app model, and that's a good way to think about the infrastructure. So in that context, the first thing is how do you make things efficient? This time around it's that tokens per dollar per watt, right? So the token factory has to be super efficient, so there's a lot of work that goes on, I think is going on, at the systems level all the way to power, right? And so that's one side of it. Then on the app server side, it's more like an agent factory. So if you have a token factory, so think of it just as throughput's got to be most efficient. But then you got to harness that to build very efficient effective agents, and that is an entire different ball of wax, right? Which is you got to have a set of application services so that any agent that you're building delivers on some eval metric that matters in the world, and so those are the two, at least I think, as the infrastructure pieces.
Yeah, I think you have such an engineer point of view of this and I remember you mentioning the sort of complete thought from silicon to operating systems to applications. How do you think that cascades down to the core part of how networks are designed in all this buildout and across the WAN and the edge? And how do you think about that?
Yeah, I mean it's fascinating. I mean obviously what you all are doing, in fact the first time I remember you and I chatted when you said you're going to vertically integrate build this network as a service platform. It's pretty impressive to sort of say that's I think a little bit of what we are also doing, right? When people say wow there's a lot of capex being spent. I mean the reality is the data center is the computer is becoming more real today, right? I mean that's kind of where you want to really think about the complete thought is how do you really go deliver on that tokens per dollar per watt? But just take one of the new data centers we just brought live last week or a week before called Fairwater. I mean interesting enough we built it as a double-decker data center, this is the first time we did that, and we did that because to in fact for the network which is can you have the 5 million network connections that are there in that particular data center optimized for cable length, right? Because we wanted the most dense GPU environment, and so in order to do that you kind of had to even rethink the construction, if you will. And so to me, and then not only did we do that in one data center but we also built an AI WAN. And I was sort of very pleased to see you all working with Lumen and building your own sort of connection between what you're doing and what they're doing. In fact, that's kind of a similar thing we did. We now have a Wisconsin Fairwater and Atlanta Fairwater and both of them are sort of worked through this AI WAN. So you can now run distributed training across these. So to me that's I think what you kind of take the workload. I worked for this gentleman who at Microsoft called Paul Meritz a long time ago. He used to always teach us as you know systems folks growing up saying look at the end of the day you really want to fall in love with sort of the what the new workload is and then design infrastructure and application servers. So that's what we're doing. You look at these new data parallel workloads which kind of are unlike anything we've seen before, a training inference data gen data pipelines for all these eval rubrics what have you, you go build for that, and that's where I think the infrastructure is being rethought end to end.
Yeah, is here somewhere in the audience but I watched the interview you did with him and the scale of the buildout. How do you feel as an engineer like these are mass scale projects? How do you think about this? And you know as an engineer I know like you understand these systems, are you in awe just as much as the rest of us are to look at the scale of this?
Yeah, I mean I must say we're in awe of it but I'm also quite frankly humbled by one fact, Anel, which is the amount of expertise that is required to pull something like this off, right? If you think about the electrical engineers, the mechanical engineers, the construction civil engineers, the obviously the computer engineers, all of them coming together, composing on a project timeline, making something like this happen, delivering on an SLA. So to some degree it is what I'm in awe of, and I also feel that this is one of those places where we've got to get it right, that not only do we have to build stuff that works and works super efficiently, but it's also to do what I think that's the main thing as engineers. Because for the first time as I mean when I say tokens per dollar per watt, if you're really going to consume a lot of energy, one thing that I think we all in this era have to be more grounded on is the social permission to go take that type of energy convert it into tokens is not going to be about celebrating like some zeal that you and I can get hyped up about. It has to be something that the world actually benefits from.
You guys are building out the entire stack like you mentioned which is amazing to watch. How do you think you maintain that while also thinking about reliability and trust because so many companies depend on what you guys are doing and how much you're building up?
