Systems & AI | Kevin Scott, Chief Technology Officer, Microsoft

Yeah. I mean, it's hard not to be surprised. I think things have accelerated probably faster than I was expecting. So, like, none of this stuff is surprising. Like, what is happening? You know, I think it was relatively clear, even those, like, seven years ago when we were having these conversations that, like, you know, scaling laws were going to work and that, like, the models were and the systems that we built around them were going to be like very, very, powerful in a general sense. And they were going to behave like a platform and like people were going to be able to build lots of exciting things on top of them. So it was just a question of like, how fast is all of this stuff going to happen? I think it's happening faster than the new, yeah, that I would have predicted. But like, what's happening doesn't seem like too terribly surprising, you know, and I think the thing that things that are ahead of us are, you know, I'm sure you you all have been talking about this already. We really are not yet at the point of diminishing marginal return on, the like capability increases and the and the platform infrastructure that is powering all of this change. But at the same time, there's a capabilities overhang where people are using.

Yeah. Yeah. I've been like diatribe about this inside and outside of Microsoft for a few years. Like the models are already way more powerful than what people are using them for. I think the closest glimpse that you're seeing to, you know, like, true, full utilization of model capability is in coding and like, is an absolute frenzy right now in the state of software development. Like you have very, very senior people and, you know, like best coders you've ever met in your life who are just completely overwhelmed trying to keep up with the rate of progress that's happening right now.

Well, I think it becomes more of what, like a good software engineer, good software engineers have always done. It's like less about the mechanics of, you know, like, what characters are coming off of your keyboard and going into a text editor and more about, you know, like, do you really understand what it is that you're building and why? And like, how is value being created and know, like, how can you move faster and recognizing what value you're creating for people and just sort of accelerating that rate of improvement. Like that's what the best software engineers that, you know, you and I have always worked with, like I've been able to do and, you know, I think that is still going to be, like very true. Like it's critically true right now because, you know, like, you can produce a lot of code with these coding agents right now. There's nothing to say that it's good code. So, like, you can just sort of spree and review has become the bottleneck.

And you know, like you, you know, people need to really make sure that they're not getting confused between, activity and progress.

Yeah, it's it's always changed. You know like I what I hope honestly like this is Kevin's opinion not Microsoft's opinion. But yeah, I've been grumpy, like an old fart for a whole bunch of years now that, you know, like, we, we have turned, computer science education into, like, vocational education, like, we, you know, like students, like, go get a C degree because they want to learn how to be programmers and, like, what I want people to go back to is learning how to be computer scientists. Like, be thinkers about. Yeah. Can you think algorithmically? Do you know how to decompose problems? You know how to choose the right problem to go spend your time and energy on. Like, do you understand, like how the science of what you're doing fits into society and, and, yeah, the like the broader ecosystem of science that's happening around you. Like, do you possess the capability and the curiosity, like, punch down through the layers of abstraction to like, debug what's going on when the machine doesn't give you the result that you want? I mean, it's like all of those things are like the critically important things. And I think we'll get back to a lot of that because like a bunch of the vocational aspects of the software engineering job are just going to change or go slowly over the next handful of years that it's going to be unrecognizable.

Yeah. So I think the optimistic story is probably the more likely one, because it's going to be the necessary thing that has to happen. Certainly it starts with like, I think that doesn't sound optimistic, but I was chatting with a friend the other day who runs, like one of the premier, educational institutions in Japan. And like he said, this thing sort of off handed to me that like, took me a back. And he was like, yeah, this year is, peak high school graduation in Japan. So Japan will graduate as many high school students as they're ever going to graduate in a single year this year, which means, like, from here on out, fewer high school seniors graduating, like, fewer are going to college. And it goes like down. And there's nothing to do about it. Yes. Sans, you know, like some crazy thing with immigration and like that, there's just, you know, by looking at what the birth rates are, that's just precipitously going to go down.

Just demographics, just pure demographics, pure, pure demographics? And so, you know, like we, we know in abstract. Right. Like the demographic data is super clear, that, Japan is in population decline. It's a little bit ahead of where, like other places are bit like China, Korea as well, like a bunch of countries in Western Europe. The United States sans, immigration would also be, like in population, or would be in population decline, over the next handful of decades. And so, like, you have this aging population, it's actually a much, much bigger issue than people have.

And like, you can even see it in the United States, like, they're these rural places, like where my mom lives, where the population is aging faster because, you know, just like the demographics are kind of unfavorable in, like, rural parts of the country. And, you know, it really has these massive implications that we don't talk about all that much. And like one of the implications is if you don't have as many people to do the work and you have more people who are going to create new forms of work for society to take care of, like this elderly population that is growing very, very rapidly. Something has to happen with the very nature of like how you get work done and what productivity looks like in order for you to maintain just the level of, you know, quality of life in society and like what we think of as normal, in society and like it doesn't happen for free.

And so you have to have technological interventions. This is always the way that we've solved our productivity problems as a species. And so like I think the optimistic case is like, thank God I has come along when it has, that like we actually have this and a handful of other technological things happening in the world right now that will give us at least a partial answer for what you do when, you, over the next handful of decades, don't have the same labor dynamics as you've had over the, you know, like, actually the whole course of recorded human history, you know, it's like a crazy thing that's happening that we just don't talk about all that much. And AI becomes that kind of unlock that you need otherwise, like, yeah, look, it can't do everything. And it's not a substitute for, you know, it's not a substitute for all of the things that human beings do. Like, I don't, you know, like, I don't think anybody here wants it to be, like, but but it gives us an option, for, you know, like, how do you go sort out some of these crazy problems that otherwise would be very zero sum and very, very challenging and destabilizing. And so, like, I am optimistic that we can, you know, put our heads together, like, identify all of those things that feel zero sum. We need to turn them into non-zero-sum problems and then go tackle them and like, have the good sense to have taste about how you, you know, put the technology into practice. So that's the optimistic case. You know, the pessimistic case. Like I won't give you the white. Like you can just sort of pick your, you know, like there's a palette of them right now like Terminator happens.

Or like, you know, many, many things like so my, my pessimistic scenario is that, you know, like we get into some superficial mode about AI that, you know, like we instead of using it to go solve some of these, super important problems with urgency that, like, we use it to just further distract ourselves and, you know, like a bunch of superficial, like I look at, you know, my my kids are like, half and half, right? So like, hey, you know, half the time they're using AI in like, really super ambitious ways to, like, do things, you know, with, you know, the biomedical engineering projects they're working on or like, you know, whatever their technical fascination is and like, they're just way ahead of where I was when I was their age. And then the other half of the time they're using it to like, you know, make pictures of, you know, green llamas with big butts that they paste someone's face on. And it's like, okay, like, you know, hey, that's, that's a, that's a good use of computer resources, right?

So like, I, I hope that, like, we can resist the temptation to make the whole narrative about AI is like, okay, well, here's, you know, the the new sensational thing that happened over the weekend or, you know, like the new way that, you know, a bunch of powerful people are going to throw darts at each other and like, we can just make it more about like, what does society really need from this technology?

A long time. Like, this is the beginning of my 10th year, so it's the longest I've ever worked anywhere.

Well, look, I think, Maybe this isn't misunderstood, but, like, you know, you have to understand about Microsoft that it's a platform company. So, like the we don't really think about doing anything unless it's building something that someone else can pick up. And then build another thing on top of, it's like, wired into the DNA of the company and in your both for good and bad. Right. It means that, like, we're really good at some things and like, we're we're less good at, at others. And, you know, as a platform company with five decades of experience doing platforms, you know, hyper successfully sometimes and less so, other times, we just have infinite patience for the messiness of what comes with technological, you know, transformations like the one that we've got right now, like, a lot of patience, like we we're not waiting around to go do stuff until the ideal conditions exist for things to go maximally fast, like we're going to jump in early on a bunch of things, like we're going to get a bunch of things wrong. We'll get a bunch of things right. And like, we will deal with the world as it exists. Like, not as, you know, we wish it might be, and, you know, like, you know this as well, like, you spent your, you know, your entire career doing enterprise software, like it's, you know, it's a messy business sometimes, like, you have to go make the thing that the customer wants and needs and in some cases, must have, it's like it's not about changing someone's mind. It's about doing the thing that is necessary in a situation like, what's worse, the what's really bad thing about enterprise software, what's worse than having no customers, having one customer, you can never turn something off. You can only turn them on. Yeah. And so but look. And at the same time, like, that's an enormous privilege. It is if you sort of think about all the things that Cisco runs in the world and all of the things that Microsoft runs in the world, you know, they, you know, if you took all of those things away, like it would just be a grinding halt. And so, yeah. And I think you have to appreciate the privilege of, you know, like, all of that messy complexity, you know, obligation that you have to make sure that all of these things work, and they're up and they're available. And like you, you know, when they are not available that you like, go dig in and understand what went wrong and make it better so it doesn't happen again. And, you know, it's, for, for me as an engineer, like, it feels, it feels great because this is what I've always been drawn to as a curious technical person is like these platform problems, like making tools for other people to go. Used to like, make things like it's even what I do in my hobbies. I know that, know it is so, so like I it's why I've been there as long as I have. Like, I just super enjoy the challenge of, like, all of these messy platform problems.

Yeah, I look I, I am proud like maybe the proudest thing, I've done is helping to recognize the shift. The shift in AI from being a very narrow specialist thing where, you know, like the way that we all did machine learning 20 years ago, like to the extent that we were doing machine learning at all, is you had a group of quantitative experts who were working on a very narrow problem, like predicting whether someone was going to click an ad or not in a particular context. Like, that's a problem I worked on for a long time. And like you would have your experts in your data related to that domain and like, you pick an algorithm and you go train the thing and like, you had your experimental pipeline and like, everything was sort of specific to that problem. Like being able to recognize that, like, okay, well, finally, the promise of more general AI is about to be realized. And then getting it through the partnership with OpenAI and a bunch of the things that we built together with them, like really out into the open.

Yeah. And whether or not that's true, like, you know, the thing that I'm especially proud of is like, I like I like a world where, you actually have the capabilities of these platforms, like out in the open, like where we can have a conversation like this, where it's not just like one Silicon Valley company that is building this thing on their infrastructure with their smart people, and then they decide what is going to get done with it. And it's like out there for anyone who can go sign up for an API key to like, go start building something on top of and like that, that, that democratization of like powerful AI capability is like a thing, like, you know, whatever small role I had in that, like that, that I'm proud of, you know, so

Yeah, I don't know. Like I think it's going to be constrained for like a while. Like I keep thinking where, you know, we're about to pull out and then demand keeps, you know, sort of exploding. And like, I look at what's coming capability wise over the next, 12 months or so and, like, I just can't imagine, like, with the capabilities that I know are coming online, that there's going to be less demand for the, for the product, or for the, like, the capabilities that are coming. And. Yeah. So I think we're. Just looking at coding agents like the, the most ambitious teams at Microsoft right now who are, like, fully using coding agents. Where the thing that limits them is they're, they are available attention to manage the full complexity of what the agents are doing cost about 150 grand a year in inference. And so a very tiny slice of even the community of software developer developers, like, have that degree of access and ambition right now for the product, but like, they all could benefit from it.

And then you could apply that to a whole bunch of other things as well. So I just don't see how the demand for inference is going to go down. And it's hard to imagine, like how you get into a state just given, you know, like what the, silicon situation is, the hardware situation, like how difficult it is to, like, build data centers and deploy power and all the things that you want to do, like how you get ahead of that anytime soon.

Yeah. Like I this is the other, you know, going back to your, question earlier about like a thing people don't understand about Microsoft. Like, are we also for know 50 years now have only been successful because we have all of these partnerships, that we're doing. And so, you know, like we enjoy a great one with you. Yeah. So we, we yes, we, we have our own chips and like, the chips are amazing. But like, we also have gigantic fleets of Nvidia hardware and gigantic, fleet of AMD hardware and, like we, we have a huge amount of silicon diversity. Like part of that is driven by, like, what people want us to provide for them. And like part of it is driven by what we know about where our workloads are going and like what we need to build to, like, be maximally efficient. But like at any point in time, like we we really, like we built an infrastructure to manage, the complexity of all of this stuff and, you know, like whatever is most cost efficient, is like the thing that we are going to go deploy at scale.

So say that one more time.

So I just want everybody to remember that, like, the technology hasn't transformed into anything other than a tool. And like it as a tool. It is there for us to do interesting things with, and so, like, I don't think that there's any kind of inexorable trend line about the technology that sort of dictates one path versus another. It's all about choices of like what we choose to do with the tool and how we prioritize things. And I think it's like really, really, really important for us to focus on like how we as individuals who are wielding the tool can do that in service of like our fellow human beings like that. That's I wish every day, like we could all go into our jobs and think about what we're about to do and think, you know, it is this serving my fellow human beings well or not? And if we did that, like we would be in a like a great place at the end of the day.

Yeah. Like the zero sum problems are miserable. Like they're all about like, scarcity and constraints. And like, winners and losers and like, the thing, the thing that I think we really want from our technology is for it to take as many of those, any of those, like, zero sum things that are like, real challenges for us and turn them into non-zero-sum things like, it's like, honestly, it's the reason to go do technology.

Yeah. Pleasure. Thank you. Thank you,