5 Ways We're Confusing AI Capability and AI Reality with Microsoft CTO Kevin Scott

I hope you all are here to see me talk, otherwise I can just imagine somebody wandering into this bar and then this is what they get—worst bar experience ever. But anyway, thank you all for coming out today. Hopefully I can see this little weird setup for a tech talk. But pros and cons. I think the pro is you all get to drink while I'm talking, and the more you drink, the smarter I'm going to sound. So the thing that I wanted to chat with you all about is just sort of five observations that I have had about how it is we are thinking about AI and some of the expectations I think that people have and where I think there may be confusion or gaps between the expectations and reality. There are five of these things. I'll go through them relatively quickly. And we'll sort of start with a thing that I've been chatting about for a while. So I think a few years ago at one of my Build keynotes I talked about this thing that I was calling the capability overhang. And it has been the case for several years now that the AI models are actually more capable than the things for which we are using them in the real world. That has gotten a little bit better in some places and actually a little bit worse in some other places. And one of the reasons is that just because an AI model is highly capable of something does not necessarily mean that quick deployment will follow. The thing that we're seeing right now, probably the place where the AI capabilities are furthest ahead and we'll talk a little bit about why in a second, is with agent coding. So the AI is like helping all of us write more code and like helping people who were already great coders be even more effective at their job and helping people who've never written code before in their life to actually write their first programs, which is kind of amazing to see. Even then, in the place where AI is at its most capable, we still have a whole bunch of things that are making deployment go slower than it might otherwise go. I know even for an organization like Microsoft, we have about 100,000 developers inside of the company and there are just so many reasons that deployment of these tools into the everyday working processes of folks just doesn't go quite as fast as you hope it will. And so this is one of those things. It's not necessarily like Field of Dreams—build it and they will come. In some places yes and in some places no. And so we just shouldn't have uniform faith that as the AI model capabilities improve that we're going to get this crazy fast deployment everywhere, which may be a little bit of a controversial point of view to have here in Silicon Valley where, and I am a Silicon Valley resident, where sometimes we just sort of think if you make the technology better and better and bigger and bigger and larger and larger scale that all of the deployment that we hope for is going to happen.

So another interesting thing as we think about capability is we are seeing models improving most quickly when there are closed feedback loops and fast running feedback loops helping the models and the AI systems themselves improve faster and faster and faster. And again if we look at what's happening in coding, this is the clearest place where the closed feedback loops have been most effective. And the feedback loop in the case of code is for a while now models have been able to emit code. I remember when Sam called me a handful of years ago and was like, hey you need to see this, GPT can write code and it only works 35% of the time, which again is like half glass half empty, glass half full sort of situation. From that point forward we have had really an effective closed feedback loop for making models better at coding. So the model can generate code, we can run the code through all of the static development tools that we already have to see whether the code is well-formed, whether it compiles. We can use the models themselves to generate test frameworks and testing for the code. So we can sort of see if the code that you generate from a prompt is performing the task that you prompted it to do. And increasingly we're getting more feedback loops just in terms of experts helping the models in post-training to learn stylistically which generations they should prefer over others. And then increasingly even the feedback loops that we get from developers using the product are making the models better and better and better. So the thing to be aware of is you all are just sort of imagining the future and forecasting what's about to happen is the capability of the models is going to improve fastest where these feedback loops exist and in some cases the feedback loops, to the extent that they exist at all or can exist, they are not going to necessarily be as fast as some of the ones that we have with things like software development.

Yeah. So for instance, one of the things that the models can already do is they can postulate new ideas for particle physics experiments. And the problem with particle physics experiments is they actually take a lot of expert technical labor to set up and run and they require the use of extremely expensive oversubscribed experimental infrastructure to run. And there really isn't a convenient way other than publications in the scientific literature to get the output of those experiments back into a model. And so there, you shouldn't really expect without additional interesting breakthroughs in how we build models—like building simulators that are very accurate to the physics of the world, being able to produce data that the AI systems can then use to train—you just shouldn't expect some things to go terribly fast. And so it's a good lens to look at the world through. Because in some cases we're just going to see capability roaring ahead super fast and our big problem is going to be this deployment issue that I talked about a minute ago—how can we figure out how to make these capabilities more valuable to people so that they can get into people's hands and be useful. And in some places we're going to maybe hope that the AI systems are going to be able to solve problems and structurally they just aren't going to be able to. And I think it's our job, I'm guessing most of the people who are at this conference and in this room are developers or product builders, as we are sort of thinking about what to develop and how to build our products, this is an important thing to keep in mind.

So another interesting thing and this is kind of at the root of some of our interesting deployment problems is just because we can make software go fast doesn't mean we can make organizations go fast. And I think we're just going to see this an awful lot this year. We're going to have people who are super excited and really enthusiastic about how to go use these tools and I can just sort of say myself, I build a lot of prototype things that are green field where I have no constraints whatsoever—I've just got an idea and there's nothing stopping me from using an agentic coding system from producing a brand new thing. But in many cases the things that we want to produce are fairly highly constrained. They could be constrained because you have a regulated environment that you're trying to do something in like finance or healthcare. It could be that they're more constrained because you've got a bunch of infrastructure inside of your company that isn't agent accessible. So it's just sitting inside of a bunch of things that have walls around them that are going to be really really hard to build mechanisms for the agents to get in. It could be that they're last mile problems. So one of the things that we see all over the place is we use these tools, we resolve a problem, we just get much much faster, sometimes near infinitely faster at a particular thing and then you just encounter the next thing that's going to slow you down. And so it's just going to be a lot of work and a lot of the work is going to look like plumbing to get some of the deployment issues resolved. And I think that's both bad news and good news for all of us in this room. It's a lot of opportunity out in the world right now dealing with a bunch of these plumbing issues that we're going to have to make things more accessible to agents. And then some of the stuff is just pure human psychology that is throttling things. Like I know I'm kind of grumpy old man. I know how to do things a particular way and I find myself increasingly in these modes where someone tells me, hey dude, you should give this a try. And I'm like yeah, not at all. And so that is also a normal valid thing to expect like if you are someone who's been super successful in your career and you just know how to do things a particular way. Sometimes you are going to have to see a lot of proof that a new way is better before you're going to change your mind about how it is you want to do your work. And that is a perfectly reasonable and valid point of view to come from. There's also just a forecasting problem I think that we all have with something that's going exponentially fast. So all of the stuff that I'm doing right now using agentic coding to build things wasn't even possible November of last year. And so it's just hard, I think, for us to in the places where you have these closed feedback loops where things are going really really fast to get people to necessarily notice the rate of change and snap to quickly enough because it's just hard to see and it's hard to keep up.

So this is maybe one of the more important things, which is just because you are using AI to create a lot of activity does not necessarily mean that the activity that you're creating is valuable. And so I'll give you a good example of this. I was kind of bored yesterday. I was flying back from a meeting that I had in Japan. And I thought it would be a really good opportunity on this long flight to go build this super frivolous app that I've wanted to build for a while. So I've got two teenage kids and for a while I was doing this thing to irritate them by only communicating with them via meme. So I would send them a meme and they'd be like, Dad, what are you doing? And then I'd send him another meme and he's like, Dad, really irritating. And then I'd send him another meme. And I was like, this is kind of labor intensive, me having to generate all of these memes. I want a meme chat app that generates these things for me. And for a while, that seemed like an extremely frivolous thing to do and a horrendous waste of my time, but I was like, let me give it a shot. So I coded this whole app on the plane. I think I had 65 PRs over the course of this flight. And yeah, it was kind of fun to do. Completely other than my own sense of enjoyment, valueless. Do not expect Kevin Scott to be launching a meme chat app anytime soon. And so I do think that we have to pay as developers especially close attention right now to this difference between we now have this new tool and we can just do a lot more with it, we can have a lot of output, we can build more complex things than we built before. That doesn't necessarily mean that the things that we're building are super valuable, that they're going to land in users' hands, solve problems that they have, increase the top line on the businesses that we're running. And I think it just becomes more important than it ever has been before that we pay attention to how we measure value and that we're paying very very very close attention to the feedback that we're getting about the work that we're doing. Because otherwise you could get very very very confused because it kind of feels good to go from I can make things at this speed and at this level of complexity and now that's 10x or more than it was before.

So the last thing that I wanted to talk about is just because you are able to build systems that are autonomous, they can do complicated things end to end, all the way. And so this is talking about things that are at the end of solving all of the problems that I just talked about, does not necessarily mean that we're going to be able to trust those things so that we can go use them. So I think there are just tons and tons of things from a regulatory perspective that you're always going to have to have human oversight on where this notion of autonomy is a little bit of a pipe dream. But the big thing that we have to do and it has to be done very authentically is you have to build systems in a way that are doing complex things where people can trust that the systems are doing them correctly and in a way that's aligned with their interest and values. And that's a new way of thinking about software. It's not simply that your job anymore is make a beautiful user experience or make a product that passes its whole test suite and you've got good test coverage. It's us having to think really quite hard about what does it mean to build trustworthy software? Because until you get to real trustworthiness no one's going to give work to these software systems to do fully autonomously on their behalf. And I think there's some pretty exciting stuff that we're going to talk about, I've been yelled at for saying too much about these things that are supposed to get announced tomorrow. So I'm not going to steal my boss's thunder. But there's some super good stuff on this front and on all of these fronts that Satya is going to be talking about first thing tomorrow morning in his keynote. So I do think we're making really good progress.

So the net of all of this is I think there's just a lot of work for all of us to do over the next handful of months and years in order for us to really be able to unlock the potential of this crazy tool that we've built collectively over the past handful of years. These problems that I enumerated don't go away just as a function of scaling up an AI model. There is no silver bullet that gets all of these things sorted out. And so it just means a bunch of technical work, a bunch of social work, societal work, a bunch of organizational work, a bunch of just sort of dealing with legacy systems and plumbing work. And in some cases I think that's going to be more work than we've ever seen before. So we will have to have a whole flurry of activity and intensity around some of these things that requires all of us to engage in ways that are even more intense than what we see right now just because we can see the promise of what we're able to do for the world if we're able to overcome some of these things and really get the value out of this infrastructure. That's my talk. I will let you all get on with your hopefully fine drinks that you're consuming and I hope all of you have a really amazing Build experience this year. Thank you all.