The AI Breakthrough Nobody’s Talking About ft. Kevin Scott (Microsoft CTO)

The thing that's been true for those four decades of me programming is the job of the software engineer is constantly being reinvented. My daughter, who is in the 10th grade, texted me and she's like, 'Dad, how do convolutional neural networks work?' But I was in the car on my mobile phone. I was like, 'All right, how am I going to answer this question quickly for this child?' And I asked Deep Research the question. It took about 5 minutes of compute time to find 23 resources, including medical literature, and then it wrote code for the approach that it thought was best, and the code worked. You would have thought that we would have closed the gap more than we have between what the technology is capable of and what people are using it for. I think if anything, especially over the past 6 months, the gap may have widened.

Is that what you'd call it? I call it a maker studio, a shop. What would you call it? Yeah, it's sort of an eclectic collection of things. We're sitting right now in a ceramic studio. Next door is a big wood shop, got a machine shop and a sewing studio. It's just brimming with possibility. You have an idea, you can go make a thing. I think it's really important. Spending time with your hands making things makes you take the world of things that you consume less for granted. It's like a meditation, but object done. After I'm finished, I love that. I think that's great. But it's also interesting because with being a builder with pottery or construction, woodwork, code, or technology, there's a thread that runs through all of it: this impulse to make. The great makers that I know are all curious about how things work, attracted to challenging problems, and trying to figure out how things fit together to make something new and interesting. In that sense, no different than coding or building a piece of furniture. It's really funny the number of software engineers I knew who had wood shops in their garage. It's crazy.

It's really tough given how fast the technology is evolving. You really are having to do two things simultaneously. You are having to explore enough so that you understand where the frontier actually is, because if you don't understand where the frontier is, it's very hard to set your ambition in an appropriate place. But at the same time, and this is the struggle I think that all great entrepreneurs and makers have, I'm biased towards finishing things. There's some aspect of making for making's sake that I enjoy, but what I really like is doing the best that I can with the raw materials and the technology that's available and then having something that I can give to someone else, and then I can see whether or not it's useful. So at some point you do actually have to stop fooling around and start making something that you can start getting that feedback about whether or not this thing is going to be useful or not. I think we're at an interesting spot right now because you would think after several years of Gen AI just being at a breakneck pace, you would have thought that we would have closed the gap more than we have between what the technology is capable of and what people are using it for. I think if anything, especially over the past 6 months, the gap may have widened. The technology is capable of so much more than we're using it for right now. It's really remarkable. I've got a bunch of little shocking anecdotes that I can share if folks are interested.

Well, one of them is I was in the car with my wife a few weeks back coming home from dinner, and my daughter who is in the 10th grade texted me and she's like, 'Dad, how do convolutional neural networks work?' And I was like, 'What? Why?' 10th grade. She's working at a cardiothoracic research lab at Stanford, and she's got this project that she's doing trying to build a blood oxygen saturation meter. The idea is take an image of a retina and be able to reverse from the retina what blood oxygen saturation levels are, and trying to see whether or not that can give you a more accurate reading than a finger blood pulse oximeter, which are not super accurate. If you are in a surgery, you need super accurate. She was asking about these CNNs because one of the sub-problems of designing such a thing is you have to be able to do this process called segmentation of the image. You need to take the pixels that are in that image of a retina and just get the blood vessel pixels out of it, which is one of those computer science problems that is easy to state. I just told you, you understand exactly what it is I intend. You could probably even sit down with a pencil and look at one of these images and you would be able to draw around the blood vessels. You could do the segmentation yourself, but that is super challenging code to write. She was just trying to figure out what technique do I go use to write this code. I know how convolutional neural networks work, and I took image processing back in the dawn of time when I was a grad student, so I knew a bunch of other image segmentation approaches. But I was in the car on my mobile phone. I was like, 'All right, how am I going to answer this question quickly for this child?' And I asked Deep Research the question. It was kind of mind-blowing. I'm in the middle of this every day. I see what's going on. I've been paying as close attention to what's going on as is possible to pay for the past six years or so, and even I was blown away. It asked me a bunch of follow-up questions so that it could get more precise about what it was that I wanted. I did another turn of specifying what it was that I wanted, and it took about 5 minutes of compute time to find 23 resources, including medical literature, a bunch of citations for a bunch of different approaches, and then it wrote code for the approach that it thought was best, and the code worked. If I had been given that task when I was a computer science grad student, it would have taken me days, maybe a week worth of wall clock time to figure all of that out, to go to the library, do all the research, read the papers, make sure I'm understanding what I read, go write the code, test it. Five minutes worth of compute time. I did it on my mobile phone. I texted a link to my daughter, and she was like, 'Whoa.' If we hadn't had this technology and she hadn't had a computer science father, I think that would have been unreachable. This is not like, 'Oh, what's going to happen to software developers?' It's like, 'Oh my god, what an unlock.' This made a thing that would have been completely impossible for someone who isn't a programmer possible, in the same way that it would be possible for you. It really is doing that thing that I've been hoping for for a while with AI, which is getting past this point where getting something done computationally is incredibly complex because the abstractions that we have for doing them and the low-level details of the machines are super complex, to having a tool where you can just sort of say what you want and it can kind of understand it the way that an expert person would, and then go do what it is that you want. Because ultimately, it's not the actual journey that's important. It's got a kid who needs to solve a problem.

There are a whole bunch of things about the software engineering process that I think are going to fundamentally change that I'm excited about because I'm a software engineer. We can talk about some of those later. But the things that I really am most excited about are these sort of scientific discovery and healthcare applications of the technology. I think the history of technology has been one where peak benefits occur when the technology helps increase productivity somewhere where you have a real absence or shortfall of this thing that you need. If you look at one of the big problems that we have as a society right now, we just don't have enough high-quality healthcare to go around for everyone. It is at the moment and will stay like this until there is some kind of technological intervention. It's a zero-sum problem. All we're doing is we have a finite number of healthcare practitioners and a finite amount of our GDP that we're willing to spend on provisioning healthcare for people. You can argue about it being more expensive than it ought to be in one place than the other, but it really is very zero-sum right now. The best you can do without some kind of technological intervention like AI is hope for things to just sort of move around, which means that the thing that you improve comes at the cost of suboptimizing something else. That's not me saying that there may be a better equilibrium than the one we've got right now, but the even better thing than moving that equilibrium around is really having something like AI come in and just create some abundance where it doesn't exist right now.