Satya Nadella: Microsoft's AI Bets, Hyperscaling, Quantum Computing Breakthroughs

What are the tools that we can put in the hands of people that will give them that sense of empowerment? That's what I would love to work on. I'm not into this anthropomorphizing AI at all. I come at it as it's a tool. There is going to be a job called a software engineer, it's going to be different. When I look at it, you are really taking a software engineer and saying you're now a software architect.

This is the home crowd, all right man. San Francisco, you should move to Seattle. I started my career in Seattle, very fantastic place. Anyone who's successful starts at Microsoft, that's right.

At Microsoft, I feel we are a platform company, a product company, and a partner company. So I think of those three dimensions. In my 35 years, I've lived through client-server, web, internet, mobile, cloud. This is the fourth, so that's just how I pattern match. The first thing I think about is the platform opportunity. When I look at all the folks here, the interesting thing is the compounding effects of all these platforms. This AI piece, the reason why I think the rate of diffusion is so fast and so wide is because it builds on the previous generation. If the cloud was not there, we wouldn't have been able to build the AI supercomputers, which then led to the models, which then led to the products. That compounding effect is the interesting thing to me. That's why you always take the previous platform and build the next platform, and you want to be able to get that right, and then you got to build the next generation products on top of it. With each one of these platform shifts, there's a workload. When I first remember looking at the large-scale training job, it's a very different workload to what we built for the cloud. It's a data-parallel synchronous workload, which is so different than a Hadoop job. The platform itself then completely gets relitigated and changed. So to me, that's the exciting thing on the platform side. It's a golden age of system software. Today, if I had to think about anybody who's building at the infrastructure layer, not just the hyperscalers but even the startups, I think that's a tremendous opportunity. Obviously, there's a tremendous opportunity in the model side and then the products on top of it. We think of these, and ultimately what's it for? It's for one thing and one thing alone: to drive economic growth and GDP growth. If I had to ask me my benchmark for AI, is it creating surplus in the world around us, one community, one country, one industry, one company at a time?

It's a good question. One of the questions is, is the model like SQL, or is it the SaaS app itself and the model? Where does the model end and where does the product begin? If you sort of say model with some scaffolding and tool calling in some infinite loop is the product, then that's where it gets a little confusing. But that's like saying a bunch of SQL business logic is what an app is. So I think it's still possible for anyone to build an app tier on top of a model, and you have to abstract yourself and say the model is just like SQL was to me. I always dreamt of a moment when AI and machine learning will have a SQL moment, because we never had a stable platform layer in the past. Everything was vertically built and integrated. For the first time, in this model layer, we now have something like a SQL engine that we can then use to build pretty sophisticated products. The inference time compute plus tool calling is giving us a pretty robust harness to be able to build pretty sophisticated products.

I think that's a good observation. At least my read of the situation is the model is an important piece. The model scaffolding and all this tool calling, there's a real app server that you kind of need in order to build sophisticated applications. But the interesting thing is the feedback loop, the data path inside the product that then is used in order to post-train, in order to do the right tool selection. That seems to be the place where product creation is all going to happen.

If you sort of really step back and say, first, if you go with compute or intelligence as a function of compute, and then ask how much energy does compute consume? In the United States, maybe 2% today, 3% tops. Let's say it doubles to 6%. That's massive, because the amount of extra energy that needs to be produced for AI to use it is pretty high. I think that's why we all have to keep in mind that if there's one lesson history has taught us, it's that if you're going to use energy, you better have social permission to use energy. So you've got to make sure that the output of this AI is socially useful. If we really are not creating social surplus, economic surplus as measured by countries and communities, then we just can't consume energy. That to me is the bigger thing. Everybody's today hot and bothered about what to do about energy production. I think the real question in the next five years is we've got to produce enough products that are creating great value, which I'm very confident of. In healthcare, education, productivity, there are many domains. But that's the real challenge for us as a tech industry: to prove unequivocally that what we have created is showing up in real stats, not just an AGI or AI benchmark. The hope is that this will show up in the real things you interact with on a daily basis. You go get a mortgage loan, and instead of three months or two months of waiting around and not knowing if you're going to get approved, there are so many things that are important parts of your life that get drowned in paperwork or bureaucracy. Those things could potentially go away. If you take some of the public services, any country's GDP, or healthcare in the United States, 18-19% of our cost is healthcare, and a lot of it is workflow. If you take something simple like discharge, the back end of an EMR system with just an LLM and a prompt, that itself is going to save so much time and money and energy that it would pay for itself. Every dollar spent on clerical work could have been spent towards treatment that would have saved someone's life, or the simple time allocation of a physician away from paperwork to the patient is right there to be had.

Here's the interesting thing. This audience is so young that none of my metaphors would work, but nevertheless. If you came in the early part of, let's say, you were a multinational company pre-PC, how the heck did we do forecasts? A simple sales forecast: you would send faxes, people would take those faxes and send interoffice memos, those interoffice memos would be annotated, and a forecast would come hopefully before the quarter end. Then suddenly people said with email and PCs and Excel, let's just send an Excel spreadsheet in email, people enter numbers, and you have a forecast. What happened was the work artifact and the workflow changed. That is what needs to happen with AI. When someone says I'm going to now do my job but with 99 agents that I am directing on my behalf, the workflow is not going to be constant. You now are really going to have to change even the scope of your job. That change management is a real rate limiter, because you're now taking the means of production in an insurance company, a financial services company, a healthcare company, a software company, and saying we are going to change everything in the way we work. We're going to change what jobs are. At LinkedIn, they took multiple functions—the design function, the front-end engineer function, the product function—put them all together and said we're going to have full-stack builders. That's a change in scope of even a job. How do you then rebuild the product team with new roles, new scopes? That to me is the social rate limiter. There are other issues like power, but I would say change management is key. When I look at even a lot of the AI startups, everyone has forward deployment engineers. That's the exciting thing, the Palantir model, which I think is fantastic. That is because of that change management. You really need to help customers and partners understand the benefits of any product you're creating, not just the technology but even how to use the technology in a workflow.

I think one of the most understated things as an opportunity for anyone creating products or fundamental breakthroughs, even at the model layer, is the amount of drudgery there is in knowledge work. In software engineering, we saw that. We're taking the joy out of software engineering because you were out of your flow. To be able to stay in the flow to complete a task is a great example of what is going to happen to all knowledge work. The amount of cycles you spend out of band collecting information is pretty low now. Having a sophisticated reasoning model and your prefrontal cortex work together, whereas a lot of the mundane stuff is getting done by some agent, that is definitely the frontier.

This field is changing so rapidly. I had not even imagined last year at this time that we would get this far with RL and with test-time compute. It seems pretty limitless. The way I think about it: pre-training worked, all the post-training techniques on top of it were fantastic, then this inference time compute seems to have really added a massive scaling law. Now I'm interested in whether there is some new algorithmic breakthrough. I always say this entire regime could be changed by one person here who comes up and says I have a more efficient thing to do. You have to be open-minded that the last big breakthrough algorithmically has not yet been found. The other one is what is the next step up? What is the pre-training to RL end-to-end training loop? That is the next big sample that I think will happen in the next year. If that is another scaling law breakthrough, all labs will be working on a more integrated response reasoning model. There's something very interesting in that. If you think about an LLM instance as a consciousness, which some people are starting to say, it's instantiated, you do a bunch of work with it, then it goes away, and you open a new chat box. I'm curious if that loop needs to be completed.

I'm not into this anthropomorphizing AI at all. I think of it as a tool. It's not trying to replicate how we think. It is definitely showing signs of intelligence, but it's not intelligence that I have. Human agency will still matter, and we will use these as tools. That said, a memory system is a good thing. If I look at the next frontier, there are three things: memory, tools use, and entitlements. Entitlements are basically if I'm going to take action, what entitlements do I have to take action? These three systems have to be built as first class around the model in order for us to build more sophisticated applications.

That's right. When people say scaffolding layer, you have a model plus scaffold. The scaffolding now gets first class by thinking of these three things: tools use, memory, and entitlements. Put that together, and you can create an agent. An agent has an ID, management, and provision control. That's the way to think about it.