has generated $250 billion with a B in market value for Microsoft. Scott Nadella, chairman and CEO of Microsoft. Since you've been the CEO, three and a half years, the stock is up about, I guess it's about 120%. I'm good for my 80 billion. I am going to spend $80 billion building out Azure. Maybe after the industrial revolution, this is the biggest thing. That's our goal with our frontier model. Our model should be the best model that they can use as a base. We create technology so that others can create more technology. That's who we are. We're tool maker.
Please welcome Satya Nadella.
Hi guy. Good to see you coming out.
Good morning guys.
How are you?
Crazy weekend, but here we are. Do we need to pace the frontier?
So, let's start with the common sense part first, which is we should do what it takes to build stuff that serves humanity first and is in human control. You know, it's kind of crazy that we have to start with that level of common sense, but I think it's a good place. Then when I think about pacing whatever the first thing that at least I believe is the broad diffusion of this technology is the most critical thing because the benefits of this tech showing up everywhere is really what's all about right so at the end of the day if you sort of say serving humanity let it actually reach humanity in ways that it serves humanity and that means you got to have choice you have to have competition. You have to have all kinds of business models whether they're open weights, close weights, what have you.
Then the other aspect I think that is not talked about when we talk about control is actually the control that for example customers have, enterprises or businesses have around this technology because sometimes this is so opaque, right? I want my privacy. I want to be able to embed my knowledge in a set of weights I control. I want to see all of the chain of thought that's being generated. I want to use it to do fine-tuning of my own models. My IP shouldn't leak. So, there's an entire body of things that nobody's talking about as much, which is I really want to make sure that this tech is in my control.
Then we get to what is I think a real issue of safety and we should take it seriously which is we should take all the time we want to test things. In fact I love this idea of having third party testers. Oh wow. You know I grew up in a company that's always done testing. So it's novel that we should say wow they're having embedded third party testers. Why not? It's a great idea. In fact, the only thing I would say is we should avoid like these, you know, cozy arrangements of who's testing what, who has access to what, and it should be broad.
Were you surprised though when both the essay landed and then it seemed like there was a circling of the wagons amongst the frontier companies?
I think that it comes, my suspicion is it comes genuinely from this place where when you start seeing, in fact it's fascinating, right? We are when you start seeing reward hacking and what's happening in these environments right with these agent swarms there is the mundane there is some DevOps error where somebody misconfigured a container.
Right, right, or these API keys.
Or an API keys or yeah exactly there's no monitoring, there's internet access, there's sort of classic I would call it basic devops and then there is real novel new stuff right which is what is this reward board hacking that you know with these persistent agents and so on and that's a place where I'll admit that the science is not there it's I thought Jakob's post which is a good one which he said he called it we're growing intelligence not building intelligence so it's an experimental science and so the more experimental sciences then you really need to make sure you're doing those experiments in controlled environments.
If anything the place where I would love is taking even the Hugging Face incident in other places more transparency on what would it take in fact one of the fascinating things right now is the insider risk I mean think about it right if you're sitting in an enterprise this is all test time compute by the way right so it's not like oh it's going to only happen when in some training run it can happen for a very mundane task that I give one of these frontier models inside an enterprise where I say you know I don't I was you know telling David this suppose I say hey go optimize my working capital it may fake my books right because this is like a new type of insider risk.
And so what is the way to do that I would say oh go build a maybe a causal model like a semantic model that actually checks and verifies so I think there's a lot of product building I would say making things more robust which is classic engineering that we should be talking a lot more about transparently versus saying hey this is so mystical that you know we can't figure this out.
Do you buy this argument that it's mystical?
I mean I buy the argument that we do not understand the latent space. Right other than I thought you know as you said like do we understand the brain? We don't. We do functional MRIs and do neuroscience and we're trying to figure this out continuously getting a little better understanding. So I do think that in that sense we don't exactly have a complete understanding that's why by the way I also I don't believe in new release right so that's why I think making sure that the chains of thought are in language that we can all understand in fact they're transparent so that when I go back to an enterprise that's using all these models and if you have the full chain of thought then you can really go look at it deeply in fact you can have multiple models and you can look at the chain of thought across those I think these are all things that I think will become very important.
Satya you've worked with technologists for decades and when you see as a leader of one company Microsoft which has very crisp communications with the public and you see what's happening with Dario and his team people coming out saying 10% chance we all die what do you think is going through those technologists minds. Do you believe they actually believe that this is going to kill humanity or are they going through some psychosis or are they seeing something working on those frontier models that is terrorizing them? You're not a psychologist, but you have worked with technologists for a long time. Handicap what's going on in these organizations that's all the making people feel the need to resign and say we're all going to die.
Yeah. You know, it's hard for me to speak to what's happening in any of these places, but let's just say how I grew up even inside of Microsoft, you know, for example, one of the biggest things you learn as an early sort of engineering lead is how to deal with a showstopper bug.
Right. I mean, that's kind of like 101, right? Which is why you're faced, you're like, you know, you have a bug. What do you do? Do you stop and fix or you defer or you go in and say hey this is such an edge case that's kind of the judgment so I do think and as the stakes go up you want to like transaction processing I remember working on databases right you know wow like you know you got to take very seriously any bug where if the transaction is going to get lost right data loss is a thing that you stop the thing for so I feel a little bit culture culturally in the AI industry rediscovering maybe because when you see and it's possible that they see stuff which are showstoppers before the rest and if you see a showstopper stop the show right to fix the bugs.
Yeah when you saw the Hugging Face run and it was super performative Dario did his whole post civilizations what do you think what's your take on that testing they ran because they could have run a test where they had 3,000 agents defend a bunch of websites. Instead, they instructed them to hack websites and you know the hiding of information all this anthropomorphizing whatever of the agents.
I mean the way at least I understand it was it was actually you know basically trying to do an eval for cyber gym and as I understand it given that eval it sort of figured out a way to say let's just say reward hack and that's what led it to Hugging Face in fact it speaks to I think what's the clear issue right now which is you can have these things if they're long-running persistent agents become essentially like new insider risks. And so that I would start from the very basics of saying okay what is containment look like. So for example like one of the things that I think is going to be really an issue and a thing that needs great solutions is true aggressive monitoring of agent activity.
That's behavioral evidence and so everything has got to be auditable. And then every object it access, right? If it goes and gets a secret, oh, it's going to go chain a couple of things, you should be able to see it when it's starting to chain a couple of vulnerabilities to go hack. And so I think that these are the ways that you really have to sort of deal with these situations versus saying in fact I think the core of my take is we will have to get the engineering process around building out this experimental science to be more robust.
Yeah. Thanks. So I think that's a great point. I love how you differentiated in the Hugging Face episode between the mundane things they got wrong like the misconfigured sandbox and Hugging Face had credentials just sitting in a public repository and there was no monitoring and then you have the genuinely novel behavior, the swarms of agents, the reward hacking. That's the stuff that has everyone freaked out. I agree that, you know, we have to now figure out how to fix the bugs or, you know, fix the deeper problem that's coming from that reward hacking. What do you think that means for and I think to their credit I think what the Frontier Labs are saying is we are now going to slow down the pace of let's say raw power and shift towards reliability and predictability and you know what they call alignment which I think is good business practice I guess what do you think that means for what we see in terms of new products for the next year or two does it mean we just kind of improve what we already have or do we see new capabilities what do you think this going to mean?
A great question, David. I do think there's already a massive model overhang, right? I mean, capability overhang in the sense of the models are very good except the broad diffusion requires a lot of things, right? even requires essentially if you're compressing workflows and changing workflows to happen differently the amount of change management that needs to happen in order to even incorporate these systems is sort of what's taking time so to some degree I would say the and also the ability to create these new form factors right I mean if you think about coding agents and coding agents became really usable when you discovered that you could have an agent loop with a file system and that was the breakthrough that just made coding agents work.
And I think now maybe with computer use right so which is with Astra with computer use could be a way for us to even do computer use or we just use long trajectory tasks that can get completely automated. So I think these type of product innovations where the model plus the harness allow us to do things that then lead to broad adoption. Right? I even go back to the chat GPT moment for me, right? Which was it was that RLHF at the very end that made a chat conversation possible.
And so I think that yes, so there's some science, there is some form factor that then leads to broad diffusion and we now need to find the next level of these things that are doing real work in the real enterprise. And in that context by the way the other thing is it's going to be a multimodel world right so at this point just out of resilience right I mean think about right every enterprise now comes to me and says hey this model does refusals here this model I want weights here I don't and so the people are going to want multiple models so one of the other things that we have to get right is some standards of interop right like even KV cache like why the heck can't I use multiple model families and have KV cache reuse.
Right we've had document standards you and I lived through it right but we've sort of you know you kind of have things that are interoperable in the real world everywhere else so I think this industry also has to wake up and say hey in fact if I were talking about the most important pressing things is how do I have more standards on interoperability how do I have a harness that is external to a model so that my memory is not tied to one model I mean this is the first time you're going to have a technology where your use of it and the exhaust in the data could not be yours. I mean that you know like it's like if I sold you a database and said hey the data you put into your database is not yours and it's mine. It goes away if I took away the license. How would you feel about it? So therefore I think we have some serious issues like that to deal with.
I think that's a good segue. Sorry. Let me just ask one question to connect the economic incentive argument on what's going on. The argument is the Frontier Labs are facing token compression. 50 bucks for OpenAI's kind of million token output versus I think someone estimated DeepSeek's new is like can go as low as 15 cents for a million tokens of output. Let's call it 60 cents. 99% cost reduction. If that is the big kind of economic crux of what the frontier labs are facing, why would most tokens be paying 50 bucks? Most enterprises pay 50 bucks when they could pay 60 cents for most of their tasks. Doesn't that also beg the question, are they in the wrong business model? And I asked this for you as the CEO of Microsoft, what's the right business model? Do you want to be making the frontier model? Do you want to be running the compute and charging for rent on your compute? Or do you want to be in the application layer? I know you talk about this a lot, but I just love your perspective from where we sit today and how this all kind of.
Yeah, I think the fundamental thing that I think we're observing is good old-fashioned competition, right? I mean, for me, if I look back at it, we had like some real great closed source assets, Windows. What was the check against it? It was of course the Mac, but Linux. We had a great closed source product called SQL Server. What was the check against it? there was always a substitute called Postgres or MySQL. So I think that's what's happening a little bit of it is there's real competition between closed source and the open-source check is real. And that's good quite frankly because without it I don't think we're going to have a broad frontier ecosystem or broad diffusion because otherwise we'll just be back to some mainframe lock-in that's just not a thing.
To your point about if anything given that we will now hopefully continue to have a much richer choice in every layer. Right. So to me hopefully we can start building these AI because today the royalty of an AI product all going to just the model layer doesn't make sense if you really want to build a product company right it just cannot be in fact if anything like that's the same thing right which is if you take the database if there was no open-source check on closed source the prices wouldn't have been at a place where people could have built the app tier successfully and with a margin and so I think the apps are going to become you know much more viable economically which is great for the ecosystem. there are going to be all these other layers of middleware call it right which is hey what's my memory system what's my harness and orchestration layer so there's going to be a very rich tools ecosystem there the model companies will do fine in fact you know the power they can manage the token pricing based on their model family if anything I want them to work on even the KV you know these standards.
Such that we can use multiple model families in fact it's better for them in fact I worked on Windows interop with Unix first. In fact, it was counterintuitive, right? We used to think, oh my god, this interop means we'll be less used except we were more used. In fact, we became weirdly enough because there were so many variants of Unix at that time that Windows interop made Unix better and Windows better. And in fact, we were able to penetrate the enterprise primarily because we did that interop work. And so that's at least how I think about it.
Satya one of these we're in this interesting moment where on the one hand you have these experts asking for regulation asking for oversight governance it typically always leads to some restriction of freedom and general society are put in a position where now we have to opine on whether this is right or wrong but then on the other side most people's lived experience is not this magical productivity boost of AI. At best, it's integrating our Apple Watch data to tell us why we're sleeping less. That's like functionally the bar for most people. Or why is my kid into chat GPT? So can you just help us bridge this? I mean, you see so many enterprise applications. Where's the magic? Like where are the gains in profits? Where are the huge upside breakthroughs that AI is creating that will somehow make all of this tension understandable for everybody?
Yeah, it's a great point. I mean, I think this is the real question which is how do we truly see this in the productivity stats? How do we really see it in the GDP growth? That's broad-based. It's not just supply side. I mean the one example that I love and I get back to in fact healthcare is a good one right if you think about healthcare and even the simple doctor patient interaction in our case we have this thing called DAX Copilot that's the place which is the most tangible example I can always point to when a doctor can spend more time with the patient caring for them versus just the entry into an EMR system that's a good productivity gain If it can triage the inbox for the doctor so that they can be more responsive that's helpful for the patient and the care system the administrator in fact keying like the insurer like because it's the triangulation of the pay patient and the health system.
That's of all in fact most of healthcare is sort of all workflow cost. So taming of that workflow complexity that's a helpful thing.
But do you see that in Microsoft with the people that you're helping?
Yeah, absolutely. We see that and by the way even in simple copilot cases, right, which is if you look at the amount most people think about jobs which I think there is going to be displacement there is but the bottom line is what are the new jobs that get created is going to be one of the key aspects of it. But also a lot of knowledge work unfortunately is drudgery right who you know I get up in the morning and I think about like man all I do is email triage right you know what if even just these workflows that are taking away time from things that you could be spending time on.
Okay well you're bringing up this great point if you go all the way back to like the turn of the century the industrial revolution when we had a 7-day work week you know a lot of people forget why did we introduce the weekends it was to sort of manage the tension between different religious groups that had to work in the same factory. And then when you look at long run GDP outside of some exogenous events, it sort of is, you know, between two and 400 basis points. And so what happens is as productivity boosts come in, human work steps back and you kind of accomplish the same amount of work. Do you think that that happens here? Is there a risk that we have a three-day work week and we're just still growing at two and a half%?
Yeah, that's a great question or will we find new things and this is where the excitement at least I have for what the real impact of AI would be is instead of just thinking about hey it has helped me augment some workflow or simplify something that's happening today is it inventing new things is it speeding up drug discovery is it taking the I don't know let's again go back to my example of okay the working capital management of a small business has become so much more efficient that suddenly it's no longer just oh I have an ERP or a QuickBooks like thing but I truly am making decisions based on the ability to introspect my invoices my emails and what have you and somehow optimize my working capital that's productivity that didn't exist and so I do hope that we will start seeing GDP growth which we did see in the industrial era during the first phase of it.
