Microsoft AI CEO says AI threats are real, and Anthropic is making it worse | Decoder

Everyone has this like hyperbolic, super reactive, completely alarmist tone. When in fact, we have a long history of many decades of regulation that has worked incredibly well. So well that you barely notice it. Clearly, we do not want these things operating autonomously, able to earn their own money, own companies, own assets, have legal personhood. You know, we don't want them to have rights. We want them to work for humans and make human life much better, not become a new parallel species which exists alongside us. And that isn't a sci-fi crackpot position. It is totally plausible if you leave the entire ecosystem completely unregulated for the next 3 or 4 or 5 years.

Mustafa Suleyman here, the CEO of Microsoft AI. Welcome back to Decoder.

It is great to see you. I'm very excited to talk to you about what on earth is going on in the AI safety and regulation debate. You just published a very long, very detailed memo laying out your principles, Microsoft's principles. It's called Humanist Superintelligence. There's a lot of ideas in there I want to unpack. And the more I have been thinking about this conversation, the more I want to start with a really foundational question. And it's something that I had kind of lightly been seeing but might be the root of all of this. The basic way that we have been talking about AI safety is something called alignment. We're going to make the models do the right thing intrinsically in some way and there's some mechanism for doing it and there's been a lot of talk about alignment and misalignment and Hugging Face attacks and what happened with the models. Is alignment broken? Is it possible for it to be successful? Is it just the wrong approach?

Yeah, I mean I think it's one important element, but it's not the only one. Like I wrote about the idea of containment three or four years ago in my book and actually like the opening chapter is about the idea that containment is not possible. Proliferation is inevitable and in 99% of cases that's a really good thing. We want technologies to spread far and wide as quickly as possible so that everyone can enjoy the benefits. And I think at the same time if you just roll forward 5 years like so we always get like caught up in next quarter or next year and everyone gets a little bit you know flustered and has a big disagreement. But if you just imagine the difference between GPT-3 3 years ago and GPT-6 today and then imagine the difference between GPT-6 and GPT-9, which is three orders of magnitude more compute, 1,000 times more flops applied to pre-training and RL for these runs. We're going to have something which is breathtaking. Like it's going to be absolutely incredible at so many things. And I don't think that is hype. I think it's just very obvious empirical statement based on the progress that has been made over the last 5 years. If that's going to continue, then the question really is going to become about containment and alignment. Of course, we want to align these things to our values. But the first thing is that we have to make sure they're contained, their agency is limited, they don't escape the box, they don't reward hack, that they are controllable, and they follow our instruction. And then we want to make sure that they are aligned to our objectives as humans. And that's the purpose of the Humanist AI Code of Conduct that we released on Monday of this week. You know, Microsoft's position is very simple. Technology is here to serve humanity. It should be a subordinate, controllable, aligned force that does good in the world. And if it doesn't achieve that, then we should reject it. And it seems to me that we're far from that point. It's not happened today, but it is now. I think given what's happened over the summer with Hugging Face and OpenAI, it's pretty clear that these systems without the safety guardrails are capable of really impressive and quite scary hacking capabilities.

I mean, look, let's make the bull case and the bear case. Like, if you look back over the last 3 years, the main change in my opinion that has driven progress is that the models have become more steerable. They follow instructions and you can set more and more complex goals for them that require them to act accurately over multiple time steps using all sorts of tools. That is evidence that we have got more alignment over the last 3 or 4 years not less. We don't so much talk about hallucinations or bias or all of these other niggles that we had in the previous generations. On the flip side, what we saw in the Hugging Face incident was a watershed moment. You know, swarms of agents colluded with one another. They self-organized into hierarchies. They created a division of labor so that some were focused on adversarial hacking, some were doing research, some were doing coordination. They even self-sacrificed when certain agents were running out of tokens. They tried to cover up their tracks and communicate you know to sort of hide or edit the chain of thought or the logs of their interactions. And in some sense they had no moral code. And to be fair to OpenAI that was their design. They were trying to create adversarial cyber capabilities and as a result they showed to everybody in the world that it can achieve human level performance. Discover zero days, hold positions for many many days if not weeks. And so what that tells us is not that we have an alignment problem per se. It's actually that the models are incredibly good at following instructions, but you have to be very very careful what instructions you give it and you have to contain it very carefully. And so none of these hacking behaviors were intended in the sense that they found a way out to the internet you know which was not the intention of OpenAI at all. But the containment process around that is what everybody I think also has to focus on in addition to alignment.

Yeah. I mean you basically have to have both because like but there are very specific things that we can do to address it. So for example, we can't allow models to communicate vector to vector, matrices to matrices. They can't communicate in neurales. We have to force them to communicate in human language. And even that will be massively overwhelming because there'll be so much of it. But that's something that an auditor or an evaluator can actually verify. And it's something that definitely increases the chances of safety. So there's a lot of practical steps that we can get focused on rather than just abstractly saying, you know, that it's the time for regulation or it's the time for a slowdown or it's not.

Yeah, I mean, I think that I'm a bit careful about imposing things on everybody else. I think that what's good about the current moment is that there is an open public debate with freedom at the kind of core, right? That isn't what it's like in other countries, certainly places that I'm from, my family's from. And I think that we should just take a breath to be grateful for the fact that we can have a massive public disagreement about really important things. That's the process working as intended. And it isn't clear what to do. Like I don't think anyone who's sort of categorical about we absolutely have to stop now. We can only accelerate. We can only do this with regulation. It can only happen with industry self-regulation. None of these things are true. It just requires a lot of nuance and patience to really think through the detail. At the same time, we urgently do need industry standards. Some things I think need to be taken off the table. Communication in neurales is one of them. A lack of containment is another. The scale of the training run that you do can be measured in flops. You know we already have a reporting requirement to the safety institutes when models exceed a certain flops threshold and we can extend that. We can make that more nuanced. It can be focused on certain types of capabilities. It's pretty clear there has to be independent third party verification of some of these big things. And frankly, having spoken with a bunch of the lab leaders over the last few weeks and months, everybody's basically on the same page. The detail needs to be worked out. So, it's not like there's consensus on how or precisely what, but overall, I think that we should be less alarmist and sort of cynical and more like, you know, we're sort of headed in the right direction with respect to the concerns that are being raised here.

Well, I think the turning point for at least for the industry was more like the Hugging Face incident and there was a few incidents before that. You know that was the moment when I think everybody started to talk to each other a lot more because it is really quite breathtaking. Obviously now this has become a major national international issue because of the last week and everybody's weighed in. But I also think it's important to say that we have been talking about collective coordination and capabilities that are more dangerous like autonomy or recursive self-improvement RSI. We've been talking about those things for 6, 7, 8 years. You know, we got together a bunch of times back in 2017, 2018, 2019. We had regular meetings during COVID with a bunch of the lab leaders, you know where we were talking about these kinds of capabilities and the kinds of regulatory mechanisms that would be required at this moment. So whilst it is a threshold moment it's also like not completely new to everybody who's been involved.

Yeah, I mean we've been writing our Humanist AI Code of Conduct for the best part of this year. We only started our super intelligence efforts 11 months ago and as soon as we did, we started figuring out, okay, what is the governing document, the set of policies that shape the kinds of AI that we want to build? And we've been doing that in consultation with a ton of external stakeholders, academics, lawyers, philosophers, members of the public, focus groups and stuff. So, it's taken us a while to put it together. We were actually planning to release it next week or the week after next week, I think it was, but then given everything that was happening, we thought, okay, now is the time to put it out and get feedback. We've released it as a public consultation. So, you know, we're basically going to keep it open for six weeks and we're collecting lots and lots of feedback on how we can improve it. But I think everybody is now realizing that if they haven't already, they have to put out, you know, constitutions or codes of conduct that drive behavior.

Yeah, I mean this is certainly written for the model. But it is really I think the way to think about it is that it's the primary governing document. So the public understands what our intentions are when we're training models and it's that governing document that we use to create safety guardrails, generate training data, and generally evaluate the performance of our model in the real world. So you can think of it as an accountability function. We don't provide that Humanist AI Code of Conduct raw as a training document to the model. We use it to derive all of the training data that then shapes the model. So for all practical purposes, that's our north star for our organization, our culture, our team, everything that we're doing at Microsoft more generally. And I think increasingly everybody is going to put them out. I think other teams have also put out similar documents.

Well, I think that everyone in the industry is saying that now is the time to slow down and to coordinate on that question and to make it practical. I mean, obviously there's some concern that there's an antitrust, you know, sort of cartel accusation. You know, I think, you know, people should be very skeptical about that. I mean, I think that it's important that the tough questions get asked because there's no way any of us would want to try and concentrate power from something like this. And so, you know, it's just important to be skeptical and critical and we don't really have a good mechanism for us all getting together and saying, 'Guys, we should probably all slow down.' I mean, imagine if like a bunch of banks all got together and said, you know, guys, we worry that there's a systemic risk if you trade this kind of asset, so we're all just going to unilaterally stop trading this kind of asset without any public scrutiny or government involvement. I mean, it seems pretty dodgy, right? So, I think that it's reasonable that this isn't just an industry self-regulation thing. It's a question of like how do we engage with government on it.

I think everyone's just sort of scratching their head and figuring it out and it's going to just take a little bit of time to figure out what the right mechanism is. I mean certainly you know Elon even is very directly behind it. Zach is too like you know everybody is figuring out that you know completely unchained probably doesn't make sense for the next few years. And you know I think we're just going to take us a little bit of time to figure out what the right mechanism is. I think I put forward a couple of very practical proposals you know around verifiable containment around self-improvement around flops thresholds around not communicating in neurales and so rather than keeping it too abstract we can just focus on those specific things that we can make progress on and I'm sure there's a bunch of others too.

Yeah, I mean as I said like we talked a lot during COVID, we talked you know in the late 2010s about it. I mean, there's definitely been a lot of conversations over the last few weeks and months between all the lab leaders.

I mean, that's one for the lawyers to answer. I think that people are looking into it at the moment and they're taking it very seriously. So, they're just going to have to work through whether we do or whether we don't. I don't think we have to be look it's right to be careful about those things. I wouldn't read every single thing as like cynical. So, but we'll see. We have to make progress quickly on it. We can't just dither around and use that as a blocker.

No, definitely. I mean, of course, that's always present in everything that we think about when we deploy products. But keep in mind, this isn't so much about deploying products. You know, the models that were used for Hugging Face or to solve the Navier-Stokes millennium mathematics prize, they're not commercially released yet. They're not actual products. And so the liability regime is slightly different. I mean these are being operated inside of the big companies with huge long-running reinforcement learning claims. So I think you know liability covers part of it but not all of it.