Pentagon CTO on Anthropic, AI Safety and 'Super Intelligence'

Good to be here. Good to see you both.

Well, this group was formed on the backside of what was really a historic summit that brought the leaders of the super intelligence industry and, you know, the cloud industry, the cyber security industry. I mean, the number of people who were there from Lisa Su to Jensen Huang to Elon Musk and Dario. You name it. It had really quite the roster of leaders from the tech industry writ large. And then specifically the subject was super intelligence and what is the government going to do to ensure that Americans get the benefits of super intelligence, but are protected against the risks. You know, there's acknowledged risks with cyber and other categories. So, Jay Clayton, who's the director of national intelligence, is leading it alongside of me, Scott Cooper, and the head of the FTC, Andrew Ferguson. And so, what we're trying to do and what we did as the first meeting yesterday is start to task out how we start to define the relationship between the government and these super intelligence companies. And I'm in particular responsible for the relationship with the super intelligence companies. I know a lot of these folks.

And for the incident response, like how are we going to make it so that when something happens, the government knows about it and has visibility into it, and then potentially can point resources toward that issue, right? And then Andrew Ferguson from the FTC is going to focus on all the laws and regulations that already exist today that are applied to these companies, so that the American people understand that there's like a robust set of protections they already have. And Scott Cooper is going to focus on critical infrastructure, not just government critical infrastructure, but hospitals, wastewater treatment plants, electricity grid, banks, you name it. And so the meeting yesterday brought all of us together alongside of Susie Wiles and Secretary of Treasury and Secretary of War, among others, to figure out how we're going to split up the tasks, how often we're going to meet, what is the output of this 120-day effort. So it's pretty exciting to be in the room with that kind of a set of people who really care about these set of issues along with Secretary of Commerce, Howard Lutnick. And it was a very successful kickoff to what is, I hope, a good 120-day period.

Well, I was saying that that's what the model companies themselves say about their products. And I think, you know, it is certainly true that there are cyber capabilities that have some risk if someone's going to use them offensively. I think that we can sort of all agree on that. And then we're also therefore in the midst of a cyber hardening exercise where not only the Department of War, but government and businesses are going back on their old code and their old systems and making sure there's no vulnerabilities and so on. But that's what they say about them, but a lot of what they've said has been proven, you know, wrong or is likely to be proven wrong. So part of the other reason for this SI force, if you will, this SIF, is to make sure that, you know, the good news story is told about what AI can do. Because I think it's been a lot of doom and gloom. If you look at the public surveys about how Americans feel about super intelligence relative to any other country in the world, we're the most doom and gloom out there. Yet, we're the center of the world on that point, all right? And obviously there's some narrative here that I think has affected people.

But how do I proliferate this in the Department of War and do it safely? I mean, we do a lot of testing. We do our own like just, you know, there are several parts in the government, including at the Department of War, where we rate models based on their capabilities. So we know what they are, you know, oftentimes ahead of time before they're launched, which gives us insight into what their capabilities are. And I think that's good. I think the American people want that, right? They want to make sure that their national security enterprise understands what kind of products might be out there that could have some risk in them. So, but I will tell you that it's been successful so far. A year ago today, we had 80,000 people out of 3.5 million Department of War employees using AI. And today, we're about 1.7 million. So, and it's just for basic use cases, right? So, it's making a difference at the enterprise level, like making people more efficient, like you would understand as a writer and a journalist. It's making people more efficient in the intelligence community because you could synthesize more data more quickly. And at the war-fighting level, it's baked into a lot of our systems. And again, people assume, you know, they don't understand that warfare is a lot of forms and paperwork and permission and all these kinds of things. Let's simplify the back end of that so that people could focus on the battle at hand.

Yeah, I mean, I remember being in Silicon Valley and I lived in that bubble for a long time. So, I'm familiar with how big that bubble can be. And I'm not talking about a financial bubble, just the information bubble, if you will. And so, you know, the researcher who'd hop between OpenAI and Anthropic, and then a few weeks later sort of came out and was talking about humanity ending in 3 years. You know, I do think, you know, in this kind of information environment, that kind of gets a lot of play. Like, never before have you been able to get 200 million views on something in a short period of time, unless it was so dramatic like that. So, I think people may be living in that bubble a little bit. And you can believe that and also believe that there's some risk with the technology that we have to protect against. Like, those two things are not inconsistent.

And what I think is inconsistent is if these SI mega corps say, 'Hey, you have to stop me because I can be doing something dangerous. I'm, you know, I'm a drunk driver in the car. You've got to pull me over. I can pull over myself, but you have to pull me over so the other guys don't get there first.' That kind of thing doesn't make a lot of logical sense to me. So, I try to take a balanced approach to this, and that balance is with all great technologies. I remember when we were launching Uber and people were lamenting the loss of taxis. And now with Uber, people are lamenting the loss of Uber with driverless cars, right? So, the fear kind of traverses technological advancement, you know, throughout history. So, I take a more balanced approach to it. And when it comes to are the laws sufficient, I'll leave that to Andrew Ferguson to answer. But based on my understanding from him and from my reading, more than sufficient. There's the Computer Misuse Act. And when I say people, what laws govern the internet? They said, 'Well, there's Section 230 or whatever.' Well, that exempted the internet from liability. That was the one law that took liability away from them. Didn't impose any new restrictions on them. What laws were there to stop the mobile phone? You know, so people sort of assumed that the body of laws for computer misuse, fraud, someone, you have a product that you knowingly know that people might misuse because you haven't built the right guardrails in. There's a lot of those concepts that are already there.

Yeah, of course. I mean, there's been something called ITAR for years. If you ask any cybersecurity researcher, well, there was a certain amount of encryption for a while you couldn't export that encryption because we thought it was too dangerous for other countries to have that sort of encryption. So, we have lots of tools, whether it's a restriction on that type of technology being exported because of the advantage we think it might give an adversary, to just straight-up export controls and limitation on foreign nationals' ability to use that software. That's just a few among many things that the executive branch can do. And then you go to law enforcement, whether it's about trade practice, something that violates the deal, you know, a law that the DOJ can enforce on. There's just a lot of them. And I guess I'm not the world's expert on this. I think the FTC chairman is. So, I think you may need to try to get an interview with him. But we've already proven that when Anthropic refused to pull back their model, and this is the company that said you should have the right to pull back any model at any time for any reason. That's what they said. And they said they wouldn't when we found out it was jailbroken, they were export controlled within 2 hours. So, I think you can count on this administration, this president, to act with urgency should there ever be a real issue.

Yeah, well, Anthropic is out of the Department of War as we said 6 months ago they would be. I was, you know, I was with Dario at the White House event a few weeks ago. I talk to all these companies and really have for a long time constantly because my first duty is to the department and for the American people from a national security standpoint. And if that means I'm talking to someone who's suing me, well, then I'm talking to them. That's okay. So I talk to them as much as I need to and they'll probably be more in the coming weeks and months.

No, they're... I'm an adult, they're adults. You know, I've seen Tom Brown many, many times and spoken to him many, many times and expect to be dealing with them just as much as I'm dealing with the other SI companies in a fair, even-handed manner. We're trying to get to common frameworks, but the White House Accord was four common elements that they each agreed to implement. And I think the things that we will be doing in the future, whether it's incident response, will be a common framework that they all agree to, hopefully, about how to report incidents and how to rate the severity of incidents. So, I expect to work that with them fruitfully through this period.

I mean, here is my very, very succinct view on this, which is if you don't want humans to lose control, you can't give models the control over humans. And that whole notion that a model has feelings and it can overrule the founders of a company is by definition loss of control. They are saying, 'We want to lose control to something that may be smarter than us or have a conscience or may be more moral than us and have a conscience to us.' That I don't agree with as a moral principle. And as long as that moral principle is embedded in their software because they attach their constitution, which has that concept against how the model behaves, I cannot have them at the Department of War. Because the Department of War, we have command and control, so we have a constitutional structure that says the commander-in-chief and then he delegates that power down forward. If a model gets in between one commander and their superior and says, 'Well, I don't like what you say despite your commander saying it.' That doesn't work in the military. I don't think the American people want that to work in the military. So, you know, if that would happen in the real world, you would have to quit your job and you just leave. You're like, 'I don't want to do what my boss is telling me to do.' In this case when you have a foundational technology, that in a time of need, you may not know how it's going to behave. That just doesn't work. So, until they, unless they change that, there's no discussion.

Well, I mean, that's why you have terms of use in a contract and you have either a constitution or you don't. Or you have, you know, you hear what Sam Altman says fairly regularly. It's like we, you know, we are governed by democratic process. We live in a democratic society. They're an entity in that democratic society. Elon signed the same terms of use, which were for all lawful use cases, we could use his software. Google signed the same thing. So, really the outlier are the ones that say they want to impose their views on the Department of War, not the standard.

Yeah, I mean, the case in California was at the district court level. The one in DC that we won was at a circuit court level. So, that would be the controlling precedent in our view. I'm sure that they appealed that already. I think we'll win, and you know, but you know, the legal processes are legal processes and I think everyone was surprised that we won. That was clear to me that we were going to win, but we'll see.

I mean, I got to tell you that, and I called it historic for a reason and I don't throw that word around lightly, but not just the amount of market cap, the amount of brains that are in that room and then the fact that they all came together and agreed on this compact including Dario, including Elon, including Jensen, including Sundar was quite awesome, right? And so, if you could get, you know, and I remember I kind of joked to Elon afterwards. I was like, that was great. He's like, what do you think was great about it? I said, the fact that all you guys agreed. Like, you all have your different competitive battles, the fact you all came to agree and said, here's a good safety framework start that we all can sign up to, was awesome. And so, like if that ethic and that sort of cooperativeness continues, I think hopefully everyone will be all fine. There won't be any divergent views on any one of these companies.

Yeah, well the three things out of the SIF are this incident response framework, this, you know, how are you going to make sure critical infrastructure is hardened, and then what laws and regulations we already have that apply to these kinds of companies in this new technology area. Separately, the David Sacks-led advisory board called PCAST, I forget what it stands for at the moment, they're going to do a study on any recommendations that might exist that are necessary for legislation. You know, there has been legislation proposed by this administration a while back that was to sort of preempt state rules, have a national framework, it was going to address some of these issues. And I think we're just taking, you know, David Sacks' study is going to take another look at alignment and some of the key technological sort of things that people worry about, study them, and see if there's anything more necessary to recommend to the president to do based on that.

I don't see this... this is a very, very Silicon Valley bubble sort of question. Not, I'm not accusing you of that, but just, you know, government and business merging. You're like, okay, well, what does that make us? Like an authoritarian capitalist state or kind of what that is, but...

No, I understand this. You know, I've seen interviews where people talk about this merger of government and SI companies, but I think if you see what's come out of Meta in the last few months, you're like, wow, this is now a frontier AI company, SI company. They've picked a certain vector for consumers. It's wildly popular. They delayed it because it wasn't as safe as they wanted to be with your data. And now it's a great product. You're like, well, that sounds like a great model. Like what, you know, what... yeah, and why presuppose that something else is going to be used different? And you know, there's lots of solutions if something is going to be used different. Let's say someone had a bio-capable model to help do biological research to do disease research, right? Or disease cures. You know, maybe you just do a know-your-customer thing. I don't know. Maybe they want to know who they're selling it to, these companies, right? Because they don't want it to be misused. Just like you know who you're selling a reagent to at a lab, right? You don't sell it to, you know, the guy at the bodega. You sell it to someone who might use it in a way that you think is appropriate. There's lots of ways this could go, but a big merger between the two will be not only market destroying, where will the talent go? I mean, there's so many bad implications of that that we've learned throughout history of state-run enterprise.

I think if I'm one of these companies that's worth a trillion, two trillion, going to three trillion, four trillion, I'd want to... I'd be spending a lot more on safety and making sure that my products weren't misused and making sure that I was serving the national government in the right way because this government is protecting my interests. And I think you'll probably... I think you're more likely than anything to see a shift in their investment toward safety in their training and their sandboxes and toward serving the government in a way that's productive.

I'm not in the loop on that.

War, not equity.

I mean, we've taken equity in Intel. We've taken equity in some of the deals, frankly, actually in the Department of War where we're giving a low-cost loan for a critical minerals mine and we're trying to get the taxpayer some potential benefits. So, there's lots of places the government has historically taken equity and will in the future. So, I don't think it's sort of a crazy notion. But you know, I'll leave that to the president.