I worry about other countries using AI to take humans out of the decision-making process. They don't trust their generals.
If you were so close to being willing to work with them, then how could they end up being a supply chain risk?
It's just we don't want them in our supply chain. We don't want to use them. The President decided that he doesn't want the government to use them. If I went back to my office right now, I was like, how would I order a pizza from outside to be delivered in? I'd have no idea.
So, you're not a believer in the Pentagon pizza index?
I'm not a believer in the Pentagon pizza index.
We're here at the Pentagon because the AI story that we talk about on this show has escalated quickly, very quickly into a core national security issue. And you saw that of course when the Pentagon banned Anthropic earlier this year. So let's talk about it with Under Secretary of War Emil Michael and speak with him about how AI might change the future of warfare and how it might already be doing so. Mr. Under Secretary, welcome to the show.
So AI's capabilities are increasing exceptionally fast and you're the man tasked with implementing them at the Pentagon. So I want to know from you, how is AI going to change war? How do you hope it will change war?
I think one of the analogies I like to draw is having been at Uber, and you look at an autonomous vehicle, and people were scared of Uber from taxis, and then they were scared of autonomous vehicles from Uber. But in reality, if you look at FSD from Tesla or even Waymo, the safety statistics are amazing.
Yeah. So, and it's like people are afraid of the change, but the change is better than what we had. The same thing with Uber. People were afraid of the change from taxis, but it made service more reliable. There was less drinking and driving, more availability, more reliability. So if you would apply that to the war context, you could do much more—be more precise, be more specific about what you're going after, what you're defending. You know, and the precision is really what's interesting to me. Because if you can use AI to detect and discriminate—and by discriminate I mean discern a decoy from a non-decoy—you could be more precise. And the example I always give is like drone swarms coming at a military base, and you're trying to determine what are they, are they armed, are they not armed, what are these things, how do I deal with them? Well, some of the visualization, these models can help you do a better job of taking them down or not taking them down because they're not a threat, where one human can't really absorb multiple hundreds of inputs at the same time and make an action, make a reaction that's as precise.
Yeah. I want to make this concrete for folks, and recently the public has been lucky because, you know, in a world where sometimes we don't get the most transparency into how this technology works, we did get a demo, and this came from Cameron Stanley, the Department of War's Chief Digital and AI Officer. And he showed what a program called Maven Smart System, which is the Pentagon's core tech platform, looks like, and I'm pretty sure it was called Target Workbench. And this is where they select targets and then end up going and sending—it seems like they use the word action for them. My understanding is they end up going and sending the attacks to these targets through the system. So the way he described it is it's this single unified visualization that allows you to look at live images and then be able to select targets.
Well, that, and then imagine the context around that. Where are my assets? Where are my planes, my boats? What might happen if you took that action? What might be the reaction? Subsuming all that information but still having a human make the decision at the end means that you're increasing the human context window, is one way to think about it, right? When you talk about context windows and AI, well, think about a human that's trying to absorb all this information and make the best decision they can. If you could synthesize that information so they can make that decision, and you're using more sources, by definition almost, the data and choices are going to be better choices.
Yeah, and he showed it in action. You're seeing this data overlaid on this map, and then he says when you find something that you want to target and you'll see the information, he says—it's very interesting—he says left click, right click, left click, and then it ends up in a targeting workflow.
What's happening in those clicks?
I mean, you know, what's happening in those clicks without knowing exactly... I mean, it could be everything from if it's an air—let's say it's an air-oriented thing, there's an F-35—it could be what's the weather, what's the drag, what do I have on board the airplane, what am I going after, what is the collateral, potential collateral effects? I mean, it's again, it's less giving you one example of how that might work and more just imagining how much input you could have into that decision when you have a computer basically able to gather that information, help you synthesize so that you can make the right choice.
Yeah, it's interesting. He shows that there are toggles that the military can select, whether it's an optimization for how much fuel you want to burn, what munitions you want to use, the distance that you need to travel to hit the target, and then you can optimize.
Fuel, weather, you know, where are other assets that the adversary might have, and where and how might they react. Just the amount of information that you could absorb is almost infinite. So the idea of taking one person and giving them the power of 10 people makes them better at what they do by potentially an order of magnitude.
Yeah. And then within there, once that's in the workflow—
The last step is whoever's looking at it can act, assuming they have the permissions, can action on that target, which means send the assets.
Which they're doing anyway. So you have people whose job it is to do this right already without any computers. It could be with paper and pen. It could be with whiteboards. It could be with PowerPoints. And now you're accelerating that and giving this person the power of more tools so that when they do the right click, left click, right click, or left click, right click, left click—some of those clicks—they're way more informed, and then you're going to lead to better outcomes.
Right. And it is interesting to see what's happened with this digitalization, whereas before—this is from Pirate Wires—they say by the start of the conflict with Iran this year, targeting processes were connected with PowerPoint, email, and Excel files. I'm paraphrasing: target lists were relayed in spreadsheets, sequence maneuvers sat in Gantt charts and PowerPoint.
I mean, that's probably the case historically because when did AI models start to become generally available to consumers, right? We had the ChatGPT moment in '22. And then you say, when was it available to enterprises? And then when was it available to government on the networks that government uses for warfighting? And you're talking about a fairly recent phenomenon where these tools are even available. And then we went through protocols, safety testing, the modeling and simulation for how would you use this in a conflict. There's a lot that leads up to actually using it in a way that we feel responsible for.
What's interesting, I don't see an LLM in there. Are large language models or today's generative AI layers baked in that system?
Yeah, I mean, the genesis of what Palantir does is an orchestration layer on top of data streams that we—data that we put in it—and say, here's the data we would normally use for any battlefield operation, plus an AI to help you synthesize it. So all those things are combined, and they provide the visualization.
But there's not like a chatbot on the side window which is like, "Lay out a list of targets that I want to hit. Here's my objective is to win this war. What are my targets?" It's not a Skynet thing.
No, it is a tool like any other tool that you might have on your computer, or in your war room, or with your team, except it's on your computer visualized, where you still have checks and balances. You still have to get all the authorities you need to do anything. It just surfaces the choices in a way that's more consumable, if that makes sense. And it's good to have this discussion because I think, as this is a fast-moving technology, it's good to be able to talk about it so everybody understands how this works.
And I think that this is again going through some of the—
What's actually happening versus misconceptions.
There's been some talk, and we're going to get into the Anthropic situation in a bit, but just to talk specifically about what an LLM can do in this process: there's been talk that the LLM was involved in the kill chain. But that is not exactly what the LLM has been...
So let's talk about the extremes. And I talk about this in the way we're deploying AI in the department. There's the enterprise corporate level: tons of PowerPoints are generated in this building, memos you couldn't imagine, it's like nothing you've seen in the corporate world. And that could all be made more efficient, and that's sort of the mundane work that people would prefer to do less of so they get more interesting work. Then there's the intelligence layer, which is imagine all the intelligence we gather from satellite imagery all over the world. How do you synthesize that? So right now you have to have a human analyst look at everything and make a judgment. Imagine you had the historical data of all satellite imagery. Then you could look at it and say, "This is an anomaly." And the AI can learn what it was, so it could tell you what the anomaly detection might be, which is a totally different paradigm for intel analysis, if you will. And then third is for warfighting, where it could take all the paperwork and modeling and simulation, all those things, not only be able to have you react faster, but react in a more precise way. And those are some more tangible ways of AI. And that's why I think if people understood that better, particularly in Silicon Valley, they'd say, "Oh, that makes sense, like any big company would do or any big organization: efficiency, how do you be strategic about what you're doing and allow more analysis, and then know how to use it to execute on whatever operation you have in front of you."
Yeah. And this is a big reason why we're here, is I wanted to speak with you because I read so many stories, and they didn't comport with what I was hearing from people close to what was happening, and I thought let's—
"Lay it flat," we say here in the department.
That's right. So just to confirm, the LLMs, what they're doing is they're summarizing different reports—
Synthesizing, interpreting, taking in different forms of data and giving you alternatives.
And most of these are very mundane because, again, you have to imagine that every single thing that the military does has to be audited, has to have the right command and control structure—like who's authorized this and that, has it been checked through the legal system, has to comply with all our internal memos about ethics and sort of the laws that we follow in conflict. And that doesn't change; it's just the tools to make that better and easier, if that makes sense.
Now, there's an argument among those who watch this tech in action that sometimes a little friction is better, right? That was the one thing that made me feel somewhat uneasy when I looked at this Maven Smart System demo, is like, maybe we want the Excel spreadsheets and the Word docs and the PowerPoints when it comes to something as serious as making a decision to attack a target. Like maybe you don't want to make it that easy, because the easier you make it, the easier it is just to hit action and send it away.
Well, the friction's there regardless. This is a key point: you have the same rules of engagement, the same approval system. What you now have is better aggregation and synthesis of the data that you would already use to make that decision. So it's partially about speed, but it's more about more data points, right? So if you think about it as we're taking as many data points as we can to make a better decision, yes, it's going to be faster if you were going to go hunt and peck for all those data points, but there's no military in the world that doesn't believe in speed. Speed wins the game. Look at what happened in Venezuela: the speed at which the execution of that operation happened meant that we didn't have any casualties on our side. That's amazing. If you had to spend way more time, you weren't able to synthesize information as well as one could, maybe you had to be there for 48 hours instead of 3 hours, right? So you think about that: speed has to be one of our prerogatives, but better information is the goal so that the decisions are more precise and more consistent with the operational objective we've got.
Is there a limit to what this can do for you? I mean, I'm thinking in the context of the war with Iran. Obviously, there have been many air strikes, lots of them quite precise, the entire echelon of Iranian leadership taken out, but the IRGC is still in control. There's a new Ayatollah with the same last name. So, isn't there a limit?
Yeah, there's a limit. I mean, I don't believe that there is some all-seeing, all-knowing answer to human conflict, which has been happening since humans existed, right? I think that ultimately what you want is clear objectives. You need the manpower and machinery to do it, and you want to do it with the least cost, with the least amount of damage and the quickest time, right? That's the goal. And I don't think AI or really any technology becomes the answer. It's just one of the tools.
Yeah. And that's sort of one of the fundamental questions here, is does AI just become something that is a speed up, is a friction remover, or can it fundamentally change war?
I don't worry about that from our side because I believe the way the United States has structured our command and control is you have a Commander-in-Chief in the Constitution. He appoints a Secretary of War who's confirmed by the Senate, and you have the generals and all the ranks. So all the procedures to make sure that decisions we're making are the result of a democratically elected leader and a Congress that finances these things. I worry about other countries who don't have that using AI to take humans out of the decision-making process. They don't trust their generals because of graft, because they don't have the expertise, and they start to use machines in place of humans as opposed to using machines to augment humans. So that's more of a worry for me. And I think that one of the things I've tried to explain to some of these companies is think about the alternative: what would an adversary want to do with AI that we wouldn't because it's not consistent with our values? And we have a chain of command, a constitutional government. If another government doesn't and wants to use AI to eliminate human risk, where we're looking to augment human capability, it's a totally different way of thinking.
Which governments are you referencing?
I think if you think about the biggest military buildup in world history in China, and you think of—we've seen the purge of the generals and sort of the military hierarchy there—you start to wonder, well, how do you replace all these people? What is the command and control? What would your AI strategy be if you were running that country relative to ours? It's just a different mindset. And so the uses that we've talked about right now are largely—when we talk about LLMs in this world—largely chatbot uses, or I put them in the chatbot bucket, right? You have information, you synthesize the information, you get something that saves you time to make a decision. But now the AI industry is moving towards agents, which is, like the word connotes, letting the AI take some action for you. Do you have a plan for agents here? Is that where this goes?
I think not for things that require human judgment, no. Again, you have to have an endpoint where it ends with human oversight and human discretion on the most consequential decisions, right? But you could imagine scenarios like I described, with a drone swarm coming in at a military base at night, and how do you deal with that? But again, that's not an agent use case per se. That's like a visual discrimination or discernment use case, and maybe you have a directed energy laser that could take them down, and it's a lot cheaper than the alternative, a lot safer, a lot less collateral damage. But in terms of agents, we have some agent pilots at our enterprise level. Remember I was talking about the enterprise corporate level, just to do the mundane things we have to do every day. But those things are not sort of where we're at at the warfighting level.
Okay. So, if I'm hearing you right, basically the plan here is not to automate warfare.
But the question is here, if you have your adversary who's doing that, let's say you're in a direct conflict—maybe it won't be China—can you really afford to sit still and do it by the book? Because that's the worry, right, is that these capabilities are out there, they're integrated, and it becomes tempting to go into, let's say, a Maven Smart System and say LLM is getting me 99% of the way there, just finish it off.
I understand. I'll tell you why: number one, that's the reason that the US has to be AI dominant, so we're never faced in a position where the counterforce AI is better than our AI and therefore we have to face those choices at all. Secondarily, people confuse automation with some sort of automated army, right? Automation I just described to you in the drone example. What about an automated minesweeping or mine detection operation? There's no human underwater. Do you want to find the mines? There's no human involved at all, but there's an action you want to take to do that. Well, everyone would say, "We don't want mines on our shores. Sounds like a good idea." Or there's a missile coming at you and you want to take it down from space, like Golden Dome, like we've talked about. How do you do that, right? You have to do that in 90 seconds from when it's launched. So those kinds of things, in the most extreme circumstances, you want humans to be able to rely on some automation capabilities. But in terms of mobilizing a whole army or whole fleet of jets or whole fleet of suits, that's not in anyone's mind. And there's a 35-page directive at DoD that talks about human oversight and how we manage these systems, and we're constantly updating that and making sure we have the right controls on it.
Yeah. One more thing about LLMs. One thing that I heard is that they could be useful potentially in being another layer of data on top of strikes before they happen. So for instance, the school in Minab, Iran, where there were markings outside of playground and hopscotch outside, maybe an LLM in the future, if something like that becomes a target, can basically flag it and say, "Hey, maybe don't shoot here."
Yeah, this is the point I was trying to make with the driverless cars: if a driverless car ends up detecting a jaywalker better than a human, isn't that a better option? So when I say it's there to augment human decision, it could be on the front end or on the back end, which is check and make sure this is something that we want to go after or hear warning signs. It works both ways. But ultimately, humans have to make the decision. That's the end state. How that decision is contributed to at the GLMs, especially the ones that are trained on visual—and, you know, Google has your Nest Cams, it has YouTube, has a lot of human movement—all these things have different datasets that they're trained with to some degree that are proprietary, could be very valuable. So that's why "LLMs" I think is going to go away as a term, because they're not large language models only; they're visual, they're going to be used for robotics, they're going to be used for a lot of things.
Yeah, and that's the general side of the whole AI part.
Let's talk about drones briefly. You brought it up a few times. I feel like it's worth discussing since it's part of your remit. Very interesting uses of drones in Ukraine right now, and we saw, I think, unprecedented uses of drones in the Iran war. Different use cases: one is an air war, one is a ground war. What are the main things that you've learned watching this in action, and how do you think it changes, again, the way fighting might happen?
You're right to point out two different scenarios. In Russia-Ukraine, you have a battle over territory, and so that battle over territory where the lines are drawn means that with the drone warfare, the robots are the front line and the humans are back. And the idea is, well, why risk a human going front if you could send a machine first and see if you could fight it that way? Still a lot of destruction and death there—that's obviously sad and unnecessary—but I don't know how much more there'd be if you had a Civil War-style thing where you have humans on humans. In Iran, the drones—I think the lesson from that is the imbalance of costs, right? You have a cheap drone going against very expensive targets.
Right, and also millions of dollars to shoot one of those things down.
And to protect your exquisite targets on your side against a very cheap drone, you have to use expensive countermeasures. And so the lesson there is how do you turn the dial from maybe we should have more mass-attritable weapons, like drones or counter-drones that are affordable so that the cost ratios are similar, as opposed to a country that can afford cheaper stuff being able to threaten expensive assets on our side. And for me, that's been a big push in this department, which is how do I bring what we call mass-attritable weapons that are not exquisite, that can be delivered quickly, that are designed for manufacturability, that are cheap, that you can afford to lose, as opposed to the big stuff that we build that takes 10 years to build that costs billions of dollars.
Yeah. So let's tackle both of the ways that the US is working to head off these threats. Let's start with the bigger drones, shall we say. So there's a program here, LUCAS, right? Low cost, $30,000 a pop. You can send them out. And do they crash into other drones? What's the point of... or do they do the same thing that the drones do?
The idea is that the Shahed drones that the Iranians had—a one, we call it a one-way attack drone, long distance, can go fast, but cheap to manufacture. These are sort of that. And they can do a lot of things: they could be defensive, take out other drones, or they can be offensive. And they're designed to be cheap to manufacture. If you lose a couple, you're okay, right, just from a financial standpoint. And they're used in the same way, in theory.
Are we working with the Ukrainians on this project? There were some headlines that the Ukrainians offered to help and we turned them down. Now what's the story there?
There's two levels of this: sort of the grand United States-Ukraine relationship, but we just launched our Drone Dominance Program, and I think there was two Ukrainian companies in it that were going to onshore manufacturing here and take some of the learnings with them here to help us with our smaller drone scenario. And so we're sort of agnostic to that, but we want to divest of supply chains from adversaries. So that is one of the requirements, is that the drones that we use at the Drone Dominance Program don't have a dependency on adversaries.
Okay. And that touches on the smaller drones, the ones that are being used in the land war, the DJI-style drones—
Right, exactly. China has been putting on these displays, epic displays of drone art in the sky—
Drone swarms, right? To call it what they are. And at first look, it's like, man, China is really innovating on fireworks, but then you realize this is completely a military simulation.
Could be. I mean, I think that's the scenario that I've tried to explain, and I do think it's something that these AI companies understand once explained to them. You could say, like, you see that drone art displayed that you saw—imagine those were armed drones and imagine that they were communicating with each other and they could therefore form and reform in ways against your defenses. How do you defend against those? And depending on where you are, you may have a fully defendable garrison, let's say, but let's say it's a small military base, let's say it's over the border. How do you deal with these things? And that's something that's a new challenge that wasn't present, or we weren't thinking about before the Ukraine-Russia war.
So what is the answer? Is the US working on the defense side of that and on the offensive side?
All the time, right? Drone Dominance has both elements to it. We have a counter-UAS (counter-unmanned systems) task force that's looking at everything from lasers, directed energy—which is one of my critical priority areas—to how do you do electronic warfare on these things to take them down? They all run in some way, so there's lots of different measures and countermeasures. That's what makes this time in the department so interesting: the danger of warfare is changing, technology is getting more capable, the actual ability to access technology is becoming cheaper, and the need to have these systems interoperate is never greater. Because what is a drone swarm? It's a set of interoperable drones that work together, and you could see them in the sky like you're talking about, and you can imagine what their military utility might be. So the tech problems there are super interesting, right? They're hard, but they're interesting.
Now, briefly on the cyber warfare side of things, I imagine AI could really impact that side of warfare.
It seems so. It seems that models that are trained on code can learn vulnerabilities in code, is what these companies are saying, and that presents risk and opportunity. But yeah, I mean, obviously, what you've heard in the news yourself and that's been released about the cyber capabilities that are almost here are certainly going to come from every frontier model company at some point, certainly going to be tried to be distilled by the adversaries, and are going to be the next wave of innovation from these...
from these companies. Okay. So, it's clear, I think, from the beginning of our conversation that AI is becoming critical in what the Pentagon does. It's helping synthesize information in some areas. It's helping with targeting. Clearly, you need it for drones and you also need it if this is going to be a new cybersecurity front.
So, I want to talk about how you pick the AI vendors. So, we're going to talk about the situation with Anthropic and then a few other topics when we come back right after this. And we're back here on Big Technology Podcast with Emil Michael, the Under Secretary of War for Research and Engineering. Emil, appreciate you being here with us. Let's talk about Anthropic. So, I just want to hear from your perspective: describe the culture of Anthropic versus the administration.
Well, I would say this, which is they were the first to aggressively try to provide service to the government after the Biden administration's executive order about AI, because they were—and again, you see this in the marketplace too—OpenAI was more focused on the consumer with ChatGPT and the subscriptions. xAI sort of hadn't been started really until 18–24 months ago. And then Google was also focused more on the consumer. They were focused more on enterprise. When I say enterprise, I mean enterprise at large: an enterprise like the Department of War or an enterprise like a big company. And so they naturally started sooner here.
And I think there's a certain portion of people at all these companies that all now have a government division that are all going to start understanding the vernacular a little bit, and we can have conversations like you and I were having about what are the meaning of some of these advancing capabilities of the world. But yeah, from a culture standpoint, I think we think about, you know, we live in the bureaucracy of what we have to do every day to innovate and to reform. And I think the image that they might have—and this is not unique to them—of the Department of War or the administration, is that we don't have the safeguards that we do, that we're not paying attention to sort of the risks. We are, if not more so than most Americans would understand that we do, because of the procedures that have built up over decades and decades of being careful and smart about what we do. And so that culture clash, to the extent as you call it a lack of understanding, a lack of confidence, lack of trust in us and our ability to do things in a way that's consistent with our values as a country and the laws that are passed—and that is how I describe the difference.
Okay. And just to recap what's happened between the Pentagon and Anthropic recently: they were in Maven Smart System like we discussed. There were all these provisions in the contract that the team here didn't like. So there was a renegotiation. It almost came together, but there were two things that Anthropic wanted to include in the contract: a provision against mass surveillance, a provision against autonomous warfare. And ultimately, there was not an agreement there, right?
There was, although I would say the following, which is just clear: the provisions that were in the contracts that ultimately served the Department of War said you can't use it for planning kinetic actions, you can't use it to develop weapon systems—so all the science, engineering, aerodynamics, all—
