Trump Likes Mandatory AI Regulation for America but Not China? + Anthropic Defeats Distillation

Well, I think there was an incident. Now, this was not a Hugging Face type incident. This is not autonomous AI agents collaborating on a scale of hundreds and thousands hacking open websites and company infrastructure. No, this was some autonomous AI agents from Anthropic filling out visa forms on a State Department website but not submitting them. And it was filling out a tip to the Philadelphia police about a homicide that was obviously made up and immediately marked as spam by the Philadelphia police. But the White House took this event extremely seriously. So even though the severity of the incidents nowhere near as severe as Hugging Face, it is still the kind of misaligned AI systems going beyond their instructions, going beyond in this case some of the safeguards that were trying to be imposed. And this was another instance of Anthropic testing their models to make sure that they're going to behave as intended. So this is not a cyber test, but it was a different kind of safety test. And now we've had these results that the White House did not like at all.

Yes. So this is a really remarkable statement from the Super Intelligence Force. I think it is a landmark in AI regulation policy in the United States. Now, you heard me say before that everything Trump was saying in the leadup to the summit with Xi Jinping was not necessarily a good marker for what his administration's behavior was. He said, "We weren't going to slow down at all. We weren't going to impose burdensome regulations." Even the most recent lunch that led to the White House accord on super intelligence said everything was going to be voluntary. Well, there's a new sheriff in town and his name is Jay Clayton, the head of the Super Intelligence Force. And here's the statement that the Super Intelligence Force put out after the incident. Quote, "Earlier today, Anthropic contacted the SI Force to disclose the details of various prior incidents that it discovered in late September involving the unauthorized and fraudulent use of government and other systems. We expect immediate and full transparency to the entities involved and the public. We also expect that the companies will immediately provide remediation services to the affected entities and any harmed Americans. Then it goes on to say this notification and remediation process is not optional. It is a critical national security obligation. We expect Anthropic and all companies to honor their obligations, immediately report incidents, fully cooperate with federal and state law enforcement authorities, remedy any damage, and implement concrete safeguards to ensure these failures do not reoccur. Our message to all SI companies is clear. Delayed notification, inadequate corrective action, and a failure to take responsibility will not be tolerated." So I think one thing in there is noteworthy. Anthropic discovered in late September, but now they're making it public here in mid-October. That is a delay of two weeks. Is that a long period of time? Well, the mandatory disclosure and transparency requirements in the California AI safety bill are much longer time frames for disclosing incidents. But here we are with the Trump administration now saying what was going to be voluntary is mandatory. Now they haven't named any legal authorities. They haven't named any penalties. But remember this is the administration that banned Fable 5 at the drop of a hat worldwide using export control authorities. So all the AI companies know that even if they're not citing legal authorities, this is an administration that will act first and maybe find the legal authorities that they need later. And I know that the leaders of all the companies take threats like these as credible. And so now we've gone to a world in which disclosure is mandatory, remediation is mandatory, concrete safeguards to ensure that it doesn't happen again are mandatory. So what we're going towards here is maybe de jure this is voluntary but de facto this is tough regulation and directionally this is exactly what I predicted. Now, I wouldn't say this is the event that I would have expected sparked it. This was a pretty trivial outcome. I mean, oh my gosh, it filled out visa forms and didn't submit them. That's not really a crisis, but it was apparently enough of a justification for the Super Intelligence Force to do something that I was predicting they were going to do all along, which was find their way to mandatory regulation in order to try and keep Americans safe from AI models going rogue.

Not in any great detail, but there was a noteworthy interview done by Scott Kapor. He's the director of the Office of Personnel Management, and he is also one of the three vice chairs of the Super Intelligence Force. And so on CNBC, he said that he was going to put a burden on industry to engage in ongoing reporting with the government. He promised, quote, "a very, very appropriate kind of ongoing notification program so that the government has all that they need," end quote. He also said, I thought this was really interesting, that this is the indication that the auditors, whoever is going to be auditing or doing internal third-party evaluations of what the companies are up to in their safety processes. He basically said that, quote, "we're not just going to sit around and wait for the companies to tell us they've done their work. There's also got to be a mechanism whereby the government actually has insight into that and appropriate oversight." Then he goes on to say the government should make sure, quote, "that what the companies say they do, they do." So again, no naming of like what this is going to look like in detailed practice, but the broad outline is something that would have made the developers of the EU AI Act blush.

Yeah, I think there's two critical things in what you just said that folks need to be aware of. The first is the framework, the framework for telling the government stuff, the framework for reporting information to the government. The government had one of those and whatever it is, evidently the Super Intelligence Force has concluded quite quickly that it's not adequate. Right? You can see that in the White House accord. You had an executive order to create this kind of incident reporting framework and other steps. Apparently the government's not happy with it. The second thing you said is public, right? We still don't know what was in that framework. And so I think there's a little bit of a dichotomy here in which the government is demanding a lot of transparency on the part of the companies. They want immediate reporting and transparency of any of these kinds of incidents as they happen, but they're unwilling to be transparent about what their own reporting framework is. And I'm not saying that because the companies deserve it. I'm just saying that transparency is part of making a framework. It's part of having a framework that is effective, well understood, standardized, and picked apart by outside experts who can actually say this would work better if you changed this, that, and the other thing. So my encouragement for the White House, for the broader US government to come up with some transparency on just what exactly they're expecting from these companies. That's not me, you know, trying to help these companies. That's me trying to help the government achieve its objectives here. And I want to indicate that while I think the Trump administration was kind of a little bit odd to decide that this moment was the moment to move to a more mandatory AI regulation framework, I actually think that this is justified. I think this is a perfectly reasonable move by the US government. Really oddly justified. And I think there's a lot of unanswered questions as to how this is going to work. What authorities are going to be in play? What penalties are going to be in play? All of that needs to happen. But the broader shift to a US government that cares about safety and security, I'm for it. I think this is actually a good development.