China reaches Mythos-tier Cyber AI + White House AI Accord + Trump/Xi Debrief

I'm just kind of letting the events of the past few weeks just blow right past me. It's been incredibly crowded in terms of AI policy work.

We could very plausibly make this a five days a week podcast. I mean, with the exception that I don't think we could pull it off at our current staffing level, but there's certainly enough worth talking about to allow that to happen.

So I think we basically predicted correctly that there was not going to be any major new developments following the meeting between Treasury Secretary Scott Bessent and Chinese Vice Premier Hu. And that was where we agreed to have a dialogue with the Chinese on AI incidents. We also said that there was going to be some kind of notification mechanism and those two things are the most substantive things that came out of the talks at least with respect to AI. It's clear that Trump and Chinese Communist Party chairman Xi Jinping did talk about AI but doesn't look like any meaningful progress was made. I mean there was a list of items in the statements by the various governments that came out after the meeting and on China's eight deliverables AI is number seven right between counter narcotics cooperation and pandas going to the Atlanta Zoo. So not especially treated like it's the high-profile high priority issue. Barely nudged it above pandas on the list and nothing at least in public on chips, export controls, distillation, open models in either the US or the Chinese readout. Now, one thing that we did get is a little bit of a formalization of the United States and China agreeing to hold an upcoming dialogue on quote risks and benefits end quote and establishing a channel for AI incidents. Now in the US statement it says this quote the two countries established the US-China super intelligence dialogue to exchange views on risks and benefits related to SI. The next exchange will occur by November 2026. End quote. And it also says quote the United States and China also agreed to establish a bilateral communication channel for SI incidents. Now, neither of these readouts defines what would qualify as an incident or says what information will be included in these communications or even what form they'll take. Whether that's going to be a fax machine or a phone line, who knows? But it does appear that at least for the time being, we are stuck with the term super intelligence. It's in the official White House readout. And then the White House even put out a statement saying, quote, "The two leaders agreed to use the term super intelligence rather than artificial intelligence." End quote. And tragically, this is just not true. And we know this because China's version came out saying, quote, "The China-US AI dialogue to exchange views on risks and benefits related to AI." End quote. And it also talks about quote a bilateral communication channel for AI incidents end quote. So in other words, the Trump statement says that China agreed to use super intelligence and the Chinese statement makes no mention of the word super intelligence. It's frankly embarrassing that the United States is spending so much time on this. Now, I'm going to go into more detail than this topic actually deserves, but just so folks understand, the term artificial intelligence was coined in 1956. And it really gets to the facets of intelligence, thinking, learning, reasoning, planning, all of those things being done by something not human, not biological. That's all artificial intelligence means. And the field of study has been very successfully using this term for 60 years. It's a helpful term even though it's a little bit of a moving target in common discourse as a field of scientific research. The people in it know what it means and use the term correctly. Now super intelligence is also a term that really does mean something in this field. And the term was coined by Nick Bostrom in a 1998 paper of his and massively popularized by a 2014 book called Superintelligence: Paths, Dangers, Strategies that was massive in the effective altruism community in the AI safety community but just among the AI labs in general it was taken very very seriously and super intelligence in Nick Bostrom's formulation is artificial intelligence but also vastly more powerful than human intelligence and in most forms of the extrapolation of what it means. It's a general intelligence which means it's flexible, works on a lot of different stuff. The point here is super intelligence already means something in the world of AI. And the difference between how the Trump administration is now using it and how the field uses it is just a space between super and intelligence and capitalizing the I so it becomes super intelligence SI. But I think the Trump administration and Trump himself is trying to rebrand artificial intelligence which is fair to say is extremely controversial here in the United States. And he's using super because it's a positive adjective. In the same way he calls his bill the one big beautiful bill or tariffs were dropped on liberation day, not tariff day. This is just fundamentally like a branding exercise that it's difficult to take especially seriously and it's clear that the Chinese aren't taking it seriously and one has to ask is the rest of the AI community taking it seriously. Well, I guess we'll get to that when we get to that lunch.

Yes. And I think this is a huge thing to keep in mind. Just imagine for a moment that there was some agreement that the United States and China could reach on AI that was acceptable to both sides. I'm just making this up, but let's just say America gets rid of all export controls and China agrees to allow in American inspectors to all of its AI facilities to make sure they're safe and not being used for military purposes. That's not going to happen. Nobody's going to propose that. Just imagine it for a second. Well, even if both sides liked that kind of agreement, the fact is that Trump's ability to make such an agreement is very much in doubt just by realities in terms of what the politics in the United States are right now. Now, the Republican party is obviously quite loyal to Trump and quite dependent on Trump. And part of the reason for that is that the upcoming midterm elections, even though President Trump is more unpopular according to polling data than he has ever been in either of his two presidential administrations, he nevertheless has a massive campaign finance war chest that all kinds of Republicans are relying upon his fundraising power and his financial power for their contested races. So you might see Republicans who are repudiating some of the things that Trump is saying in public. That includes on AI and data centers, a very controversial political issue here in the United States. But after the midterms are over, it's very unclear just how loyal Republican senators and members of Congress are going to be to Trump. Certainly, they'll agree with him on most issues, but are they going to defer to him on every issue as they did in his first year of his second term in office? Pretty much. So, that is very much in question. The second thing that's in question is who's going to be in charge? The Democrats are favored to win the House. And just recently, Nate Silver, formerly the editor-in-chief of 538, changed the odds of the Democrats retaking the Senate from toss up to lean Democratic, reflecting a 60% chance. Now, I am not the kind of person you want interpreting your polling data, but the basic point here is that Xi Jinping has to keep in his mind as he's negotiating with President Trump that America is not a dictatorship. Trump is not the only person who matters when it comes to making a deal. I mean, anybody who has taken American history in high school will probably remember that President Woodrow Wilson basically proposed and did most of the design work on the League of Nations during the final negotiations to end World War I. It was his idea and he could not get the United States Senate to ratify the treaty to make the United States a member of the League of Nations. And so the difficulty in getting the United States to adhere to agreements that only a president makes as opposed to a president with the agreement of Congress. That's really tough. And so I think a statement that Senate Minority Leader Chuck Schumer that he made while Trump was meeting with Xi really does bear reading. And this is a long quote, but I think it says an awful lot. Quote, "Well, today as Trump wines and dines President Xi at the White House, he has a choice to make. Once again, bend the knee to the CCP, meaning Chinese Communist Party, or stand up to Xi and defend our national security and keep the United States ahead of China on AI." That's why today we are calling on Trump to stand up for the nation's security by once and for all blocking China's access to our AI chips and manufacturing equipment. President Trump must draw a hard line. Do not weaken America's export controls. Do not give China greater access to the advanced chips and chipmaking equipment it needs to close the gap with the United States. Why? Because Trump is currently giving away our greatest advantage in the AI race by greenlighting the sale of advanced AI chips to China. China has always been desperate to get our AI chips. They have set up elaborate ways to smuggle them, something the Trump administration has failed to stop and has only made easier for them to acquire. As we've been warning for years, Trump cannot let the CCP get its hands on our most advanced chips and the manufacturing equipment needed to produce them. Trump could take action today to make this happen. A bipartisan group of senators has introduced legislation to update and strengthen our export controls. They are the Match Act, the AI Overwatch Act, and the Chip Security Act. We have worked with Republican partners. Each of these is a bipartisan piece of legislation in both chambers to include this critical legislation in the NDAA negotiations. The Senate could pass all three bills today and protect our national security and keep US AI tech out of our adversaries' hands." Now, that's a remarkable statement because there was a lot of things that Schumer could have said about Trump's meeting with Xi Jinping, but he really emphasized these export controls and he really emphasized the bipartisan nature of these export controls. And if you are Xi Jinping, you have to keep that in the back of your mind that there is this really significant repository of opposition to Trump's policy. So imagine that Trump says, "I'm going to sell you all the chips you could possibly want." Does that mean that you as the Chinese government stop supporting Huawei's attempt to indigenize, stop supporting their attempt to stockpile, stop supporting their efforts to reverse engineer? No. Because simply the chance that America would go back on any agreement that Trump reached is in and of itself enough to adequately scare the Chinese to keep working on these reverse engineering and indigenization efforts. So there's no apology that America is going to make. There's no deal that we're going to cut that's going to persuade China to slow down its catch-up, copy, and surpass efforts. And that's why we might as well try and make them effective rather than try and persuade China to not want the things that we shouldn't let them have.

Well, I don't know if it's significant, but let's just start once more and I apologize to folks out there listening with the term because super intelligence, it is going to be something that you're going to continue hearing for this administration for the foreseeable future because right after the lunch, Trump signed an executive order officially replacing AI with super intelligence across the federal government. Agencies must use super intelligence and SI instead of artificial intelligence in official correspondence, public communications, websites, reports, and policy documents. Here it is. It's here to stay. Now, this does not change artificial intelligence as already defined in federal law, which is in a non-trivial number of places. So, I think what we're dealing with here is analogous to the changing of the name of the Department of Defense to the Department of War. Basically, the congressional legislation still says Department of Defense. The appropriations budget still says Department of Defense. The true legal name of the entity is still Department of Defense. But you have this executive order saying always, forever more, call it the Department of War. And I think that's what we're going to be dealing with in super intelligence. Now, Trump is also hoping that corporate America gets on board with this term, which I think is pretty unlikely. You have all the people who attended this lunch. If you go to their websites, do you see super intelligence on them? No. Nvidia is not calling it super intelligence. Neither is anybody else.

Yeah, we're not changing the name of the podcast. Don't worry about that. But let's get on to the substance. As you asked, who was there? Well, President Trump actually posted on Truth Social the seating chart, which I do think was meant to be a flex in terms of communicating just exactly how he views the various people who are trying to influence him on artificial intelligence. Jensen Huang of Nvidia and Elon Musk of SpaceX, they're flanking the president. Vice President Vance is directly across from them. Also, you've got Frontier Labs folks. Greg Brockman, the president of OpenAI was there. Dario Amodei and Tom Brown of Anthropic were there. Microsoft, Amazon, Meta, Google, all sending CEOs, as was AMD, as was Micron. I could go on, but I think that about does it in terms of the industry folks and the government. You had the president, vice president, and speaker of the house, Mike Johnson. You also had Treasury Secretary Scott Bessent, Commerce Secretary Howard Lutnick, NASA Administrator Jared Isaacman, the Director of National Intelligence Clayton. And this is one that I thought was kind of interesting. The chair of the Federal Trade Commission FTC, Andrew Ferguson. And that one I can only speculate, but I think it has to deal with the inclusion of antitrust as a part of the conversation of AI safety of late. You may remember that Dario Amodei in his public essay specifically said that some of the stuff that he was proposing might need an exemption to antitrust law. You also had Susie Wiles, the White House chief of staff, Michael Kratsios, the director of the White House Office of Science and Technology Policy, and David Sacks, who is formerly the AI Czar, but now is the co-chair of PCAST, the President's Council of Advisors on Science and Technology. There's other folks that I'm just skipping over for right now, but gives you a bit of a who's who in AI policy in the United States. Notably, Pete Hegseth of the Department of War was not in attendance, even though the Department of War actually does have a lot of equities in the AI safety and security debate. Not sure what to make of that, but I thought it was noteworthy.

Well, the group of folks who were in the room and the group of folks who were part of what the White House is calling the White House Accord on Super Intelligence. It's not the same list actually. So the deliverable of this meeting is this voluntary joint commitment and it's out there on the internet but it's actually I would say meaningful progress on AI safety even though it's short of what I would advocate for. So the premise in the Accord's own words is that quote every company is responsible for developing its own technology safely and in a way that builds trust with customers and the public. And then it asks every company training and deploying frontier models which I thought was an interesting term of art to implement four layers of controls and audits. One, internal controls quote to monitor the capabilities and alignment of its models during training and deployment around areas like cybersecurity, biosecurity, and chemical threats and to ensure that its models do not hack or access technical systems in unintended ways. Two, an internal team to quote ensure all of the controls, monitoring, and detection are operating as intended and that any issues are remediated. Three, quote, "Partner with an independent external auditor or evaluator to carry out independent assessments of whether the controls, monitoring, and detection are operating as intended." End quote. And then finally, have an independent committee of the board of directors to oversee and receive reports from the internal teams and the external auditors and to ensure issues are being remediated. And one more thing I think is probably worth including on regulation. Quote, "Over time it may make sense to codify these steps into laws or regulations. Regardless of whether this is required of companies, we believe that implementing these controls and audits is critical to ensuring a safe future for everyone." So that's signed by President Trump. It was also signed by the CEO of Google, Sundar Pichai, Anthropic, Dario Amodei, Mark Zuckerberg of Meta, Greg Brockman of OpenAI, Elon Musk of xAI, and Jensen Huang of Nvidia. So, I thought there was a couple things that were noteworthy. Number one, as I read, the specific point here is around training and deploying frontier models. Well, Microsoft was recently bragging about a model that it had trained cyber capabilities and cyber is one of the risks listed in this document and yet Microsoft was not a signatory to this document even though Satya Nadella was present. And then separate from Microsoft training and deploying its own models, it also deploys a lot of other people's models including Chinese models. The same is true of Amazon and AWS as hyperscale cloud providers. So it really wants you to ask like what do they mean when they say deploying frontier models? It appears to be the case that serving them through the cloud does not meet their definition of what is going to be required who actually bears these kinds of obligations. Zuckerberg who historically has put Meta opposing most ideas around AI safety or AI regulation. He's generally been opposed and quite publicly so to a lot of this stuff. What's interesting is that he did sign on to this document and he also put out a statement afterwards was quoted in Politico. Quote, "The idea isn't that this is the only thing we will ever do. It is that this is a start and an accord that the whole industry could come to." In other words, Meta and Nvidia typically some of the skeptics of the need for AI safety efforts even they are now in a position where they agree that this is a good idea. Then on the other side you have Anthropic and OpenAI who are calling for more and more mandatory regulation but at least everybody can agree to these voluntary commitments. So as you mentioned this is a voluntary agreement. What do you think that will mean moving forward for these companies that are training and deploying AI models under this framework?

Well, I think the reality is that Anthropic and OpenAI and xAI had all expressed as well as Google to a certain lesser extent, but those four companies to a greater or lesser extent had basically agreed with what Dario wrote in his public essay, which was more than this document calls for. And this document added three signatories, Meta, Nvidia, and the president, but it dropped a lot of those more significant elements of what was already agreed to. So the White House has used its convening power to bring two of the industry laggards to at least agree to some extent of additional safety practices. I think that's legitimate progress. I commend the White House for securing this, but I also don't want to overstate what has actually been accomplished. I mean, when asked if this was going to be binding, Trump's response at a press conference after the event was quote, "I think it's morally binding." End quote. Quote, "There's going to be a tremendous self-policing aspect. It's almost like a constitution in a way and it really is a form of protection." Uh, no it's not. I mean, it's definitely not nothing. It definitely counts as progress, but it is not meaningful protection for the United States population from the plausible dangers of AI. Not only is this less than what Amodei called for in his letter and what the other CEOs said that they supported as an idea, I think it's less than what we're going to get from the California safety regulation at the state level, which all of these companies are going to be subject to. And that's true, including the executive order that Governor Gavin Newsom recently passed on this topic. So between California's on the books legislation, the executive order on this topic, and what I expect to be updated legislation that OpenAI is itself now openly calling for, it's unclear whether this voluntary request is actually more meaningful than the on the books mandatory accountability and transparency that we're going to get from California in the not too distant future.

Well, AI was not the only thing he was talking about, as you said, but what he did say on AI was quite a doozy in terms of rhetoric. Listen to this. He told the General Assembly that the United States quote totally rejects international AI governance. Here's the full quote. The United States also totally rejects any attempt to construct a globalist scheme to control artificial intelligence being spoken of so much now. Then he goes on to categorize AI risk as a hoax. Quote, "These are the same people that are now saying that AI is going to kill us all, that robots are going to attack us, and that everything is going to be a total disaster. Same group of people." End quote. So, this is a pattern of President Trump's public contempt for AI governance. And I think what you kind of have to say is there is a bit of a disparity between how President Trump is talking of these issues in his public messaging versus what's going on in a lot of his private messaging. So in public he said effectively AI is going to be great. There's no need for safeguards. There's no need for regulations. And then he's actually going out and trying to secure commitments from these companies to do something. To do what? I mean, to address bio risks, to address cyber risks, to address chemical weapons risks. It's all in there. And so what we have here is a disparity between the style of the Trump administration and the substance of the Trump administration. And I think the style of the Trump administration is to express contempt for a lot of the ideas of AI safety. But you have to say that substantively they have come a long way from January 2025 during the inauguration of the second Trump administration. And then if you look at Michael Kratsios's stated position also at the UN, here's what he talked about when it comes to AI, which he uses the term super intelligence. Quote, "As the world leader in frontier super intelligence, the United States has taken actions to ensure that we are able to keep the frontier moving safely and responsibly. We have engaged frontier labs on testing and evaluation of new model capabilities, established cybersecurity clearing houses for coordination on vulnerability mitigation and clarified the existing law enforcement tools available to address harmful uses of super intelligence. Internationally, the United States is committed to diplomacy and dialogue on emerging technologies, but international dialogue in this forum and in others cannot be allowed to drift towards

global governance. So, I think that actually makes some sense and you can kind of get the reconciliation of what is Trump's style versus Trump's substance. And here you have the position, which again, I think this is real progress from where the Trump administration was in January 2025.