The State of the AI Debate

The next era of the AI debate is officially upon us. On the one hand, you have President Trump calling for a new AI force, as well as suggesting his own rebrands of the entire industry. On the other, you have regulators and politicians all around the country proposing AI kill switches and new bills and changes to data center policy. And behind all of that, you have the lurking question of China. If China isn't on board with any sort of pause or pacing the frontier, do those actions even matter? This week, Chinese President Xi Jinping is in Washington and already the trade delegations have been discussing AI. So, will we actually see progress on some of these questions or will the state of the AI debate remain firmly divided between these two very different countries? The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. Welcome back to the AI Daily Brief headlines edition. All the daily AI news you need in around 5 minutes.

We kick off today with Anthropic punting their planned blockbuster IPO into November. Although insiders say the delay has nothing to do with AI safety. The Wall Street Journal reports that Anthropic will not in fact go public this month or next as originally planned. However, sources said the decision to delay was made before CEO Dario Amodei called for an AI slowdown. Instead, Anthropic's advisers have said the delay would allow them to share third-quarter financials, which are expected to be strong. The Information confirmed the reporting, adding that Anthropic has not begun meeting with Wall Street analysts and institutional investors. These road shows usually take place around a month before the IPO. However, some potential investors have been meeting with Anthropic as part of so-called testing-the-waters meetings. Leaking a few financial details, sources said that Anthropic has gone from spending $2.30 for every dollar in revenue in Q2 of last year to being slightly profitable on that basis in Q2 of this year. That would line up with recent reporting from the Financial Times, which said Anthropic claimed they had positive operating income after stripping out stock-based comp. That report repeated a claim that gross margins were above 80% if you strip out revenue-sharing deals and the cost of model training. Industry critics were perhaps unsurprisingly extremely negative on this framing, suggesting that stripping away major costs to get to profitability on paper is not a great indication of financial health. This is one potential issue with the Anthropic IPO as the disclosure documents will give investors their first look at unadjusted numbers. The Information wrote that some investors also walked away from meetings with the impression that profitability would be short-lived. The company has signed a ton of new data center deals that will start to add structural costs over the coming quarters. Kevin Shu of Interconnected Capital suggested the market could have a difficult time signing a blank check, commenting, "Anthropic going public means it is elevating itself to the entire financial market, making it more palatable for them to get different options of financing. But the public market community still wants to trust the fact that Anthropic can fund its own capital expenditure and that debt is a bridge to that, not that the borrowing has no end in sight. That's the problem the entire AI complex has right now." The New York Times reported that Anthropic said they're on track to reach $100 billion in annualized revenue by the end of the year, up from $65 billion at the end of July. What's more, beyond the numbers, some on Wall Street are beginning to view AI safety risk as a market risk. Former NASDAQ executive turned consultant Karen Snow said that following the calls for a slowdown, the Anthropic IPO, quote, "went from being a no-brainer to not being a no-brainer." She added, "The markets can be finicky and sentiment has shifted a bit on AI. The issue is there's a bit of a gray cloud over AI right now." To that end, Anthropic could be delaying their IPO in order to get a new model out the door and try to shift the narrative from safety to progress. AI leakers are starting to report sightings of new Sonnet, Opus, and Fable models in stealth testing, meaning we could see a complete model overhaul in the coming weeks. It seems Anthropic are still on track for an IPO by the end of the year.

Maybe the even more interesting discussion around Anthropic, however, is that the company has reportedly set up a wet lab in the Bay Area to begin biological experiments. Reuters reports that Anthropic has quietly established their own biology lab to conduct in-house research. Anthropic head of life sciences, Eric Cadur Abrams, told Reuters, "We believe that to do biology, the final test is still and will be for a while in real lab work. We absolutely are doing that today." He said that the lab operates similarly to other biotech labs in the Bay Area. And while Anthropic didn't divulge exactly what they're working on, they did note the focus was on fundamental biology rather than drug discovery. Now, any biology lab that conducts physical experiments with live tissue and blood samples is considered a wet lab. However, we don't know the lab's biosafety rating, which determines whether it's able to work on risky experiments. Still, many latched on to the obvious safety concerns with Flexport CEO Ryan Peterson asking, "If Anthropic is worried about rogue AI killing everyone, why did they give it control of a robotic biology lab?" Investor Chamath Palihapitiya wrote, "The group behind such hits as 'We're All Going to Die' and 'Regulate Me Now' are building a wet lab in San Francisco. I do not recommend this." There is another really interesting question around what this means for Anthropic's business model and the business models of the labs more generally. The reporting noted that Anthropic has been careful to avoid the appearance of competing with partners in the pharmaceutical industry like Novo Nordisk. However, investor Nick Carter wrote, "This is a very cynical interpretation, but could it be that the fear-mongering of recent weeks is a way to gently introduce their investor base to the concept that Anthropic will be internalizing all major breakthroughs rather than letting the unwashed masses share in the spoils? Selling tokens is probably a better business model as opposed to relying on very lumpy biotech and other scientific discoveries. But if distillation really is an unacceptable margin compressor, Anthropic is incentivized to keep the frontier private and capitalize themselves on new IP rather than letting their clients have them. And instead of outright admitting their token merchant business model is unsustainable, Anthropic could announce they are pacing the frontier and hold the general public six months or a year plus behind the true frontier available only to their own internal researchers. Of course, this would require the cooperation of their arch-rival OpenAI, so Anthropic didn't appear to be too behind. All the better if they created a legal cartel and voluntarily agreed to slow the public frontier. They'd have to ask the government for an antitrust waiver, though. Really activates the almonds, doesn't it?" Nick concludes this is speculation. Perhaps the fear really is genuine and this all happens to be an extremely convenient state of affairs. I think there are two very different things going on in Nick's analysis that are actually separable. The first is whether there is a concern on the part of the labs that token prices get competed down and in the long term the business model of selling that basic commodity doesn't look as good as it does today and that it may be beneficial instead to take advantage of having early access to the state-of-the-art and not just selling tokens to people to do things but doing those things themselves. That is, I think, separable from whether pacing the frontier is a vehicle for that strategy. And even if one doesn't take that cynical view, the first part about whether this signals a sort of business model hedge among the labs is something interesting to consider. And for what it's worth, there are plenty of non-cynics out there who see Anthropic as a lab that has clearly been excited about the possibility of the medical solutions use cases of AI simply taking an obvious step in that direction.

Now, moving back over into markets, we got some first indicators of a potential data center debt crunch. The Information reports that bonds tied to a data center leased by trading firm Jane Street have sold off. The bonds were issued in August at an 8.9% interest rate, but they're now trading at 11.3%, a large increase that can't be explained by the Fed's rate hike. Now, there are a few important caveats to fully understand the implications of what's happening in these funding markets. Firstly, this doesn't change what Jane Street's data center partner is paying on the debt, which was issued at fixed rates. But it does imply that investors will demand a much higher rate for the next tranche of funding. This is also specifically about deterioration in the junk bond market rather than investment-grade debt. Jane Street carries a double-B rating and S&P Global apply the same credit rating to the joint venture building their data center. Essentially, this is just a sign that data center funding is slowing down at the margins. Another big story over recent days was that Meta has tapped the junk bond market for the first time in a deal with CleanSpark. The deal raised $2.3 billion at an interest rate of 8.25% and saw more than $10 billion in demand from investors. Like many recent deals, the debt was structured with CleanSpark as the borrower and Meta providing a guarantee as the data center tenant. These deals have allowed hyperscalers to effectively rent their credit rating to smaller data center developers. In this case, we're seeing that investors are more than willing to keep lending to hyperscaler data center projects, but they need a little more premium. The Financial Times has also reported that $18 billion of debt tied to an Oracle data center in New Mexico has hit stress levels, quoted as low as 89 cents on the dollar. Oracle was already at risk of being downgraded from investment grade. And if that happens, it could trigger a wave of institutional selling. At this stage, it's just market jitters, but there are clear signs that people are on edge. Over the weekend, infrastructure-focused venture investor Melt Demir posted, "Hearing banks stopping all compute lending, credit crunch is beginning." The post gathered 800,000 views and a ton of engagement with many financial folks asking her exactly what she meant. Melt later added, "Nuance: private equity firms and specialist lenders active. Investment-grade borrowers will continue to get funding, but more scrutiny. The long tail is drying up very quickly. Extreme divide between haves and have-nots. Compute is an underwriting game, and most credit quality is bad and getting worse." Still, investor Shaw suggested it's a little too early to panic, commenting, "Folks are freaking out about debt spreads widening for AI data center buildouts. No need to freak out. Yes, two- to three-year buildouts are hard, but many are seeing sub-25-megawatt data centers as a way to build 50% cheaper with 120-day timelines." Still, for those who are trying to keep an eye on where there are signs that the AI trade might be rolling over, this rather than stock prices is the corner of the market to keep an eye on. For now, however, that's going to do it for the headlines. Next up, the main episode.

Welcome back to the AI Daily Brief. Right now, we are in the midst of a fairly significant transition in the AI debate. Specifically, of course, I'm referring to the regulatory debate as this issue becomes, in the lead-up to AI 1, with significant political resonance. Importantly, however, this week brings with it something unique in the context of that conversation, which is a chance to more directly understand and hear from China about where they stand relative to all these questions that are coming to dominate discourse here in the USA. Given how much of our regulatory debate is backstopped by assumptions about what China will or will not do, that gives this week the potential to be fairly significant. But if you thought the administration might take a weekend off in advance of those talks with President Xi, you were sorely mistaken. Over the weekend, President Trump announced a new AI-focused arm of the military, a search for a new AI Czar, and a rebranding to address AI's flagging popularity. In a Truth Social post on Saturday, Trump railed once again against the, quote, "many hoaxes" that he argues are culminating in the, quote, "decimation of AI." He declared, "I, as president of the United States, will not stand by and let this happen." After talking about how, quote, "wealthy and prestigious" data centers were for the communities in which they were built, Trump announced, "I am forming the AI Force, much like I did Space Force, which has been a tremendous success in my first term. To that end, I will be announcing in the near future the AI Czar. Only high-IQ individuals need apply. AI is the next industrial revolution or internet, but will be even larger and more impactful, possibly as much as 25% of our country's GDP. We are leading China and the rest of the world and I intend to keep it that way." Following the announcement, the president spent the rest of the day workshopping a brand overhaul, writing in a post on X, "Many people think that the words artificial intelligence are inaccurate and very ineloquent relative to AI or artificial intelligence. A far more elegant and accurate description of this new phenomenon would be superior intelligence, SI, or extreme intelligence, EI, or supreme intelligence." The post even came with a poll and after more than 230,000 votes, Superior Intelligence was edging out the others with 41% of the vote, with Trump later cutting Supreme Intelligence from the running, declaring it was likely losing because of its association with the Supreme Court. In that second poll, which at the time of recording has 178,000 votes, Superior Intelligence is just barely ahead of Extreme Intelligence, 51 to 49. Now, I probably don't have to go all that deep on the jaw-open, aghast reactions that this was what the president was spending his time on, with, by the way, those critiques coming from both the left and the right. Right-wing commentator Matt Walsh wrote, "Why the hell is the president focused on this?" But at least a few folks were taking it a little bit more seriously. Signal on X wrote, "Once again, Trump's branding instinct remains undefeated and should never be ignored. He is absolutely right that the entire field is poorly branded. Taco Bell has spent decades teaching America what supreme means. It means more, better, etc. And artificial has almost always been used in a negative context in the American lexicon." Of course, even if one does take that branding mission seriously, no amount of rebranding is going to deal with the actual issues surrounding AI. So, what did people think about AI Force? Well, as you might expect, many lampooned and derided the idea. And yet, some weren't so sure it was worth dismissing out of hand. The argument for taking it seriously basically looks back at the example of Space Force and notes that while it too had a lot of fun poked at it when it was first announced, when you strip away all of the nonsense, basically what it did was to consolidate satellite operations and defense between the Army, Air Force, and Navy, which were each operating their own assets and maintaining specialist personnel. It is not at all inconceivable that some sort of talent and resource combining, particularly around cybersecurity and cyber defense, might actually be valuable. In other words, hold aside all the other debates about X-risk and AI kill switches and things like that. One thing that's for sure is that we're going to be dealing with a new era of cybersecurity concerns thanks to advanced AI. Having a central body that provides support around cybersecurity to other arms of the military might not be a crazy notion.

Now, speaking of kill switches, in other areas of the AI debate, kill switch bills are becoming a very popular policy. California Governor Gavin Newsom, for example, has signed an executive order to accelerate policy around an AI kill switch. The order convenes a working group to provide policy guidance within two months and includes other measures like a requirement for third-party safety inspectors. In a video posted to social media, Newsom said, "We're not waiting to act. We're going to speed up our work on substantial and responsible AI oversight before it's too late. We're going to do this thoughtfully but with urgent velocity. The stakes are too high to wait or delay action." Now, hold aside the substantive merits of that policy or any thoughts you might have around Gavin Newsom himself. Newsom is, if nothing else, going to be a useful bellwether for understanding AI politics for the Democratic Party. Newsom is expected to be one of the leading candidates for the Democratic nomination for the presidency in 2028 and yet also comes from California where he has in the past vetoed AI safety legislation, meaning that shifts in tone in the way that he justifies things are going to provide some interesting signal about where the winds are blowing. There has certainly been quite a bit of a groundswell around an AI kill switch over the past week. Andrew Yang presented this as an AI policy preference during an interview on CNBC. California Democrat Ted Lieu is renewing his push for a kill switch bill in Washington. And it's not hard to understand why something like this would be popular, although how much it does to address here-and-now types of concerns is a totally different question. We also got some new data center policy out of Virginia, where Democrat Governor Abigail Spanberger has signed an executive order placing new restrictions on data center construction in a state that has been one of the prime areas for data center building for many, many years now. As far as data center policy goes, it's fairly middle of the road. It's not a moratorium, although some of the policies could function as de facto bans depending on how they shake out. The order bans government officials from signing non-disclosure agreements, which as we've seen is basically going to be a political non-starter from here on out. It also instructs the state government to expedite noise regulations and commissions a review of backup electricity provision across the industry. Alongside the executive order, Spanberger unveiled a new data center accountability framework for the state. Among other things, it removes the by-right approval that allowed data center construction on certain land without additional approvals. That streamlined approval process was a huge contributor to the creation of Data Center Alley in Loudoun County. Now, so far in terms of the response, it kind of looks like no one's happy with data center builders having new constraints and local opposition groups arguing that it's not going far enough. But then again, when it comes to democracies, sometimes nobody being all that satisfied is an indication that the policy has found the right sort of middle ground.

Still, lurking behind all of this conversation is the specter of China. The concern with all of these recent pause and slowdown types of talks is that if China doesn't agree to those sort of policies, they're effectively DOA and not actually doing anything other than cutting the US out of the AI race. With that in mind, California Congressman Ro Khanna has urged China to join the AI slowdown. Khanna has sent letters to Chinese AI companies including Alibaba, DeepSeek, and Moonshot to join in a binding international agreement to pace the frontier. He's also convened an emergency congressional hearing in his capacity as the ranking member on the House Select Committee on China. That hearing will be held on Wednesday just ahead of President Xi's state visit, which is of course what I was referring to when I said that this week could involve some actual progress, not just more undirected chatter. By way of background, Chinese President Xi Jinping will be visiting the United States between September 23rd and 25th. This follows President Trump's visit to Beijing back in May. The Washington meeting between the two presidents is reportedly set for Thursday, September 24th. And while AI is getting a lot of the headlines, the immediate economic priority is preserving the current trade truce, which is set to expire in November. Officials are discussing an extension and possible tariff reductions on US agriculture and energy exports, and rare earths also remain a major pressure point, shipments to the US falling 20% month over month in August. Still, AI is big on the agenda and was a key part of preparatory talks that happened on Sunday of this week featuring key negotiators including US Treasury Secretary Scott Bessent and his Chinese counterpart who oversees economic relations. Bessent heralded very successful talks with China Vice Premier He Lifeng and specifically noted the proposal for an AI incident alert system, which is sort of an AI version of the Kremlin-to-White-House red phone of the Cold War era, said Bessent. "The US has proposed that we have a notification mechanism between the two countries and we want a shared vision of common goals and common threats." Now on the one hand this sort of bilateral communication could look like just the lowest-hanging fruit. But Bessent basically said don't underestimate its significance. In a briefing with reporters he said, "We think that just like any cross-border activity moving from opaque to more transparent between the number one and number two AI powers in the world is very important."

So, where does China stand on all these issues? Keep in mind, me trying to give you here a quick summary of how the country overall feels is no less incomplete and reductive than if my Chinese AI podcaster counterpart was trying to give their audience a single point of view that represented the United States. But still, it does seem to be the case that there are some fairly key differences between the worries here and the worries there. One of the biggest differences is that the Chinese AI community has almost no, or at least almost no, professed fear of existential risk. To the extent that there are any X-risk concerns, it's the Chinese government's fear that AI could be an existential risk to societal control and ultimately party rule. One of the few official policy positions came from China's Minister of State Security, Chen Wenqing, who wrote on the topic a few weeks ago. Paraphrasing his communication, the New York Times wrote that, quote, "Mr. Chen called for more party control over AI and stricter government oversight. This, he said, was necessary to protect political stability at home, to defend against increasingly sophisticated cyber attacks and misinformation campaigns from hostile forces, and to compete militarily with countries like the United States." Henry Gao, professor at Singapore Management University, said Beijing is going into these talks viewing AI safety not as a shared humanitarian mission but through an adversarial lens. He added that the government is making it clear that, quote, "data sovereignty and political security will never be traded away for international safety accords." Last Monday, a spokesperson for China's Ministry of Foreign Affairs made their position on an AI slowdown abundantly clear. He said, "The development of artificial intelligence concerns the common well-being of all humanity. All parties should jointly promote an open, inclusive, universally beneficial, and ethically sound approach to AI. Fear-mongering and engaging in confrontation and malicious competition will only disrupt the global AI governance process and serve no one's interests." To that end, one of the biggest issues for China is Anthropic's stance on data security, stemming from a couple of incidents that barely made the news in the US. In July, researchers uncovered a backdoor in Claude Code that traced Chinese IPs and reported on activity. Anthropic presented this as a necessary measure to block distillation attacks, but Chinese officials viewed it as similar to shipping malware to their largest tech firms. Following the revelations, Alibaba banned the use of Claude Code and government officials issued an industry-wide warning. More recently, in a risk report, Anthropic disclosed that Chinese government officials had been using Claude with no regard for privacy. They reported the officials uploaded sensitive government documents, including details of Chinese military operations. On Saturday, a social media account linked to Chinese state TV warned that Anthropic was sharing data with US intelligence agencies. Part of the claim was that Anthropic had altered their terms of use multiple times without proper warning. Basically, heading into this week's meetings, it seems that setting the terms of engagement for AI-powered espionage will likely be a far higher priority for Chinese officials than anything to do with existential risk.

The other big difference between the two AI superpowers is how the technology is impacting the broader economy. In the US, to put it bluntly, the AI buildout has been keeping the economy out of recession. By most accounts, AI capex has been the core driver of economic growth over the past year. In China, however, the economy isn't seeing any significant pickup from AI. The New York Times writes, "Economists in China, even those closest to the state, have openly warned that the government is pouring too many resources into a technology that creates relatively few jobs while doing too little to save the broader economy." Now, the reason that the Chinese economy is flailing has nothing to do with AI. The issue is that there has been no sign of an AI-driven recovery to help the struggles of the past few years. Youth unemployment has reached 18.9%. Domestic consumption is in a full-on downturn with automobile sales falling 20% in a year. And housing continues its decline with another 14% annual drop. Li Daokui, a former Chinese central bank and economics professor, said that the economy is, quote, "running too cold and a booming tech sector can't lift the larger base." Economists have proposed quintupling basic pension payouts to shore up a cratering consumer economy, meaning that right now, Chinese officials are dealing with an economy that is closer to crisis than carving up the spoils of an AI boom. Now, for some, none of this should really matter when it comes to US policy. In an interview with CNN, for example, Microsoft AI CEO Mustafa Suleyman said, "I don't think we should use China as the boogeyman for not making progress on our own efforts." He basically argued that each AI lab is responsible for their own safety measures within a pluralist AI system. And said that he didn't view regulations as negative, but instead should be seen as a way of, quote, "creating shared norms and standards to increase safety. It shouldn't have to slow us down." Whether that view becomes more widespread remains to be seen, but what's for sure is that this week will advance this conversation in some way, shape, or form. And we'll check back in at the end of the week after those talks have actually happened. For now, that's going to do it for today's AI Daily Brief. Appreciate you listening or watching as always. And until next time, peace.