All right. Hello everyone and welcome back to our next episode of the Next Big Thing podcast. I'm joined by my colleague Klay Hyman and Dylan Patel, our new partner and founder of Semi Analysis, the research group. It's an exciting day that we get to go through the current state of the AI infrastructure landscape with Dylan. You've probably heard Dylan on many different podcasts. I follow his content all the time, as well as the newsletter on the Semi Analysis site. Most recently, one of the ones I saw was about data centers in space. So if anyone is interested in that topic, they have a great piece with a lot of detail on that. But Dylan, I would love to hear a bit about where the idea for Semi Analysis came from. I know in the Substack community these days, people are talking a lot about the revenue of the firm and the success it has achieved, but I feel like a lot of times you sometimes see the current level of success and forget about the journey and the origins and how much work probably went into all of it.
Yeah. So I think the origin of Semi Analysis really comes from ship posting online in a less serious manner. When I go back to my first posts about semiconductors, they were when I was in my tween years, posting on the internet about chips, smartphones, displays, and SOCs before I even had a smartphone. I was obsessed with gaming hardware and PC hardware. By the time I was 12, I was moderating and creating forums on Reddit about Android, Apple, Google, Intel, Nvidia, AMD, and other hardware topics. I've always been a poster, always replied and taken comments. Now we're a 90-person organization, and my marketing team tells me to stop replying to random bozos on the internet, but I have that burning desire to respond to anyone who criticizes me. Throughout my teenage years, I moderated forums, started investing as soon as I made money, was a quant for two years, and then started my firm. But the whole time I was posting. I had anonymous blogs. In 2020, I got fed up with my job as a quant—it's not as amazing as it seems—and I quit to start my company. I posted on a WordPress website under my real name, mixing technology, business, finance, and supply chain. I grew up in a small business; my parents owned a motel in rural Georgia, and later gas stations, so I always loved business. My first post was about how MediaTek was the biggest winner from the US banning Huawei from TSMC, because geopolitically China would rather buy from a Taiwanese firm than a US firm. Over the years, I converted my WordPress into a Substack, started charging, and wrote about the entire semiconductor and AI supply chains. I traveled to 40 conferences a year for four years, going up and down the stack from AI researcher conferences to niche chemical conferences. I developed a whole ecosystem of contacts and learnings. I would call inflections before anyone else. In 2022, I started hiring. My first two hires were from Discord. My third hire was Myin, from a hedge fund background, who reached out after I posted that memory was the biggest loser from AI. With him, we started building models and converted the business from a newsletter to selling information services, reports, and data sets. The ball started tumbling: from 2 to 7 people in 2023-2024, then 7 to 20, 20 to 60, and now 90. We've added sectors and experts from ASML, Applied Materials, Intel, TSMC, Nvidia, Microsoft, Amazon, OpenAI, Tesla, and more. Half the firm are engineers from the industry, half are ex-hedge fund or passionate people from the internet. Now Semi Analysis has many lines of business: data services, consulting, information services, the newsletter, media, and a conference. It's been a hell of a ride.
And speaking of the ride, I had a moment, Dylan. Wisdom Tree and Semi Analysis have been on this journey for a number of months working together. So Nvidia GTC happens in March as it does every year. I'm sitting watching the live stream here in Charlotte, North Carolina, and I think there's 55,000 other people watching the live stream. And suddenly...
And there's 20,000 people in a stadium. There's 20,000 people in the stadium, dude.
And he's referring to you directly, that you were like, 'sandbag,' you said he was sandbagging a certain number, and your charts are right up on the stage. So admittedly, I had a moment, I guess probably on your behalf, watching the CEO of the biggest company in the world basically refer to your research and how you were criticizing his take on some of the numbers. I'd love for you to sort of—I guess you were in the stadium from what it sounds like there.
Yeah, that moment was quite surreal. One of the things Semi Analysis does is we have engineers and we do open-source benchmarking of all AI models and hardware. We have over $50 million of hardware donated from companies like OpenAI, Microsoft, Amazon, Google, CoreWeave, Nebius, Crusoe, Oracle, and others. We run benchmarks on the latest software every single day. When Jensen originally launched Blackwell, he claimed it would be a 25x improvement. No one believed him. Even we thought it might be 15 to 20x based on simulations. But as we built out our inference benchmarking called Inference X, we found that in DeepSeek V3, Blackwell is 30x faster than Hopper on some parts of the continuum. I emailed Jensen and said, 'Hey, you were sandbagging it. It was 30x.' I didn't know he was going to use it on stage. He had our charts up for five minutes, talking about how Dylan said he was sandbagging but he wasn't. He also had the WWE-style belt we created that says 'Inference King' on the slide. He talked about us longer than anyone else in the entire presentation. The only other person he talked about as much was OpenClaw. It was an incredible moment.
Yeah, Dylan, so you mentioned a couple things there. I think it's quite interesting that you mentioned open source and now maybe transitioning towards some of the recent developments and the markets. There have been discussions around the inference efficiency of open source models versus closed source models. And there are still a lot of investors questioning the ROI on all of this. Just in the last week or two, we had a Bloomberg economist talk through how a lot of these AI initiatives may be potentially failing at some firms. I know you've highlighted how your firm is using AI extensively and giving your employees lots of access to tokens. And obviously you just highlighted that you're hiring. So I'm curious: what's your take on this end demand and the fact that it drives this big buildout we're seeing, full of constraints that have been driving up stocks tied to these themes?
Yes, I would say a few things. When you look at the overarching question of ROI, first and foremost, Anthropic is free cash flow positive and profitable in Q2. In April and May, they were profitable and free cash flow positive. June looks the same. Their recurring revenue has soared past $50 billion ARR. They're printing money. OpenAI's revenue is inflecting as Codex adoption grows. These companies are getting much more profitable. Anthropic's gross margins are above 70%. On the other side, companies are spending on AI. At Semi Analysis, our annual recurring spend on AI went from less than $100,000 in November to $4 million by the end of January, and now it's about $11 million. That's about $1 million per year for a 90-person firm. More than a third of employee spend is on AI, and it will probably reach half by the end of the year. The ROI has been huge because we've been able to build product, sell more, and increase efficiency. Many companies are questioning their AI budgets because they blew through them in Q1 and Q2. Some are cutting other SaaS products, some are cutting employees, and some are clamping down on AI. But those that clamp down will be left in the dust in terms of productivity gains.
Gotcha. And then one of the ways to mitigate some of the incremental cost is choosing cheaper, sometimes less intelligent models, not always being at the leading edge. I'm curious: is there a point where firms like yours decide there are some use cases more optimal for using a DeepSeek V4 type model for certain work, and then you might need to rely on things that cost more for tasks that require more intelligence? Is that part of the calculus?
I think that's absolutely part of the calculus for some folks. You have to break out AI workloads into two types. One is AI integrated into a process, like checking a customer document. There you just need to hit a quality level and then you can wait for cheaper models. AI models improve at about 60x per year in cost. DeepSeek V3 was 600x cheaper than GPT-4 in two years. So for integrated workflows, you get to a quality level and then go cheaper. The other type is AI assistant, where you're interacting with the model. Cost optimization there is not about going to a cheaper model; it's about using the newest model because it's more token efficient. For example, Claude 4.6 Opus might take 100,000 tokens and 10 minutes for a task, while Claude 4.8 Opus can do it in 25,000 tokens and one back-and-forth. So the cost is actually less. When we upgraded from 4.6 to 4.7, our cost fell for a week before soaring back up because people did more work. Token efficiency is key. That's why Anthropic has been beating OpenAI—their models are more token efficient. OpenAI models can sometimes do tasks Anthropic can't, but they take 3x as long and 4x as many tokens, so the human-in-the-loop feedback is slower. That's why we remain a majority Anthropic shop. Some tasks we use OpenAI for overnight runs, but most tasks stay with Claude Code.
