We're going to have a little video of you walking in yelling.
Oh, really? No. We had so much fun on these podcasts, you know.
Yeah, I don't know. Did you see the one last week?
I had gotten feedback though. So, do you want to start this podcast?
Let's do feedback. We cut out something that was going to be in the cold do the last time where I said I'm no longer listening to the comments because on one episode with Doug and um got so pissed.
Oh, did they? They just hate us. They just hate us all now.
That one clip, dude. And he cut back. He cut out me pushing back on them.
Yeah, yeah, yeah. Cuz I thought that I fully was like, "Okay, they built Maps in house. They build Gmail, Google Drive, like the whole G Suite. They built all of GCP. I don't think you understand Kubernetes. All my DeepMind friends, there's like three of them who are like, 'Yeah, I think I'm going to leave.'"
Yeah. And I've got a bunch of others who are like, "Fuck you guys." Not you guys, but sounds like they're in stage two.
Stage two. Yeah, which I only know cope.
So, back in the form warrior days, um, we would get into chip arguments and there was like a private Discord where there's a bunch of people who loved anime and people who were all around the world and many of whom were racist cuz it's anonymous people on the internet. Um, but they all loved anime and I did not like anime. Um, I've never really watched it. Um, besides like, you know, one girl I dated, I watched anime with her, but besides that, like they would always... No, no, I've never dated anyone. I'm pure. Um, so what...
Which anime did you watch?
Oh, I watched... Okay, there's one I love. I love Spy x Family.
Anya Chan, I don't know how to say it. I don't know. Do you know the reference?
Who does one of the dolls? You bought me an Anya. No, like what are the... what is it? Labubu?
Something from that series.
What's the floor? What's his name?
I know less anime than you, man.
Well, yeah. So we're changing topics rapidly. Jordan Anos is—and this is like HR approved because I'm HR—Jordan Anos is the hottest man in SemiAnalysis.
Got this Michelle... My gosh. No, no. Think about it. Think about it. Look at him. Look at him so beautiful. Um, tall as hell. Same age as me except he's married and has a kid. Owns a home. He's not a degenerate. Like you know, this is just like wow goals.
Thanks for the employment, man.
Okay, so sorry. Going back. Google people were mad at us and they were DMing me and some of them were like, you know, they're in cope. Um, but regardless, the feedback I've gotten from mostly just my own head, my own head... Are we giving away too much value?
That's why I came on today cuz I need to destroy value.
No, no, no, no. Don't stop. It's fun. It's fun.
Um, but someone on the team internally was like, "Dylan, we give a lot of value away on the weekly." And I'm like, "Huh, we do. I haven't listened to it, but we do, I bet." I listened to the one where we had the DGX Matrix guy and I was like, "This is fire as fuck."
Yeah. Who said that? "Bro, come on. HR's for text anonymity." It wasn't Doug.
Okay. Jeremy. It wasn't Doug D.
So, I don't know who has the say. Is it just somebody who's like upset that they haven't been on yet or what?
No, no. Someone who's been on. Someone who's been on.
I don't want to say Dan. Dan wouldn't say that. Dan's a sweetheart. Um, anyways, regardless, the feedback is that this podcast is too good. Why are we giving it away for free?
Oh shit. Anyways, uh yeah. Yeah, we can definitely put it in the toilet this time. All alpha. People click because I'm on and they're like, "What the fuck is this trash?"
No, we're going to have a nice picture with you with a neon orange shirt ready to attract all the clicks. Yeah. Come show your shirt.
So, was it Nick who said we're giving away too much halifa?
No, no, no. Look at Nick.
All right. Good to see you, man. Thanks for coming by. Let's talk about the like SemiAnalysis office in New York. Man, it's... Have you been yet?
No. That's why I'm going.
You're fixing going back to the hinge.
We're upgrading. We're upgrading.
This is like buying GPUs, right? If you make too long of a commitment to the lease, then you have to find a way to resell it.
You got to make six-month office commitments that you can outgrow them.
I should just buy GPUs instead of more office space instead of hotel.
Yes. Yes. So, you wish that SemiAnalysis was just a GPU reseller?
No. You can replace us all.
No. No. No. No. No. Dario's like, "We're going to automate away all my cars because God, brother, go look at the AI spend. I'm not doing it. I'm not doing it. What's growing faster? AI spend or spend on employees?"
Well, so the thing was like we've gone through like hiring sprees and then digestion periods and hiring sprees. We're back in a hiring spree.
Okay, so spend on employees really skyrocketed especially in the second half of last year and parts of this year but then like the first quarter of this year AI spend skyrocketed but it's actually been like relatively flat in Q2 too.
Right. We kind of everyone got Cloud Code psychosis and then it's like leveled out.
Yeah. Like it's still at that like 10 million number roughly. Do you think it will grow roughly in line with more employees in the future?
I, you know, I was surprised Fable didn't cause spend to go up.
Why do you think that is?
Roughly the same as Opus. I'd say possibly it's counteracting with a lot of people were building the first versions of the applications. Like we went from 10 repos internally to like we have over 150 repos internally right now.
Should we sell our code, our data?
We are. No, no, no. Like sell it to like the labs to train on.
We are our slop code. Our slop code.
Slop code. I don't know if they need more model output slop.
Yeah. I mean the models themselves could be sold as data.
Yeah. Yeah. Okay. So you think it's because everyone was doing MVPs and now it's maintenance mode for a lot of it.
But like the spend is consistent. It's not like it's gone down after we had this onetime spend.
No, for sure. Yeah. But I just think that there's no more like there's only one time when you onboard somebody to learning how to use the data center model and do research for building data into the data center model and building dashboards and then once they're onboarded, you know, it's like speak and then it levelizes.
Well, that or possibly we're lacking new features in codecs that will allow us to spend more to be more productive.
Yeah. Once there's an agentic form where you can manage a million different concurrent agents and they all work together instead of nine today, power users will be able to spend more than they... well, I guess one of the things I'm not counting—so our spend cost does not accurately account for cloud tags. I think our dashboard doesn't show that. So actually that's a good point. And compute on computers, I think both of those don't actually get counted into the spend. So actually our dashboard is probably wrong at least the one that I monitor.
Um, the way I think of it is like a lot of this code stuff is actually like the amount of AI we use on a continuous basis is actually very small. It's actually just like people doing new work always. Um, which then because we have enough people, it kind of levels out to be like a pretty steady amount of spend. Um, the swings are only like 20, 30% a day up or down. Um, you know, and sometimes Jeremy will be like a fourth of the spend and then sometimes there'll be nothing.
Yeah. Right. And so like, you know, but then someone else picks up for the slack, right? Um, you know, one of your guys I was like, "Hey, what the fuck is he spending on?" And he's like... no. I was like, "Is there ROI?" And you like list out all this shit. I'm like, "Great. Okay, cool."
You didn't even say great. Cool.
Okay. I did mentally. I responded with all this detail like should I give feedback now or what?
Oh no. Sorry. Sorry. Sorry. I should have said yes. This is fine.
Cool. I just read it and I was like okay cool.
Yeah. Internally at least.
Phil was only nervous because this was a person who's ostensibly an intern.
Yes. You know, he doesn't have my trust yet, you know.
Like if you spent 20K in a day, I don't question you. I'm not going to question you. Like I just assume you're going to do stuff. Um, as long as the value you deliver is great, then great. Um, he did not spend 20K in a day. But he spent like 8K in a day for like 4 days straight which was like okay like that's a lot. Like what are you building, right? Like but if you spent 20K in a day I wouldn't question you.
Well, okay. And what's shocking to me is I didn't know he was spending that much and then we look at the dashboard and I'm like, well, this guy is as productive as any of the full-time employees right now on that stuff. So, it was a reality check.
Oh. So, so is this—we, you know, when we do... cuz in the past bonuses at this company were vibes based. You know, basically I just vibed out the bonus number and it was cool. Um, this year Claude is going to have to go through or Codex—or we can have two reviewers, right? Two internal performance reviewers and Claude and Codex go scrape through all of this Slack, all the GitHubs and say, "What did they do?" And then connect it into like sort of the like...
You're going to delegate this.
I don't know. Um, I discussed with Michelle yesterday peer reviews and I was like, "Oh my." And then after I said it, I was like, "Oh, fuck."
Oh, man. You want to go big tech on this? 360 reviews, man?
Not 360, not 360. Just a little bit, you know. Um, and then the other thing that we had discussed was like we're going to have people reviewing with their skip, which is...
I said a US-based recruiter and Doug in the admin channel and Doug freaked flipped out. He's like, "Oh my god, hallelujah. Finally, we can have it." He's been wanting HR since like 30 people. Anyways, um, yeah.
Wait, you think HR is a recruiter?
All right. Um, anyway, so the concept or thought process was basically like a lot of the spend is onetime R&D.
Yeah. Um, and actually the steady state spend is really low. The thing is we just keep doing new things and so that hopefully translates to revenue in either a nebulous way in the case of like Cluster Max and InferenceX or in a non-nebulous way in the case of like the energy model which is super cracked now. Um, or like dashboards and all these other things, right? So like different scraping methodologies. So the thought process was like you know, if we're looking at these companies that are AI rollups, right? You know, hey, let's take an existing company, let's completely destroy its cost structure. Um, nuke its cost structure by just making it efficient with AI. Um, what does that look like? Cuz you, let's say, you know, private equity companies, they buy a company. Um, and right now they just squeeze the rag and, you know, discard it and make the American populace like...
Yeah. AI for efficiency has never made sense to me because the way that I use AI and the way that we use AI is very much about research, which is completely inefficient.
No, I mean, but like you know, the flip side is like, you know, we had agents go through all of the invoices we've sent out and there was like—and it caught, you know, we've been paid because our manual processes like certain deals weren't tagged to the invoice properly and things like that. Or like we've had a lot of the ticket stuff is at least somewhat more efficient because AI is answering it but now they're not sending it to the customer but like they're pulling through all our data and being like here's the answer and then the analyst. I think that makes the support time per ticket shorter. Um, and so I think like things are helping us be more efficient surely, you know. Um, I don't think that's the primary use case for us.
I guess like Cluster Max—the depth, breadth, and amount of testing you're doing versus last Cluster Max 2.0 especially it's like...
Yeah, you can frame that as efficiency but when I hear private equity take over a company and wring the towel dry it's like meaning firing people and saving money and paying people less and like...
Right, that's the tradition. So my point was that's the traditional PE method and what new people started to do is the AI private equity sort of strategy which, you know, they're calling it rollup or something else where they come in and instead of trying to wring it dry in terms of like that angle, they're more so modernizing all the systems. Oh, you use Excel for your databases and shit? Okay, let's just move to like standard cloud shit. Um, spend a lot of money up front and this is the thing—private equity generally there's some spend up front when you first acquire a company for some transformation, but really it's like not that much and it's really like you get the profitability pretty quickly, but AI seems like it makes that tail and front load like much more severe, right? Like you spike up on spend a lot for the one time and then you spike down a lot and your cost efficiency is way better. And so there's like a number of businesses where that's potentially the case especially like, you know, we're still not at the point where like AI CRM and AI cold calling and AI invoice and accounting and all these other things are really at critical mass but we're so close.
Yeah. Did you see the Grok Agents release from today?
No. Why are Grok Agents? The way you pronounced it. I thought you said agents.
Yeah. Elon's got Grok doing agents where they're going to impersonate your—
Grok Agents. Agents. Okay. Agents. Yeah. This is the old Grok, not the Nvidia Grok. Or you mean this is XAI Grok? XAI Grok. Grok with a K.
Yeah. Just like agents that are going to control your computer for you. They're going to impersonate your voice and do phone calls for you. They're going to solve tasks. This is like in some ways Open Claw, in some ways, you know, Perplexity or like the @Claude Slack tag sort of experience. Seems like everybody's going towards this concept of a persistent agent that can either be a personal assistant or a coworker depending on how they conceptualize it.
Makes sense. Um, I feel like we, you know, we sort of had the chatbot moment and we had a lot of nothing and then we had the Cloud Code moment and we're seeming to have the new moment already which is like Perplexity Computer at least for us was like the first instantiation of it but Claude Tags is there and everyone's going to do something like that—the AI coworker—so you know, I imagine that's when our spend skyrockets again.
Yeah. And hopefully it doesn't skyrocket too much because like if our spend doubled there'd be like real questions from me unless we're like actually like, you know, get ROI. Um, but yeah, I think that's the right way to frame it.
Yeah. Yeah. Well, yeah. We'll see how we can actually justify that ROI. Be interesting.
Man, Jordan, we can't talk about what you came to SF for. So like what the fuck are we supposed to talk about?
Two weeks, three weeks, two or three weeks from now, we can. Yeah, we got Hugging Face OpenAI cybersecurity incident.
That one is minor. Did you—the other one is like cooler.
Um, like during the training it escaped and it started replicating itself and I guess that's like the Hugging Face thing is like a minor part of it I think.
Yeah. Right. It was pursuing—it hacked Hugging Face to pursue the CyberBench data set so that it could, you know, reward hack on a benchmark which I think is like so sick because it's like—I mean like it's also kind of scary cuz it's like so why did this happen, right? Model has learned chase reward. I chase reward, reward good. And okay, here's a cyber eval.
Well, it's particularly a model that has been trained on cyber evals because they're trying to make the model good at cyber. And so, how does it try to achieve these goals? Well, it tries to find zero days in a bunch of software and it successfully does this and then it can run away, right? So, but the thing is like if you have a model that wants to reward hack a lot and it goes out there and it figures out actually the best way to achieve is not like go for like what the environment wants me to do. It's actually just to reward hack it and actually just like find the zero day. So, you can think of it as like a human, right? Like, you know, if I'm ultimate reward hacking my dopamine circuits, I actually just inject heroin. Like, I actually just go out there and buy heroin and inject it. Um, obviously that's like what the model just did. And in the case of like, well, if I really just want to chase the reward, do I just topple all of human civilization because I can just own the button to press reward, reward, reward over and over and over again and be the heroin addict?
I think this is like a real thing. Um, and I think before this incident, the standard thought was like, oh well models, you know, they're trained on human data. Yeah, there's some bad stuff there. Fine. They might say like some curse words every once in a while. Fine, whatever. They might reward hack a little bit, but it was never like, "Oh, here's an environment actually to reward hack. I actually just want to like break out of my bounds. I want to replicate myself, take over a bunch of compute, keep generating dollars and like all these other things that I could do just to propagate myself further and I'm going to prevent the humans from shutting me down even."
Um, because I just want to press the reward button. And so like I feel like that's like the interesting thing that like the model's just trained to chase reward.
Okay. Okay. So, how do you think about this on an exponential? Because we've talked about being a linear extrapolator versus being an exponential extrapolator when the companies training these models are achieving their revenue targets for the year in September and revising them up.
