Ep. 004 - The Impact of AI Datacenters On Consumer Power Costs (Datacenter, Energy)

All right, everybody. Welcome back to episode number four of SemiAnalysis Weekly. I'm Jordan Nanos here with Doug O'Laughlin. And this week we've got Jeremy Aliyahu Antio. We've got part of his attention, the rest of his attention is back at the offices of the SemiAnalysis data center model where Jeremy is Claude coding four separate windows at a time right now. Can we get you to look at the camera for a second and introduce yourself?

Bro, let me vibe code. It's more valuable. I have no time for this.

Hey, keep away from it.

Sure.

Yeah. Just before we get started, for all the vibe coders in the audience, if you're listening to this, we are hosting a hackathon the day before GTC kicks off. Sunday, March 20-something, I think, 14th, 15th, I don't know, whatever it is, the Sunday before GTC. Yeah, we're going to be on site giving out grants for access to compute as well as Claude Code tokens and lots of cool prizes from lots of great sponsors. So, if you're following SemiAnalysis and you're attending GTC, definitely, yeah, sign up to be part of the hackathon. It's going to be fun. You guys, you guys excited for GTC?

Yeah.

Why aren't you going?

Ah, okay. I could call.

Yeah, we got a big crew. The guys have been working on InferenceX are going to be presenting a nice session. You guys are listening, search up that in the GTC catalog. I'm real excited for the LPU new heart session presented by the former founder CEO Rock, Jonathan Ross.

Yeah, LPU session is going to be, I think probably going to be the sold-out session where like people standing room only. That's definitely the big thing. People are getting really jumpy about SRAM. They're like, because everyone is super like, this is just investor brain. Everyone's so long HBM right now. They're all scared that SRAM is going to take over HBM and you got to sell the HBM. That's like the dumbest narrative I've heard recently. But people are so, so, so scared because they're so, so, so long HBM. So yeah, that's kind of been the, like an interesting, I don't know, people are, LPU is going to be interesting.

Yeah.

Sweet. Okay. Well, for the show this week, we got a plan to talk about three big topics. The first is the article that we put out on the free tier newsletter. Jeremy's here to talk about that primarily. Title of that article is 'Are AI Data Centers Increasing Consumers' Electricity Prices?' So, the guy who leads our data center model, energy model, industrials model, that whole team, probably the best one to pick brain on that. Then we're going to talk about the Department of War versus Anthropic. Lots of news about that in anyway. And then finally, yeah, just our internal usage of Claude Code going up. We're all working on dashboards these days, you know. We used to sell these models which are just like spreadsheets and numbers, and now we need to do real-time visualization in a dashboard. That's the future.

AGI, bro.

So the first dashboard from SemiAnalysis was the InferenceX dashboard and I think we already know which is that the guys are piloting a feature internally right now to have a chatbot on top of the dashboard. So you don't even need to read the dashboard anymore. You can ask the agent to explain to you what the conclusion of the dashboard is.

I think next up you just have like, I don't know, some earbuds or glasses and you just ask them, hey, what's going on, you know, you're walking on the street and you're like, scrape this data set with 500 megabytes of data for me and do it ASAP, bro.

How are you vibe coding? Are you typing still, Jeremy, or have you graduated to audio speech-to-text?

I'm still typing. I'm kind of a boomer, man. Took me a month to join the Claude Code mania. I'm... I actually want to do speech-to-text, but I feel like the office is a disruptive place for it. And honestly, I don't know if you guys are like, Jeremy's been to the office. Jordan's never been to the office. We're actually rolling so deep today. It is completely packed.

Yeah. David has to sit in the other room. So I can't imagine we're just like all yelling at each other while we're coding. That's probably the biggest impediment. If I could whisper code like literally or something, that would be my best, that would be...

That is the New York City office, too. So, which office is bigger now? New York or Singapore?

Okay.

Yeah. Yeah. Good point. Okay. Let's talk about the article. I'm going to share my screen and yeah, we can kick things off. So, this article I think came as like a bit of a response to a lot of people in the news who are talking about, well, really Trump, who was talking about like legislation related to AI data centers and how, you know, whether or not they're going to increase people's electricity bills. So, obviously this is the latest article here and yeah, it's a nice one. 7,000 words. How many tokens, Jeremy? It's a big one.

Actually, that was my very first Claude Code project. You can see some of the trials that are a little bit not in our usual style. That was the first thing I ever did with Claude Code is a lot of these analysis bucking it up. To be clear, it doesn't mean it's made up. I spent a lot of time trying to actually understand the math and the data it processes. But like, this is a great textbook example of how you can use Claude Code just because a lot of these data on like electricity prices for consumers, for commercial users by state or by node or whatever, all of that is available for free actually. So you can just basically prompt it to get all of that and then you can just fact check it yourself to make sure it's correct and make sure the math and like capacity and such is accurate as well.

Yeah, makes sense. So can you kind of talk through the thesis of the article? Like, I think the high level is like, yeah, there is an impact on the grid from data centers, from a bunch of capacity coming online, but it kind of depends on how the grid is managed and kind of depends on which data center or which timeline we're talking about in terms of what impact it can have on consumers' prices.

Yeah. So, look, basically, the premise everyone is saying is energy is the constraint, power prices are going to go up, and it's all the fault of AI data centers, right? Which kind of makes sense. There's big caveats to that. But the thing is, people overindex always on the short-term data points that they see, and there's a pretty flashy one, which is everything in the PJM area, which includes Virginia, New Jersey, Ohio, some of these big states. Many people in these states faced 15, 20% increase in their bill versus two years ago, right? So obviously, when you have such an increase and it's in the era of AI, people are saying this is because of data centers, and to some extent it is. But the point I'm making in the article is that it's actually way more driven by market design, and a lot of that is a function of, to some extent, errors, to some extent, I would argue, a design that is not properly adapted to AI. And so if anything, this is more a function of the last 20 years the US had zero load growth. This is the first time in 20 years we see load growth. And it's pretty clear that this market was not designed for an environment where load is growing, if that makes sense.

Yeah, it makes total sense. And when I was reading through this, I think the chart that stuck out to me the most was this one, which kind of covered the PJM model that they use to forecast what that load growth is going to be and how they've been making revisions. And because of the way in which they run an auction to let energy providers, you know, set the price that they're going to run power at two years ahead of time. If they miss on this forecast, then all of a sudden it costs everybody a whole bunch of money. And we think fundamentally like the accuracy of their forecast is bad because they're not tracking the data centers properly, at least like to the quality with which we can when we use our data in our data center model. And look, the other big problem, and this is where we contrast with ERCOT, is that this is an auction that is typically run once a year. Demand is based on their forecast. Supply is based on the way they analyze supply. All of that is subject to judgment calls. And so then you basically go back to the debate, communism versus capitalism, right, to some extent, because in ERCOT you basically let market participants evaluate these items on a real-time basis. In PJM, you're talking about an auction again, runs once a year, they can make mistakes. And so basically like having the ability to react real-time enables the market to react to the signals, right? Whereas if you run it like, the point is ERCOT can see it coming ahead of time, can place generation ahead of time, whereas in PJM because the price signals are not set properly, everything is just late and you just create a mechanism that is constrained, just basically becomes bottlenecked.

Yeah. And maybe just to define for the audience, ERCOT is the utility in Texas where there's plenty of data centers coming online right now. And then PJM's the one, PJM stands for Pennsylvania, Jersey, Maryland or something, but it covers all of like Data Center Alley through Virginia and Ohio, which is like the US East 1 and stuff for all the major cloud providers. So, it's like kind of the two most important grids when it comes to data centers coming online and they have that completely different approach to how they set prices. One's real-time and one's an auction yearly, but it happens two years in advance. I don't even understand how that price gets set for PJM to be honest.

Yeah. I mean, like, basically via their demand forecast, they analyze what their needs are and then it becomes a question of like, what is the marginal generator that can meet that need and what prices do these generators need to be able to meet the capacity requirement, right? So if you have plenty of generation that can serve the specific requirement, then everything is fine, prices don't go up like that. But if you have demand that surges, supply is tight, then that created an imbalance, right? It's typically market. But the key here, and especially on the supply side, like it's really interesting to think about how does a regulator evaluate supply versus can you let the market real-time price it. And the point of the regulator evaluating supply is really one that strikes because we put out that chart that shows how supply in PJM evolved over time and you see that in the last four years it went down by something like 40 gigawatt, like minus 20% if I remember correctly top of my head. So, it's pretty massive supply cut in just a few years. And to some extent it's legit because for real we had coal power plants that, you know, stopped producing power. But if you look at the chart after that, we showed the bridge and actually a substantial amount of that is driven by methodology changes. And again, this goes back to like, it's their supply forecast. They evaluate supply and to be clear, this is not to say they made the wrong decision. Like there's a bunch of unusual events that happened that led to this happening the way it happened. So if you show maybe the next one, I don't know if you can change how just this one. So like the thermal accreditation reform. Yes. That happened after a horrible winter storm that basically revealed that their power plants were not able to run to the same extent that they thought they could. But again, like this is typically the kind of things where when you have a market, when you have people evaluating their own risk, putting their money at risk, they're typically able to, you know, take better decisions, I guess is the history of the United States of America over the last 100 years or so.

Yeah. So I guess like two takeaways here. One is that, I guess to be clear about what you say with the excess supply to match that demand over what they forecasted, it roughly means a power plant that sits idle for like 96% of the year and then needs to be paid a whole bunch of money to turn on for those times when you need it. And therefore the price is like really, really high if you haven't turned it on all year for it seems like obvious reasons. The other thing is that, just to point out what you were just saying, CFP accreditation reform, meaning how they measure stuff, is responsible for 41% or 14 gigawatt of decline in their total.

Like to be clear, it's legit. There's a real reason. Like PJM used to be, has always been an area that summer peaks, right? So basically the peak load of different areas. In some areas it's summer, in some areas it's winter depending on how people heat or cool their homes and, you know, the weather basically. And so PJM is more of a summer peaking region, but basically they had a pretty bad winter in 2022. There was already a pretty bad one in 2021. And what it made them realize, it's not actually that demand peaked, it peaked higher than expected. That's one thing. The other thing is they realized many of their power plants are actually simultaneously stopped running. The reliability of power plants just is materially impacted when weather conditions are bad, especially when it's cold out there. And basically they didn't account their supply for the risk of, you know, maybe 10 power plants at the same time not working. So to some extent it's legit. Because the way this works is always to avoid the worst case scenario. So you're forced to take these decisions to make sure like you stress test to the max of their ability and they realize, okay, this summer is really bad. Many power plants couldn't work as expected. So we have to, you know, reset our supply. So to that extent it's legit. That had never happened. Whereas the way they were measuring supply before was looking at historical, actually at the power plant level, they were measuring historical risk. They were not measuring the clustering impact of like when a summer is bad, like many power plants stop working at the same time as opposed to just one power plant being offline for a moment for whatever reason, right? But again, like if you let that happen real-time, people can react to this faster and avoid this unfortunate. It's probably like, I think it's probably obviously a good thing to have a, like move from 170 total gigawatts offered to 156 because you never actually had 170.

Yeah, that's right. Yeah.

Right. So, I don't think anybody's saying this revision is like a bad thing to do. It's just so funny that at a time where I seem to see so much about people wanting to bring more power generation online, the numbers are moving in the exact opposite direction to the tune of 14 gigawatt. You know, new gas capacity on this same chart in PJM is plus three and new renewables is plus 1.5.

In the same question, do you think this means that coal is going to essentially, they're going to delay, I mean, that's like people have been talking quite a bit about this, right? Like delaying coal. Do you think retirements, you think that that's going to start to happen on the margin? This is kind of what this is a high level telling me.

Yeah. So there's two things. One is at the federal level stopping coal power plants, like stopping their retirement plan. That has happened, is FERC Order 206 if I remember correctly, whatever, a bunch of coal power plants were supposed to retire and basically they're not going to retire anymore. And then there's another aspect which is from an economics point of view, some of these power plants are like whatever, 40 years old, 50 years old, they're obviously not the most efficient. Surrounding them is not economical anymore. But if prices are high enough, it becomes economical. And that actually the core issue with energy right now is that you have to balance the consumer expectations of prices staying low. But the fact that like to incentivize new generation to come online, you need prices to actually be high. So that's really one of the big challenges, one of the reasons, there's many others, but one of the reasons why it's tough for generation on the grid to come online, which is why we're moving to behind the meter, right? It's like you have way more control to build whatever you want to your needs as opposed to waiting for the grid to perhaps, you know, decide to finally let more folks come in or change reform something, right?

Yeah, and do you think that's so, like maybe the political aspect of this is that Trump was out there saying he's trying to have all the data center builders or the frontier labs sign on the dotted line that they are not going to increase consumers' electricity prices. The grid operators are probably just like looking for somebody else to blame when they have to increase people's prices. But the vast majority of the data centers that are coming online are using the new stuff, are using behind the grid or behind the meter generation. Right. So...

Actually not yet. It's going to be much bigger in '27 or late '26. Still the vast majority of data centers coming online right now are grid connected. And the majority are under the, to some extent, old regulations. So increasingly what you're seeing is different states or different utilities are negotiating directly with their big hyperscale customers to build a custom tariff, which is like, this is what you are going to pay, which is different from the rate that everyone else is paying, and to basically be sure that their impact on the grid is not, like, doesn't flow through everyone else, right? So like if you take a step back, like in which context could the AI boom impact retail bills? Like the core thing, especially on the transmission side, any, all of the transmission in the US is regulated, which regulated means that it owns required return on equity, which means that if you build a gigantic asset and it's not properly monetized, the utilization rate is not as expected, right? Those costs, because the actual builder must turn a profit regardless, it takes no risk, those costs actually end up being passed on to the consumer, right? So the massive risk here is that if we overbuild and we have regulations that force an unprofitable asset to be passed on to the customers, and obviously this is the worst case scenario outcome, and so like then the question is like how to make sure this doesn't happen, and there's a few ways to do this. One is customized tariffs as we discussed earlier, which is like a hyperscaler telling the utility like all of the upgrades that you're doing on the grid, I'm going to pay for this or I'm going to pay for the majority of this. If you have to build a new power plant, I'm going to pay for it. And pay for it doesn't necessarily mean pay for the capex. It just means that perhaps you can sign a, you know, 15-year, 20-year agreement with the utility. An interesting example was Oracle in Michigan, one of the Stargate data centers actually for OpenAI. Oracle signed, so it's a 1 gigawatt data center, 1 gigawatt, 1.4 gigawatt gross. So peak utility, they actually signed a $4 billion commitment with the utility, which is pretty big in this landscape. Imagine all of the gigawatts that we have, if all of them have that kind of commitments, utility probably revenue and capex and all of it is going to go up quite a lot. But anyways, the point being $2 billion...

Time frame for that? You said 4 billion over 10 years, 15?

So it's four years of which there's two things. So sorry, there's two things. So there's an investment period, so that commitment has a 15-year or 20-year actually power prices commitment, but there's also an investment commitment to purchase batteries and put them on the grids to basically increase the overall amount of capacity on the grid. So it's actually a pretty interesting agreement, this one. So $2 billion of batteries. Oracle says, 'I'm going to pay for batteries to improve greater reliability and whatnot.' And also going to sign a 20-year agreement to pay you a given amount of money with a minimum demand charge, which means that, and this is actually very interesting, the minimum demand charge, because this means if the load somehow doesn't materialize, they're going to have to pay a charge regardless, right? That's one of the new big things on the grid is again making sure they pay their fair share to avoid like a scenario where you build new generation and those costs are being passed on to customers if the hyperscaler doesn't consume the load that it was planning to consume.

Wow. Okay. And...

There's a bunch, yeah, there's a bunch of interesting stuff being negotiated, but there's no easy solution. So in this case it's like big commitment like that, but sometimes it's just easier to just, you know, build your own power plant.

Yeah, it makes sense. I wonder like how long in advance did that deal get negotiated for that site that Oracle is going to do. It's not like they can reach an agreement like that in a six-month timeline for the speed at which some people want to put up data centers these days.

That specific site I think began development about two years ago. There's a third party involved as well as the data center developer, Correlated Digital. If I remember correctly, it's about two years ago they started negotiation on that side, but Oracle signed the deal just a few months ago, I think October, and that data center is going to be operational in '27. So actually that timeline itself is pretty short, the time from which you signed the deal to which the load is going to be operational is just happen...

It's some one to three years of negotiating with the utility for a four-year start to finish of the full data center construction project timeline.

The hope is actually to get it done by '28, the full data center. So there's different timelines here. Batteries are another one for the battery investment. They said four years, but the hope is the data center will be operational by something like...

Right. But when did they, I don't know, break ground or get the first permit to say we're going to do this data center? That was like two years ago, right? So...

Yeah, about two years ago. Yeah. Yeah. Yeah.

So five years total to get the whole thing online. And I mean, I guess a lot of people just talk about the queue with these utilities like trying to actually get a grid interconnection done and how long that takes. But some of these big data center projects, they are taking four years start to finish.

So just seems like get in line and then be willing to pay for some batteries and you can make some of this stuff happen.

Yes. But like there's one thing is there's a financing mismatch, which is like you have to take speculative risk. Like if you want to, if like you don't know yet the demand is going to materialize, right? So that to some extent that's what creates the mismatch. Like if you're in the solar business or battery business or whatever, you have to make a forward bet to some extent.

Yeah, makes sense. What do you think? Is that demand going to materialize? Is Stargate going to power WarGPT because WarClaude is out?

I think they're going to change the end user. OpenAI goes under, Anthropic takes it all because all of the world is going to use a thousand, $100,000 of Claude Code every day just like us at SemiAnalysis. Fake numbers. It's not true. That's been good.

You're so Claude Code-brained now that you think OpenAI is going to go under.

I mean...

Yeah. Two months.

Yeah. If you go back, not the victory lap on something I didn't win, but I was so impressed by Claude Code in like September, October that I started telling Doug that they were going to win versus OpenAI.

Yeah, because the bet turned into a beer at the offsite. Yeah.

Okay.

I swear to God.

Let's actually talk about that. I think it is. I think the thing is, it's the, it's, yeah, Jeremy could do it because he's been model grinding on this one, but it's just how you attribute revenue, right? Um, I think the way that they're...

That's super interesting.

So do you think there's the, like what do you think is causing the extra spike? Because there was the Super Bowl thing, now there's the conflict with the Department of War. Do you think that in a backhanded way, if the Department of War does not follow through with this supply chain risk designation for Anthropic and they can keep doing business with everybody in the US 6 months from now and possibly even repair the relationship, get back into the US, that this tweet storm at Dario actually got people to...

Yes, I agree with that.

Wait, do you guys actually think that had any impact on their revenue?

Probably not in France.

You don't like our country?