Welcome to Next in Tech, an S&P Global podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, chief analyst for industry research at S&P Global, and this is the next in our AI CEO series, where we're looking at places where AI is being put to work. With me today is Dr. Varun Sivaram, founder and CEO of Emerald AI, formerly the group chief strategy and technology officer of Ørsted, a Fortune 500 energy company, a physicist, and Rhodes Scholar. Welcome to the podcast, Varun.
Eric, thanks so much for having me.
It's great to have you on. We've talked about a number of different applications of AI and where it's being put to work, but this is a particularly interesting intersection that actually hits a number of the things that we've talked about on the podcast, both on data centers, energy consumption, grid operation, and a lot of those interconnected challenges that we face, really in helping to manage getting sufficient energy where we need it, when we need it, to be able to manage AI demand, but also to more flexibly work with data center build-outs, and to be able to meet what is that ever-expanding need for capabilities. But, what you've been doing is actually leveraging AI capabilities to be able to help manage some of that. Could you give us some details on some of that?
Absolutely, Eric. Look, I think there's a collision course today between the AI revolution and this unprecedented historic growth of AI data centers, and what the energy grid is really capable of delivering. And because of this collision course, because we can't get enough energy to the AI revolution, we risk the United States' competitiveness and innovation in the AI race, and we also risk raising energy bills for communities and making AI a historically unpopular topic. That's not good for anyone. It's not good for the AI companies, it's not good for America, it's not good for communities. Now today, United States wants to add 50 gigawatts or more of AI data centers just in the next 3 years between now and 2028. And you might recognize that number, Eric, it's because I'm quoting S&P Global statistics. But between now and 2028, just half of that capacity in AI data centers can actually get connected to the grid, 25 gigawatts or less. So that's a very difficult position that we're on and we need a serious course correction in order to achieve all of the goals. Get enough AI data centers connected as fast as possible, keep bills low for communities and grids reliable, and advance our position in the AI race.
If you think about what the depth of current interconnect queues are for the grid, we're seeing such tremendous delays in order to both bring on new capacity as well as to interconnect new data centers to the grid. That presents some, I guess what seemingly are insurmountable challenges.
Would seem insurmountable, Eric. And the reason for this is that you can think of the power grid as a highway. This highway that you probably take your morning commute on has rush hour at 8:00 or 9:00 a.m., right? In fact, the grid is a highway with horrible utilization. It has rush hour maybe once or twice a month. The rest of the time, that highway is relatively empty. There's plenty of room. That's how the US's electricity grid functions. There's plenty of spare capacity 99% of the year. And so because the grid isn't able to serve new data centers for 1% or less of that year, they make them wait, as you said, for 10 or 12 years in that interconnection queue. And that's what's holding up the AI revolution.
Well, that really speaks to the challenge of design of the grid, right? Which is that issue of the capacity has to be able to meet peak demand. That's the design criteria. And that's really what's been guiding design principles and, as you're pointing out, is one of the things increasing that the time to get connected and the way in which we manage grid capacity broadly.
Exactly. But, I would flip the paradigm there, Eric, and I would say, this is not just about grid design and about how you operate the grid. This is about how the users use the grid. Because, if there is 99% of the year plenty of spare capacity, wouldn't it be great if users could just use that spare capacity? Wouldn't it be great that during those one or two rush hours a month, instead of an 18-wheeler, you could shrink that 18-wheeler down a little bit? That is what flexible data centers represent, and that's what Emerald AI, our company, is doing. We have developed technology, backed by Nvidia, backed by seven Fortune 500 companies. We've developed technology to help data centers to control their power demand during these rare rush hour periods. And if data centers can flexibly modulate their demand in response to those rare moments of peak grid strain, you could fit 100 gigawatts worth of data centers on today's power grids right now. So, again, recapping, the US wants to connect 50 gigawatts in the next 3 years. We can only connect 25, but really there's 100 gigawatts of capacity that we could connect right this second if we had flexible data centers. That's the opportunity. It's $4 trillion of AI data centers. And the good news, every time you connect a flexible data center and you avoid having to build out another lane of that highway that's already a massive superhighway most of the time, you save customers money. Local communities will actually see their power bills go down and not up with more flexible data centers. And policymakers are taking notice. Around the country, states and even the federal government and the Trump administration saying, 'We should really find a fast lane for these flexible data centers because if you're flexible, then you're going to bring rates down and you're going to make the grid more reliable and you're going to use all that spare capacity without having to build out extra lanes on the highway. Then you should get a benefit. You should get the 10-year head start in getting connected to the grid.' And that's what we're seeing. Just June 18th, we saw a major new Federal Energy Regulatory Commission announcement that flexible large load, flexible AI data centers, are now potentially going to have a benefit. And Texas has passed a similar rule to give flexible data centers a fast lane onto the grid.
But that's a complicated problem. The data center design philosophy has always been that we're going to be able to go meet demand for our full capacity all the time and there's been this disconnect between both the power demand side and the workload automation side of what do you run and where do you run it and it seems like that gap has been pretty vast. There's some specialized applications like crypto mining. You take a look at how crypto miners have set up camp in various places, not only electric grid, but also in natural gas environments where they're looking at stranded capacity to be able to manage that. But crypto mining is one of those workloads that you can start it, you can stop it, and shut it down. Typical data center workload. So, there's no coordination between when that workload runs and what power demands look like. And it seems like in most data center environments that there isn't even a mechanism for it. And you look at most cloud providers, you got spot markets to be able to take advantage of spare capacity, but there's no view from the workload perspective into what the current availability of energy happens to be. How do you bridge that gap?
Eric, that's such a good point. You're absolutely right that making data centers flexible and controllable power users sounds like an alien concept. And you're right, Bitcoin miners can do this, but it's relatively trivial to ramp up and down in how much Bitcoin you're mining. Traditionally, with data centers where you might be running PayPal and Venmo transactions or a human maybe controlling a surgical robot, you really don't want to mess with the availability and speed of any of those transactions. And so, traditional cloud computing has been a terrible unsuitable candidate for this power modulation. And if you just tried to slap a bunch of batteries onto a data center, it would be an expensive way in order to modulate the power draw from the grid. But let me tell you that although this sounds alien and hard and something that data centers may not want to do, compare it to the alternatives and you'll realize how attractive this solution is. The alternatives for data centers are you continue down the current path. You wait 10 years, and over the meantime, the grid has to build out additional lanes to that highway and raise everybody's power bills, and then you've got pitchforks coming out from communities saying we don't want you here. Or another alternative is you just say, 'I'm going to go off the grid.' It's technically complicated and expensive to build your own natural gas plant and your own microgrid. You happen to raise prices on everybody else in the process because you're buying up all the supply of electrical equipment. And then you've got to maintain and run your own little electricity grid when right next door there is a massive power grid that is built literally to do that function for you. Or you go into space. And space is great because there's a lot of literally space. However, it comes with its own engineering challenges of getting the data centers to space and operating them and using photovoltaics and doing heat dissipation, et cetera. All of these are an order of magnitude more complicated than making the data center power flex. Making an AI data center controllable in the same way that a Bitcoin miner has been controllable. And I believe that the rise of the AI factory, these specialized data centers filled with GPUs or other accelerators are actually purpose-built to be exquisitely controllable because they do one thing really well. They convert watts into tokens of artificial intelligence. They may be running training or fine-tuning or inference. And Emerald AI has built a series of AI tools, AI agents that will in a very intelligent way ramp up and down the power by orchestrating these workloads. By keeping the critical ones running at 100% while slowing down or throttling some of the more flexible ones or by moving certain workloads between locations at the speed of light so that in Virginia, as we demonstrated with Oracle, you can reduce power when the grid needs you in the middle of a winter peak period by moving them to Chicago during a period where there's excess grid capacity in Chicago and excess data center capacity. So, there is technology to do this. Yes, it's complicated and no, it's not more complicated than the other alternatives.
