Welcome to the Energy Capital podcast. I'm your host, Doug Leuen. My guest this week is Dr. Varun Sivaram. Varun is one of the most interesting guests I've had in the what, three years now I've been doing podcasts, both the Energy Capital podcast, go back to the Texas Power podcast. He is the founder of Emerald AI, a company which is transforming energy-intensive data centers into grid assets and grid allies. We talked about all the different ways that data centers, if integrated right, in the Texas reliability entity brought this up in a report, the possible disorganized integration of data centers into the grid is one of the biggest reliability risks. I would argue, and clearly Dr. Sivaram argues, the counter is true as well. The organized integration of data centers can actually make grids more reliable and spread costs out to more customers. We got into all of that. Just a couple of notes on Varun. He was formerly the chief strategy and innovation officer at Orsted. He was the chief technology officer of India's largest clean energy company, Renew Power. He was a diplomat at the US State Department. He is currently a senior fellow at the Council on Foreign Relations. He was named as one of Time magazine's Time 100 Next for the next 100 most influential people in the world. MIT Technology Review named him one of the top 35 innovators under 35. You get the idea. He also has a PhD in condensed matter physics from Oxford. This bio is kind of ridiculous. Clearly one of the smartest people out there and this company, Emerald AI, is really doing some super innovative things with some really high-level partners including Nvidia and others. I think you'll enjoy this conversation as much as I did. Please leave us a five-star review wherever you listen. And most importantly, if you are not already a subscriber at dougleuen.com, please go there, become a subscriber today. Your support for the podcast really makes it possible. And with that, here is my conversation with Dr. Varun Sivaram.
Varun Sivaram, welcome to the Energy Capital podcast.
Doug, thanks so much for having me. It's an honor.
Hey, it's great to talk with you. I have been reading article after article about Emerald AI. I saw your presentation to the large load task force a couple of months ago in Texas and been meaning to do this for a while. So, thanks so much for taking the time. We're gonna obviously talk about Emerald AI. We're going to talk about Texas and data center growth. We're going to talk about all of these things, but I just want to start from a very high level. Varun, we are recording here on November 7th. Yesterday, Heatmap News had a very provocative headline: 'Everyone hates data centers.' I don't know that that's actually true, but I know what they're trying to say. There certainly is a lot of opposition to data centers right now. You are doing a lot of work, obviously around data centers, data center flexibility. Just from a grid perspective, thinking about affordability, reliability, lower emissions, all these aspects of data centers. Why should people not hate data centers?
Well, look, Doug, I think we're at a critical juncture. And that juncture is between what I consider to be the villain path for data centers and the hero path for data centers. And I don't think either one of them are pre-ordained. I think that folks may be right to say that they're worried about the impact of data centers in their community. The way things are headed today, right? The average annual household power bill in Columbus, Ohio rose by $240 directly attributable to data centers in 2025. And you've seen NERC studies, for example, and other reports showing that the advent of AI data centers could cause grid reliability issues. So there is a scary villain path that I'm worried about in which data centers come to town and communities don't want them. They raise rates. They destabilize grids and as a result you just have fewer data centers getting built. I think the villain path is not just bad for the AI industry. The villain path is very bad for America because America needs AI infrastructure and AI data centers to compete in the 21st century in the most important economic sector we've ever seen. And data centers can provide economic development and they can help us to compete with China. So we absolutely need a lot more AI data centers. The hero path is the one that I'm obsessed with getting us on to because I actually think AI data centers, far from being the thing that undermines the grid, can actually be the asset that saves the grid. And in that hero path, if we get on it, data centers come to town. They actually lower your rates or at least they arrest the increase in rates because they're more efficiently utilizing your existing system. We can connect far more data centers much more quickly to existing power systems and defer the massively expensive overbuild of infrastructure and more prudently expand our grid and expand our generation. And by the way, data centers and this hero path are going to contribute to grid reliability and help us to avoid rolling blackouts. I think we can get there, but we're not on that path right now. And folks are right to worry. And this is the moment where we switch from the villain to the hero path.
Yeah, I definitely want to talk a lot about both paths because I think we're seeing elements of both of them, but obviously to me anyway, the hero path is much more interesting. Maybe that's just my sunny outlook on life or something, but I think that there really is an opportunity here when you see the scale of investment that you see and we know that the grid has been underinvested in, this is an opportunity to bring a lot of investment to the grid. Where I think I want to go next is I do want to ask you about Emerald AI and I think the way I kind of want to bring that in here is this test you guys did in Phoenix recently. Can you guys talk a little bit about what you did there and connect that obviously to the hero path?
Absolutely. So Phoenix, Arizona deployment, one of now four Emerald AI deployments. In that first deployment, we went to Phoenix with a range of very credible and authoritative partners like EPRI, the utility association that runs DC Flex, Oracle in whose data center we were operating, Nvidia, who's both our investor and partner in this demonstration, as well as a local utility, Salt River Project. The goal is to prove that on a grid that faces a summertime strain, let's say, you know, you've got a peak moment sometime in the summer where a million air conditioners are straining in Phoenix, Arizona, that a data center can actually flex its power consumption. It can reduce its power consumption at that very moment to provide badly needed relief. And in doing so, that data center demonstrates the kind of behavior that you say, 'Ah, this is one of those grid-friendly data centers that if one of these comes to interconnect to my grid, I would love to have this hero of a data center connect not in seven years, requiring me to build out transmission lines and power plants, but right now in 7 months, because it can provide badly needed relief when I need it, and it's not going to raise my peak load unsustainably.' And so we went out to Phoenix and we worked with our partners as well as the chief scientist of Databricks to design a representative set of customer workloads running on this cluster of Nvidia GPUs and this representative set of workloads across inference, fine-tuning, training of large language models. We went ahead and said, is it possible that if we get a signal during the peak demand on that day from the local utility that we can then reduce the power consumption by 25% for 3 hours? And those were the parameters set to us by our utility partners. The test succeeded. In fact, the test didn't just succeed once, it succeeded many times. And we're really pleased. I think this is the first time I'm sharing this publicly, Doug. We're very pleased that those results have now been formally peer-reviewed and accepted for publication at one of the world's top scientific journals, Nature Energy.
So, we're delighted that this is kind of a, you know, inaugural first demonstration of AI computational flexibility where the data center itself is changing the way it operates in a way that is supporting the reliability standards of the AI customers. We made sure that our AI customers and partners were happy with the performance of their AI workloads while at the same time the grid got exactly the performance it needed to see. That reduction over a controlled ramp rate, that 25% reduction over a 3-hour period, which is what the instruction to us from the grid was, and then a controlled ramp back and no snapback beyond the baseline energy consumption. That's the kind of behavior that if you replicate it across many data centers can save a grid from a blackout. Just very briefly, I'll say Doug, we also simulated a real California event a little over 5 years ago in August 2020. We simulated what happened in California where a 500 megawatt gas plant just tripped offline. Had we had Emerald AI deployed on data centers in that service territory, we could have avoided the rolling blackouts that ensued. And what we demonstrated in this trial, again, this was in Phoenix, but now responding to a CAISO emergency need, we demonstrated that we could first reduce the consumption by a little bit and then reduce by a further amount if that's what the grid operator signals that we need to do. And so this kind of dynamic ability to respond to the grid's needs as they evolve while protecting the performance of the AI workloads, the most valuable workloads in human history. That's the dual optimization that Emerald AI enabled in this Arizona test and it's just the first of many deployments that we're super excited about.
Now, you had I think a few months ago again at that large load working group said you guys were at least considering doing some kind of a test in Texas. Is that happening or on the road map?
It is my deep desire to get that test up and running. You know, ever since the Arizona demo, we went ahead and did another commercial demonstration again with EPRI DC Flex and we'll be excited once we finish EPRI's independent technical validation to present those results to the public. We have a test that's been announced and that will be done in the United Kingdom, our first international expansion to London with National Grid, the national utility there and a large data center with the most advanced Nvidia GPUs. And then last week we made a major announcement about our fourth deployment. Happy to talk more about it. A commercial scale of nearly 100 megawatts in Virginia with Nvidia. So there's a lot of excitement for what's to come. I really would like to do this in ERCOT. And I'll just share that one of the most impactful statements I have heard came from Pablo Vegas, the CEO of ERCOT, who shared with me. He said, 'Look, Varun, I would like your technology not just to try and relieve the grid when we're in an emergency moment.' Imagine, you know, all signals flashing red, but rather when they're flashing yellow.
When they're flashing yellow and it looks like, you know, we might be approaching a scarcity event, that's when our grid-friendly data centers can really help bring the grid right back to that green zone and therefore avoid ever coming into the emergency situation to begin with. It's why I'm so enthused about this hero pathway. If you have grid-friendly AI data centers, and Doug, we should talk about why Nvidia calls them AI factories. If you have these grid-friendly assets, they help every day to keep you in balance and avoid you entering that emergency condition from which then you have to take drastic action to recover from.
Yeah. So I think there's a ton of applicability here. What I'm trying to think through, and I do want to come back to talking some about AI and why they're data factories and not data centers and all of that is really interesting. I just want to get grounded a little bit more in what the technology is. So, I'm going to repeat it back and then you're going to tell me where I got it wrong or and/or expand on it. You're certainly going to tell me where I got it wrong. I'll get it wrong somehow. But, let me see if I can break this down. So, with an AI data center or data factory or however you want to call it, you've got a lot of different things going on. You've got some inference that could be doing things like, for instance, routing calls to a 911 call center or helping an autonomous vehicle interpret that that thing moving across the street is a pedestrian and it needs to stop. And so there's certain functions that you can't shut them off. Those need to run and they've got to run quickly. You can't even necessarily move them to a different data center. Latency is a major issue for those kinds of applications. That's kind of on the extreme end. On the other extreme might be a large language model that's training over a long period of time, could be done from anywhere, could be done almost any time. And then there's like a whole lot in between there. And what you've done is developed a software that understands all of those different use cases and can within those use cases kind of move workloads around and even sort of maximize the efficiency of the chip performance. This part I don't understand. This part seems like magic. And maybe this is just the magic of the technology you've developed. And maybe you don't want to say too much about it, but it seems like from what I've read, you're maximizing the efficiency of that GPU in that moment such that for those use cases where they can't be shifted, you're still getting the exact same output but at less energy use, which is basically the definition of energy efficiency. Okay. So, if I got that all wrong, you could just start over and describe what you do. If I got it partly right, you know, then you could correct the other parts I got wrong. What I get right, what I get wrong. Grade my paper, Varun.
Doug, you get an eight. You did a great job there. I'm really impressed. We got to bring you on staff here. Look, let me just go up a level just to explain the broad framework of flexibility here. I think of kind of four components of flexibility for a data center. Component number one is what we call temporal flexibility within a data center. You might have just like you said Doug, you might have some really mission-critical, time-sensitive workloads that you can't pause or slow down. You might have some other ones you can pause or slow down and everything in between. And temporal flexibility takes advantage of slowing or pausing certain workloads. And you can do that in many ways. And we take advantage of all of these different ways as we demonstrated in Arizona. Whether it's changing the clock frequency on the Nvidia GPUs or it's rescheduling workloads or it is changing the resource allocation, what's called in the industry autoscaling the GPU utilization for particular workloads, etc., etc. There's lots of different things you can do but basically in one data center temporarily over time I can slow or pause to create flexibility, reduce the energy draw from the grid in time. The second way is what I call spatial flexibility. This is a very unique trait that data centers have but other economic users like electric vehicles don't have. Data centers can move their workloads from one location to another at the speed of light over the fiber optic network. Again, this works for some workloads, but not other ones. It's probably not going to work for a large training run because there's too much data to transfer, but it can work. You can move queries for an inference query, for example. You can move queries from one location to another so long as you have had the model set up in multiple locations. So the second is spatial flexibility where if you have a problem in Arizona, the grid is strained, you move your query over to Dallas, right?
Before you go to the third one just real quick on that so because I think that's where a lot of listeners will have some familiarity and direct experience with AI. If you've done a ChatGPT or Gemini or Grok or whatever kind of search you prefer, all three, it will often say like on OpenAI, you know, it'll say on ChatGPT it'll say 'thinking' right and you get that little lag there, right? And sometimes you want to, you can change that and optimize it to give you a real fast answer. Sometimes you want it to think more. But that thinking, you know, when you're dealing with fiber optics, like that could go around a world multiple times in a second. So like you think 15 seconds is a long time or a short time or whatever you think it is, like you could move that around a world a bunch of different times. So, and the reason I want to dive into that for everyone because one of the things I hear so often in conversations with energy people about data centers is there just is no flexibility in these things. The data centers are paying so much for these GPUs. They've got to run 100% of the time. Not even 99.999. They just got to run all the time. And that is true for some use cases. But for some of them, and perhaps the ones that are the biggest use case, maybe you can expound on that a little bit. There is a lot of this spatial flexibility because somebody's asking a question and 10, 15 seconds is a perfectly fine response time.
Yeah, absolutely. You make a great point, Doug. The latency or the delay that you'll face if you move a query from Phoenix, Arizona to Dallas or San Antonio is not going to be a second. It's going to be measured in milliseconds. Right? You will not notice it. You absolutely will not notice it. And that's important because AI workloads in many ways are different from the historical class of workloads that data centers used. And again, we should talk Doug about why we're moving toward this paradigm called AI factories. Yes, a factory that's optimized for converting electricity into tokens of artificial intelligence. Historically, data centers have done all kinds of heterogeneous or different things. Those include routing 911 calls. By the way, no one ever wants to mess with routing a 911 call. No one ever wants to mess with Doug Leuen trying to send a Venmo transaction to Varun. I know you haven't done this yet, but no one wants to mess with that.
Do I owe you money? Are you trying to tell me I owe you money? Oh dear.
Sorry, inapt example, but no one wants to mess with a transaction like that, right? Going forward, the set of AI workloads, there are these new sets of flexibility parameters. If you're fine-tuning a model, for example, you might have some temporal flexibility. It may be okay for that fine-tuning operation to pause for an hour or two. When it comes to inference, you mentioned these chat bots. Yes, the time to first token, which is the time you wait around waiting for that first response or that first word back to you from ChatGPT, that can take a second. It can take a few seconds. And so the few milliseconds of moving the query is not noticeable. I'll also say that although Doug, you and I interact with ChatGPT every day, that's not the only use case for AI. There are lots of use cases. You might be a scientist, for example, and you might send a request for protein folding configurations and expect that request back in the morning or next week, right? That is what we call a batchable request, which you might be able to pause again for an hour if the Phoenix grid is strained. So there's a range of different ways we'll use AI and almost all of them have some kind of temporal or spatial flexibility. So again, we had four ways I told you about. The first two are temporal and spatial. They're computational flexibility. The third one is what I call resource flexibility. This one's intuitive. You might have some batteries on site and you're able to reduce your grid draw because you've got fully charged batteries and for that limited amount of time that your batteries can run, you can locally power your data center or part of your data center. And the fourth kind of flexibility is what I call adjacent flexibility. Adjacent means not on site. You might have, for example, at the same transmission node, you might have neighborhoods with batteries or Nest thermostats and you may be able to aggregate them all together and provide a little bit of flexibility to that substation, for example, and the utility might be willing to treat that as flexibility that counts toward the data center's own flexibility. So these are the four types. The first three are on-site. The first two are computational, temporal, spatial. The third one is on-site resource flexibility. And the fourth one is adjacent flexibility. You put all those together and I sincerely believe we can make AI a flexible resource and Emerald AI is the software layer that sits above and enables this flexibility. We are orchestrating the temporal and the spatial computational flexibility. We are co-optimizing that with the resource flexibility, the batteries on site for example, so that you can best harmonize what you're doing on the compute side with what you're doing with the resources on site. And my sincere hope is over time we will also harmonize all of this with adjacent flexibility and all the other resources that are off-site.
So is your software already doing this or set up to do the adjacent part of it? Because that part I'm really fascinated with because you talk about a hero path, right? The potential for data centers to pay for reductions from customers. And to be very clear, and people will get tired of me saying this, but I'm going to say it every time I talk about this on a voluntary basis. Nobody will ever be required to do this. If you just want to pay a lot for electricity, you just don't care. That is your right as an American and I will defend it. But if you would like to lower your bill by participating, like you could actually have data centers paying for thermostats and batteries, and this is what markets are all about, right? There's a price for that. You figure out what the price is, what's the price to add another increment of battery on site. If the price is lower to put money into the pockets of the people at the neighborhood just across the way and be a hero and save money, right, the data center, I mean, you talk about a win-win-win all around. I'm really fascinated by that piece. I'm really glad that you laid it out with because I was going to ask about it, but you gave it without a prompt. I'm curious though, is that something that is like part of the software or is your software more like one and two, temporal, spatial, and then that's done by somebody else?
So look, right now, if I'm being perfectly transparent, Emerald AI is focused on flexibility at the data center. So we're doing temporal flexibility, spatial flexibility, and coordinating with on-site resource flexibility, right? Emerald AI makes it possible to co-dispatch an on-site battery alongside with your computational flexibility. So, we do the first three, but I'm a big fan of all the different buckets. Let me just say a moment on this hero path. Look, I think data centers are already striving to be heroes. My friend Chase Lochmiller runs Crusoe. He's an investor in Emerald and they're a standup example of a company that, you know, comes to town to Abilene, Texas, for example. They invest in the community. They invest in the workforce. Chase actually has this vision of building even more generation than the data center will need so that you're actually reducing overall power system costs. Oh, and by the way, to your point, Doug, it would be wonderful if data centers through their high willingness to pay for their compute costs are also willing to therefore subsidize flexibility in the adjoining communities. And another good friend, I was recently on a panel with Justin Lopez at Base Power. Base Power is an example of a company that is putting together a range of battery resources in a neighborhood or in a community and they're able to bid that in or dispatch that as an adjacent flexibility to data centers. So, I'm just delighted that all of these great innovators are coming with solutions. Emerald seeks to be this glue that makes it possible for the data center itself to flex and to play nicely with all that adjacent flex out there.
