Turning AI Data Centers Into Grid Allies with Emerald AI

Cody, thanks for having me.

Emerald AI seeks to make AI data centers, what NVIDIA calls AI factories, power flexible. That means is we want these AI data centers to modulate their power consumption on command. So when the grid is stressed out and the AI data center is able to reduce its power consumption by a certain amount, 25% for a few hours, well that data center is then going to be able to connect to the power grid more quickly, access a larger power interconnection, help support the power grid stability, and oh by the way, potentially reduce power bills for the neighboring communities. We believe that flexibility should be one of these core capabilities that every data center can have because data centers in addition to being the biggest new user of electricity for power grids could actually be the grid's greatest ally.

Well, first of all, it's always helpful to think through historical analogies. And so, as you've said correctly, Cody, like America for the last couple decades has had flat electricity demand, and now we're starting to see this ramp up in demand. And so, we think to ourselves, when was the last time we saw something like this? And you're right, the 1950s and 60s, we saw air conditioners and other forms of industrial demand drive up electricity load year after year. And our electricity system had to expand to meet that load. So in that sense there's a historical analogy. You know we've done this before. We move mountains to make sure that everybody could be powered by an air conditioner and now we need to move mountains to make sure that AI data centers can also be powered. The single most important economically productive electric load in human history. But I think the historical analogy probably ends there. You picked out one reason why the analogy doesn't work because air conditioners were diffuse but AI data centers are particularly concentrated. Now to be clear, maybe AI data centers going forward will evolve where there many more edge or smaller inference focused data centers providing low latency services and perhaps it will look more diffuse and less concentrated in the form of what we see today 1 gigawatt or more campuses mega campuses focused on AI training. But nevertheless, setting that aside, there are a bunch of other reasons why I think the historical analogy can kind of get in the way of preparing for what is an unprecedented surge. First is I personally don't think AI is what we call a normal technology, a technology like any other. I don't think when Elon Musk says that what we need to do is secure more power than America uses in entirety today or secure more power than the world uses in its entirety today to power compute. I don't think he's exaggerated. I don't think that the demand surge from AI data centers will resemble any demand surge from any technology before because AI is likely to use more energy than everything else combined by orders of magnitude. And the second reason that the analogy probably fails is AI is a fundamentally different energy user than any other technology that's come before. AI can move its computations around the country and around the world at the speed of light. You can't do that with air conditioners. Can't even do that with electric vehicles, right? Can't really decide to charge your car in Indiana one night in Virginia the next night depending on where the grid conditions.

There are, and we should talk through some of those constraints, the constraints that limit what flexibility operations you can do. But just to close this point, we do ourselves a service by remembering hey in history we have managed to meet large loads. We do ourselves a disservice by saying we just need to do what we did in the 50s. What we're about to face is something unprecedented. And for as many people who talk about whether AI infrastructure is a bubble and whether it'll pop, I encourage folks to think about the opposite question. Are we underestimating compute needs and energy needs? And are we therefore being too small-minded in what sorts of changes we need?

I've spent the last 15 years in the energy space building large-scale energy infrastructure. But I will say when I was an undergrad at Stanford, I had a left or right decision. Study AI, which was one of my passions, or study energy and physics. I chose the latter and that set me on one trajectory for the next 15 years. I did my PhD in condensed matter physics focused on solar technologies and then worked on building large-scale solar and other energy infrastructure around the world. I was at ReNew Power, India's biggest clean energy producer. I was their chief technology officer. They're a publicly traded company. I was at Orsted, as you mentioned, the world's largest offshore wind producer, and I was their chief strategy and technology officer. And we built multi-billion dollar pieces of energy infrastructure. So, most of my life was build big projects to produce more energy. And the big cognitive shift for myself was, well, can we build our way out of this? If AI is truly going to be this unprecedented energy user, it's certainly going to need far more energy generation. But if we only build energy, I fear that we will face a twin dilemma. And we should talk a lot about these two twin problems. Problem number one is can we build AI as fast as humanly possible? And problem number two is can we make energy affordable for everybody even while AI eats the grid. I realized that just building our way out of this wouldn't work. That instead of just focusing on the supply side where I have been for the last 15 years, I also wanted to focus on the demand side. So, Emerald AI was born out of this shift. Actually, about 10 years ago, I wrote a book called Taming the Sun: The Future of Solar Energy. And I had a kind of throwaway half a chapter about wouldn't it be nice if data centers could move their computations to where solar energy was abundant? In fact, you could follow the sun over the course of 24 hours. That idea has kind of come full circle where we're not quite doing that at Emerald, but it's a related concept where by adjusting demand, you can actually squeeze far more juice out of your existing energy system and solve those twin problems, the speed to power crisis and affordability crisis. So, that's where I am. About a year and a month ago, I founded Emerald AI. I was fortunate, you know, I consider as co-founder Radical Ventures the firm that originally seeded us to build this company and very quickly we built an impressive and I'm so grateful for the syndicate of investors whether it's Google's chief scientist Jeff Dean one of our angels or Fei-Fei Li at Stanford one of the founding professors of the AI revolution. NVIDIA was a crucial partner and investor and ever since we have been working with partners and investors to build Emerald AI into what I consider to be an ecosystem play. I don't really think we have competitors. Instead, we seek to be a picks and shovels play to enable everybody else to make money. We want NVIDIA to sell trillions of dollars more GPUs. We want Digital Realty to build far more data centers. We want Oracle to be able to offer far more cloud services. We want everyone to make money by building AI much more quickly and squeezing more juice out of our existing electricity system.

Exactly. We built an end-to-end software product. This end-to-end software product enables a data center to communicate with the power grid and be power flexible. Now, to be clear, our end-to-end product also involves functionality for the utility or the system operator to use in order to observe, monitor, and dispatch the flexible data centers on their network to modulate their power consumption. So we are that software layer that unites these two multi-trillion dollar networks. The multi-trillion dollar power grid and the multi-trillion dollar network of emerging AI data centers and fiber optic networks. Those two basically don't talk to each other today.

And the power grid seeks to always supply whatever that data center needs. We seek to be the intelligence that connects those two and acts as that interface layer so that the power grid and the data center infrastructure can be co-optimized and we get the best out of both infrastructure networks.

The product is indeed called Conductor. Cody, you're a product guy. You're probably also a product marketing guy. So, let me say this. I have put not a great deal of effort into the nomenclature. So forgive me because a lot of this came off the top of my head. But yes, the platform is called Conductor because what we aim to do is conduct an orchestra of available resources and flexibility tools to achieve the power behavior that we want.

You absolutely are. Let me just back up and say we stand on the shoulders of giants. Look, although we're a one-year-old startup, there have been these pioneers like Google who have been developing tools for very sophisticated orchestration of compute resources for goals such as, you know, how do you achieve 24/7 clean energy? They've done a lot of great work on this and power flexibility is one of those advances they've made. It's one of the reasons we were proud to invite Kate Brandt, Google's chief sustainability officer, and Jeff Dean, their chief technology officer, as our angel investors. The point you made, I think, is absolutely right. There's already a lot of sophisticated work on making sure that within one of these two multi-trillion dollar infrastructure networks, the data center and fiber optic network, it's highly optimized, but there isn't a lot of work to co-optimize the power grid network and the data center and fiber optic network. So, as you correctly said today, if you're Meta and you're running a service like Instagram and for whatever reason there's congestion in one of your data centers or you need to reroute traffic for one reason or another, they do that with an extremely sophisticated set of tools. That's because what they want to do is optimize the performance and uptime of their service for all the users who want to scroll Instagram while taking best advantage of all the server infrastructure they have, server utilization and, you know, best utilizing their assets. But nobody today makes the kind of decision that ah today the power grid is facing stress or may face stress in the next few days and those power lines are running at well below full utilization today but tomorrow they might be running at full utilization and we should make decisions on what computations to run at what speed and in which locations based not only on whether the servers are available and the fiber optic network is available but on whether the power line network is constrained and the power plants are hitting their capacity.

Let's back up there because you mentioned that what we do might relate to the cost of power, which it actually really doesn't. And so I want to back up and just make sure your audience knows what it is we're talking about here. I brought up early on the twin problems. One problem is you can't get data centers connected to power grids fast enough. The other problem is everybody seems to be paying larger rates, higher rates for power. In Columbus, Ohio, in 2025, the average household energy bill went up by $240. Largely attributable to data centers coming to town. Why does this happen? The reason this happens is because a data center comes to a particular community and says, "I want to plug into this part of the electricity grid." The grid, the system operator, the utility running the study to interconnect them, says, "Well, you're a 200 megawatt data center." And on the worst moment of the summer here in Phoenix, Arizona, when a million air conditioners are running, we don't know if we can serve you at that moment. It might only be an hour, but we just don't know if we can serve you. So, we're going to make you wait for seven years, and we're going to build out our network. We might have to reconductor a power line here or build a substation or build a, you know, expand a peaking natural gas power plant's capacity so that we can serve. And if that data center is completely inflexible, the grid's doing the right thing. Now the result of this is you get fewer AI data centers built in America which is bad for American competitiveness in this race with China which has 400 gigawatts of spare capacity by 2030 to build AI and you raise everybody in Arizona. You raise everyone's power bills because everybody pays for that reconductoring of a line, the expansion of the grid network. So this is a bad outcome on all counts. But if that data center is willing to be a little bit flexible, that data center says, "Look, actually on that worst day of the year, we are actually willing to ramp down our capacity, our instantaneous power draw, we won't do it for most of the year, 98-99% of the year, leave us alone." But for that rare moment where you're really congested, we will help you out. Well, then you can connect the data center more quickly. The utility may say, "I can fit you on today's network right now. Come on in." And I don't have to increase everyone's bills. Actually get to reduce them because everyone's bills comprise two components. One is the supply of energy, the fuel cost for example of natural gas and the other is advertising paying down the cost of the fixed asset base. The grid lines have already been built and that you're paying for it over the next 25 years. If the fixed asset base remains fixed, right? We're not building out more grid or at least we're building it out more slowly and the variable revenue that comes in increases because these data centers are paying more dollars for every kilowatt hour of energy they use. Well, then there's more money in the system to pay for the fixed asset base and everybody, your grandmother's power bill goes down. It doesn't go up. Just to back out, this has nothing to do with the cost of power. That really is irrelevant to a data center today. A data center is using GPUs that are 100 times more expensive to idle for an hour than the cost of the power that you would save. Right? We don't care about the cost of power. One day, by the way, we will care about the cost of power. That day may be in a year or three years when the cost of inference and the demand for inference rises so much that, you know, we really need the inference costs which are now approaching the marginal cost of energy. We need the inference cost to be rationalized. And so maybe you'll want to save energy cost by computing at a time when energy is abundant and not computing at a time when energy is scarce. But today that's not the problem we're solving.

As you correctly said, there are these three kind of domains of flexibility that we seek to harness. The first is temporal flexibility. So you might have certain computations that you can just slow down or pause for a certain amount of time. Middle of the summer in Phoenix, Arizona, all the air conditioners are running and you need to give the grid 3 hours of relief by reducing your power consumption by 25%. This was the first demo we did that was published in the scientific journal Nature with the Electric Power Research Institute and Oracle. And what we showcased is well the one way to do it is to look at all the jobs that are running and see are there some of these jobs that customers are okay slowing down. Maybe you're fine-tuning a model and that model fine-tuning doesn't have to happen right this second just like you mentioned the operation of enhancing the quality of images is another batchable operation that you can probably run overnight.

In that particular trial, we partnered with our friends over at Amperon. Sean Kelly, the CEO, is our adviser. They run fantastic grid forecasting services. So thanks to Amperon, we kind of knew a week in advance when the stress event was going to happen. So if you're a data center operator and within your four walls, you've got as your tenant a cloud provider, let's say it's Oracle, and Oracle has some customers, Emerald software is going to receive some forecasting data and relay that data and say, you know what, this is going to happen a week from now, two days from now, etc., and give you a little bit of heads up so that you and your customers can prepare. Now, let's say you've done some of the preparatory work and you have at least a couple workloads running that are possible. When the event happens and the grid says, "Oh my goodness, I'm reaching the stressful point." They're going to send a signal and our software Emerald is going to receive that signal. We have a module we call GridLink. So, GridLink receives the signal and it says, "Okay, thanks utility for letting us know what you need right now is a 25% power reduction for 3 hours. We're going to work on that. We're also going to communicate back to you that we've achieved that target by proving to you the auditable log of the telemetry so that you are very confident that we did what we promised you we would do." And then to the data center now Emerald provides software throughout the stack to provide them visualization tools, dashboards, monitoring simulations. We have a simulator that simulates, hey, how would you actually achieve this? Some compute customers may want everything in addition to our orchestration software that Conductor offers that actually slows some of the jobs down. Other customers may say the visualizations are really helpful and I'll just go ahead and do my scheduling and orchestration myself. There's like a menu of options for customers. In this particular demo, we went ahead and did everything. So there were a range of jobs that were running and the chief AI officer of Databricks which was the simulated customer in that case ran us through a bunch of representative workloads that were running and the representative priority levels for each of those fine-tuning or pre-training jobs or inference jobs. Some of these inference jobs were very high priority or very latency sensitive like don't slow them down, don't pause them. And some of the other jobs, fine-tuning jobs were lower priority and could be slowed down. So in that particular demo, we slowed down the ones, you know, down to an acceptable threshold for the customer. We didn't slow down the ones that weren't allowed to be slowed down. We had various operations involved. This is all public. We stopped some jobs. We reallocated the number of GPUs working on other jobs. And in other cases, we actually slowed down the actual clock frequency of some of the GPUs. So there many different levers we had available to us. And the net effect was that we kept the jobs running according to the customer specification. So the customer was happy. That's the number one priority. We also exactly met what the grid needed, which was a 25% reduction for 3 hours. And we were able to prove it all to both sides, to the compute customer and to the grid. And that's all been published.