Powering the AI Era

Hello everyone. Great to be here.

Well, thanks Rob, thanks Molly. I'm so delighted to be here at Radical's new headquarters. Congratulations to you. I know you're helping us inaugurate it so this is a good way to do it.

Emerald was founded on the intuition that AI doesn't just have to be the biggest new user of electricity on the grid. It actually could become its biggest ally. And so even as we go from today where AI is or data centers broadly, AI being a small component of that, are roughly 5% of America's electricity grid, by 2030 this decade data centers account for somewhere between 9 and 17% of the power grid. Within a decade from now, they could be 25% or more of America's power grid. They're a huge new user and we're ill-equipped to bring on all those data centers because in the worst case, those data centers can ask for so much power that the grid can't supply them. The intuition though, the reason we exist and want to make data centers flexible, is because almost all of the time, 99% of the time, there is enough power. There's so much power because the grid is kind of like a superhighway that is facing peak rush hour just a couple times a month. And so if we can take advantage of all that spare power during all of those hours except for the couple peak hours a month, well, we can just add the data centers to the grid today and not have to wait 10 or 15 years. So Emerald is the software platform as you said Rob that takes an AI factory as Jensen Huang calls these specialized AI data centers and makes them controllable or flexible so that during those rare peak hours you can reduce the consumption of an AI factory and we'll talk a little more about how you can do it. There's three main ways to do it. But if you can reduce the consumption of the factory during a hot summer day in Phoenix, Arizona when the temperature is over 100 degrees and all the air conditioners are running, well, then you could have fit that data center on that grid, right? Then...

A lot of people look at data centers and they say these are the most inflexible entities you can imagine. Right? Because every data center in history has said, 'Leave me alone. I want my power connection and then I never want to think about the power grid again.' And this has been justified. If you're a large cloud compute campus heterogeneously doing a lot of different things, you know, serving a hospital, running a PayPal transaction, you really never want to think about the power grid. It should always be on. But with AI factories, I believe that these are maybe the most transformatively flexible assets that we've ever even had on our grid. They're large and massive entities compared to the small assets like your home thermostat that utilities might ask you to flex during a flex alert on a hot day. It's really hard to get a million thermostats to all act in the same way. But one single data center is like a city. They're electronically controllable in contrast to other large industrial loads like factories. And so we can, you know, using computers, using a laptop, we can virtually control the power load of an entire data center. And they can move their economic activity at the speed of light from one location to another, which no other economic actor can do. You can't charge your electric vehicle one night in Indiana and one night in Virginia unless you drive from Indiana to Virginia. But we can do this rapidly. So the three ways that you can make an AI factory flexible, the first is what I call temporal. You can slow down certain computations. Computations that have a little bit of delay tolerance. They might be batch inference. There might be a fine-tuning run that doesn't have to terminate right this second. You can slow them down by slowing the clock frequency of the GPUs by pausing or rescheduling a job. So that's temporal flexibility. In one data center, you reduce its power use for a couple hours and you do this in a way that's acceptable for the customer based on the workloads they're running. The second way is called spatial flexibility. This is where you might have some serving inference workloads that you really don't want to delay for a couple hours or minutes, but you could delay for milliseconds because the latency tolerance is acceptable. And so you'll move an inference query from one location to another location. So that's spatial inference where in the one location, Phoenix, where it's really hot, you move an inference query over somewhere else where the grid is not constrained because not every grid faces rush hour at the same time. And that helps you to meet the needs of the local Phoenix grid and therefore fit far more data centers. And the third way is the one that's most intuitive to most people. It's what I call resource flexibility. Resource flexibility is where you might have some batteries on site or a generator and you're taking these actual physical energy resources and running them, discharging the battery for example so that you draw less from the grid at that moment. Right? If you only relied on that on batteries for example, it would be prohibitively expensive. You'd have to overbuild a massive battery to account for every possible scenario of the grid facing peak load. But if you can use that alongside the computational flexibility, then you have this holistic flexibility solution where either one data center or a network of data centers, AI factories I should be clear, are flexing in harmony. And the platform we have, I said harmony for a reason. The platform we have that makes this all possible is what we call Emerald Conductor that fuses together these different instruments. The on-site resource flexibility, the computational temporal flexibility and the computational spatial flexibility putting it all together. So customers of AI compute are happy with their performance. They get the tokens they want at the rate they want, but the grid gets precisely what it wants when it has a peak load event it needs to reduce by a particular amount. And we can talk more about how we've demonstrated this in the field, but more importantly what that means for what the grid can handle going forward.

You're absolutely right. There is a mess of actors, right? But the economic value is so large. It's almost like a magnet where all of the paper clips are kind of aligned toward that inexorable magnetic force which is you take a data center and you get it access to power just one year early or you get a larger power connection, a 200 megawatt data center goes up to 230 megawatts and we get to deploy liquid cooling and next generation Nvidia GPUs and suddenly you've unlocked billions of dollars of net present value. And so I believe first the value proposition is so compelling that we're getting the whole ecosystem to work together. Second, it's been wonderful to have really terrific partners, many of which you have helped us find. Nvidia is maybe the most strategic and key of our partners. Nvidia is an investor and they're also a key strategic partner. And so they have what's called the reference architecture for AI factories. This year at GTC, the Super Bowl of AI, Jensen Huang announced this DGX architecture, and said that going forward, every AI factory should have DGX Flex, a capability that makes an AI factory capable of ramping up and down to meet the grid's needs. And we were delighted that they announced us, Emerald, as the partner, the software partner for DGX Flex to make this possible. And so with them and then later on we announced a range of additional strategic partners who have invested in Emerald, everybody from the folks who develop the on-site energy resources for that resource flexibility, whether it's GE Vernova or Siemens or Eaton, to folks on the other side, the electric power utilities like National Grid, one of our investors. It's been important to bring together this entire ecosystem and I think our goal at Emerald is to serve as that picks and shovels accessory, the software platform that helps everybody else make a lot of money because there's so much value here. We want more AI deployment. We want electric power utilities to get new lucrative customers. And at the same time, we want local ratepayers in communities not to have rising prices because the AI factories are such good grid citizens that they actually come onto the grid, avoid really expensive grid upgrades and therefore end up subsidizing the bills of all of the communities. And so rather than having a data center come into town and raise your bill, it should actually lower your bill. And every governor in the United States and every world leader should say, 'Wow, I would love to have a power flexible AI factory in my community.' The value unlock is on the scale of trillions if we can do this right. And so Rob, your point is absolutely valid. There are so many actors that have to come on board, the AI users, the cloud operators, the builders of data centers like Digital Realty, our partner, as well as the electric power utilities. But thanks to these partnerships, thanks to Nvidia's leadership and thanks to investors like yourselves or Energy Impact Partners which brings together utilities from around the country, I think we've been able to get everyone kind of on the same page and focused on that magnetic attraction.

Totally. We've been delighted to get to do five commercial demonstrations now at five commercial data centers around the world. We started in Phoenix, that's why I kept bringing up that hot summer day in Phoenix with the utility Salt River Project. And our partners Oracle, which had the data center, and EPRI. EPRI is the research organization that brings together electric utilities and they have a consortium called DC Flexibility. DC Flex, data center flex. So, we went to Phoenix and we showcased that at one data center when the grid is spiking, it was peak load, we can reduce consumption by 25% over a three-hour period for that cluster of GPUs. We then went on to Chicago, Virginia, Portland, Oregon, and London, the United Kingdom. And in each case, we tried to build on what we had previously done, right? In London, for example, with Nebius, which is one of Nvidia's cloud partners, this brand new AI factory in London needed to prove to the national grid utility there that no matter what grid event it faced, it could support that grid. So, just to give you an example, one day in the middle of our night, so all of our engineers are sleeping, 4:00 a.m. comes the signal from the National Grid utility that there's been a lightning strike. There wasn't actually a lightning strike, but they pretended like there was a lightning strike and they said within seconds we want this AI factory to ramp down so that we're supporting the grid during this, you know, terrible weather catastrophe. And because the Emerald Conductor and its autonomous agents are operating independently, that is exactly what happened. The power reduced in line with the lightning strike request. The workload performance actually across the dozens of workloads that were running at this Nebius AI factory cluster of Blackwell GPUs, the ones that had been designated high priority, please don't touch these workloads, they have to finish right now, they weren't touched, they continued to proceed unimpeded whereas other workloads that had been tagged as more flexible, those were throttled in accordance with what was acceptable. Another fun example in London was the soccer game, the football match in England draws a lot of people to watch television and then at halftime they go to their kitchens and turn on their tea kettles to make tea. And during that 1 gigawatt spike on the grid we showcased that the AI factory can actually ramp down perfectly during halftime and during the 2 gigawatt spike at full time when everyone finishes watching the game in order to stabilize the grid. National Grid, our partner, said after this particular demonstration, they said, 'You know, if all of the AI data centers acted in this way, we would be tripping over ourselves competing to attract them. We wouldn't be making them wait years. We would do everything in our power to get them connected immediately.' In fact, we've now spoken at the highest levels of the UK government as they do their connection reform process to make it possible for these flexible data centers to get far faster connections. And so whether that's there or in, you know, the last demo I want to talk about is one in the United States where with Oracle we showcased moving AI workloads between Virginia and Chicago during the Virginia winter peak load moment. And this was a great demonstration because it showcased with our utility partners, Dominion Energy and Commonwealth Edison in Chicago that if one utility has an issue, but there's a facility elsewhere that can help to take the slack, then in any localized setting, the AI factory grid or the AI grid that's connected by fiber optic cables can act as a superstructure over and above the physical electricity grid. Look, Molly, you and I have talked about this a lot. We're never going to build out our physical grid backbone as fast as we'd like. Even as America re-industrializes, even as we invest heavily, it just won't happen quickly. Fortunately, we have this virtual grid that lives on top. It's less visible, but no less important. And that superstructure can make it possible to move photons at the speed of light. Bits rather than watts become the mechanism to guarantee reliability and low costs. And the last thing I'll say is there is a risk and maybe we'll come to this in a moment. There's a risk that America's two grids, the physical electric grid and the virtual data grid, they move apart. And if that happens, it's a terrible outcome across the board for our ambitions on energy and AI. So that's kind of why we exist to unify them.

In many ways, Molly, as you'll certainly appreciate given your deep policy experience, this is a wickedly hard policy problem, even alongside a wickedly hard technical problem. And the wickedly hard policy problem arises from the fact that the electric utility industry in the United States and around the world is just fundamentally different than what you would intuitively imagine. For example, the chief strategy officer of National Grid in the UK, their utility said, 'In any other industry, if you tell me, here's a new well-paying customer. Would you like to have him on your system?' We'd say, 'Yes, in a heartbeat, let me do everything I can to connect him.' And yet, the way the utilities actually work is they say, 'No, no, no. Please stay far away and wait 5 or 10 or 12 years while I think about how to connect you.' So I believe that the most critical unlock from the regulatory side is an offer that the electric utility industry can make and that offer looks like this. Hey, if you're a data center, you're an AI factory and you're willing to be flexible, I will give you some remarkable benefit. You will get a larger connection amount or a faster connection speed or both. In return for that, I expect that AI factories will jump at the opportunity to be power flexible. Now, electric utility regulation is extremely complicated, but there's a simple political calculus over and above this, and that's at the level of governors and their, you know, politically elected representatives. They say, I'd love to have three things at the same time, but I can't have them right now. I'd love to have a lot of AI data centers that will drive economic development, construction jobs, tax revenue, and innovation and competitiveness. Two, I'd love to have my residents have low energy costs. Three, I'd love to have a grid that works so that when a winter storm comes that there's not a blackout. If any of these three fails, I'm a politically elected representative. I get in trouble. Flexibility, power flexibility of AI factories is the only way to achieve all three of them. And so even though it's wickedly complex at the utility regulatory level with many layers and years of regulatory rulemaking to allow this new offer to come onto the table, which is if you're willing to be flexible, we'll connect you, there is a strong and immediate political reason to get this done. And that's why I'm seeing across the US, across 3,000 utility territories in the UK, in India, I'm seeing more movement on this faster in the regulatory world than I've ever seen. Texas, for example, is currently actively debating whether they're going to allow flexibly controllable data centers to connect at a larger capacity than those that aren't. We are currently working with a utility to offer a first in the nation program where we'll flexibly connect larger capacities of AI factories if they're willing to offer that service and the utility is very excited to serve more customers and we'll do this utility by utility state by state until everybody realizes this is by far the best way to go.

For everyone watching, I actually did

No, no. I'm glad you went off piece because where you grew up, Virginia, your home state, is where we're going to deploy with Nvidia the world's first power flexible AI factory at commercial scale. We didn't tee this up, guys. We actually, right, it's happening later this year. It's going to be on the scale of 100 megawatts and it's in Dominion service territory, which I think is your home utility.

And Dominion is also, you know, the capital of data center development. And so what we're going to showcase in collaboration with PJM, which is the largest US grid, it's on the east coast, along with Dominion the utility and EPRI, this consortium, and our partner Nvidia and Digital Realty, which is building the data center, is this 100 megawatt data center can do whatever you want it to. And so when there's a winter storm, when there's a lightning strike, when there is a peak load event in the middle of the summer, PJM last year had record demand, I think on July 16th, well, we'll showcase that this AI factory will ramp down smoothly and precisely. That's what I think needs to happen is a true commercial scale demonstration of this capability. You know, we've done this at five data centers around the world, but we haven't yet done it at the 100 megawatt scale. I think this will blow the doors open on the potential for this technology once we showcase this. I recently testified in your home state, Virginia's public utility commission proceeding to say this technology is real. It's coming. And as regulators and the chair of that regulatory commission is a tremendous ally of these sorts of forward-looking regulations, you know, I said to Judge, Chair Bagot, hey, this is coming. This technology is real and Virginia can be the leader. I think I called Virginia, you know, using a poor state metaphor, beacon on a hill, in terms of showcasing to the rest of the country and the world that you can be first in bringing these flexible data centers to your territory and expanding the effective capacity of the Virginia grid without building a single transmission line. And that's huge because that allows them to keep rates low because there's not expensive new infrastructure that everybody has to pay for while at the same time bringing in all the economic development benefits of data centers.

Look, I'll say this and I've said this elsewhere, so it's not just because I'm on your podcast. This company doesn't happen without Rob, right? Sure, there were...

No, no, I'm actually not. I actually think you changed my life. And I'm glad that we had some great times at Stanford. I won't talk more about them for fear of incriminating both of us.