Raise Summit: Sovereign AI Piece by Piece - Hydra Host, NetApp, Radiant, Coatue & More

Excellence, good afternoon everyone. I'm Thomas Kurian, managing director at Bank of America. And with me on the panel, you know, George Kurian from NetApp, Maya Matari from Radiant, Aaron Gibbs from Hydrohost, Ben from Polarize, and Max from Coherent. So, it's pretty clear that we are going through what one could recognize in one of the most fundamental transformational technological shift the human race ever experienced. And as we going through this shift, it's increasingly recognized that the implications go far beyond technology and affect everything we do. And in particular, relationships between states. We've seen in the recent past basically governments curtailing access to frontier models. We've seen governments taking proactive policies in order to reduce their dependencies on other countries when it comes to AI. And so, what is sovereign AI? What does this entail? And how does it affect the stack? This is basically the purpose of today's panel. First question, and we had quite a bit of a debate with my panelists on this topic in preparation of today. I'll ask first each of them to give their definition or their understanding of what is sovereign AI. And maybe George, I ask you to start on this.

Maddy, you want to comment?

Yeah, I think I mean the key thing with sovereign AI, the way we see it is that it's not really it's not an end state. Right? It's a continuous spectrum. And different organizations or nations will take decisions in terms of how much control they want to take off the different components of AI in general. And how much do they want to de-risk as they go through that process? So, it's a continuous spectrum in that regard, I think. And as AI develops, that spectrum is constantly developing as well. But the key point of it is around control and putting controls in there.

Aaron?

I would probably start with data residency. I think that that's going to become an increasing very durable trend, mostly because the technology shift that happened with GPUs is fundamentally anti-cloud. And so now you actually have the ability to do it, so you will do it. So I would probably not focus too much on the model layer or even like AI native layer like companies in that zone. Probably focus on like the actual data center infrastructure, the chips, data residency, things that reflect actually into national identity because those are like the precursors to the AI native ecosystem. And like within those countries. So I'd say like chips, data, money kind of staying in the country as opposed to like you know Amazon running a data center space in you know Luxembourg and the money going back to Amazon. I would say it's much more about developing the fundamentals around the ecosystem in that country and then from there you start seeing more of something specific to whatever kind of expression they want to put into AI that reflects their nation state.

So we'll pick up on obviously you started going into my second question so we'll definitely come back on that. Ben, can I ask you from your point of view?

Yeah, sure. Maybe from a European perspective especially, I think the Europeans are catching up now getting the importance of the AI workloads, right? So somehow especially from Germany we are a little bit in a recession. So where the workforce is getting older and older so we need to watch out that we not losing the train when it comes to AI. So what I'm heading to is that AI is in the end a kind of critical infrastructure especially for Europe nowadays and the next years and that we should see it as such, right? So the token production, so the underlying utility in the end. So I see the tokens as a utility and it's same like gas, electricity, water and being dependent on the production of your utilities is a key or should be a key. Right now, we are heavily dependent on hyperscaler infrastructure. So, coming back to Aaron, I see the same that it's also the physical infrastructure that needs to be sovereign in the end and not run on hyperscalers or I mean we won't avoid that, but that we should come up with alternatives for that. Yeah.

Thank you. Max, to you then.

Yeah, I mean I think all of these gentlemen said exactly how I had thought about it up until maybe 2 weeks ago and the only difference is there's maybe now another layer. Like we're speaking at the nation-state sort of macro layer of where things are located, where it's hosted. The new one which in panel discussions over the last 2 days I feel like has come up a lot more is sovereignty as sort of data sovereignty and do you as a company own your data? Not necessarily do you as a nation own your AI infrastructure. So, it's just another angle that I think seems to come up more and more. Are you using open models? Do you own those models? Do you own the weights or are you not? Where is your data going / who owns it is another angle of sovereignty which as a tech investor I find to be interesting.

Perfect. So, it kind of informs the next question because you and Aaron you started alluding to this, right? Which is that one could take the view that part of the stacks which are much more critical than others in order to achieve sovereign AI to the extent it is achievable then should look from all the way down to the chip layer all the way up to the application layer. I'm sure that each of you will have potentially different views as to what is more critical in terms of reaching this level of independence and therefore sovereign AI. And so, maybe Madi I'll ask you to start with that and then I'll ask some of you to add to this.

This is a very long debate, man. Look, I mean, for me, I always think about it from a control plane perspective. And regardless of where the physical infrastructure is located, regardless of what you're running on top of that control plane, as long as you have sovereignty over that control plane that is managing everything that is above it and below it, I think that's the most critical part in the sovereign AI stack for any organization, for any government, or nation doing that.

Okay. Maybe on this very topic, can I ask you, Ben, to comment as well?

Yeah, as said previously, so being in charge of the infrastructure, so I mean, what we do at Polarize is trying to think it end-to-end for our customers. So, from owning the real estate, owning the infrastructure for it, owning the GPUs, I don't know, Kubernetes layer on top of it, and what we're also building is a Polarize Drive where we hosting open source models. So, of course, we could have a debate in each of these layers of how sovereign and independent we could be. But starting maybe with the infrastructure and being maybe not dependent from a European perspective as well, not dependent on hyperscalers, owning and running it is a really critical thing, I think, because the US Cloud Act might claw everything in the end. So, we are not fully secure of that, yeah.

Okay, maybe Max, I ask you to add to this. And just to add to this debate, right, which is that depending on indeed there are potentially parts of the stack which are more important than others. But maybe if you want to integrate to the answer an element of how achievable it is when you know the level of complexity of the supply chain, right? And so, even if we were to agree on this panel, there are some parts actually which are much more important than others. Actually, is it just achievable knowing that, you know, just if you look at chip manufacturing, the complexity of the supply chain is such that, you know, how independent can you really be?

Yeah, I mean it has, I think, governments have seen it as an evolution of trying to figure out what sovereign AI actually means. Clearly, chip export controls were part of that equation. Now, there's discussion of what companies are benefiting from what models. Are American companies benefiting from Chinese open-source models more than the Chinese companies that created those models are? Is a question. The actual location of the data centers, who built the data centers, is a question. Then I think where I had spent a lot of time and continue to find to be an interesting question is like we talk about all of the bottlenecks in building out AI infrastructure, power and energy being one of them. Just to take one example of gas turbines, which is a type of equipment that a lot of people use to build power plants in the US and abroad. There's three major suppliers and one of them is American, two are not. And a lot of that power is being shipped to American data centers and I don't know. It's an interesting question. We're talking so much about chips. We're not talking about where is American equipment allowed to go and where is European, German equipment or Japanese equipment allowed to go because right now a lot of it is coming to the US still.

Maybe George, if you can add on this and maybe as well from a NetApp perspective. On this point of the part of the stack and how, you know, maybe with the current stack, what you offer to your customers allows to deliver an element of sovereignty.

And maybe to finish on this topic, Aaron, if I can ask you to comment and maybe just to explain from your point of view in terms of what Hydra Host offers, to which extent the solution also helps achieve this goal of sovereign AI.

Yes, so we have a number of sovereign clients. And it's actually governments that either their telco arm, their energy company, or some part of the they sometimes create a new agency. And so we provide the OS at the data center level so that they don't have to have any expertise in running the video gear. And also allows off-takers a standard way of accessing deployments in far-flung places that they have no interest in co-locating or didn't even know existed. But, you know, you'll take a GPU from, you know, El Salvador or you'll take a GPU from Panama because you're like, 'Why not? It's a GPU.' So, I think that the way that people view sovereignty is like one is like there's in economic speak like there's the opposite effect of the public commons tragedy fallacy, which is that like once you kind of break the shell of like someone who doesn't want to participate, it kind of like unravels kind of quickly. And so they obviously the positive effect of that is the like idea of like the environment of like that's why we have to have laws that, but also like goes the other way. And then so in some respects we have to admit that like China won the internet battle. Because one is that Americans have to realize that I'm an American. That and I'm almost Chinese. So that much of what we consider today free markets or like platforms that people build on is actually American goods being exported. And we don't view it that way. It's sort of like for example, the open oceans, right? So, all of the laws that regulate the way we do free trade, it's because we have, you know, 11 different navy fleets that sail around the world and says that this is open. So, we deploy someone in the Strait of Malacca, we deploy in the Mediterranean, we deploy in the Gulf region because we say this is open, right? So, that there's not like this kind of secular free trade thing of like we just all agree. It's like, 'No, that's actually not how this works, right?' And I think that what the Europeans have realized over the course of several administrations, you know, the most commonly accepted one is around NATO, that, you know, the Europeans just outside of the polls like, you know, they just kind of been not doing anything. And so now they realize that like, oh crap, we got to do something, right? And I think that that's good for the world because it reestablishes like the what he was just saying about like local economies being able to like develop. And I also don't think that this is the end of free trade. I think it's an iteration of like it's almost like kind of a reestablishment back to what it was in like the '50s and kind of pre-World War II stuff. And so I think it's like a healthier balance of that nation-state should be able to express themselves in their own local phenotypes and how they want to trade with the world. But that fundamentally the world all doesn't have to be the same. And I think it took American policy to kind of get there. And I think they're doing it in ways that are helpful and doing it in ways that are not helpful. And but I think everyone at this conference has to realize that GPUs are regulated in the US government as if it is a missile defense technology. Is it a missile? You know, can I put it on an F-35 jet and sell it to Germany and you can shoot it at somebody? No, right? But that's where it is. And once the governments decide that something is regulated that way, it generally doesn't go the other way. So like, governments have will that is exogenous to markets. It's a free agent actor in the market that is completely removed from market forces. And so the current posture of not just the US government, but Chinese government, the European governments, that this is a regulated technology that fastest ever in the history of probably Silicon Valley. And this is just something that everyone at this conference has to accept that like, there's a third player in this go to market and they're when is the government ever relinquish control of anything? So it's just something we all have to admit that it's going to be a durable trend.

Thank you. So next topic and I introduced the concept of the part of the stack which are potentially more critical to this sovereignty element than others. Obviously one question that everyone has in mind is the role of hyperscalers in that sovereignty element and knowing that not every nation or economic block has a proper hyperscaler. So the question is really in simple terms can nations achieve sovereign AI while using existing hyperscalers? And so maybe here obviously we have very different points of view but there is a concept behind this which was discussed during our prep sessions which I'll ask Max to comment first because this very interesting concept around rerouting which I'd love you Max to start with and then we'll obviously go to Maddie as well as Ben and finish with George if you're okay.

Yeah I mean I think for me these guys will be able to speak maybe more intelligently to it but for me as an investor looking from the outside I realized as I was thinking about this question oh there are already products that exist like an Amazon bedrock or a Microsoft Foundry or Google Vertex or Base 10 or Fireblocks or a lot of these businesses that are routing your model requests to specific models and so you may not whether you're directing exactly where that goes or whether you've built an agent that directs based on best possible use case this is I guess if we're starting from the dirt at the bottom all the way to the ethereal model layer at the top I think this is as far up as you can go to think about, hey, where what model is my data actually getting sent to and who created that model? That was a question I had thought about. I think model routing has become a much much much bigger topic of discussion in the last 6, 9, 12 months and it's a lot of what people have been talking about on the stage. And I think that plays into the sovereignty question as I talked about in the beginning. So, I don't necessarily know the answer on it, but it again complicates at what layer are you able to designate where things are and George, I think his business helps to solve that problem, but it is one that people are now just starting to deal with think about. Yeah.

Okay. So, knowing that and being conscious of the issues, I mean what does this mean and maybe Ben and Maddie, I'll ask you to come first in terms of is there a way to actually deliver properly sovereign solutions, knowing in particular that context or do you have to basically go through a completely independent infrastructure?

So, I think it's the second, yeah, having the completely independent infrastructure. So, looking at use cases from the past, I know that there are products from AWS or Microsoft or Google saying it's the sovereign package or whatsoever, but I mean, you had when was it last year? Was it I think Microsoft France the general counsel in front of the French Senate stating under oath that he cannot assure that the data stays with France or you have the Amsterdam Trade Bank blocking on the Google Cloud last year. You have OVH Cloud still being in a lawsuit with Canada regarding French data because they have a Canadian subsidiary. So, I think in order to avoid that there's only that one way from our perspective. And so, a clear no on the basic question. Yeah.

Madi?

Yeah, I think it goes back to how much control and how much de-risking do you want to do in achieving or going towards sovereignty. And if you believe that I need to control the infrastructure, then you can't achieve that on the hyperscalers. But then there's other camps that say, 'Well, hyperscalers build data centers in your country.' So, that's physical infrastructure in your country if that's how you're defining that. But then they control the control plane for that and that's where they so again, it goes back to like especially governments today, I mean, I see in sovereign AI they go through three phases. There's rhetoric, then there is policy, and then there's execution. I don't think we got to the execution phase yet in any of the different governments. I mean, maybe some of them have moved towards policy at the moment, especially around Europe in a few places. And that shift from policy to execution is where a lot of the decisions need to be made. And again, the decision I think should be very simple. It's around control and how much de-risking do you want to do for the future? Because if you have less control over a certain thing, whether it's the data or the infrastructure or the control plane or even the models you're using, then you're giving sovereignty away to somebody else potentially, right? And that's the spectrum that I think a lot of countries and even companies, I mean, sovereignty it's associated to nations, but it is a something that large organizations and enterprises should be thinking about as well. So, whatever they sit on that spectrum, that's where decisions get made in terms of the infrastructure hyperscalers and so on, I think.

So, maybe to conclude on this topic, George, from your point of view, the ability to deliver an element of sovereignty, again, which is discussed, right? There's different versions of it. It's gradual. It's not black and white, but using hyperscalers and going back to what you see from your clients and what you offer to your clients.