How do you have a killer app for confidential computing and have that application take off if it's not completely trustworthy and completely seamless to be able to adopt? We call it confidential computing now. In a decade, it'll just be computing.
Welcome to AI Confidential. I'm Aaron Fulkerson, the CEO of Opaque. This is the podcast where I sit down with industry experts to talk about the promise and potential of AI to create breakout industry leaders or to disrupt the current leaders. After all, whoever controls the data controls the industry. For this episode, I'm really excited. I had an opportunity to sit down with Mark Russinovich. Mark Russinovich is the Chief Technology Officer and Deputy Chief Information Security Officer at Microsoft Azure. Also who joined us was Mark Papermaster, the Chief Technology Officer at AMD. These are two pioneers in technology and computing, and I've looked up to for decades, really since the beginning of my career. I've known who these gentlemen are, so it was really exciting to have them on the pod. In this episode, we discussed everything from trust and attestation for sensitive data to the innovations that confidential AI can unlock. We even looked into our crystal ball to predict how confidential computing will eventually become the standard across the industry. I hope you enjoy this conversation. If you do, please subscribe to AI Confidential wherever you get your podcast, and also be sure to subscribe to our newsletter for updates. Thank you for tuning in.
So Mark Papermaster, Mark Russinovich, thank you both so much for taking the time to speak with me today. I've known who you guys are for as long as I've been, since I started my career. I've known of both of you, so it's a real honor that you'd take the time to speak with me today.
Well, thanks for having us.
Absolutely, it's a pleasure. And Mark Russinovich, you are the CTO of Microsoft Azure and the Deputy Chief Information Security Officer. You also have authored many books, including five novels, which I only learned recently, and I will be reading some of your novels because I read the background on them and they were pretty exciting. Furthermore, you were the creator of Winternals, which were like my favorite nerdy configuration tools for Windows way back in the day, so I give that a shout out because I love those tools. They were fantastic. Mark Papermaster, you have been a pioneer for decades leading technology and computing. You were a leader at IBM, then you went to Apple where you directly reported to Steve Jobs while you were responsible for the iPhone and the iPod, and you've been at AMD skillfully leading AMD into the AI future as the CTO of AMD for the last 11, well since 2011, correct?
That's correct. It's been 13 years. I've also had a chance over that time to work closely with Mark Russinovich, so thanks for bringing us together with you today, Aaron.
Why don't you guys talk a little bit about your collaboration over the years?
Yeah, well Mark and I have been working on confidential computing, primarily just in the collaboration between AMD and Microsoft in general, but with a strong focus on confidential computing for many years. Just in terms of a partnership where we actually act in some sense as almost as one company working on a project together. It's been an incredibly fun working with Mark and the rest of the AMD team.
Yeah, Mark, I share a passion for what we do together. I mean, you know, we at AMD we completely value our customer collaboration and Mark's given us great guidance. But what I want to add is we both share a vision of what does the industry need to do. So what might need to be one company working with another, but what really requires the industry to come together. And that is rare to find folks that really want to partner on going over and above and bringing the tide up so that all the boats can rise and how the whole industry can benefit. And that couldn't apply to any better topic than confidential computing.
You both have been very consistent in your focus on the customer, the downstream end user, and the benefit. So when you're talking about building up the ecosystem, it's a real demonstration of your desire for the world to be a better place through technology and the customer experience to be improved. And it's a testament to the character of both of you gentlemen. I'd like to just jump right in. We'll start with Mark P. Kick off by giving us your high-level point of view on confidential computing.
Well, confidential computing's been a long time incoming because we've been thinking, this industry, how do we provide more security broadly to customers? And it starts with the cloud because you think about people wanting to run their applications in the cloud. That means it's leaving their control, leaving their immediate premise. So it's really the first application. And it's great being on your podcast with Mark Russinovich because that's how I got to meet Mark and got to start working closely with them. Because of course, Microsoft Azure is a leader in understanding that need for security and really driving this confidential compute capability. I think Mark's incredibly articulate to probably kick off your podcast with even the definition of confidential compute.
Thanks, Mark. Yeah, so confidential computing, just fundamentally what it's based on is a hardware root of trust that creates an enclave or trusted execution environment into which code and data gets inserted or loaded, and can provide quote or attestation about what that code and data is. That code, the application, can then present to a key management service or to another application that allows it to establish trust, give it a key to decrypt the data, and then that can get processed inside the enclave free of visibility from the outside and also free of tampering from the outside. And the whole point of this is that you want to minimize what the code needs to trust to execute, and also provide the attestation that can allow another service to trust that it can decrypt and use that data without the risks of the outside environment, whether that's the operating system, the server administrator, the hypervisor, even, or a malicious insider or a hacker that manages to breach the infrastructure. The whole point in a cloud like Azure is to minimize the attack surface for our customer data. And so we've been working on confidential computing for many years. In fact, it goes back a decade to when one of our papers on exploring early use of confidential computing for analytics coined the term confidential computing.
A lot's been rapidly changing with the advancement of AI, both in the ease in which people can adopt AI, but then also with generative AI, novel uses like for weaponization or attack vectors with generative AI. How is this change the privacy and security landscape? Has the need for confidential computing and confidential AI evolved accordingly? And which either of you feel free to jump in on that one.
Well, I want to say, you know, Mark and I have talked about this quite a bit. I mean, confidential computing has been steadily growing in adoption, but AI is the killer app because you've got to know that you can trust your crown jewels. Because what are you doing with AI? It's your knowledge base of your company, or it's your own personal knowledge base of information you have. So it's clearly the killer app. And I just want to, before Mark will talk more broadly about it, I want to even build on the earlier comment that Mark Russinovich made. So you think about this killer app of confidential computing. So if you're going to put those crown jewels and run it in the cloud, and you want to know that it's safe, it puts an urgency on what we just talked about a moment ago, in that you have attestation. So every device knows that they're talking to another trusted device, so your whole computing layer is trusted. But honestly, the other key to confidential computing broad adoption, and this is what AMD and Microsoft have worked very, very closely on, is you can't put all these hurdles that people have to do to be able to run in a confidential computing environment. You need it to be run seamless. So you've got to have complete ease of adoption. Otherwise, you really limit how you have a killer app for confidential computing and have that application take off if it's not completely trustworthy and completely seamless to be able to adopt.
So confidential computing is fundamentally important to the security of in general. In fact, I've been talking with Mark Papermaster about this for a while, that we call it confidential computing now. In a decade, it'll just be computing. But for example, in Microsoft's own infrastructure, we're adopting confidential computing starting with their trusted computing base of Azure and moving into other services, including things like our payment systems. I completely agree with Mark that AI is the killer app. If you take a look at different AI scenarios, whether it's training or fine-tuning, the data that is going to go into training or fine-tuning can be a very high value that you want protected. The model itself that comes out of the training or the fine-tune model is also very high value, and you might want to protect it with confidential computing. And on the inference side, using the model, the data that goes into the model and the data that comes out of the model is also going to be very high. A great example is in speech transcription, where the data goes into the model and that speech can be extremely sensitive, and you want to make sure that whole process of the inputs and outputs are protected with confidential computing. And then finally, another scenario that is relevant for AI is multi-party computations, where you bring multiple data sets together from different parties and compute on them, train a model on them. It's the killer app because of the sensitivity, the value of the data, the value of the models, and also the fact that the architectures for these different use cases are fairly structured and universal. That means that there's not a lot of variability, and you can design systems with confidential computing specifically for it that protects that data end to end. It's much simpler than workloads where every workload has its own specific way of data flow and what sensitive data versus non-sensitive data that you have to go design a system for if you really want protection on a case-by-case basis.
And the point you make about data and models is a really critical one. We see often with our enterprise customers and large organizations that they get stuck, their AI projects get stuck in pilot. Because they'll do an internal pilot project with sensitive data, which is of course the most valuable data in most of these situations, but then when they move to production, they have concerns about how do we ensure that the data is not going to leak, that we're going to retain the ownership of the data, or as you pointed out, the ownership of the model. So this being the killer app is totally consistent with what we're seeing in the market. In my 20-some year career in tech, I've never seen technology with such tailwinds in terms of driving adoption. You guys are old hat in the confidential computing space. I've been in it for a year now having joined Opaque a year ago, but it's been remarkable just what the pickup and secular tailwinds have been in driving this. So what innovations can confidential computing and confidential AI unlock? Is there a potential impact on the economy and society? Mark P, would you want to start?
Well, you talk about autonomy. It's one of the drivers of confidential compute because to have trust that we talked about earlier, that you truly trace, you know, safe chasing your data, your models, this incredibly sensitive information. The question is who has control? Mark talked about earlier that this is not just the data that is being computed or the models and the algorithms that you're running that are being encrypted. It extends the boundary of trust extends to the hypervisor. In fact, it was a tough bar. We were very proud at AMD we came out with secure encrypted virtualization, so that meant the data or the models or any application that was in computation, data in rest, data in motion, we encrypted it. But in our first pass, Mark and the Microsoft team said not good enough. You need to extend that trusted boundary to be the hypervisor because you can't even have the administrators have access and be able to have a rogue agent. So that trusted boundary had to extend. And then I don't think people realize that the key that really unlocks, for instance, a sovereign government from saying why is it that I can trust that I can run in the cloud and not on-premise application, is that the keys are held in key servers by the end user, not by a cloud provider or the administrator of even an on-prem installation being done. So when you talk how does it unlock, it's frankly unlocking the very basic needs for some sensitive use cases. And so it unlocks the innovation of how computing can be deployed because without that foundational trust, you cannot run your applications where you have sensitive needs.
That's a great point, Mark P. Mark R, would you like to add on to that?
Yeah, I think I touched on this with one of the scenarios, which is the multi-party computation. But I think when you talk about value that AI is going to, the confidential is going to unlock, specifically in the AI space, besides just security and supporting these sensitive workloads, I see that it's going to unlock enhanced privacy for consumers. For example, in the advertising space, where it's a multi-party computation by definition, where you've got retailers that want to sell targeted ads to consumers, and then you have the advertising platform. In the case of, for example, a bank, a retail bank that issues credit cards, now you have the bank credit card information mixed with the retail information and consumer history, and you want to protect the privacy of all the parties. Multi-party confidential computing clean rooms can support that and provide the kind of privacy that I think the whole world wants. But then you take a look at specific domains like healthcare or finance or government, and multi-party computations and the value that it will be able to unlock when you get past the hurdle of I can't share this data with the other parties, it's very sensitive data, it's high IP data, but I want to combine it. I would love to be able to combine it with these other parties so that we can get shared insights from combining our data, whether it's coming up with a new pharmaceutical and drug trials, or it's coming up with looking for fraud in cross-border payments. All of those are multi-party computations that are very difficult today because of concerns about data leakage and data privacy and even regulation that I see confidential computing being able to unlock.
Yeah, just on regulations, Mark, I mean, think of the regulated industries like finance or medical applications, HIPAA. Without those assurances, those applications frankly couldn't be run in a trusted manner.
Absolutely. So what you guys are saying is consistent with what I'm seeing. The thing I'll add to what Mark P said, which he said the optimal word, which is control. When I first started in this category learning about it, the first thing that came to my mind was security, regulatory compliance, through privacy. But what's turned out to be the compelling event that's driving a lot of customer adoption that I'm seeing is the ability to control the data unlocks new business models. So Mark R talking about multi-party data collaboration in those scenarios, the data is so sensitive in a lot of these scenarios. Let's say it's bond trading data from the last four decades that a particular trading firm has and only they have. They wouldn't dare share that with other parties because then they would lose the ability to monetize it. But this creates an entirely new business model where by virtue of confidential AI, they can share the data without showing it, doing confidential analytics or machine learning on the data with their partners, and they're able to monetize the data in perpetuity, effectively leasing the data rather than doing a one-time sale. So there's a lot of these new business models that are being created by the ability to control the data that seems to be creating the compelling event above and beyond even more than the security or regulatory compliance. That's important obviously, but it seems that the pace and demand for adopting AI is creating these new business models that confidential computing is creating. That's really exciting. So one of the things that this also unlocks too that I've seen, and there's recently a music label that's now suing all their distribution partners because their distribution partners have collected data that's now going to disrupt their business. So in the scenario in the before time when there wasn't access to confidential computing and organizations could not control the data with their distribution partners or supply chain partners, the next five years are going to be pretty wild because this data, it's like the toothpaste has been squeezed out of the tube, there's no putting it back in. And there's going to be a proliferation of these companies that are going to get disrupted by their distribution partner, supply chain partners, and they're now clamoring. Are you guys seeing this in your businesses on Azure, Mark R? Are you seeing companies clamoring for this now, clamoring for control of the data?
Yeah, absolutely. And you know, Mark Papermaster talked about the sovereign cloud, which is like a macro example of what you're talking about. Sovereign clouds by definition are governments that want to protect and control their data that they're processing for national security purposes. And so that's just a macro scale of what you're talking about in the corporate space of wanting control of data and wanting to make sure that it's only used for whatever you want to use it for, and you want to protect it for the value that that data has and the monetization capabilities that you get from all the investments you put into even getting that data and generating that data. So that's absolutely what's driving a lot of the business cases that are out on top of the security cases that are driving adoption of confidential computing.
The European Union is an organization that we work with around cyber threat emerging cyber threats at Opaque, and as you pointed out, it's under the umbrella of the sovereign cloud term. What's interesting is historically sovereign cloud meant its physical location, the data center was located in a country. But in this scenario that you guys are talking about, the sovereignty of the data is determined by who owns the encryption key. That's the fundamental shift. If anyone listening to this takes away one thing, that's the fundamental shift in control. Once you see a broad adoption of confidential computing, as Mark said, eventually all computing will be of this kind, the controlling point is the keys. When you have the keys, you are in control of the content.
Yeah, and I think from a practical technical control perspective, it doesn't matter if the data center is in country X or country Y. Of course, as you know, Aaron, I'm sure that laws are a concern about which laws are applicable to a particular data center depends on which country it might happen to physically reside in, unfortunately, which kind of interferes with this. Yeah, technically it's fine if it's in A or B, but that's what we strive for in the design of confidential computing systems, as with Mark and the industry, is to get to the point where technically it doesn't matter.
