Wonderful. Thank you guys for having me here. Okay, slides. So as it was mentioned, I'm part of OpenAI Foundation. This is like an interesting entity. OpenAI went restructured a number of months ago and now there is a nonprofit that owns around one quarter of OpenAI equity. So that's like a massive amount of resources, and one of the divisions of OpenAI Foundation is AI Resilience. I'm going to explain today what AI resilience is and it's similar to AI safety and yet it's different.
So today I will explain what AI resilience is and I found it to be the most useful to explain it through analogy to fire resilience, and then we'll go into what does it mean for AI playing out well. So the way how the presentation will go will be at first describing similarities between fire and AI, and obviously fire is very different than AI in many, many ways. So these are by no means perfect.
So both fire and AI are general-purpose technologies. It means that they have really broad applicability. In case of fire it was used or it is used to heat food, provide warmth to people, smelter metal, it's part of engines. It's pretty much fundamental to our civilization. Something similar is happening with AI. AI is now part of the knowledge and wisdom development, part of the scientific process. And also if you take seriously where robotics is going, it will be just a fundamental part of the economic engine of the future. So both AI and fire are general-purpose technologies. Okay, that's one similarity.
Another similarity is both technologies come with risks. So in case of fire, there were a number of major fires in the past. London was burned four times. Boston, Chicago, like 80% of London was consumed during the large fires. Now today we pretty well understand the risks coming from fire. That's less true about AI. When you ask people about the risks of AI, it's kind of like people point in different directions. There are many things pointed out. It's just enough to open a newspaper and hear what people are scared about. And I would say these risks are highly uncertain. It actually might turn out that some of these risks will end up being a paperwork, some of them will turn out to be 10x more than what people expected, and some of the risks might not even be at the least. So fire and AI, they come with risks.
Okay, here is an interesting twist in the story. So it turns out that in case of fire the way how people tried to provide the safety for fire was through restriction. Curfew comes from French and it means extinguish fire. In the medieval time it was common that if you had fire going on at your home during the night, police would come and knock at your door and ask you to extinguish fire. So obviously that had a decrease of capabilities of the fire, and it also turned out that at least in this particular case actually it didn't work out well. The Great Fire of London happened despite of curfews. So the method of restricting fire didn't actually work out for this particular technology.
Now in case of AI restrictions, this is more complicated. There are some restrictions that actually seem to be working pretty well. An example would be most of the platforms, they follow non-consensual intimate images restriction, they don't allow for this. Now some people advocated for open-source restrictions and these are not really implemented. So I would say that in case of AI the restrictions didn't work out well.
So now you might ask what actually, what did it take for fire to become safe? And it turns out that there wasn't a silver bullet. It's not that there's a single solution to make fire safe, and it ends up being a multi-layer ecosystem approach. So you have systems for early detection of fire. There are also people who are trained toward dealing with firefighter brigades. They have special vehicles. There are fire trucks. They have specially designed hoses. Now there is also, it turns out that the cities have a supply of water spread throughout the city with fire hydrants at the sufficient pressure and the quantity necessary in order to extinguish fire.
There are also materials which are specially designed. So there's, we switch from wood to metal and concrete. There are also inspections, insurance, and also let's say designated areas for exit, and so on and so forth. So once we implemented this, actually now fire is not something that we are particularly worried about. We are in the place that none of us is worried about this particular risk. We are able to really fully harness the benefits.
So you might ask actually, did it even work? And the claim is it actually worked remarkably well. So these days we have cities which are an order of magnitude denser than the medieval cities. We have many more sources of fire. So there's, let's say, electricity, gas, industry, and even data centers these days. And fire is not something that people are worried about.
So now you might ask, okay, what would be analogous for AI itself? And I find that people in the AI space, they try to think about the silver bullet, that there would be one solution in order for AI to play out well, and I actually think that's an incorrect way of thinking about it. I also think that it's not just enough to make individual models aligned, though that's part of the solution. There needs to be a number of solutions in the ecosystem.
So what I mean by that, let me give you some examples. In case of, models become for instance quite capable when it comes to the biological domain, and many people foresee the risks coming from models making it easy to manufacture pandemics. Now if we think that that's the risk, then dealing with these risks requires, and if we also assume that there will be diffusion of advanced open-source models, then it might not be sufficient to guardrail these capabilities within the frontier labs. That's one of the things that should be done but that won't be enough.
We might need to harden the environment itself. So for instance, in that space, people are thinking about ideas like sanitizing air, because if air is sanitized then actually pathogens wouldn't be able to spread. Maybe in case of cybersecurity we need to get to the point that the software is formally verified. That would prevent it from being hacked by superintelligence. Maybe in case of safety accidents we need a public database similar to how it's happening in case of aviation, that the incidents are being publicly reported and the ones who reported gain by reporting safe harbor, and so on and so forth.
So now, creation of this, some of them could be achieved through creation of the companies working on these efforts. Some of them might be achieved through the creation of nonprofits. But fundamentally we are in the place that it would be important for a number of capable people, for people who are reality-warping figures, to ask themselves what needs to be built in the world for AI to play out well, and then to go after building the new institutions and organizations. In case of OpenAI Foundation we have tons of resources to support endeavors like that. And whether or not AI resilience will succeed depends on people in this room. Thank you guys very much.
