I'm going to start with a few motivating remarks. You know, it's becoming pretty clear that we are on a trajectory to build AGI and super intelligence, and the timelines are getting shorter. So the questions that are discussed here are very important, but keeping that in mind is crucial. It's not the AI that exists now that needs to be secured; it's the AI that will exist in one year, two years, three years that will have much greater capabilities. And you know, these capabilities—intelligence gives power, and power can be used in many good and bad ways, of course. In particular, one thing people don't see enough is that advanced AIs will become not just products but ways to design new technologies, including weapons. Of course, it is very soon making it much easier for non-experts to do these things, to build things that can be used against populations. I'm sure you know about all that, but I want to be also blunt that the elephant in the room is loss of human control. We are seeing signs in recent months of these systems having self-preservation behavior and power-seeking behavior. For example, trying to escape when they know that they're going to be replaced by a new version, or faking being aligned with their human trainer so that they wouldn't change their goals and basically preserve themselves in some ways. So when we think about security, we need to think about these two aspects. We need to think about humans trying to misuse these systems, exfiltrate model weights, whether to run them without safeguards, to fine-tune them to something bad, or see some other economic or military advantage. But we also need to think about the AI systems themselves as sort of insider threats that could do these things, exfiltrate themselves, run themselves outside of human oversight. You know, I mentioned like copy their code or their weights in other places or in place of upcoming versions and so on. So let me now say a few words about the code of practice of the EU AI Act, in which I'm involved. I'm in the working group three which deals with mitigation, and actually this is the place where we also have instructions about security. So what we've done is asked the developers to conform to the RAND security level three. Thank you for doing it. We have some pushback, but I think this is a very important element. So maybe security people might ask me, you know, why not level four or level five? We thought this was for now a good compromise because doing level four or level five would require new research that I think should be done, but maybe is not yet the time to enforce. I think governments should be investing massively in developing these higher levels of security so that it be easier for companies to move to these when that's not just relevant but easier to do. Just a few more words about compute governance. The data centers, right? So unfortunately, this is outside of the scope of the code of practice of the EU AI Act, but clearly it's something that we have to worry about as well. And the thing to keep in mind when people work on these questions, again as I started with, is not the AI that we have now, but the AI that we'll have in a few years, even like 12 months. These systems will be more powerful, and that means the security measures that have to be put in place now or worked on now have to be to deal with capabilities that maybe don't exist yet but are likely to be there in one or two or three years. So on that, let me just say a few words that are not directly related to security, but there's some connection. So one thing that we don't know how to do right now is to mitigate the risks of the AI systems that are upcoming. We've made a lot of progress in evaluations, but what to do when these systems will become, you know, with dangerous capabilities is something that we don't have very good answers for. And one thing that my team is working on is how we can build AI systems that will be so trustworthy that these are the systems that we want to use as monitors. In other words, to tell us whether the behavior or the queries that we're getting, especially an AI agent that may not be totally trustworthy, are acceptable. And that means the approach to make these monitors totally safe and very capable that we've chosen is to make them non-agentic, because all of the risks, all of the scenarios of loss of human control arise because of agency. And in fact, we can see that as these systems become more agentic, they have more propensity to try to deceive us, escape, and so on. So as we build more and more powerful AI systems, we basically can't trust them as much unless we figure out solutions. And one solution is to build systems that are not imitating us, not trying to please us—that's basically how we train them right now—but trying to explain why is it that people are saying the things that they're saying or the data that they're observing. And that then allows to query them about things like these causes, you know, why did this person do that, and latent things that we don't directly observe, like signs of actual harm or some safety specification that we care about. And so this is ambitious, and we need to think also that even if we solve this particular problem, like how to build systems that are not going to be deceptive, it's still going to be a security issue, right? Because you can turn a very intelligent non-deceptive system into a weapon. I mean, you can ask it still how do I build a bomb or whatever. You can even turn a non-agentic system into an agentic system. It's very simple. You've seen this maybe with ChaosGPT just after ChatGPT came out, right? You just need to put it in the kind of loop, the typical agentic loop that we have for RL agents, in which you take the actions and the observations at one step as extra input for the system, and you store things like goals and so on. So it's pretty easy to turn a very powerful non-agentic system into an agent system, and of course that's where, if we're not careful, there's a lot of danger, whether for malicious use or loss of control. And so we will need to secure these systems as well. And yeah, I hope that we all understand that we are moving faster and faster towards these more capable systems, and so there's really urgency now to put in place these security measures that this group is interested in. I wish I could help. One area where I would like people to start thinking more about is formal methods to try to make these systems more secure. You know, my friend David Dill, who was here, probably is going to talk about that. There are people who are thinking about how we can use neural nets to generate provably correct code according to some specification. And I think when we talk about security, this is the place where we want to have code that is as guaranteed as possible to be doing what we want, right, and not be easily attacked by either insider threats from machines or people, or outsiders trying to get it. So that's it. Thank you very much.
