It's a pleasure to present this current state of AI safety and open weight models. Thank you for inviting me at the Concordia AI frontier and the Gentex AI safety forum. Let's maybe put things in my personal perspective. About 3 and 1/2 years ago, I completely shifted my activities to start to focus on AI safety. I realized that I had greatly underestimated the rate of advances and what it could mean for society. And that I wasn't really sure that we were on a good trajectory, and thinking about my children and grandchild, I decided that I would do everything I can to mitigate potentially catastrophic outcomes of future advances in AI.
Maybe the most important piece of information for most people to digest, and it could be feeling like science fiction, but it's not what the scientific data shows, and that is the incredible trajectory of advances in AI capabilities, even exponential on some metrics like this meter curve that you see that shows progress in agentic capabilities where the vertical axis measures the duration of a task that is measured by human time that AIs can deal with, and that duration has been increasing very rapidly. We're now at around 16 hours, and we were just about an hour in at the end of last year. So this is something that should force us to think about what it means for the future, especially as these advances focus on things like reasoning, math, coding, science, agency, and what this could mean in terms of AI being used for advancing AI research itself.
I would like to tell you about first the International AI Safety Report that I have chaired for the last 2 years. This is an international report backed by 30 countries, the EU, the OECD, and the UN. And it's making a synthesis of the scientific literature on these general-purpose AI systems that are essentially at the frontier. Thanks to a very large group of about 100 AI experts from around the world, including from China, of course. And there are many conclusions. I encourage you to actually read at least the executive summary of that report. But from the point of view of safety, a crucial observation is that the current safeguards are improving, but not as fast as the AI capabilities. And they remain fallible. In particular, none of the companies can give us guarantees that those systems will not harm humans in catastrophic ways as they become more and more capable. Unfortunately, the incentive structure based on competition between corporations and countries is making things worse and leading to a race to the bottom. It's difficult for policy makers to take these kinds of required decisions under uncertainty because the scientific studies will typically lag the advances in AI. It takes time to do these studies. And so there's maybe not enough evidence to be sure of exactly where we are and where we are going. But if policy makers wait too much, then we may end up in catastrophic scenarios. So I think what should guide these decisions is the precautionary principle. By that, there are several possible future scenarios. Maybe the trend will continue, maybe it will saturate, or maybe it will even accelerate. And we need to be ready for all of these cases.
Now, let me focus on one tricky question. Open-weight models, because from a safety perspective and especially misuse, they pose distinct challenges that are different from the proprietary models. Now, they come with great advantages. I've been a big proponent of open source, for example, for all my career. They make it easier to do research. They accelerate startup ecosystems. They facilitate deployment. But there's also increased risks. And in particular, once you've shared the code for running a model, you can't take it back because it is going to be copied by people. And so that deployment and sharing decision is irreversible. In contrast, if you had a proprietary model, you could pull it off or you could patch it. But you can't do that with models that have been shared in open weight mode. Another issue is that a malicious user who has access to an open weight model can change it. They can remove the lines of code that have to do with guardrails. And they can even fine-tune the model so that it's going to be doing things that it would otherwise not do. So the safeguards can be removed. Now, we'd like to benefit from the advantages of open weight models, but not suffer the risks. And so the logical thing to do is to find a good evaluation of those risks, and share the models that are not too dangerous, and not share in open weight models the ones that are above a threshold of risk.
So let's go a little bit deeper into the vulnerabilities of open weight models. Once you have the weights and the code for running the inference, you can change those weights. You can edit the code itself. You can change those weights towards particular goals. For example, using adversarial examples that are optimized, you can optimize more easily if you have the code to make the systems do something that they're not supposed to do. And you can also provide new data to fine-tune them for a particular task so that they can be even better at a malicious task. From the policy perspective, this raises very serious challenges. In particular, if you think about the kind of software export control protocol like was imposed on Methus, it wouldn't work if it were an open weight model because you can say, 'Yeah, only citizens of this country can use the model and see the model.' But once it's copied by someone else, then you can't take it back. Furthermore, another difference that's quite important from a policy perspective is with a proprietary model, you can always check all of the queries that are made to the model because you can monitor the model as it is being used. But once you've shared a model and it's run on the user's own machines, you don't know what they're doing with the model. So a lot of the risk evaluation methodologies that have been developed for closed weight models are not appropriate for open weight models. And new methodology needs to be developed.
So this concludes my presentation. And you have to think of this issue as part of, you know, one of the risks involved in more and more capable models. Somehow our policy makers need to navigate across all of them.
