What Frontier AI has done is create a much more powerful way of uncovering vulnerabilities. I don't think we can stand in the way of technology. What we have to do is make sure, as society, we say: look, there have to be controls on it, because in the wrong hands — and there are wrong hands, let's be honest — it's a very powerful, potentially very powerful weapon.
You know, we've heard a lot about threats to the whole of humanity, but is the most likely risk that we could all actually experience, perhaps very soon, its impact on the fragility of the financial system?
Well, we take it very seriously. You know, when we first found out about, you know, the latest sort of frontier AI and what its capabilities were — which is back, you know, I can sort of date it almost to the day, it was back in April actually, when we were in Washington for the IMF meetings — I mean, it was very clear this was a very, very big development. And we've spent, you know, a large amount of time since, you know, sort of coming to terms with exactly what it is, what its capabilities are, what we know, what we don't know. And I think the way I would look at it is: look, there are great opportunities here. I do think that this has a great potential to actually, you know, to strengthen growth in our economies. We need that. I think it will be in some ways transformative in the economy, but it also brings with it substantial risks. And so we have to, you know, we have to be on top of both of those.
Okay. So we'll talk about both of those things, but in terms of the fragility, does it create new fragilities or does it sort of speed up existing ones in your world, in the banking system?
I think it's the latter. I think it speeds up things that we've known about for some time. I mean, I've said for quite a long time, if you ask me to sort of, you know, say which risk has gone up the league table of risks that we face in financial stability most rapidly since the financial crisis, it's operational risks and particularly cyber risks. And what Frontier AI has done is create a much more powerful way of uncovering vulnerabilities. It's, you know, it's revealing, you know, things that have been in bits of operating software that we've had, you know, all of us have had. It's not just the financial system, by the way. This goes across the whole of society, really. We've had for sometimes, you know, 10, 20 years. It's uncovering these things, and of course, you know, we're working very hard, you know, in our world to then fix things. But in the wrong hands — and there are wrong hands, let's be honest — it's a very powerful, potentially very powerful weapon.
So in particular, bad actors using this as a weapon — cyber attack on the bank or the financial system, which then has its own special set of vulnerabilities that can expand.
Yeah, well, this is people's money. I mean, this is people's money, and of course it's also the bedrock of the economy and the financial system.
But it wouldn't just be bad actors, would it? I mean, there is also — you now have a sort of new development which has been very focused on in terms of threats to humanity, which is the autonomy of what we call agents, essentially workers that are AI, kind of operating, doing work. One example put around by the LSE and the Bank of Canada is about people or perhaps businesses having AI agents decide where their deposits go in real time and suddenly moving them around, increasing the risk of a bank run, for example. Is that something that you're concerned about?
I think it is important, and it's something I, you know, I've been thinking about it a lot. And I would just draw out two — I think there are important points there. The first one, and this is something that goes across the whole of society, really. I think the thing that really marks AI out in this respect is that it can learn itself, and it's doing that in what I call a closed loop. So it's got this huge, you know, it's got this huge stock of information, which is because it's got from everything we've all done in the past in humanity, if you like. But then the real — and that's not new, because obviously the internet's been there for a long time — it can marshal that, and then it's got this, if you like, self-learning capacity. And I think the thing that's really changed this year, and this is what the people, you know, the people who run the companies who are developing this say to me, they were not expecting this so quickly, it's the power of that learning capacity, the fact that it's a closed loop. So in other words, it's very difficult to then do what I think, you know, society would always want to do, and we would always want to see happen, is that there will be points when you want to intervene and say, no, you know, not that, please, not doing that. It's the power and speed of this thing that is creating the challenge of how do we do that when we feel we need to do that for the public interest, how do we do it? So that's, you know, I think a very important challenge we face. I think things that are now happening in terms of saying, yeah, we need testing, we need evaluation of these models, is a step in that direction, one I very much support, because there is a public interest here.
And so is your emphasis here. The balancing act is very, very tricky, though. I mean, it's obviously — even the 10% chance of a threat to humanity has captured the headlines and attention over the past month. But actually, even, you know, some of the threats we're talking about to the financial system would be worthy enough of some sort of brake pedal. You advise we don't need to sort of ban this type of frontier superintelligence, but we do need to slow it down.
I think we need to make sure that, you know, humans work with it, and that we have enough points of intervention and can, in a sense, manage it in that sense that, in the interests of the well-being of society, we can say yes, we can intervene. We understand it. You know, we need to understand it first of all. You know, we read all these things — it's done this, done that. You know, you read these things that have happened. We need to understand how these things happen, because I think then we need to say we want to capture the benefits of this technology, but we need to have the understanding and then the points of intervention where we can say, in the interests of society, in the interests of people, there are points when we will need to intervene.
And just to put it bluntly, AI developments have been so rapid this year that it does increase the risks to financial stability.
Oh yes, definitely. I mean, you know, we've published our Financial Policy Committee's latest report today, and we are highlighting this. There's no question that this has moved on. We are devoting a large amount of time, as are other banks, as are institutions in the financial system, as are institutions in other countries, because this is a big new development.
And we're talking high chance of bank runs, scams, deepfake scams. In terms of personal finance, trading — you know, AI agents trading — are the banks actually in control of what they're doing? Can you intervene?
I don't want to, you know, alarm people unnecessarily, because I'm not saying there will be bank runs tomorrow. Let me be very clear: we're working very hard. I think, you know, a lot has been done, a lot has been achieved, you know, working with the institutions, with people in the private sector, to say how do we deal with this? My staff here at the bank are working every day to fix the things that AI is now revealing, and that work will go on. And, you know, I'm confident that that is the right thing to do. You mentioned agents, and this is very interesting, and I think this is another area where we have to focus very hard. Yes, I mean, you can set up, you know, agents to do things. That's building on things that have happened in the past using trading models, but it's a whole new thing. What I would say is this: this comes back to the point about people, about humans. An AI agent isn't a legal entity. So the people who are controlling these things, the humans, are still responsible for these things. And I think we have to have a better understanding of what that means and therefore, you know, in a sense, how we operate that system, because they don't exist in the sense on their own as legal entities. You can't take the thing to court and say...
But they could, for example — I mean, they could, for example, check out what the highest interest rate is in a bank account and try moving that round much more quickly than we're used to, change the structure of the banking system. And that will be, as you say — we've seen those sorts of things develop over, you know, my career, it's 40 years we've seen — but this is, as you say, this is a big step up in the capabilities.
And again, so as institutions, and us as regulators, and, you know, those of us responsible for financial stability, have to respond to that, because that's where the technology is taking us. I don't think we can stand in the way of technology. What we have to do is make sure, as society, we say: look, there have to be controls on it.
Are you sure you can interfere when you need to quickly enough? Because just the sheer speed of this will have changed. I mean, we take the trading example again — AI agents trading, that's happening already. If they were to, you know, build up a certain position in a book, the human may not even know before you're phoning up saying, 'What's going on in this bank?' and raising your eyebrows.
The first thing that we need to do — and I'm encouraged by what's come out of the US with the meetings that President Trump has held. I'm encouraged by — I talk a lot to the UK government. It's something that the UK government is very, I think, alert to. The Prime Minister spoke on it in New York last week. The first thing we need to do is say we've got to improve and work on our understanding of these models. We don't want models going out there before we really understand what their capabilities are. And by the way, the companies developing them — all the conversations I've had with them, they agree with this. So what I'm encouraged about in the last 24 hours is I think we've reached a point where there is an understanding now that we need that type of mechanism in place. Now, the UK is very well placed. We've got the AI Security Institute. They are world leaders in doing this work.
Okay. The more direct channel around financial stability right now is just the sheer amount of money going into AI — hundreds of billions in terms of debt issuance, trillions in terms of the capital markets generally. Are we prepared if some part of this may be exuberant? I know what you think — if a bubble pops, is the financial system prepared?
Well, that's one thing that we're watching very carefully. We test it through all the mechanisms we have for testing, for putting under stress tests, for institutions obviously that we oversee, for the system as a whole, because it's important. I mean, there is a very large amount of investment going into the sector now, and of course that's natural, because it's a major area of growth. I do think that it will be a major contributor to growth in the world economy and the UK economy, and it needs to be financed as such. And of course you see that the asset prices of the companies that are developing it have gone up a lot, and that reflects the fact that there are high expectations of what it can deliver. Now, of course, we have to then — we're always in the sort of position of being the people who bring the note of caution, if you like, and say, well, you know, there could be things that will dent that in a sense. So, you know, it could contribute a lot, but not as much as financial markets expect. It could take longer to come through. This is what we've seen in history. Sometimes these things take longer to come through. It's true with electricity. I think it'll be faster this time because the world's moving faster now, but it's happened in the past. The third thing I'll say is everybody is currently priced to be a winner. Not — again, you look back at the past, not everybody is a winner. I always say to people, you know, Google was not the first market leader in internet search. It was Netscape. Nobody can remember Netscape today. It doesn't exist. So, you know, not everybody always wins. So we are prepared for the fact that there will be, I think, some shocks come along to markets, and we have to deal with that. We have to make sure the system is resilient.
Well, that would — I mean, that's a form of it. I mean, you could see some correction of asset prices at some point.
