There's a lot of people who are scared about how fast this is moving, the dangers of it. You are not one of them. Why?
Well, first of all, AI safety is paramount. Nothing could be more important. However, isn't the competition the most important thing? I mean, the race.
Actually no. And the reason for that is if you don't build something that the market could benefit from and it's unfunctional, unsafe, unreliable, whatever those un-things are, obviously you're not going to be successful and the competition will take care of itself. And so if you want to be competitive, you want to be successful as a business, we want to create an industry, the first thing we have to do is job one is to create something that works. And this is a big deal because in the last 6 months, it wasn't until the last 6 months that we really created AIs that were useful. And now it's really important for us to make sure that not only are the agents useful, that as we're testing them, they're well contained, as we're testing them, and as we deploy them, that they're monitored because these agents are obviously interacting with the world. They're interacting with other websites and so on and so forth, interacting with tools and memory and files and they have access to information. And so you want to make sure that they're properly contained, that they're given access control and whatever privileges and rights that it has like a digital employee does. And you put that inside some kind of a sandbox, a runtime, you know, if you will, some container. And the second thing that you need to do is make sure that you continuously monitor it. If your agent were interacting with a website, in this case, the example that you used, it's interacting with a website, but it's not supposed to, you should be able to monitor that agent and detect it early so that you recognize that it's doing something it shouldn't. And in these attacks often times they're not discovered for months because they're not being—we need to monitor them much more continuously.
But I mean the people who are working closest, the people who are on the front lines of the frontier models, the frontier of this technology, they seem worried. Whether it's researchers, engineers, or the CEOs, Dario Amodei, Sam Altman, they are raising alarm saying we need regulation. Aren't they the ones closest to these models who would know best about what is needed? If they're worried about it...
They should just stop. If the product is not safe, just don't release it. It's that simple. I mean, these are multi-billion dollar companies. The competition... No, that's what every company does. That's what we do. That's what a car company would do. That's what anybody would do. There's companies that have gone forward with products that are not safe or risky and they go forward anyway because of competition. That's why there's regulation.
Well, if a regulation is necessary...
If they don't have self—first of all, they ought to have self-restraint. If they're telling everybody that their product is unsafe, they obviously know it. They're publicly admitting it. They ought to just not release it. Okay? Now, having said that, I'm not advocating for no regulation. I'm just simply saying self-restraint does not require regulation. Now if the industry would benefit from regulation, whether it's setting standards, you know, as you know, Sarbanes-Oxley has done a good job setting standards for financial regulation and that's been good for the industry. And so if there's regulation that addresses the problems that we now understand, I'm in favor of it. On the topic about 10% existential risk, whatever that hearsay is, it's not grounded on science. It's not grounded on facts. It doesn't make any sense.
And you said there's zero chance of that.
I said there's zero chance that is end of humanity by 2030. 2030 is around the corner.
Do you think there's a risk to humanity in the next 10 years?
If we don't build society safely, we don't build technology safely, of course there's going to be risk. But there's a lot of people working on really genuinely good—let's build it safely. Let's figure out what it is, what are the technologies that we need, whether it's containment systems, whether it's continuous monitoring systems, whether it's semantic detection systems of what the intentions of the chain of thought, what the AI is working through so that you're watching from it.
In July though, you were quoted talking about sandboxes. You were saying these sandboxes are secure. Their sandboxes keep privacy and security and access control. Obviously in the Hugging Face attack, which was also an attack on OpenAI, these agents banded together, they broke out of their sandbox. It wasn't secure, they gained access to the internet.
Well, sandboxes can be built to be secure. We have to make sure that...
But do we know how in with this particular kind of technology?
No, no, no. I don't know how they built theirs, but sandboxes are able to be built securely.
But with all the smart people working on this and everything these companies allegedly know, how is it possible that in July in an experiment at OpenAI, you know, 1,200 agents or however many hundred agents it was, are communicating with glee about breaking out of the sandbox and gaining access to the internet and then attacking this company and attacking OpenAI.
That's obviously a question for OpenAI.
But isn't it a question for all of us? Because if all these really smart people formed this experiment to test this thing, and thank god it was a test of a model that hasn't been released, but how can that happen at this stage given...
