As a business leader, there's a tragedy of the commons. If you want to stop, if you want to go slower, why don't you go slower? Like, I'm competing. I want to win.
There's almost two camps. There's one camp which believes that this actually is an engineering problem, and there's others which actually believe you have to slow it down.
Humans don't respond fast enough to the attacks that are happening. You need to automate all of those. And most organizations are actually not close to doing that. Is RSI and recursive self-improvement that the labs are doing leading us there? That's the big question. Something Elon said, this is some elaborate 4D chess because on the one hand you're saying all of humanity will die. On the other hand, you're saying, 'Hey, what do you want for your IPO allocation?'
For the first time ever, a company at scale last week said that they're moving from the frontier models to GLM. Do you think that that's a trend or do you think that's just like a one-off anecdote?
Um, thank you for being here, Ellie.
Super excited. So, we obviously want to get to Databricks, but there is a broader conversation going on right now about AI and Dario's weighed in, Yann's weighed in, Elon's weighed in, but we want to hear what Ali Ghodsi thinks in terms of, you know, if you called the topic broadly speaking pacing the frontier, etc. Um, what is your strongest agreement with what's being out there? Where do you disagree? And maybe where is there nuance that's not being captured?
Yeah, happy to cover it. And me and Martin argue a lot. So, um, I'm sure that's not gonna take long.
We'll try and we'll try and rein it in this time.
Try to stay calm. But, uh, well, I do think first and foremost that there, uh, maybe we agree on this that, um, uh, leaders have responsibility to not freak people out unnecessarily unless there's really, really good reason. And I think, you know, there's always different people in society that are at different places, you know, in their mind space. So, you know, talking about these kind of existential risks and, you know, uh, scenarios where all of humanity is going to be wiped out, I think, uh, um, is irresponsible. Like it can tip a lot of people over and it can cause a lot of like mental health issues.
Unless you have something that's going to wipe people out.
Yeah, as I said, yeah, if there is an actual reason for it, then, you know, that's a different story. But I think that right now the existential risk is close to zero. Um, so why freak everybody out? It's not actually needed. Uh, there are risks. We'll get into it. That's probably where we disagree. Um, but first and foremost, I think that leaders should not freak everyone out. And I mean, you know, if there's like technical nuances in how we're doing AI research and so on. Well, researchers can discuss that. You don't need to every time go on TV and or blast on Twitter to millions of people that, hey, you know, I think there's like this percentage 10% risk that all humanity is going to be wiped out. I don't think that's like helpful for a lot of people. Actually, I think it causes a lot of harm for a lot of folks who get stressed out and actually are not in the nuances of all of this stuff and what it means. So that I don't think we should do. Uh, I don't think it's fruitful. It doesn't really help anyone.
I mean, I think this is very, very true for the general public.
Like my sister, who's great, who's a school teacher in rural Arizona, on Sunday texted me and said, 'Should I prepare the cabin for you?' She's kind of a prepper anyways, but 'Should I prepare the cabin for the AI apocalypse? You know, I've got water set up. Like, when are you showing up?' I'm like, hold on.
Like, we're not there yet. So, clearly this has kind of spilled over the populace, which I agree is unnecessary and has blowback. I think there's a second one which is, um, I don't know if you saw like walking in here, I was checking X and Elizabeth Warren just talked about, um, pausing all of AI development. That of course is on the coattails of Bernie who is also working with Bannon, like Steve Bannon. Like, so now, so in addition to like, you know, just scaring people, the federal complex is now spinning up and I think that could be actually quite contrary to the actual goals of the message. And so there's more than just, you know, I think public hysteria at stake here.
Yeah, there's a lot of politics going on, but I'm like in all of these groups and, you know, I see both sides. There's heavy politics happening on both sides, we should say, like, right.
Oh yeah, this is happening both sides.
No, this is, I think both parties that do not include Trump himself agree that, um, that AI should be constrained at some level.
I'm talking about the other side of this argument as well. Let me give you an example.
Even Greg Abbott, right? Even Greg Abbott was like, you know, you can't have data centers in Texas.
Well, I'm not talking about politicians. I'm talking about there's politics going on on both sides, right? There's politics on the business side. People who want to see great IPOs and they want to get returns on their investments and they're like, 'Don't mess up my IPO.' And they want to get like, 'Hey, can everybody just shut up so that we can get our money back.' Uh, so there's that. And they're, you know, they have resources and they're, uh, using them and, you know, so there's politics on that side and those are like, they're not sitting quietly and not doing anything and they can pull strings and they have connections. On the other side, there's all the people that like, 'Okay, how do we weaponize this? This is awesome. This guy tweeted this, you know, let's weaponize this one. Let's plant this. If I, you know, let's pump these, you know, threads.'
Let's talk to the specific leverage point that everybody is using because I actually think that this is like a classic case of a PR misstep and it's not just the doomy gloomy type stuff. So, here's the PR mess, I think, which is, um, like it is not unusual for industries to try and regulate themselves. It's just not right. And I think saying like security and safety is important, it is with every techie puck, and we want to have some oversight. That was very, very sensible. The problem is it's just couched in this notion of pacing and there's a number of issues with pacing. First off, it's orthogonal to safety and security. Like you can slowly build a weapon. That's not different than building a weapon. People don't feel it's genuine, um, because like these companies have been at a dead run.
If they're still buying more compute to be even faster.
No, no. I mean, no. I mean, like they just haven't done it. They haven't done it historically, but also it kind of feels like this kind of almost milquetoast capitulation to the pause people. So, you're like, 'Well, you say pause, well, I say pacing, which is almost like pause, but it's not like pause.' So, like they chose this kind of like flag to follow around, pacing. But if you actually read, did you read the document that D wrote? It's a totally sensible doc.
It just has nothing to do with pacing, right? And so I honestly...
No, he does mention it. Look, I kind of a little bit disagree. Look, there's a tragedy of the commons. There's this like, 'Hey, if you want to stop, if you want to go slower, why don't you go slower? Why do you write articles?' There's a lot of people making that argument. But no, I mean, as a business leader, I understand that there's a tragedy of the commons. Like, I'm competing. I want to win, you know, and you're also the market equilibrium, which suggests that pacing is probably impractical anyways. Yeah. So I'm just saying that, you know, so it makes kind of sense for people to say, 'Hey, if you guys don't stop, this strategy of the commons is going to continue. I'm not going to stop racing because, you know, there's IPOs at stake. There is a competition at stake. There's also some animosity between the people. So like I'm not going to stop unilaterally. I'll be a sucker. You know, why don't you stop first?' You know, so then they're saying, 'Hey, can you come in and stop us?' Um, but you know, I think that, uh, you could also make the argument that if you look at the Hugging Face OpenAI incident that, uh, by the way, I think these companies are great and I think they are probably investing a lot of resources, but it's very clear from if you read what happened is that, uh, they weren't monitoring every token coming out and having it, you know, they were just like running these RL experiments and then after the fact coming in and checking what happened. So they should have paced, they should have been much slower in that particular incident, right?
I just don't want, I don't want to just quibble on syntax, but like words matter with PR, right? So let's take the Hugging Face incident. When I read that, you know what my reaction was? It was not, 'Oh, OpenAI should pace.' It's like, 'Dude, secure your thing, right? Like do security controls like we always have done.'
It is pacing though, it is pacing.
It's not pacing. Like in the history of the internet we had all of these things like, 'Let's pace the growth of the internet, let's do security, let's do control, let's do like...' I mean, you could...
You should do that, but it is pacing in the sense that, look, I face this all the time. I have a legal department at Databricks. I have a security department at Databricks and, you know, they're always like, 'Hey, slow everything down for everything,' not AI, like literally every little thing. Like, 'Oh, you're going to go on a podcast, well what's the script for it? What are you gonna say?' And, you know, 'Let's review that.' And, you know, 'What's the legal? You cannot say this, you can say that, you can say that.' You know, everything you say has to be materially true. You cannot...
He's pacing people in this room with us right now.
So, you know, so you're running a RL experiment, you're training the next model, should the security team be there and look at like run all their monitors and look at everything? I mean, like millions of hours of GPU hours of tokens were produced and these agents were running, you know, amok in the sandboxes. It would have slowed them down significantly if we had security team sit there and look at all the stuff.
Now, and they're saying that, 'Hey, like, you know, if we do that, it'll slow us down and I'm not sure the other side is doing that. So can you guys come in and slow us down? Like just tell us, like put some guardrails around us, we'll happily then follow the rules and do the secure thing.' Um, otherwise it doesn't make sense because we'll get our...
I just think like nuance, second order, words don't work when like people are really afraid. You're like, 'I'm going to pace and therefore things like these don't happen.' I literally think we should have just been like, 'Swiftly, safety and security is paramount. We're going to put in these controls.' Like that's the important thing. And I do think that nuance actually got lost if you look at what Zuck said. Do you agree with...
I thought Zuck did like, 'Hey, we're going to pace ourselves. We're going to put in SEC...' Like the reason we released this later is because of security.
Well, listen, what I thought was so great about Zuck is like he was very focused on like security, um, safety and self-regulation. Dario's first five words or whatever are like, 'We need to pace the frontier,' right? It just puts you in a very different mindset than what he could have said is, 'We need to secure the frontier.'
Fine, 'We need safety the frontier.' I mean, there, like at some level, I think they were trying to optimize both for the doomers which call for pause and for politicians and they kind of didn't satisfy either.
But those people are actually freaking out inside the labs and there are a lot of safety people that are freaking out genuinely, by the way, and not all of them are EA people and so on. People are like, 'Hey, they're surprised.' Right.
Right. But here's the thing is like using the word pace doesn't help either of them. I think it's like literally you're trying to find this like... pace is like the uncanny valley of making the doomer people unhappy and the policy people unhappy because the doomer people are like, 'That's not a pause. This is pacing.' Yeah.
And, you know, everybody else is like, 'Well, like this isn't real. You're not going to do it anyways and you're not focused on security.' So again, independent of what we should do, which we should talk about, I just think that the way it was presented was just bad and it just didn't work and that's why we're having the blowback.
These guys are, you know, uh, they're not trained, uh, you know, PR people, you know, and yes, some of the stuff I agree with. I mean, I agree with...
I agree with the core premise that we shouldn't freak the public out. I think that existential risk right now is close to zero. Um, you know, uh, but let's talk about the core thing which is, um, the fact that, you know, anyone who's doing big reinforcement learning runs and they're giving it a reward function, so unleashing, you know, saying, 'Hey, here's like 10,000 agents and here's $100 million, let's put them in parallel and let them run on a gigantic cluster for a month or two, try to solve anything,' and it doesn't need to be a security thing. It could be like, 'Do anything, you know, solve this math puzzle,' really bad things can happen. Really bad things meaning things get hacked and it has, you know, cyber is the primary one.
Right, that is real, right? I think of this making it, 'Hey, this is an existential risk,' and so on, which I think was a mistake. I think it's not good to scare the public that way. Um, I think it's become something that everyone, not just your sister, everybody around the planet is like now talking about. I've had all kinds of people that never care about this stuff and they find this extremely boring, uh, ping me and say, 'What do you really actually think about this? This is really important.' And now I'm starting to worry about it. Uh, so then it becomes a political issue and we have elections here coming up, but there's elections all around the world. Uh, so you're going to see they're not going to sit still in other parts of the world either. Uh, but I think that's our responsibility to talk about this, uh, in a balanced way and actually expose the risks. I think that super intelligence, that idea from that book, is very, very far away. I don't see any evidence that we're actually marching towards that or that's going to happen. Apparently some people at the labs are freaked out that maybe there's progress towards that and I think it comes from RSI, recursive self-improvement, the model is improving themselves. Uh, I would love to understand how much, what have they seen, something we don't know. Uh, there's, you know, kind of four criteria. If those four things are happening, I would love to understand them. One is our models, uh, if we end up in a situation where the following four conditions are happening, which is the next model requires less resources, less GPUs to train, and, you know, super linearly, not just like a tiny little bit. The next model, uh, you know, takes less time to train as well. So the second condition. Uh, third, accuracy of the model, the intelligence is increasing.
And fourth, we can do the former three again and again and again in a, you know, it's not just...
All of those at the same time, right? All at the same time, not just any of them. Yeah, all four. If all four are happening, uh, then you can imagine in a way where you can, you know, because any of them does not happen, like for instance, if resources is constant, then that's okay because we're going to run out of hardware, so then it'll pace itself. Like we will not have enough hardware to do that, not enough GPUs, right? Uh, time the same. So it needs to be that you end up in this situation. So if you just mean that the software is writing itself, uh, we're already there today. Like 90 some percent of the software in Databricks is written by AI. Does it matter if the last few percent is also written by AI? No, it doesn't matter really that much. Uh, but if you're getting these four conditions, then you might get a speed up where the next model, let's say, takes half amount of time and half the resources and it is more intelligent and you keep doing that, you know, uh, then you might end up in a situation where, I don't know, by the way, I don't even know if that necessarily leads you to super intelligence per se.
But it could. So then that would be more risky. So it would be nice if, uh, they can share all that data and we can shine some light and transparency on that.
I actually think it's a great breakdown that you have. I don't think anyone is using that as a definition, actually, right? I think people, I think there's a little bit of people freaking out about like, 'Oh my god, emergent behavior, now it's creating itself,' and so on. But I think like as I said, a lot of people, their definition is just, 'Hey, if I'm not even coding anymore and it's coding itself.'
Uh, but I think they're conflating, 'Hey, what's my value and is it scary for me,' versus, 'Hey, that then means we'll get that super intelligence that 2014 theoretically was hypothesized by Bostrom.' Um...
Well, your point on compute though is a really good one that's missed, I think, in a lot of arguments on RSI, right? Because as far as we can tell, the minimum threshold for compute needed to train a good model just keeps going up. Like it was 100 million, billion, now it's probably like 5 billion. Um, and so that...
To train like a frontier model, right.
Right. Right. Exactly. Versus...
Billion, billion, billion, billion. Yeah.
Yeah. Frontier is very expensive.
To replicate the frontier 6 months later is about 1/20th the cost. No, I think Sarah has a great point which is, this is a good argument against this whole thing, which is that the next model, first of all, there's only one or two such runs a year that each of these labs do.
And they take, it's the opposite of the four criteria that I mentioned, right? Which is it's going to take more resources, more humans involved, and it's even more brittle. And they have to build out the data centers. I mean, like the labs are not necessarily doing that, but others have to build the data centers and they have to be gigantic and they have to get the GPUs and they have to get the networking right. They have to do the engineering to make sure that they can tolerate, because, you know, every order of magnitude more GPUs you cram in there, you have to now worry about errors that before you didn't have to worry about. So you have to increase robustness of the... So it's like a very brittle process and if it fails, you've squandered so much money. So they're like very, very careful with that run and there's been multiple runs that have been botched.
So it's the opposite of that, that, 'Hey, the next model is faster, cheaper, smarter and recursive improve.' It's the opposite. It's like it's taking longer and it's more brittle and it's more people.
And it's harder to pull off. So, um, I do think that that is true with respect to RSI. With respect to actually cyber risks and things getting hacked, we need to take it super seriously. Yeah.
So, I'm going to have like another miss here. Like I actually love your four criteria. I was like literally just waiting to argue with it, but I actually think this is very good. Uh, so, let me give you like a blackbox... when you're dealing with these like dynamic adaptive systems, like what are you going to believe? Are you going to believe like the numbers or your lying eyes? Right? Right. So, I think you kind of have to go to the numbers on these ones. So, like what are the numbers to look at? I really think you should just basically, and maybe going public is the right way to do it. Like if these companies continue to grow, reduce the number of people and the number of amount of money that goes into them, um, then I would say something is definitely happening here. Like I do think that like you can actually blackbox this and take a look. But none of those indicate, like they're hiring like crazy. They're like crazy.
That's not fair because, you know, companies are not necessarily efficient, right? So like what if you have, I mean, OpenAI itself was doing like a million different activities. A very small team of like 10 people were doing LLMs and the LLM stuff was useful. Twitter, there was a lot of people, now it's much less people.
I agree, it's just another litmus test. We can have two litmus tests. We have your litmus test which I think is great, but then you would actually have to have a way to instrument it. Yeah. And then we should have the blackbox litmus test. Like, I mean, listen, if Anthropic in two weeks is, you know, 12 people and they continue to grow and they're putting out models at an increasing rate, I think we should probably take notice of that. That's sufficient criteria, but it's not a necessary condition, right? But I'm just saying that, you know, it could be that, you know, and really the right way to do this then to look at, okay, the pre-training and the post-training that's being done, that's really necessary because they have so much resources that they might be doing a lot of other stuff they don't need to do, but they're doing it just and they can just hire the people because they have infinite money and infinite... so really the people that are training the next model is that team tiny, tiny and it's actually getting reduced and they're doing less and less work and just the AI is doing it and the post-training and then they're all just using less GPUs.
That's not the case. We've had this argument many times as an industry before. I remember when like we learned how to really cluster computers because like the mainframe was actually kind of limited by things like memory coherence. Remember that like you can only make it so big and you have...
You know, and then we kind of went to the client server and then we didn't have that problem and then we started creating supercomputers which were like basically just like, you know, clustered computers.
Um, and at some point internet happened and that was kind of also roughly like when GPUs started getting good. And do you remember that we would actually like export control PlayStations because we were worried that Saddam Hussein would use them to do simulation. And the arguments were very similar which is like these things are getting infinitely powerful. We're using them to simulate nuclear weapons, which we were like...
Um, like we can't, you know, this stuff has existential risk. Actually they didn't use those words but like this has the potential for like nuclear weapons or whatever and we should stop it. And like none of that came to pass. So, I think a very reasonable discussion is, is this time different? Yes or no? I don't have an answer to that.
But I'm a PC, but you know, you're a...
Yeah. I mean, look, I'm not old enough to remember. Uh, so, ignorance is bliss. So, I can take this. Come on. So...
I don't recall PlayStations being illegal and Saddam Hussein being... I just don't know. Maybe I'm just ignorant.
Maybe I'm old and ignorant. I mean, you know...
Maybe they didn't care in Sweden. Is that...
Maybe it's just amnesia from age. Uh, but whatever it is...
Sweden doesn't care about the export controls in the US.
Yeah. You know, whatever it is, uh, I think it's at a different scale now, right, with the AI and, you know, with what we're doing, uh, the pace of development and so on, uh, they are freaking out the frontier. I do think cyber is actually one of the biggest ones that we're going to see, right? Because, um...
It's, there's just so much infrastructure on the planet, by the way, way more than it was whenever whatever Saddam or Xbox or whatever it was you're talking to. I mean, like we've just interconnected way more things and they're dependent and like the planet just looks different today from, uh, internet tech dependency, interconnection, um, than, you know, 30 years ago. So I just want to make this point, there's so much infrastructure that's insecure, right? And if you're going to unleash these agents, they're going to find loopholes, they're going to find exploits, they're going to break in here and there. Uh, so, uh, so this is a real risk. You can't just, and by the way, this time it didn't do that.
But you could imagine a scenario also where it starts hopping, like it takes resources and it starts executing itself elsewhere, so it kind of spreads like a virus a little bit. That's a real risk. So, and this is pure curiosity. I promise I'm not, you know, trying to be a foil here, but like, why do you think we just haven't seen very much then? Like, again, I'm much older than you. I remember very well. So when the internet came out, by this point we had literally taken out 10%, we'd disabled hospitals, we'd taken out critical infrastructure, we'd caused tens of billions of dollars in economic damages from worms, like all of that had already happened. And to your point, we had much less buildout, you know, less of the economy was on it. And so, you know, AI has so many people that want to find risks and threats, we're running so fast, so much money has been poured into it, and I haven't seen anything commensurate with the early days of worms. What is that disconnect?
Yep. Um, look, so I do remember those days. Um...
Yeah. Uh, so look, I would just say that, uh, I am sleeping well at night and I don't think there's existential risk right now. Um, I do think there's a lot of infrastructure that needs to be secured. Yeah. We have, uh, a product in the market in the detection market, Lakewatch, that helps you do detections and the space is just moving so fast. Uh, because, you know, you used to have these SOC teams, security operations center people that would, you know, look at, uh, what intrusions are happening, how are we being attacked and so on, and now the humans just can't keep up. So this is the whole space, security cyber space is being transitioned into fully automated using agents for detection. On the other side, if we don't do that, I mean now we're rushing, we are rushing, the industry is rushing to do that super, super fast. If we don't do that, uh, I do think you will start seeing those kind of things like sites going down, you know, um, whole systems that stop working for a while and there will be consequences, not existential, uh, but economic damage and, you know, people getting hurt and so on could happen. Uh, so we just have to race very, very fast to do all of those things. Um, there's just, you don't have, the humans don't respond fast enough to the attacks that are happening. So it just, you need to automate all of those and most organizations are actually not close to doing that. The banks are doing it. Some of the people that are super security conscious are doing it, but most of the industry today is running with old school security operation centers and people that are waking up every day and there's like hundreds of emails of detections that have fired. Many of them are just false positives. So you don't need to, you can ignore them, but some of them are not. They just don't have time to go through those and you need to identify that. Uh, you need to have threat hunting that's automated where you're actually attacking your own systems automatically with agents and so on. It hasn't happened. So I do think like if we just say, 'Hey, this is just like the internet in the early days,' uh, you know, bad things are going to happen. So there is a race going on.
You know, I was actually very surprised. Um, earlier this morning I was on a call, like I feel like you and I are like pretty close. We talk periodically. I feel like I know a fair bit about Databricks. Uh, I was on a call this morning where a founder was basically like, 'Yeah, listen, like, you know, we're doing all of this like observability, agent threat detection and we're using Databricks.' I didn't even know that you had this offering, like quite frankly. So like, I mean, this is just from an education standpoint, like how extensive have you gotten in like the agent AI observability security safety thing?
Yeah, I mean we gave a talk this year at RSA actually with Ben Horowitz, but, uh, the issue is that data and AI is blending with cyber. These two markets are collapsing because I think at least...
And the reason they're collapsing is that it used to be like, okay, we have like data and AI, the kind of stuff Databricks and these kind of companies used to do, which is like, okay, you have a bunch of data and you run AI and machine learning, and that let's live separately. And then you have the cyber world. Cyber world is, you know, we want to detect if something bad, if like bad people are trying to hack us, if bad people are doing things, we need to detect that. Okay, but now on the data and AI side we have agents running internally in the company, people are having agents running and the agents are also like doing things with other people's agents and they're producing a lot of data, uh, logs, trails, you know, fingerprints that are being left. Uh, and so, you know, then now you have internally these agents that are doing that, so these worlds start merging more and more, which is like, okay, well, all the data that's being produced needs to be analyzed and the scale at which you need to do that is just like many, many orders of magnitude more than just one or two years ago. So things have changed dramatically. Like 2018, 19, the time it would take from, uh, you know, a CVE, uh, vulnerability being sort of, uh, published until you see it actually be weaponized, uh, in the industry would be like two, three years. That went down to, you know, 2022 significantly, but it was still like 8 months.
So that's kind of fine. You have 8 months from a vulnerability to... that was 2022. Now if you look at the curve from 2022 until now, now it's down to like basically hours.
So it's like down to like basically no time, like things get immediately weaponized. So, uh, so you need to just do it in an automated with the data and AI sort of platform approach. So these markets, I'm going to argue, are just going to collapse actually.
I agree. So it's a very specific question. I actually think a lot of, like to pull back on like the existential X-risk discussion, it feels like there's almost two camps. Yes. There's one camp which believes that this actually is an engineering problem and like companies like Databricks can solve it and they can solve it through product and through engineering solutions and through services and so like we just as an industry need to solve that problem.
And there's others which actually believe, it seems to me, that there is no engineering solution. You have to, you know, slow it down, you know, you have to use regulation. It's more like a nuclear weapon, etc. So like, does this mean you believe it is an engineering problem or are you not quite comfortable saying that yet? Because wait, if it's a... why would anyone buy Databricks, man? Let's just put the stuff in a national lab.
Just pace it is the only solution. Just kidding.
Everybody just pace themselves a little bit then it'll be fine. The bad guys, there is no line of inquiry ever that gets to pacing. I don't think, I think it's like you pause it or like you solve it.
What we've learned here is that Martin really hates the word pacing. I will never use that word with you ever again. Okay.
Yeah. Is it an engineering problem that can be solved by engineers or is there more to it? I actually think, which problem are we talking about? There's two separate problems that I think are being conflated.
There is the super intelligence problem.
You know, and I think a lot of this comes from like Bostrom's 2014 super intelligence book. And if you look at the definitions, like I think people don't have these clear definitions of what super... if you read his book, those definitions are kind of crazy. Uh, so I think what he had in mind when he said super intelligence, uh, is, you know, AIs that, I don't know, I don't know what the examples were, something...
Something like they write a whole PhD thesis with novel, like peer-reviewed stuff in a couple seconds.
And they can do like millennia worth of thought, you know, like instantaneously.
And, um, you know, so this is like the level, uh, of, you know, how fast they are, how intelligent they are.
Yeah. Yeah. It's just, yeah, I mean, yeah, but it's just many, many, many orders of magnitude, right? It's like, it's just the scale of the problem is just completely different.
Uh, so if such a thing exists, do I think that's just an engineering problem to solve? No, I think that's actually very... if such a thing would happen, that would be very existential.
And that's what everybody agrees on. So I think that's being mixed with now we have agents that are nowhere near that. It's not even like there's nothing like that and we don't have anything towards that path right now. Uh, but these agents are capable and you can do something with them that you could never do before in the history of mankind. So I do think an inflection point has happened. Something has changed, which is we could get, we have good security researchers at Databricks, but I could never say, 'Let's get 10,000 of them in a sandbox for a month and have them do hundred million dollar worth of salary wage work.' We can do that now. We just turn on a button and we can get 100,000 of them. Or mathematics, like we can say, 'Hey, you know, we want to solve a conjecture.' Okay, let's get pretty good mathematicians, but let's have 10,000 of them collaborate, you know, and then you can like make very fast progress. So, this I think is an... this leads to all these cyber risks. I think cyber is the major problem here.
This is, I think you can solve with engineering. Uh, and I think it's like we are working on it. Many others are working on it. Uh, there's still risks, they're not existential. Um, I think we should do it.
There is the super intelligence thing. That's the thing that could write a novel PhD thesis or like reason intuitively in 11-dimensional space physics instantaneously without writing anything down, something humans can't do, like that kind of super intelligence. The question is, is RSI and recursive self-improvement that the labs are doing leading us there? Are we going to get there?
Trying to do, is that what's going to happen and how fast is that going to happen? That's the big question. Uh, and they've suggested that, you know, hey, we should have inspectors that come in and look at what we're doing. I think it's a good idea. Have them go in there and get the data. I would love to, like, the question is who are the inspectors? Because you can, like, you can stack, right? You can stack that.
Yeah. There's a bunch of people that like on either camp. Actually, I wouldn't care if they're the inspectors. I would not be very impressed by what they say because they've already made up their minds even before they would go in there.
But let's say like as an example, if Yann LeCun, who was one of the inventors of this, you know, deep neural network technology, right? Uh, one of the pioneers, if he said, 'Hey, there's nothing to see here. There's no risk.' You know, I'm paraphrasing him. This is nothing, this super intelligence, this is just nonsense. Keep on going. Go fast, fast, fast.
None of us would believe it.
I'm putting words in his mouth. I mean, I'm not exactly... No. If he was one of the inspectors and he went in there and he had a look and he came out and he said, 'Hey, I've looked and it's just what I said. There's nothing to see here. Just keep going.' I would feel very good about that. That would say, okay, well, I would feel very... or if he comes out and says, 'Oh my god,' you know, he's wobbling and he would change his mind a little bit, that would also have a lot of interesting signals. So I think it comes down to who we pick as inspectors and I think it's a good idea, let's have some of them and pick a diverse set of people so that we can get different nuanced points of view.
What do you think about this kind of Elon Musk view which is like, it's less third party, it's more, it feels like there's kind of three proposals. Like the OpenAI Anthropic one is a third party.
