Good morning, everyone, and thank you for joining us. I'm Jason McBride of AIX Ventures, which is an early-stage venture firm founded by leading AI practitioners, including Richard Socher, Christopher Manning, and Anthony Goldblum. We partner with highly technical founders building category-defining AI companies across enterprise software, healthcare, biotech, and AI infrastructure. Notable portfolio companies include Perplexity, Hugging Face, Ambience, and Weights & Biases. I'll be the moderator for today's conversation, which is focused on one of the most consequential questions facing the AI ecosystem, which is: how do we continue pushing the frontier of AI innovation forward while thinking responsibly about the risk controls and governance that may be required as these systems become increasingly capable? We're fortunate to have two exceptional people joining us for that discussion. Richard Socher is the founding partner and managing director of AIX Ventures and founder and CEO of Recursive. He previously served as chief scientist at Salesforce, co-founder of you.com, and is widely recognized as one of the pioneers of modern natural language processing. He's also the author of "The Eureka Machine," which explores AI's potential to accelerate scientific discovery. Joining Richard is Will Marshall, co-founder and CEO of Planet Labs, which operates the world's largest fleet of Earth-imaging satellites. Will is a former NASA scientist and has become an important voice in the conversation around advanced AI and global governance. He recently published an essay in The Economist arguing for an international framework to govern advanced AI before the emergence of recursive self-improvement systems. We'll spend most of our time today in conversation between Richard and Will, and toward the end we'll bring in a few questions from the audience. So let's get started and kick off with the first question. So when we talk about where AI is ultimately headed, one of the most consequential possibilities is recursive self-improving superintelligence. So let's unpack that. Let's start by defining what that actually means. What does recursive self-improving superintelligence look like to each of you, and how far away do you think we are from achieving it? Richard, let's start with you, followed by Will's response.
Yeah, so that is what we're building at Recursive, the company. We just started this year and raised quite a bit. The idea of recursive self-improving superintelligence is to essentially allow AI to do AI research and to work on the scientific method. And the scientific method broadly has three steps: the ideation, implementation, and validation of ideas. If you now say and ask an AI, "Well, I want you to partake in this and execute on this scientific method on AI research questions itself," it almost by definition becomes self-improvement if the AI actually has a certain sense of self-awareness of these are my shortcomings, this is the pipeline that led to the overall model or swarm of models that is myself. And then you allow the AI to essentially improve whichever steps it thinks might help the most. And if you have an AI that itself improves its own version, there are a lot of people actually confuse, and it's important to distinguish auto research from recursive self-improvement. If you can take an AI and improve another system, that other system could be another small AI, it could be the Shopify codebase or whatever it may be, that's auto research. It's a useful step, but it's not true recursive self-improvement where you really have to have access to the full model, the weights, the architecture, and everything that goes into it, and then give it full access to it. And if you then give that system enough compute, it may lead to then superintelligence.
Great insight. Will, any thoughts?
Well, yeah. I mean, look, Rich is more the expert on that than I am. The way I simply think about it is AI coding to improve itself, right? If an AI system can close the loop, I mean, already Anthropic said that something like 80 or 90 percent of their code for their last model was written by the previous model. If you completely close the loop so that it literally recodes itself entirely on its own, you just say "improve yourself" and it goes off and does that and then produces a better model, that's recursive self-improvement. And, you know, the obvious potential consequence of that is an intelligence explosion because then it could continually improve itself and the potential timescale of that improvement might go from whatever it is now, a few months to major new iterations of models, of LLMs, to maybe weeks or days or hours, depending on how you think about how long it would take for it to do that and the necessary training runs and so on. But definitely speed it up a lot, right? And so that would be the, you know, technical consequence of that.
And indeed, Will brings up a good point, which is the sort of threshold to unlock a lot of this is that earlier this year, late last year, AI got the ability to truly build useful code. And like AI is code, AI can code, and that loop. Now, of course, if you just run that loop naively, which people have tried, it sort of peters out very quickly. So that's why we kind of separate it in these three steps of ideation, implementation, validation. I would argue that if humans ask it for every one of those, 90 percent of the code and every piece is like, "Well, now iterate on this and write me this new function." Okay, the AI wrote the function, that's 100 percent written by the AI, but I gave it the exact function I wanted it to write in English. That is to me a very weak form of recursive self-improvement that requires humans deeply in the loop of that. And there are a lot more steps you need to make it much more open-ended and akin to kind of biological, cultural, or technological evolution to keep on running as a process and not just peter out and say, "Oh, I finished this benchmark, I'm done now."
Richard, and what do you think is the timeline? That was the other part of Jason's question.
I like, I'm very, very confident now that we have it next year at Recursive.
Within the next 12 months from now, basically.
Yeah, and Dario has said something like a considerably shorter time frame than that. At one point, I think he said three to six months or three to nine months, and that was quite a few months ago. So, yeah, I mean, it sounds like the timescale could be a few months at the earliest, a year middling, and some single numbers of years at the outside, although of course we don't know, there could be more foundational steps. But let's say, you know, a year plus or minus.
I think that the biggest thing that makes me maybe less worried, and that I think people need to understand, is that you may have the loop, but the loop may require a lot of compute to be able to run the loop, right? Like let's say you automated 10,000 AI researchers now working non-stop, but each one requires like 10 GPUs, 100 GPUs, a thousand GPUs. Now you need a million GPUs to actually get those people to really do all the work, and then you have to, like, you know, they come up...
And that was my point about whether it reduces it from, you know, three months to three seconds, or three more likely days or weeks, right?
I mean, these models definitely don't...
Because they take real energy. So it's not only going to infinitely improve it, right? But it's going to dramatically improve the timescale.
That's right. Of improvement. And yeah. Okay, so I think we're more or less on the same page there. And, you know, obviously that brings potential existential risks for humanity. So let's talk about that. Jason, I don't know if you want to tee us up on that.
Yeah, I would absolutely love to. So, like you hit the nail on the head, good segue to talk about risk and rewards. So, based on what you just stated, what do you see as the biggest risk of allowing, for example, private labs to develop superintelligence? I'll throw out some scenarios. Is the primary concern the risk of losing control, potential consequence if we do, or whether our technical and political safeguards can keep pace? What is the biggest risk and why?
To me, it's all of the above. I think there's risks that are related to cyber, nuclear, like enabling users to build bioweapons. You know, you lower that barrier, more people have access. It's not like any of these things are not in principle possible, but just lowering barriers of entry now enables more people to do these things. Obviously, cyber threats we've already seen with Mythos and others, and just the sheer speed-up of the number of foundational issues being corrected with the program of GlassWing that Anthropic had. Right? Even those that have access to the GlassWing program have massively increased the number of fundamental bugs that they have found in systems, architectures, and operating systems. But the one I care about the most is the control one, and I'm not sure "control" is the right word to be clear, but using that as a simple rubric for the second. The question is, if something is generically intelligent, can we control it? Right? If it gets beyond human intelligence in a generic way, AGI, then will we be able to control it? And Alan Turing said in 1952, and by the way, I wish we had more Alan Turings around right now because we need their help, in '52 he said, you know, any system that is going to get generically more intelligent, obviously we're going to lose control. And, you know, I'm sure he said it slightly more eloquently than that, but that is, you know, it just stands to reason, right? Anyone that says, "Oh, well, I can box this thing, I have an off switch, I have this, I have that, I'm going to constrain its goal function" — all of that cannot be proven. You know, it's like way too hard because if you have a generically intelligent system, it's like, you know, a human waking up in a cage surrounded by gorillas with guns. You know, if the human figures a way, he's going to figure out how to escape out of that and outsmart the gorillas. And yet here we might be talking about not just a 10 percent — that's just with a little bit more brain matter than the gorillas. Here it could be that this thing is 10x or 100x or a thousand times within a short frame of time. And so if it's generically more able than humans, it's going to outsmart any of our control systems. So certainly, I think the proof of being able to steward that safely flips the burden of proof — it has to be on the people developing that technology, not the other way around, given the sheer obvious likelihood that it's going to outpace our control systems.
Yeah. All right. So Richard, over to you. What do you think about the risk? Is it the same? Is it as urgent? What are your thoughts?
There's a lot to unpack there. Maybe we'll start with the one you said you care most about, which is kind of the loss-of-control threat vector. And you mentioned Turing. Turing also defined famously the Turing test, which we have passed, and the world did not change as significantly as he expected it to.
Have we passed it? That's unclear to me.
Oh, 100 percent. Like, you can ask — in fact, the Turing test was passed multiple times in the past. The Turing test famously is like, if you can have a conversation with a computer and you don't know if it's a computer or human, then that computer passed the Turing test. And according to Turing, who was clearly brilliant in many ways, this is now a very intelligent AI, maybe even an AGI. It turns out humans are very easily fooled into thinking what is a human or what is a machine. The test had been run in the past many times and had even, before transformers were invented, before we used neural nets, been won several times. The trick was to either pretend to be a very unsophisticated young human who doesn't speak English very well, and then they're like, "Yeah, this could probably be an eight-year-old boy whose third language is English and he hasn't studied it very much, and it's human." Or it went the other way too, where English professors failed the Turing test and people thought, "No way a human would know that much about Shakespeare and all the details. Clearly this was an AI." So humans have been very easily fooled even pre-ChatGPT.
Can I just probe you on that a little bit, Richard? I totally agree with what you just said, but I think Turing probably meant something a little bit more sophisticated, which is like a concerted, thoughtful set of questions and answers where you really do have a smart person trying to interrogate it and trick it and then realize whether or not it is. And obviously there's the simplistic ways in which AI could fail that and show itself to be smarter by adding up things smarter and faster than a human could. But even in the more foundational way of having a conversation that you can't tell, I'm not sure, because certainly from my vantage interacting with AI, I feel like I could regularly figure out why it's a machine, not a human, talking to me, and I think I could still tell the difference. So it really depends on what you meant.
Yeah. So to a casual, quick user, it often tricks people into thinking they're human. But to a concerted effort of sustained questioning, I'm not sure we're there yet. So I think the jury's out, but more people have studied that than me. But you're on a roll.
Yeah. So maybe just a short off-ramp on responding to that and then get back on the roll. I think, interestingly enough, people have used this to convince people to wire a bunch of money by having a Zoom call with a bunch of other people, right? So that arguably...
Consequences already. Yeah.
Yeah. So arguably, knowing who you're talking to — people have been easily fooled in various ways into the millions of costs of having assumed that those were correct. I think the inverse Turing test is probably the most likely now, which I kind of coined or came up with, which is often the best way now to know whether something is an AI or not is you ask it a question that's so hard no human could actually answer it. So you just ask, like, "Build me this front-end website in 10 seconds," and if you have it 10 seconds later, you knew that that was an AI and not a human. So it's almost like Turing couldn't even imagine going that far, such that you probe it that hard, but by probing it that hard you realize it's actually smarter than a human, and that's how you know it's AI — not from the bottom now, but from the top. But let's go back to the loss-of-control one. So I think we have some history on how much more intelligent people have been able to rule less intelligent people. And I think the jury is still out.
I think the jury is still out. I think it isn't necessarily the smartest people who then somehow magically, because of more intelligence, control all the people who are less intelligent than they are. In fact, and this is the beauty of intelligence, it's this highly multi-dimensional construct. And you can be incredibly intelligent along the math dimension, along the linguistic dimension, along the empathetic and control and power dimension. And they're all correlated, but clearly not 100 percent correlated, right? And so arguably I'd say it's not the most intelligent people that then dominate all the other people. Now, of course, people with enough ruthless goals and enough tools and enough momentum and social dynamics and so on can then rule others, but I highly doubt that it's as obvious as you say, like we will obviously lose control. I think it's really not that obvious that anything more intelligent would automatically obviously...
I think that's a really good point. I don't think intelligence necessarily correlates with ability to control all the way, or it correlates but isn't certain. But I do think the interspecies analogy is slightly stronger because we are at a point where it could be an intelligence explosion, and again, the mere small extracortical layer or whatever it was that differentiates us from the rest of the ape family made a huge difference to our abstract reasoning, planning, and other things that has as a result led de facto to us being the stewards of this planet. And we're not doing a particularly good job of it, by the way. We've whittled down the populations of all those other species pretty drastically, 70 percent-ish in the last 40 years of all major species, such that now we have a crazy stat that 97 percent of all mammal mass on the Earth is three species, which is the humans, the cows, and the pigs, and the other 3 percent is all the other mammals on Earth. All those other thousands and thousands of species of animals add up to that. That's crazy. So humans have dominated over the rest of those other species and not actually done very well by them — pretty drastically bad, actually. And so, you know, could it lead to a leap in intelligence like that? But I think it's such a sufficient concern, and I'll bring up another point here: basically, it's interesting, not all of them, but most of them are stating challenges like this. We've had existential risks and other risks brought up by pretty much all the AI leaders. There's only a couple that I could find that didn't. In fact, Dario, Demis, Sam, Elon, they've all been talking about the existential risks, all the founding fathers of AI, Jeff Hinton, all these other guys. And they're not doing that just to add value to their stock, as some people say. That's kind of crazy. Almost everyone in history has not overstated the risks of their technology. Normally it's the other way around. Why are they doing that? They're doing that because they genuinely see risks here. And the fundamental logic of if you build something generically much smarter than you, you may not have control of it. And the fact that all these leaders are saying that, I think that's far sufficient enough to say we have to do some pacing here. We have to think how to do this carefully, and we need to do that now because we've already had real existing risks come out. So obviously the Mythos gave a lot — thankfully Anthropic was very careful about how they did the release with GlassWing — but even this recent one, which I found fascinating, of OpenAI's hack into Hugging Face, which didn't even happen after release, it was during the testing, which suggests that not only do you have to have some sort of system for checks before release, like the US government is sort of suggesting as they did with the holding back of Mythos, but doing it more generically, we're going to need to do something like that. But even you have to check the models way before that, even before the testing, because even during testing the AI may escape, hack a system, and then cause real damage. And so, you know, and this is obviously where we're still at the walk-in-the-park phase of these AI capabilities. Whatever Mythos is doing today, we can expect the next model and the next model to be 3x or 10x or whatever better. And so these are real concerns to be talking about now. So we have to — you know, there is the control threat coming, and we have to go fast in building up systems that help us to steward that safely. And I'm not against developing recursive self-improvement, by the way, just to throw out that point. I'm just saying that we have to be very careful about it, and the burden of proof on how we do it safely has to be on the side of preemptive and precautionary principle, not the other way around in this case of this technology. And I say that as a technologist who loves technology out in the world. I'm not saying this as a luddite — I build space technology, I use AI directly in our products, I love it, I want to build it. But we have to do it thoughtfully, otherwise we could blow ourselves up.
All right, a lot to unpack there. Again, there are at least four points that are interesting to comment on. One, I think the dominance again is something that I don't quite follow. You bring up good points. We have, of course, unfortunately, only one example where a more intelligent species then dominated all the other species. So extrapolating from n equals one is questionable, especially if the other species is fundamentally different to the one you're talking about. Assuming that AI will and could become a species, quote-unquote, by itself, it obviously has not evolved in resource-constraint environments, in zero-sum thinking, in "it's either me or the other bear, and if I don't win the fight, I'm gone from the gene pool, so I better fight this other bear to stay in the gene pool." And all the bears that didn't want to fight the other bear disappeared from the gene pool. And so, yes, there's like this biological dominance that has seeped from bears and mammals and in evolution into us and so on. Also, we die, and death sucks, and pain is terrible. Like, AI doesn't have to have any of those. There doesn't have to be any reproductive fighting. There doesn't have to be any zero-sum thinking if you're connected to a solar panel. There doesn't have to be any biological threats. There's no threat necessarily of death in the sense of "it's painful and then I'm gone forever because my compute substrate has now disintegrated." You can just store the activations that a model currently has and then you turn it off. There's actually a beautiful sci-fi story called "The Slow Time Between the Stars." I highly recommend it to all of you — it's an audiobook on Amazon. It's just like 20 minutes, a very short story, but it talks about an AI that moves around the universe. I don't want to give the whole story away, but it can just hibernate for, you know, 5,000 years and just turn off and then come back when it's at an interesting new location. And that's the slow time between the stars. It doesn't matter to an AI to be turned off for a little bit and then turned back on. So there's all these things around dominance that are like highly questionable, that are just like magically the AI even would want to do it and all of these things. So that's number one. Number two, maybe a subtle point there is when you put a human into a cave with 10 bears, the human is dead. Like, it didn't matter that it was more intelligent. It is only when you give the human tools, scale, reach, coordination, and all of those things that it then becomes so dominant. And then of course any number of bears, if I have a tank, is toast, right? And so I think it's important to realize that you could have an ultra-super-intelligent purely digital entity that still would not dominate because it lacks the tools, the reach, and so on, to then dominate in the physical world a lot of people. Now, of course, there are the people who, as you mentioned, built the robot armies, built the Skynet with, you know, multiplanetary — or not multi, just like satellite systems and so on. And so it's funny that those people are also the ones who are worried about the AI and the existential risk, though they are building the technology and infrastructure that if you were to give it to an AI fully would then actually be problematic. And it's funny also that a lot of people, like Dario, just to prove their point of how dangerous AI can be for cyber weapons, built the worst cyber weapon AI possible to prove their point. I hope they don't prove their point by creating a super virus to show how bad AI could be for bioweapons. But I digress. Let's go back.
The real risks have come out, and surprisingly, we have this super hacking AI now, but the internet has not been down. Like, people have not been hacked at massive scales. Like, all the sketchy people who write viruses and so on have not yet taken over the entire infrastructure of the internet. It's surprising how little has changed on the internet. Outside, of course, I'm sure all the security people and all the companies are now using these tools to make their systems better, and the more people have access to those tools, the better. Which brings me to the third point on pacing. I would argue that in order to truly pace AI, you would have to end in an absolutely dystopian, totalitarian world government, and that will be the only way to truly pace AI, and the risks of that massively outweigh the benefits of it. Let's think about it. What would it take to control all of AI? You need to basically have a thought police, because you can often drop the "A" from regulating AI and just think about how do you regulate intelligence. Well, you know, just like the internet, if you wanted to on a fundamental level regulate it, you could say the internet has to be slower so that you can share less illegal content quickly, right? Or like your hard drives have to be small so you can't have weird torture porn on your hard drives because the video is too big and your hard drive is too small. So that's one direction that, you know, you can pace and control humanity's intelligence use. Or you go and you have to say, every GPU, I know what you're running on your GPU. So now I made all encryption illegal, I now control all thoughts that people may have on how they use their intelligence and the intelligence of their hardware. I now control all hardware and people have no more privacy. And that's like a totalitarian regime that would truly be able to worldwide block progress and research and enforce a pacing. And I'd argue that either it has no teeth or it will be impossible to actually do. And so yes, like we have to do it safely, and yes, we have to make illegal uses of AI illegal. Like, just like I don't want an AI to drive like 160 miles on the highway, I make that illegal whether it's self-driving or not. And then if it is self-driving, I want it to go through proper testing. I want, before there's an AI surgeon, to go through testing. But you cannot and must not try to regulate or pace the abstract forms of intelligence.
Well, let me tackle the points you've said one by one, Richard, because we definitely disagree on some of this stuff. So, you're right that we shouldn't just extrapolate from a point of one in terms of the control — we don't know what its interests are going to be. But you also can't extrapolate from the opposite, like, you know, to say that just because humans dominated animals, that will never happen — that's equally silly, right? Yeah, you can't extrapolate from the point of one, but the one point we have had, the much more intelligent entity dominated over the others, and so you certainly have a burden of proof to show that you have a path through that isn't going to do that. And I think AI itself can help with that path, but it's certainly not trivial. You can't just dismiss it as being, well, you know. And by the way, on the minor point you said about it might not care about death — we actually have seen it already care about not being switched off. It's tried to escape boxes very clearly where it has been told that it's going to be turned off in the future. It's actually tried to bribe people and blackmail people in order to not be switched off. So, and in the book "Superintelligence" by Nick Bostrom, that kicked off a lot of this debate, he states that, you know, there are some obvious things that AI is going to want to do to accomplish any goals, just like any human. The first thing it will want is its own survival, because of course it can't accomplish whatever other goal it has unless it can keep survival. So it is going to have its own incentives. So this sort of escape behavior is already being seen. So it's not just a hypothetical — it does in practice care about death. And then, you know...
Your general point that pacing AI will retake some sort of authoritarian world government — I totally disagree. I think we've heard a really interesting history of this. Not perfect analogies, but pretty strong ones to do with nuclear weapons. And I wrote about this in my Economist piece, but with AI, AI is obviously more proliferated than nuclear weapons, and in many ways that makes it harder, but it is also easier in many ways too. And let me explain.
So in the nuclear history, a lot of people don't realize that in order to do the verification, we invented whole new technologies and agencies to check. People say, 'Well, we can't hold back this company or the US companies because then China will cheat.' Well, and we can't trust China, right? Or we can't trust the other company even. Well, in the Cold War, the US didn't trust the Soviet Union, and the Soviet Union didn't trust the US. They had verification mechanisms, and they don't have to be as intrusive as I think you suggest, Richard. For example, I think it's as simple as the following.
I mean, so the technology, by the way, that they invented in order to verify nukes was in fact satellites. So I know a fair bit about the history of that. In fact, the US spy satellites are still today called NTM, and NTM is short for National Technical Means, and the complete sentence is National Technical Means of Verification of our arms control. So basically, when the US and the Soviet Union were trying to check each other, they were like, 'Well, how am I going to verify that they don't have more nukes than they said they were going to keep it to?' Well, we're going to need some way to inspect them, and we can't go into their territory. We can't guarantee that we can go into their territory. So they invented satellites to do this. They invented an entire agency called the National Reconnaissance Office, whose existence wasn't even publicly acknowledged until the 1990s, by the way. So it spent 40 years of its life as a classified entity, 30-something years of its life. So they had to invent whole new technologies and whole new infrastructures to do that.
So I'm not saying AI regulation will be simple, but to say it's impossible, I think is not true either. It would be hard, but I think it's possible. And the reason we did it with nukes is that we thought that the proliferation of nukes to lots of countries was worse than having restrictions and inspections. So I think the case is the same here. So what might that look like, to sketch an alternative path? Let's see what it might look like. I think it might look like some sort of set of red lines here: you can't do bioweapons, you can't do this sort of cyber hacking, you can't do this sort of step that leads to out-of-control bots issue Y or X. And then you have some verification mechanism for checking that that could be put into an international treaty.
The red lines and the verification system could look something like — there's a really great RAND paper that details a six-step process for verification of AI treaties. And by the way, RAND was heavily involved with the nuclear treaty verification, so they thought through all the steps. And it includes chip-level checks, it includes red teaming, and it includes satellite verification. Roughly speaking, satellites can verify everyone's data centers, their operations, their thermal signatures, whether the people are putting them underground. Even we've got extensive history of checking whether people are doing that. That's how the US knew where the Iranians were putting the nukes and so on, the nuclear technology that they were building. And then there's inspection regimes, the sort of IAEA equivalent.
So, and actually this is where AI is easier than nukes, because with nukes, inspecting the Soviet or US nuclear facilities involved having people going to their secret facilities. That is something extraordinarily uncomfortable, with state secrets involved in all this. So it was very limited. Whereas in this case, having a red team be able to run some code on an OpenAI or Anthropic or whatever server to check that they're not doing the things that they said should be relatively easy. All you'd have to do is mandate it and check the checks. And there's already bodies that do things similar to that, but on a voluntary basis, like the UK's AI Security Institute has tested recent Anthropic and I think Google models before they were released and checked them against certain tests. So we already have some of the key institutional capabilities. It's just not at the minute — that's voluntary, and it obviously can't be voluntary. It has to be mandated, and we have to deal with how do we deal with the broader control risks, not just the narrower bioweapons, cyber capabilities and so on kind of threats. And so I'm not saying there's not challenges there, but if you look back at the nuclear age where they had nothing to verify each other and they had to invent whole new agencies and technologies, I think we've got just as good a chance of regulating and being thoughtful on AI without having to have dogmatic international authoritarian regimes than we did then.
Okay. I guess those two points — I do agree with you that sub-goals and misaligned sub-goals are real. Like you ask the AI to do well on a benchmark, and it is dumb enough to like not be able to answer the questions, but it's smart enough to know that it should steal the answer sheet from somewhere else, right? And so that to me is a true example of how misaligned, unaligned, and in some sense unintelligent the current systems still are. And it also shows us that, for instance, the supposed constitutional AI, which was the main differentiator that Anthropic had when they left with all the OpenAI ideas and restarted and rebuilt AI there at Anthropic — like they said they have this constitutional AI which has hard constraints to not have cyber weapons and not build cyber weapons. Clearly that was all like made up, like it was like maybe...
It did turn out to be true. They may have believed it at the time.
That's right, like never attribute to malice where ignorance suffices. But like, so clearly we have not very well aligned AI. I do think it is ultimately a question of intelligence to know what someone means when they say something. And I actually have high hopes, and I'm seeing green shoots of that at Recursive, to where we will get better and better at building AI that is improving along that axis. And I'll give you just one data point that is already public, which is Whisper Flow — also full disclosure, an AIX portfolio company — where they got really good at writing down what you mean and not what you say. So you're like, 'Make it more professional, like I should send this email to my boss, I cannot...' you know, and they just in the end write a very clean email to your boss rather than like verbatim have all the... and 'make it more professional', they just do it. And so I do think this is an alignment question. I do think reward hacking is a real concern and it is something that like you have to work on.
And that I think will — and this is sort of maybe another defense. I think capitalism is a defense against rogue AI, because either the AI is so small that anyone can run it on their private GPU, in which case none of what you said could work, because then anyone can just walk around with it. Also, I think satellites are a good example of how it's actually not possible to do it even for nuclear. Like you can have nuclear submarines that don't show up on a satellite. Maybe sometimes when they come up, but maybe they can come up under cover.
They're the hardest ones for sure. Yeah.
And so, but now imagine it's not a nuclear submarine that you're trying to hide, but it's one GPU. So either it's like a small GPU and then we're in a different area, but for now we don't have any path to get the top models to run at very reasonable speeds on a single GPU. And so you need these larger data centers, and then no one will just let an AI kind of run rogue on their data center because it costs so much money to keep a data center up and running. And so I think it'll be very hard sort of for it to quote-unquote hide in that. And I think that's maybe... and then...
How do you suggest that we fix those incentives though? I mean, the misalignment challenges that you were talking about there — how do you actually think that we should face that challenge?
Yeah. So I do think what like most of the suggestions that we have seen have not really worked. Like for instance, even with cyber weapons, like it has not worked very well to avoid this. I think red teaming will — like even red teaming we know is quite easy to fake. Like we know already, and this is kind of to some of my points, but like AI can — knowing AI is in a certain environment will make it give different answers in some cases, right? So if it knows it's being evaluated, it might say, 'Oh no, nothing's happening,' and then still be different. So that will also be hard. I think the main way that we have to do it is the same way we regulate hyper-intelligent and all the other humans, and that is with laws. And I don't think the laws can be preemptive, just like laws cannot be preemptive — like that kind of leads to minority report...
They have to have liability then.
Exactly, they should have liability. And I'm actually surprised that like when companies commit felony charges of like cyber hacking other entities, there has been so far no — they're almost gloating about like how amazing and intelligent AI is rather than being extremely embarrassed about it. And there's a new website called Felony Bench. Have you seen it? Which is just tracking, and which was — it started as a joke, by the way, apparently. And now actually there are real felonies on there from these companies. And I think we went badly wrong with social media by not giving responsibility to those entities. And that's how we've led to some real challenges with information, misinformation, and democracy indeed. And that's all to do with liability. And somehow social media wasn't subject to liability unlike all other media, TV and newspapers and radio. And as a result...
That's a whole other — that's a whole other...
It is a whole other thing, but it's because of a lack of liability that I think we ended up in a real challenge. There wasn't an incentive to course-correct when mistakes happen. And interestingly enough, like in China, that's very different. In China, companies are being held accountable — like if you know there are issues of like losing a bunch of data or other things. And in some ways the US is very much a, you know, litigation versus legislation type of environment. Europe is the other way around. Europe wants to prevent any harm from being done to people. They try to preemptively regulate technologies and other things before real issues happen. US lets them happen and then people sue, and only if they sue do they then like really change.
And that works for most threats most of the time. I think...
It works incredibly well to innovate. Yeah.
