Were these dominant, super-powerful, politically connected monopolies or cartels? And finally progressives gained enough power where they regulated the railroads and everything was good. Kolko, a socialist, said, "No, that's actually wrong, and that it was the railroads when they reached a certain point of power, and they also realized that the market, it's going to get tougher and tougher to have profits, they went to the government and said, 'Hey, we need to be regulated.'" There was a moment with social media or the tech sector more generally in 2018 where Mark Zuckerberg said exactly that to Congress, where Tim Cook of Apple said, "You know what? The tech sector isn't working anymore. We need regulation." How, from a kind of libertarian or a Kolkoean socialist perspective, how can you assure me that this isn't what's happened? There's a half dozen big AI companies who are suddenly saying, "You know what? It would be a damn good thing for the citizens of America if we had national regulation that preempted all of these problems." How is that not just locking in your market position right now?
Well, I've said the same thing for 10 years. I've testified in Congress for 10 years. I said the same thing for 10 years. We started saying this the moment we started Anthropic when we were a company with no product, no revenue, nothing.
And now you have generally it's usually put around a billion dollar valuation, right? I mean you're private, but
And it all gets closer to a trillion dollar.
A trillion, excuse me. Yeah, that's a minor difference. I'm a lit major, I'm not a math guy.
Yeah, so look, I absolutely totally understand this frame. I'd say for us, look at our consistency of action, statements, and beliefs over time. Dario, I've worked with Dario, the CEO of Anthropic, for many years. We were both at OpenAI together. He's said the same things for 10 years as well. And the approach which we've always had is we believed and continue to believe this will be the most powerful and consequential technology that gets built in this century, for sure. And the answer to how you deal with it must be that we need to create new tools for us to govern and manage it. And we need to integrate it with our existing laws, and we need to integrate it with some kind of system by which it can't only be companies making decisions about it because the technology will be too powerful and too influential.
Can you allay my worries that—and this is something that Zuckerberg said to Congress—that, you know, I will help you write the regulations to control social media, but you realize that means there will never be another Facebook? Is that the price you're willing to pay? And obviously it got a little bit squirrely for a lot of different reasons, but if Anthropic and OpenAI and a couple of other companies get to write the rules, does that mean okay, these are the five firms or whatever that are just going to dominate forever?
We've shared our ideas for regulations, but I really welcome and encourage people like you and others to write your own take on it. I think you've seen people like Dean Ball, who was briefly a Mises Admin. He does a huge amount of public writing on what he thinks are sensible policy approaches. Oren Cass has done similar work.
And now that free market—I mean he—
The point I'm making is the more people you have outside the companies articulating a view here, the better it is for everyone because you want a big pile of different ideas around how to approach this. And then some of your ideas there might be commonality. Do those ones. If you can build me a table that has like companies, libertarians, free market maximalists, and there are some ideas where everyone agrees, maybe those are sensible ideas to do. And if there are ideas that only the companies propose, be really really skeptical of them because there's probably a reason the others didn't propose it.
Talk about when we're dealing with issues of governance—the US is one thing, the EU is something else. And I think generally speaking, certainly most people in America would believe this. I think many Europeans would—their approach to technological innovation governance has been bad. It's like tamped down and Europe is further behind than it was.
They've done regulations ahead of their own capacity to have the thing itself.
And then you have places like China which are kind of like, you know, we don't care. How does US governance end up affecting global governance? Because the EU is going to do what it's going to do and China is going to do what it's going to do. How does this play out in the best possible scenario?
The best possible scenario is something that happens millions of times a day, which is cars get exported between all of those markets—maybe with the exception of the US in the car case—to one another. Planes take off from airports in all of those places and land in the other ones. And you somehow have confidence. Oh, food gets exported. You have confidence in the cars, planes, and food. Why is that? It's because you have some notion of shared standards, shared testing, shared means by which you can validate that the thing has the safety properties needed for it to be exported and imported. And rather than having some kind of global governance regime, you instead have a bunch of standards bodies in different countries which basically have reciprocity. And I think that this has worked amazingly well for commerce. You know, I buy a lot of baby toys now as I have a 3-year-old and a 7-month-old. It's great to have certain labeling standards because my wife likes to buy organic food and also doesn't want toys with lead in them.
I understand, but there's actually labeling schemes that let me do that. So I think that the—
But aren't a lot of those voluntary, right? I mean, if you buy something at IKEA, a great Swedish company, it's less likely to be painted with lead paint.
Many of these are voluntary, like the Energy Star thing from the utility industry. But then some of these get codified as just like hard standards. You know, there are certain hard standards for automotive and airline security which is just baked in. And then there's a lot of stuff where it's voluntary and we compete. But that's just how the world figures this out. It figures out what is the core set of things which just has to be true.
I am worried in the sense that one of the great success stories of the post-World War II era was a general reduction in tariffs and trade exemptions as well as capital requirements that capital can roam more freely around the world. And that seems to be—if it's not coming to an end, it is very much under attack by this administration, by a Republican administration as well as Democrats like Elizabeth Warren or Bernie Sanders types. That worries me because Trump has made it clear he doesn't want particular vegetables or people from certain countries coming here and there seems to be broad support for that. So I worry about that. But if we may, let's talk—I want to ask you about recursive self-improvement, which is a big focus of material coming out of Anthropic. Can you explain what that is and why this is the kind of big thing that we should be talking more about?
AI moves forward by people gathering resources like computing data and using relatively well-understood algorithms to train deep neural networks on those resources and you end up with these amazing predictive machines, generative models, universal teachers. Well, these systems have started to get really good at also writing code for training AI systems. They've started to get good at proposing ideas for how you might train AI systems. There may come a point where we as people can step back entirely from their forward development and it's sort of a case of wind them up and watch them go. We give them the resources and we say, "Okay, Claude 10, build Claude 11." And Claude builds the architecture, does the research, de-risks it, does the training run. Claude 11 comes out, it's better in every way than Claude 10. We don't see that today, but I think there's real potential for this to arrive certainly this decade, but if I had to bet the year, it would be 2028 towards the end. And that's based on reading a lot of the scientific literature and also we published information.
Like, I mean, you said you have a 7-month-old, I have a 6-month-old, they'll be a couple years older.
A couple years older. 2028 isn't even two years away. It's quite close. Yeah, AI moves quickly. And the implication is we should expect AI systems to move even further in the future, grow even more powerful than they have so far. We should expect the progress we've felt in the last five or six years to actually maybe be compressed into the next two or three years and then compressed again.
Are there other examples of that, at the surface level, where a technology has accelerated—like the improvements of the technology have accelerated that much?
There's basically no example. The only good example is evolution of biological creatures and it takes millions of years. But, you know, you can make better dogs. It just takes many generations. Probably the best analogy I have is imagine a 3D printer that could print a finer resolution print head than the one it had. That would be amazing. And it would yield an amazing burst in productivity of what you could manufacture and make. That's kind of what we're talking about here.
How do you put hedges on that? How do you put brakes on? How do you build in brakes? Because this is part of the doomerism and the apocalypticism—that AI, you know, it realizes like, oh, you know, human—it's not just that it doesn't need humans or shareholders, but they're in the way.
Well, it's a harder problem but of similar form to what we're currently dealing with things like cyber and bio, which is you have some property of the system that you're worried about. Well, you need to know how to measure it. You need to know how to protect and actually control it, make it so that the system can sometimes not be capable of AI R&D. And you need to probably demand transparency out of the places which are building this because you as society might say, this is great and it's going to drive loads of strategically important stuff in our economy and national security. And we probably don't need it to go thousands and thousands of times faster than any human can hope to govern it. So, you need some measurement property of it. A lot of the work that I'm doing at the Blithe and Propy Institute, which is kind of a think tank with a supercomputer inside Anthropic, is studying AI research and development within Anthropic and figuring out the measures that we might use to spot recursive self-improvement if it were happening. And eventually you want to share those measures publicly and socialize them with policymakers similar to bio or cyber and say, "Hey, we're going to need to figure out a framework and set of thresholds around this because this is how we can make science go orders of magnitude faster than today. It's how we can intentionally bring benefits to today that will seem like miracles—novel therapies, incredible advances in material science." And the same would be true of national security relevant capabilities. And you're going to need to figure out what the control layer is on that.
Is there broad buy-in from other major AI companies to kind of this type of sharing of standards and transparency? Because it's also, you know, like I've got my black box up here and I'm not giving you the Coca-Cola formula.
So, we said recently in a post about recursive self-improvement, we think it would be really good for the world to have the option to potentially slow down the pace of progress if you hit something like RSI. You might want to be able to intentionally slow that down to give us time to adapt.
Who is—yeah, and who is "we" in that? Because, and I'm not trying to be a libertarian, although I fail at that on an hourly basis. But to be like, "Who's we, Kemosabe?" Because Anthropic so far has acted like a good corporate citizen, right? You delay the launch of something that you think is too powerful. Okay, so that's good. But then what about other companies or a part of a government that says, "No, we've got to go for broke right now."
I think all we can do is put these ideas out there, socialize them, and I believe that if it's truly important, the world will find ways to come together to coordinate around that data. I think that the world has done this in the past, and it may need to do it in the future.
CFCs is one of the best examples. We were like, "Oh, okay. All right, no one actually wants the hole in the ozone layer." Manufacturers, scientists, governments, let's just stop this. It's great. That was a great amazing example of global coordination.
And it's interesting that very few people talk about that. So, the big loss was that McDonald's started wrapping things in wax-coated paper rather than using their clamshell foam container, right?
And the world cried a single tear, but came for it.
