We're talking the future of AI with Jack Clark, co-founder of Anthropic. We had a chance to do this at the last second. So I'm in Miami on my laptop. Jack is in a much fancier setting, Anthropic HQ in San Francisco. Jack, welcome to Axios.
Thanks for having me on, Mike.
So you ignited online conversation by bringing a topic that is often whispered about, but hadn't been talked about this formally. And that is the early signs of AI contributing to speeding up research and development of the AI itself. And you're saying that by 2028, AI could be building itself. Take us to that frontier.
My prediction is by the end of 2028, it's more likely than not that we have an AI system where you would be able to say to it, make a better version of yourself, and it just goes off and does that completely autonomously. And the reason for that prediction is looking carefully at many public sources of information about AI development, and seeing that we've gone from just AI systems that can do basic coding to AI systems that are now involved in the production of science itself, and also the unglamorous but fundamental engineering work involved in doing any form of large-scale R&D project. And so I decided to write a post because the purpose of Anthropic Institute is to give people a sense of what's coming, and to also start talking about concepts early, to give us time to think about them. And I think you and I would agree that an AI system that can build itself has big implications, and we'd like to start thinking about it now.
And Jack, you have some trepidation about this.
Maybe this is the Englishman in me, but I've had quite a lot of change in the last ten years working in the technology industry. And this is going to create even more rapid change. I think if you talk to many Americans, there is rightfully some level of trepidation about the impact of AI and the speed with which the industry is moving. And what I'm looking at is a technological trend where, if anything, the speed will accelerate further.
And what you're envisioning here is AI building entire systems to replace human-built ones.
Yes. Although it's a bit... I don't think that humans are not going to have anything to do. In fact, we're going to have a huge amount of work involved in validating what the AI systems have done and looking at how these new systems work. But it means that the way that AI development works will change fundamentally and the speed with which the industry moves forward will go through another ratchet and move forward even faster.
And we've talked about recursive self-improvement, but is this recursive construction? Or help us understand how this is a leap beyond what the companies have been seeing and pushing for?
Let's use a metaphor. Today, it's like if all of the AI companies are 3D printer companies, we have tremendously talented people working to figure out how to make the next print head on the 3D printer that can do even finer-grained things. And to do that, we need to do a lot of scientific research to improve the resolution of that printhead. The analogy here for AI is what happens if a 3D printer could suddenly make a better version of itself? What happens if a 3D printer could print that finer resolution printhead for itself and start building better versions? Now, we haven't encountered a technology like that before. It's always been the case that humans outside the technology need to come up with the ideas that then get put back into it. What happens if we have a technology that can generate ideas within itself for how to improve itself? That's a new concept. It's a new thing for us to contend with.
So as we try to wrap our heads around this, what would this mean for Anthropic as a company? Do we need you?
We and the other companies are going to be taking this technology and trying to get it to do good in the world, right? To help push forward things like biology or medicine or robotics. And the implication of a technology that has got a lot smarter and is able to, to some extent, adapt itself to get better, is that it's going to open up the opportunities for us as people to steer that technology into domains where it's really, really, really hard to make progress because we need incredibly talented human scientists, of which we have too few. But being able to bring a technology in to say something like cancer research and have it work much like an incredibly smart human colleague, to orient itself around that tremendously difficult topic and figure out how it would need to improve itself to get better at that task, will yield amazing things for all of us. So I think we'll be quite busy if we achieve this technology.
So Jack, in your research agenda, you talk about fire drill scenarios. And there's a term, intelligence explosion, that had kind of been a theoretical term in AI safety circles. You're putting it in writing front and center. Tell us what an intelligence explosion is and why you're getting ready for it.
An intelligence explosion is where suddenly the AI systems start improving very, very rapidly. And the world has a new both opportunity and problem to contend with, which is what do you do with a tremendous amount of growth or a tremendous amount of abundance in many, many different fields of science all at once. Now you want to obviously deal with some of the potential risks of that work on things like the cyber implications or for biological weapon implications. But you actually also need to have a plan for what to do with the abundance side of it, you know, what do you do to ensure that these amazing technologies flow through to benefits to people in terms of improved healthcare services, lowering the costs of making healthy food for people, generally improving things that we would all want in our world. And it's like dealing with all of today's institutions have very, very narrow pipes through which you push new drug candidates or new ideas before they come to market. How do you massively broaden the size of those pipes in advance of this abundance? And how do you work on making the world defense dominant for the potential risks the technology creates?
What do you mean by defense dominant?
Right now, an experiment that we are doing in close partnership with others is something called Mythos, where we have a model and we have an initiative called Glass Wing, where we are trying to equip some of the world's most significant companies and organizations and the US government with the means to make a defense-dominant cyber environment using systems that have got far, far better at understanding the cyber domain. How are you going to do that for other domains as well? That's a muscle we're going to need to build, and it will allow us to make a world that's more resilient to the kind of changes that we expect to arrive, and better prepared for them as well.
Jack, I can tell you my conversations behind the scenes with the Pentagon and the administration, Mythos and its power is the reason that the Pentagon is anxious to make up with Anthropic. Why the administration is anxious to make your tools available more widely. What have you learned about the power of Mythos since you've started letting some partners use it? You knew in your lab that it had cyber penetration capabilities that were awesome, fearsome, which was the reason that you wanted to be so careful with letting it out. What have you learned now that it's in its controlled public environment?
And this is the philosophy behind forming the Anthropic Institute: we have this information that only labs know about a technology like the cyber capabilities of Mythos. How do you get that to the world so that you equip the world to make better decisions? And what we've learned is that we cannot solely centralize these technologies inside a very small number of players. We have to get better at figuring out what is the community across policymakers, companies and other organizations that we can flow new technologies to so they can work on the problem together. And I think one of the most joyful things about all of this has been we've got this serious technology, we've deployed it to a set of people. And through experiencing the power of the technology they've made their organizations safer, but they've also become conscious that this isn't a one-time thing. This is the shape of the future, and it's got us to a much more, I would say, collaborative place in working the policy problem that AI presents and also coming up with solutions together.
Have you had any scares with Mythos in the semi-wild?
Not yet, but we are continuing to treat it as a handle-with-care technology. But ultimately the goal here is you don't want to end up in a regime where all AI technologies need to be handled with care. That would lead to issues for how you diffuse them to maximally benefit society. A lot of what we're learning is how do we build robust safeguards and robust interventions so that in the future, the whole world can have confidence in diffusing this sort of technology more widely to a broader set of people to capture more of the benefits.
Now, Jack, I can't tell you how surprising are some of the elements of this research agenda. You think research agenda? You think it's going to be a snore? I was lit up by several of the points here, including you were talking about abundance for some, but there's another road where there could be real disruption, economic job pain. And one thing the institute is going to do is look for early warning signals of significant change disruption, and is going to encourage the government to help you respond to it. That is new, that is industrial policy. Tell us how you came to that, Jack.
