Hi, I'm Ilia with a special competitive studies project. During our summit on AI plus science, which we just held in Washington, DC, I had a chance to record my next episode of Memos to the President with Jack Clark, co-founder and head of policy for Anthropic, one of the leading frontier model companies that we have. I hope you enjoy the conversation as much as I did. Good afternoon everybody. I know it's always difficult to use the cliche that Jack and I are the last panel before the lunch, but I promise you this is going to be a tremendous conversation. I've known Jack for a long time. And so before we start, Jack, welcome by the way. Welcome to DC. I know you come here a lot, but it's so good to have you at our event.
It's great to be here and such beautiful weather as well.
Thank you. But I wanted to start by showing the audience a picture of you doing a demo of a large language model at one of our summits in 2022 where nobody knew here at least what large language models are. So if we have the picture up. Yeah, this is Jack, young Jack from 2022. So as I said, you've been in this journey for a long time. So I wanted to start by asking you just to reflect on where we are now and the journey you have seen to get here.
Well, maybe let's start. I mean, who here has actually used like a large language model in the last week? Just stick your hand up. So, that would never have happened, you know, two or three years ago. Three years ago, this stuff just wasn't available. Two years ago, it started to be used in a sort of hobbyist way. And now, it's being used by pretty much everyone for a wide set of purposes. Now, the crazy thing is that we all thought that this was going to happen in, you know, 2016 or 2017, but we didn't know how exactly when it would arrive. And then we had this amazing boom in recent years and now it feels like we're really off to the races with a new class of technology that's actually being used. It's a long way from where we were.
One of the things Jack that I think I admire about you is you're one of the rare individuals that have really come and visit DC frequently, you know, in the last seven, eight years I would argue. How has the discourse and dynamics changed in DC in understanding AI? We were talking about this where we were in 2018 and how people talked about AI. But you have observed these dynamics over time.
So I think a few years ago if you came to DC and talked about AI, people would want to talk to you about 5G and quantum. They'd be like, 'Oh, AI that sits alongside 5G and quantum and maybe biotech is this basket of important technologies.' Which is all true. Now, people actually have a pretty sophisticated understanding of it. I've been on the hill this week and I've been asked questions about timelines to self-improving AI, whether we could eventually have a loss of control of our AI systems, and also, you know, what to do about the pacing threat of Chinese models like DeepSeek. That's a huge level of AI literacy that we've reached. And a few years ago, I was just being asked about Terminator and how 9,000.
Yeah, you're right. I still get asked about that, but now the other stuff as well. True. Let's talk about where we are in terms of large language models, Jack. What's your observation with the current state of models? And then my follow-on question would be, how do you see the next 3 to 6 months of this space evolving?
So I think the most important thing to take away from where we are now is that language models are a lot more powerful than most of us think today. We keep on having this phenomenon in AI research where we think to ourselves, huh, could an AI model be good at say mathematics and then we give it a test and we discover that it's actually far better at mathematics than we thought. And the same has been true of physics, biology, a broad set of skills. Additionally, language models today can take actions for you over large periods of time. And I'll just try and give you some intuitions here. So, a couple of years ago, if you asked a language model to do a task for you, like translate a sentence or take a simple bit of code and rewrite it into another language, it could do that. And that would be the sort of thing that might take a person 30 seconds or a minute to do. Now we have AI systems where they'll do tasks that might take a person an hour or two hours of work to do. It's a huge change and that's leading to tremendous changes in not just how people use the technology but what kinds of things become possible. And so when you look to the future we're a few months away from having AI systems that will be kind of like synthetic employees that sit inside organizations doing tasks for you. We're way beyond interns that occasionally get stuff wrong and need extremely close supervision. It's more like you can bring a load of junior employees inside your organization and get them to work for you. And that fundamentally changes things. I have a funny story about that from my work where someone posted on our internal work slack recently, our AI system is called Claude and he said, 'What should I do when Claude is working for me?' So, his new problem is not just like how do I do my job, it's what do I spend time on now that I have my fleet of Claudes doing my job for me? And that illustrates the kinds of challenges we're going to be going through.
No, that's a good point. And so how do you even handle that in a company like yours?
Well, in a company like ours, and I think this is going to be true of everyone, the skill is less now, you know, being able to program because the AI systems can do that. It's being able to think about the problems that are important, the experiments that you want to run, and sort of how you want to run and manage ambitious projects because you now have this phenomenon where it's like everyone has the ability to be a manager of a bunch of quite competent employees. But the competent employees are ones you can only communicate with by email and they're extremely literal. So, you have to get quite good at articulating what you want to happen and figuring out how you describe projects in text. But if you can do that, it completely changes the nature of your work and it changes how much work can get done and who buys.
No, I agree with you 100%. I think the more you use them, you get better at prompting and better about managing and delegating tasks. What were some of the things that you have prompted that it really was an eye-opening moment for you?
There's a couple of things. I think just maybe a personal one is I became a father recently and so I started writing a sort of diary about becoming a father and you know when you become a parent it changes your relationship to the world and to your partner and I was saying like how can I be a good dad while doing this AI company and how do I think about the AI revolution and then I loaded all of it into my AI system and I said what is the author of this personal diary not thinking about. And Claude said, 'Well, you know, despite their references to becoming a father and running an AI company, I don't think this person recognizes the metaphysical shock they're experiencing.' And I stared at it and I got really sort of mad because it was like, you know, I've lived in California, so I had a therapist for a couple of years. It was like when a therapist said something completely dead on and you got sort of annoyed at them. And I actually went for a 5-hour walk. And then on the walk, I was like, 'Hang on, I'm on a 5-hour walk because a machine which I have built has criticized me in a way that made sense to me.' It's really unusual. The only people that have succeeded at doing that before are like close friends and my spouse. And so I think that illustrates how smart these systems have become. They can look at something and they can make inferences which are not obviously in the text itself. Now that's the kind of personal example, but the professional one I find really interesting and all of you can probably play around with this on Claude or another language model. Get it to make a simple game like tic-tac-toe or snake or breakout and then say, 'Hey, could you just make this game much more exciting for me?' and the thing will now go away and it will come back to you with a version of tic-tac-toe where randomly the board rearranges or a version of snake where suddenly other snakes appear on the game screen. And it's kind of amazing. You're seeing these things be extremely creative, generate working software for you and do it in response to very little prompting. And I think it will illustrate to you just how advanced these things are and how they're more creative than most people think.
Yeah. Before we turn to some serious topics, I just want to double down on one more question here in terms of, you have a little one now at home. How do you think the future is going to look like for them in some sci-fi vision that you can see today?
