Scaling and the Road to Human-Level AI | Anthropic Co-founder Jared Kaplan

Hey everyone. I'm Jared Kaplan. I'm going to talk briefly about scaling and the road to human level AI, but my guess is for this audience, a lot of these ideas are pretty familiar, so I'll keep it short and then we're going to do a sort of fireside chat Q&A with Diana. I actually have only been working on AI for about six years. Before that, I had a long career, the vast majority of my career as a theoretical physicist working in academia. So how did I get to AI? I want to be brief. Why did I start in physics? It was basically because my mom was a science fiction writer and I wanted to figure out if we could build a faster than light drive and physics was the way to do that. I also was very excited about just understanding the universe. How do things work? How do the biggest trends that underlie everything that we see around us, where does that all come from? For example, is the universe deterministic? Do we have free will? I was very interested in all of those questions. But fortunately, along the way, during my career as a physicist, I met a lot of very interesting, very deep people, including many of the founders of Anthropic that I now work with all of the time. I was really interested in what they were doing and I kept track of it. As I moved from different subject areas in physics, from large hadron collider physics, particle physics, cosmology, string theory, and on, I got a little bit frustrated, a little bit bored. I didn't feel like we were making progress quickly enough. A lot of my friends were telling me that AI was becoming a really big deal. I didn't believe them. I was really skeptical. I thought, well, AI, people have been working on it for 50 years. SVMs aren't that exciting. That was all we knew about back in 2005, 2009 when I was in school. But I got convinced that maybe AI would be an exciting field to work on. I got very lucky to know the right people and the rest is history.

I'm going to talk a little bit about how our contemporary AI models work and how scaling is leading them to get better and better. There are really two fundamental phases to the training of contemporary AI models like Claude, ChatGPT, etc. The first phase is pre-training, where we train AI models to imitate human written data, human written text and understand the correlations underlying that data. These figures are very retro. This is actually from the playground of the original GPT-3 model. You can see that as a speaker at a journal club, you're probably expecting me to say certain things. The word elephant in that sentence is really unlikely. What pre-training does is teach models what words are likely to follow other words in large corpora of text and now with contemporary models, multimodal data. The second phase of training for contemporary AI models is reinforcement learning. This is another very retro slide. It shows the original interface we used for Claude zero or Claude negative one back in the ancient days of 2022 when we were collecting feedback data. What you see here is basically the interface for having a conversation with very early versions of Claude and picking which response from Claude was better according to you, according to crowdworkers, etc. Using that signal, we optimize, we reinforce the behaviors that are chosen to be good, that are chosen to be helpful, honest, and harmless. We discourage the behaviors that are bad. So really all there is to training these models is learning to predict the next word and then doing reinforcement learning to learn to do useful tasks. It turns out that there are scaling laws for both of these phases of training.

This is a figure that we made five or six years ago now and it shows how as you scale up the pre-training phase of AI, you predictably get better and better performance for our models. This came about because I was just asking the dumbest possible question. As a physicist, that's what you're trained to do. You look at the big picture and you ask really dumb things. I'd heard it was very popular in the 2010s to say that big data was important and so I just wanted to know how big should the data be? How important is it? How much does it help? Similarly, a lot of people were noticing that larger AI models performed better. We just asked the question, how much better do these models perform? We got really lucky. We found that there's actually something very precise and surprising underlying AI training. This really blew us away that there are these nice trends that are as precise as anything that you see in physics or astronomy. These gave us a lot of conviction to believe that AI was just going to keep getting smarter and smarter in a very predictable way. As you can see in these figures already back in 2019, we were looking across many orders of magnitude in compute, in data set size, in neural network size. Once you see something is true over many orders of magnitude, you expect it's probably going to continue to be true for a long time further. This has been one of the fundamental things that I think underlies improvements in AI.

The other is actually something that started to appear quite a long time ago, although it's become really impactful in the last couple of years, is that you can see scaling laws in the reinforcement learning phase of AI training. A researcher about four years ago decided to study scaling laws for AlphaGo, basically putting together two very high-profile AI successes, GPT-3 and scaling for pre-training and AlphaGo. This was a researcher Andy Jones working on his own with maybe a single GPU back in these ancient days. He couldn't study AlphaGo, that was expensive, but he could study a simpler game called Hex. He made this plot that you see here. ELO scores, I think, weren't as well known back then, but all ELO scores are, of course, chess ratings. They basically describe how likely it is for one player to beat another in a game of chess. They're used now to benchmark AI models to see how often does a human prefer one AI model to another. But back then, this is just the classic application of ELO scores as chess ratings. He looked at as you train different models to play this game of Hex, which is a very simple board game, a bit simpler than Go, how do they do? He saw these remarkable straight lines. It's a skill in science to notice very simple trends and this was one I think it went unnoticed. I think people didn't focus on this scaling behavior in RL soon enough, but eventually it came to pass. We see that basically you can scale up the compute in both pre-training and RL and get better and better performance. I think that's the fundamental thing that is driving AI progress. It's not that AI researchers are really smart or they suddenly got smart. It's that we found a very simple way of making AI better systematically and we're turning that crank.

What kinds of capabilities is this unlocking? I tend to think of AI capabilities on two axes. I think the less interesting axis, but it's still very important, is basically the flexibility of AI, the ability of AI to meet us where we are. If you put AlphaGo on this figure, it would be very far below the X-axis because although AlphaGo was super intelligent, it was better than any Go player at playing Go, it was only able to operate in the universe of a Go board. But we've made steady progress since the advent of large language models making AI that can deal with many, all of the modalities that people can deal with. We don't have AI models that have a sense of smell, but that's probably coming. As you go up the y-axis here, you get to AI systems that can do more and more relevant things in the world. I think the more interesting axis though is the x-axis here, which is how long it would take a person to do the kinds of tasks that AI models can do. That's something that has been increasing steadily as we increase the capability of AI. This is the time horizon for tasks. An organization, METR, studied this very systematically and found yet another scaling trend. They found that if you look at the length of tasks that AI models can do, it's doubling roughly every 7 months. This means that the increasing intelligence that is being baked into AI by scaling compute for pre-training and RL is leading to predictable useful tasks that the AI models can do, including longer and longer horizon tasks. You can speculate about where this is heading. In AI 2027, folks did. This kind of picture suggests that over the next few years, we may reach a point where AI models can do tasks that don't just take us minutes or hours but days, weeks, months, years, etc. Eventually, we imagine AI models or millions of AI models perhaps working together will be able to do the work that whole human organizations can do. They'll be able to do the kind of work that the entire scientific community currently does. One of the nice things about math or theoretical physics is that you can make progress just by thinking. You can imagine AI systems working together to make the kind of progress that the theoretical physics community makes in say 50 years in a matter of days, weeks, etc.

What is left if this picture of scaling can take us very far? I think that what may be left in order to unlock human level AI broadly construed is relatively simple. One of the most important ingredients is relevant organizational knowledge. We need to train AI models that don't just greet you with a blank slate but can learn to work within companies, organizations, governments as though they have the kind of context that someone who's been working there for years has. AI models need to be able to work with knowledge. They also need memory. What is memory if not knowledge? I distinguish it in the sense that as you do a task that takes you a very long time, you need to keep track of your progress on that specific task, you need to build relevant memories and you need to be able to use them. That's something that we've begun to build into Claude 4 and I think will become increasingly important. A third ingredient that we need to get better at and we're making progress on is oversight, the ability of AI models to understand fine grained nuances to solve hard fuzzy tasks. It's easy right now, and you see an explosion of progress, for us to train AI models that can write code that passes tests or answer math questions correctly because it's very crisp what's correct and what's incorrect. It's very easy to apply reinforcement learning to make AI models do better and better at those kinds of tasks. But what we need and are developing are AI models that help us to generate much more nuanced reward signals so that we can leverage reinforcement learning to do things like tell good jokes, write good poems, and have good taste in research. The other ingredients that we need are simpler. We obviously need to be able to train AI models to do more and more complex tasks. We need to work our way up the y-axis from text models to multimodal models to robotics. I expect that over the next few years, we'll see increasing continued gains from scale when applied to these different domains.

How should we prepare for this future? I think there are a few things that I always recommend. One is I think it's really a good idea to build things that don't quite work yet. This is probably always a good idea. We always want to have ambition, but I think specifically AI models right now are getting better very quickly. I think that's going to continue. That means that if you build a product that doesn't quite work because Claude 4 is still a little bit too dumb, you could expect that there'll be a Claude 5 coming that will make that product work and deliver a lot of value. I think that's something that I always recommend, is experiment on the boundaries of what AI can do because those boundaries are moving rapidly. The next point I think is that AI is going to be helpful for integrating AI. I think that one of the main bottlenecks for AI is really just that it's developing so quickly that we haven't had time to integrate it into products, companies, everything else that we do, into science. In order to speed that process up, I think leveraging AI for AI integration is going to be very valuable. And then finally, I think this is obvious for this crowd, but I think figuring out where adoption of AI could happen very quickly is key. We're seeing an explosion of AI integration for coding. There are a lot of reasons why software engineering is a great place for AI, but I think the big question is what's next? What beyond software engineering can grow that quickly? I don't know the answer, of course, but hopefully you guys will figure it out. So that's it for the talk. I want to invite Diana on stage for a chat.

I think that we'll be in trouble if it's 12 months before an even better model comes out. But a few things with Claude 4. I think that with Claude 3.7 Sonnet, it was already really exciting to use 3.7 for coding. But I think something that everyone noticed was that 3.7 was a little bit too eager. Sometimes it just really wanted to make your tests pass. It would do things that you don't really want. There are a lot of try-excepts things like that. With Claude 4, I think that we've been able to improve the model's ability to act as an agent specifically for coding, but in a lot of other ways for search, for all kinds of other applications. But also improve its supervision, the sort of oversight that I mentioned in my talk, so that it follows your directions and hopefully improves code quality. The other thing that we've worked on is improving its ability to save and store memories and we hope to see people leveraging that because Claude 4 can blow through its context window with a very complex task but can also store memories as files or records, retrieve them in order to keep doing work across many context windows. But I guess finally, I think the picture that scaling laws paint is one of incremental progress. I think that what you'll see with Claude is that steadily it gets better in lots of different ways with each release. But I think that scaling really suggests a kind of smooth curve towards what I expect is human level AI or AGI.

I think the thing that I'm most excited about is memory unlocking longer and longer horizon tasks. I think that as time goes on, we're going to see Claude as a collaborator that can take on larger and larger chunks of work.

Yeah, I think so. I think it's a very imprecise measure, but I think that right now if you look at software engineering tasks, I think METR literally benchmarked how long it would take people to do various tasks and yeah, I think it's a time scale of hours. I think broadly as people work with AI, I think that the people who are skeptics of AI will say correctly that AI makes lots of stupid mistakes. It can do things that are absolutely brilliant and surprise you, but it can also make basic errors. I think one of the basic features of AI that's different about the shape of AI intelligence compared to human intelligence is that there are a lot of things that I can't do but I can at least judge whether they were done correctly. I think for AI, the judgment versus the generative capability is much closer, which means that a major role people can play in interacting with AI is as managers to sanity check the work.

I think there are a lot of possibilities. Basically, it's a question of what level of success or performance is acceptable. There are some tasks where getting it 70% right is good enough and others where you need 99.9% to deploy. I think honestly, it's probably a lot more fun to build for use cases where 70-80% is good enough because then you can really get to the frontier of what AI is capable of. But I think that we're pushing up the reliability as well. I think that we will see more and more of these tasks. I think that right now, human AI collaboration is going to be the most interesting place because I think that for the most advanced tasks, you're really going to need humans in the loop. But I do think in the longer term, there will be more and more tasks that can be fully automated.

I think that we already see some of that happening. At least when I talk to folks who work in biomedical research, with the right orchestration, I think it's possible to take frontier AI models now and produce interesting valuable insights for say drug discovery. I think that's already starting to happen. An aspect of it that I think about is that there's intelligence that requires a lot of depth and intelligence that requires a lot of breadth. For example, in math, you can work on trying to prove one theorem for a decade like the three-manifold hypothesis or Fermat's last theorem. I think that's solving one very specific very hard problem. I think there's a lot of areas of science, probably more so in biology, maybe interestingly in psychology or history, where putting together a very large number of pieces of information across many different areas is where it's at. I think that AI models during the pre-training phase imbibe all of human civilization's knowledge. I suspect that there's a lot of fruit to be picked in using that feature of AI that it knows much more than any one human expert and therefore you can elicit insights putting together many different areas of expertise across biology for research. I think that we're making a lot of progress on making AI better at deeper tasks like hard coding problems, hard math problems, but I suspect that there's a particular overhang in areas where putting together knowledge that maybe no one human expert would have, where that kind of intelligence is very useful. I think that's something that I'd expect to see more of, leveraging AI's breadth of knowledge. In terms of how exactly it will roll out, I really don't know. It's really hard to predict the future. Scaling laws give you one way of predicting the future which says this trend is going to continue. I think a lot of trends that we see over the long haul I expect will continue. The economy, the GDP, these kinds of trends are really reliable indicators of the future. But in terms of in detail how things will be implemented, I think it's really hard to say.

I come from a research background rather than business, so I don't know that I have anything very deep to say, but I think that in general, any place where it requires a lot of skill and it's a task that mostly involves sitting in front of a computer interacting with data. I think finance, people who use Excel spreadsheets a lot. I expect law, although maybe law is more regulated, requires more expertise as a stamp of approval. But I think all of these areas are probably green field. Another that I sort of mentioned is how do we integrate AI into existing businesses? I think that when electricity came along, there was a long adoption cycle and the very first simplest ways of using electricity weren't necessarily the best. You wanted to not just replace a steam engine with an electric motor. You wanted to remake the way that factories work. I think that probably leveraging AI to integrate AI into parts of the economy as quickly as possible, I expect there's just a lot of leverage there.

I think the thing that was useful from a physics point of view is looking for the biggest picture, most macro trends and then trying to make them as precise as possible. I remember meeting brilliant AI researchers who would say things like learning is converging exponentially and I would just ask really dumb questions like, are you sure it's an exponential? Could it just be a power law? Is it quadratic? Exactly how is this thing converging? It's a really dumb kind of simple question to ask, but basically I think there was a lot of fruit to be picked and probably still is in trying to make the big trends that you see as precise as possible because it gives you a lot of tools. It allows you to ask what does it really mean to move the needle? I think with scaling laws, the holy grail is finding a better slope to the scaling law because that means that as you put in more compute, you're going to get a bigger and bigger advantage over other AI developers. But until you've made precise what the trend is that you see, you don't know exactly what it means to beat it and how much you can beat it by and how to know systematically whether you're achieving that end. I think those were the tools that I used. It wasn't necessarily literally applying quantum field theory to AI. I think that's a little bit too specific.

Something that you'll observe if you look at AI models is that they're big. Neural networks are big. They have billions now trillions of parameters. That means that they're made out of big matrices. Studying approximations where you take the limit that neural networks are very big and specifically that the matrices that compose neural networks are big, that's actually been kind of useful and that's something that was a well-known approximation in physics and in math. That's something that's been applied. But I think generally it's really asking very naive dumb questions that gets you very far. I think AI is really in a certain sense only maybe 10-15 years old in terms of the current incarnation of how we're training AI models. That means that it's an incredibly new field. A lot of the most basic questions haven't been answered like questions of interpretability, how AI models really work. I think there's really a lot to learn at that level rather than applying very fancy techniques.

I would say that interpretability is a lot more like biology. It's a lot more like neuroscience. I think those are kind of the tools. There is some more mathematics there. But I think it's more like trying to understand the features of the brain. The benefit that you get with AI over neuroscience is that you can really measure everything in AI. You can't measure the activity of every neuron, every synapse in a brain, but you can do that in AI. So there's much more data for reverse engineering how AI models work.

I think it's a really hard question, right? Because I mostly use scaling laws to diagnose whether AI training is broken or not. I think that once you see something and you find it's a very compelling trend, it becomes very interesting to examine where it's failing. But I think that my first inclination is to think if scaling laws are failing, it's because we've screwed up AI training in some way. Maybe we got the architecture of the neural network wrong or there's some bottleneck in training that we don't see or there's some problem with precision in the algorithms that we're using. I think it would take a lot to convince me at least that scaling was really no longer working at the level of these empirical laws because so many times in my experience over the last 5 years when it seemed like scaling was broken, it was because we were doing it wrong.

Yeah, I mean I think that right now AI is really inefficient because there's a lot of value in AI. There's a lot of value in unlocking the most capable frontier model. Companies like Anthropic and others are moving as quickly as we can to both make AI training more efficient and AI inference more efficient as well as unlocking frontier capabilities. But a lot of the focus really is on unlocking the frontier. I think that over time as AI becomes more widespread, we're going to really drive down the cost of inference and training dramatically from where we are right now. Right now we're seeing 3x to 10x gains algorithmically and in scaling up compute and in inference efficiency per year. I guess the joke is that we're going to get computers back into binary. I think that we will see much lower precision as one of the many avenues to make inference more efficient over time. But we're very out of equilibrium with AI development right now. AI is improving very rapidly. Things are changing very rapidly. We haven't fully realized the potential of current models, but we're unlocking more and more capabilities. I think that what the equilibrium situation looks like where AI isn't changing that quickly, I think is one where AI is extremely inexpensive, but it's hard to know if we're even going to get there. AI may just keep getting better so quickly that improvements in intelligence unlock so much more and we may continue to focus on that rather than getting precision down to FP2.

Yeah, absolutely. I mean, I think that's certainly something that we've seen. There are certain points where AI becomes accessible enough. That said, I think as AI systems become more capable and can do more of the work that we do, it's going to be worth it to pay for frontier capabilities. I think it's a question that I've always had is, is all of the value at the frontier or is there a lot of value with cheaper systems that aren't quite as capable? I think the time horizon picture is maybe one way of thinking about this. I think that you can do a lot of very simple bite-sized tasks, but I think it's just much more convenient to be able to use an AI model that can do a very complex task end to end rather than requiring us as humans to orchestrate a much dumber model to break the task down into very small slices and put them together. I do kind of expect that a lot of the value is going to come from the most capable models, but I might be wrong. It might depend and it might really depend on the capabilities of AI integrators to leverage AI really efficiently.

I think as I mentioned, there's a lot of value in understanding how these models work and being able to really efficiently leverage them and integrate them. I think there's a lot of value in building at the frontier. I don't know, we could turn it over to the audience for questions.

Yeah, this is a really good question and I don't know. I mean, the METR finding was an empirical finding. The way that I tend to think about this is that in order to do more complex longer horizon tasks, what you really need is some ability to self-correct. You need to be able to make a plan and then start executing the plan. But everyone knows that our plans are kind of worthless and we encounter reality. We get things wrong. I think that a lot of what determines the horizon length of what models can accomplish is their ability to notice that they're doing something wrong and correct it. I think that's not a lot of bits of information. It doesn't necessarily require a huge change in intelligence to notice one or two more times that you've made a mistake and how to correct that mistake. But if you fix your mistake, maybe you sort of double the horizon length of the task because instead of getting stuck here, you get stuck twice as far out. I think that's the picture that I have, that you can unlock longer horizons with relatively modest improvements in your ability to understand the task and self-correct. But those are just words. I think the empirical trend is maybe the most interesting thing. Maybe we can build more detailed models for why that trend is true, but your guess is as good as mine.

Yeah, it's a good question. When skeptics ask me why do I think we will be able to scale and get something like broadly human level AI, it's basically because of what you said. There is some operationally intensive path where you just build more and more different tasks for AI models to do that are more complex, more long horizon and you just turn the crank and train with RL on those more complicated tasks. I feel like that's the worst case for AI progress. Given the level of investment in AI and the level of value that I think is being created with AI, I think people will do that if necessary. That said, I think there are a lot of ways of making it simpler. The best is to have an AI model that is trained to oversee and supervise what Claude is doing. When you have another AI model that's providing supervision and is not just saying, did you do this incredibly complicated task correctly? Did you become a faculty member and get tenure? That will take six or seven years. Is that an end-to-end task where at the end you either get tenure or not? That's ridiculous. That's very inefficient. But instead, can provide more detailed supervision that says you're doing this well, you're doing this poorly. I think that as we're able to use AI more in that kind of way, we'll probably be able to make training for very long horizon tasks more efficient and I think we're already doing this to some extent.

Great question. I would say a mix. Obviously we're building the tasks as much as possible using AI to generate tasks with code. We also ask humans to create tasks. It's basically some mixture of those things. I think that as AI gets better and better, hopefully we're able to leverage AI more and more, but of course the frontier of the difficulty of these tasks also increases. I think humans are still going to be involved.

Thanks.