I personally was super freaked out by it. I looked at it and I said, "My god, you just scale this thing up another factor of a million and it'll be better than humans at everything." This is the defining, you know, economic and scientific moment of the modern age. We just don't know it yet. This is going to determine the fate of nations and economies and the future of humanity. Like, you know, you peer into the crystal ball, you see the whole future unfurled.
I'm Walter Isaacson. Welcome to The Builders, my new series of interviews featuring the most consequential minds shaping our future. Today I'm speaking with Dario and Daniela Amodei. I used to think that some of the concerns about AI were overblown. But then when certain new models escape the sandbox, I realize that the people building these systems have got to make sure that they remain safe and that they remain aligned with human values. What I want to know about Anthropic is, are they sincere? And when they face competition, like when OpenAI says, "Well, Anthropic won't do that, but we will," do they back down or will they stand true to their principles? Hey, welcome to The Builders, you all.
Thank you so much for having us.
You know, Steve Jobs, whenever he launched a product, always had that street sign of the arts and the sciences, or, you know, the humanities and technology, standing at that intersection. And when you were growing up, Daniela, you know, you were a flautist, you loved the arts, and Dario, you were always a scientist. Did that help you gel, and do you think that combination is important?
Yeah, I think that combination is a lot of what's behind the company, right? One theory I have is that running a company is too big a task for any one person. The range of skills that it involves, it is not possible for any single person to have. So you kind of have to assemble a full complement of skills across the top, like 10 or 15 people in the company or something like that. And every very successful company splits it up in different ways and solves the problem in different ways. And I think with the sciences and the humanities thing, you know, Daniela and I kind of embody that and split that.
Yeah. And I would say, you know, it's interesting, like there was an offsite we did in maybe 2023 where like we sort of were talking about just like, what does it feel like to work at Anthropic? Like how do we kind of embody what our culture is? And we actually said, you know, it's an unusual blend of science and humanism. And it was only later that I'm like, that also is how you would describe me and Dario.
Well, when you were kids, did you know you wanted to work together?
Yes. Yes. Yes.
I don't think either of us imagined that we would start a company. That's not really how either of us thought of our lives or about what we wanted to do. I had always wanted to be a scientist. Daniela can speak to her aspirations. But, you know, I actually found companies and business to be of very little interest to me.
Yeah. I would say Dario and I were very unusual in that, like again from a really early age, we just like genuinely really liked each other. We were always just like best friends like since we were really little. And I agree with Dario. I don't think like starting a company was like ever on our bingo card. Like we were not the type...
Neither of us were particularly like company oriented. I was always really interested in like, I think I would call myself like a classic do-gooder. So I like worked in international development. I felt really strongly about just like helping people everywhere in the world have like equal access and opportunity. And I became a vegetarian when I was like quite young. It's like the only thing that I did first. Dario did everything first because he's older, but I became a vegetarian and then he became a vegetarian.
Yeah, I think what, I was like three years later or something.
Oh, not even. I think it was maybe just... Yeah. Yeah. You got on board real quick.
It seemed convincing to me.
You know, people sometimes ask me about my books, how do you become a great leader, very creative? And you said something really interesting, Dario, which is it's not just being a great leader, it's creating a great team. Tell me how you and Daniela divide up the roles and then how you fill it in with others.
Yeah. So, you know, I would say the division that we've found that has been most useful between us, and then I'll talk about the rest of the team, is I think I can function best when I think about the big picture strategy and kind of the research side of things. Like I'm best at kind of laying out a vision, seeing what comes next. I have found that when I'm freed up to focus entirely on that, that is the thing that has the most value for the company. And then running the company day-to-day, recruiting the people, just making sure that the machine doesn't, you know, the wheels don't fall off the machine while I'm trying to figure out what comes next. You know, Daniela does this, and I think this division of labor allows both things to be done to a much higher standard than either of them would be done if we were distracted by having to do the other thing.
Our chief people officer, who knows both of us really well, is in, you know, a lot of conversations with us. The way she put it is that Dario makes us ambitious and I make us successful. And I thought that was a really good encapsulation, which is I think Anthropic could be like a good successful business but not change the world if we didn't have Dario. And I think with Dario we would have the perfect vision but our ability to execute would just be more complicated. And so I think there's something special about A, the fact that we know each other really well. B, it's extremely convenient that our skills are like 100% complementary, or maybe not 100%, but I would say close to 100% complementary in this fashion. This is not even to mention the fact that we have five co-founders along with the two of us who also own like incredibly important parts of the business, the culture, like the organization. And again, I think it's hard for me to imagine how you would do this with like just the two of us or like one person. I just...
I think it's an incredible strength. Yeah. Because the fact that we have seven co-founders, they all have equal stakes in the company. So that they all carry the vision. We have inherent trust in all of them. Many people told us when we started Anthropic, seven co-founders, haha, that's going to be a disaster. There's this belief that, you know, you need one kind of singular source of leadership. In some ways you do, but we've actually managed to make it work. All seven of the co-founders are still at the company, which again is not something that's usually true when you have that many co-founders.
So go back to growing up and you have these extraordinary parents and they help teach you right from wrong. How do they do that?
I don't know that it's something that was ever, you know, sometimes I hear people say like, you know, we covered, you know, different forms of philosophical ethics at the dinner table. That was not how it was done. My mom does have a philosophy degree, but she didn't subject us to talking about Kant or anything like that. I think it was much more like inherent in terms of values, like just the belief that people everywhere in the world are deserving of opportunity and dignity and rights and respect. I think there was something special about the feeling that like we could be whoever we wanted to be as long as there was this kind of faith and belief that like, hey, that's part of what it means to be a successful human is to be a good person and to be kind to other people.
Yeah. It felt implicit. It felt very implicit. You know, there wasn't any kind of explicit like, you know, this is what you should do or this is how you live in the world.
Well, one reason I asked, as you might suspect, is now you've got a really important kid named Claude. And to what extent does that way that you grew up help inform you about how you teach Claude to know right from wrong?
Yeah. You know, we put out a constitution for Claude, I think it was a few months ago, and I think there are some interesting lessons in it, which we've learned from several years of, you know, basically trying to train Claude in successive versions. I think early on we had been imagining that there would be various rules that, you know, you would tell Claude, you know, don't help someone, you know, commit a crime or don't make biological weapons or whatever. But it turns out that when you make this kind of discrete list of rules, you know, the shape of what you're trying to teach the model — I'm speaking a little bit metaphorically here, but it's kind of grounded in what we found technically — you end up with this kind of spiky shape where you're trying to teach the model like 170 different rules that are kind of arbitrary. We found it's better to teach the model some basic principles, like what it should prioritize, how it should reason about what the right thing to do is, which, you know, of course I think that's the right way to teach people as well, right? Like, you know, if I just give someone a list of rules, thou shalt do this, thou shalt not do this, I don't think that produces as robust a morality as I say, well, this is how you should think about things, you should learn how to think for yourself. And these are the principles on which you should think, right? Principles of open inquiry, principles of care for other people, principles of fairness and justice, and then everything is derived from that. And that is frankly how Claude learns. We find that to be surprisingly effective because that allows Claude to generalize to new situations that it might never have imagined.
Daniela, I can't help myself when you said your mom was an expert on Kant. Did you actually wrestle with philosophy and moral philosophy like that?
You know what's interesting is the woman who is responsible for training Claude's character has a PhD in philosophy. It's unusual, right? You would think, oh, the person that's training, you know, the sort of character of these models must be a computer scientist or somebody who's studied ML or AI for, you know, many decades. Amanda's technical and she like technically trains the model, but I think it is very felt in Claude's character when you communicate with it that there's this kind of experience, something that's very, you know, wise, kind of the best version of the sort of better angel of your nature trying to answer your questions. But there are like, if you kind of poke Claude in these like difficult areas where maybe different philosophical traditions might conflict, I think that's where you especially feel Amanda because you'll sort of see like, oh, well, you know, you might want to consider things from like this point of view, but you could also consider it from this point of view. And I do think it's important because as the world begins to rely on these models more and more, you don't want to be locked into sort of one philosophical tradition because ethics and morality are quite complicated and different people and different cultures might have different answers about how they would like you to behave.
Dario, you were there at one of the seminal moments of modern times, which is when GPT-1 and GPT-2 suddenly burst on the scene, perhaps even surprising you. You were head of research at OpenAI. What was that moment like?
Yeah. So, you know, I think GPT-2 was maybe the first real empirical test of the scaling laws, right? So, I wasn't very involved in GPT-1, but I was very involved in the idea that it should be scaled up, that we should acquire new sources of data, more computation, and just really should run a larger scale, more expensive, more ambitious experiment on the idea that if we ran these language models at a larger scale, they would be better at language, they would be more coherent. So, you know, it's almost like one of these moments in scientific history where you propose a definitive experiment to just make a very strange prediction, and then if the prediction turns out to be correct, you're like, god, we're really on to something. So I see GPT-2 as a kind of test, the first real test of the scaling laws hypothesis. And what I found super interesting was the enormous divergence in reactions to it.
Were you kind of surprised?
I personally was super freaked out by it. Like, I looked at it and I said, "My god, you just scale this thing up another factor of a million and it'll be better than humans at everything." Like, this is the defining, you know, economic and scientific moment of the modern age. We just don't know it yet, right? This is going to determine the fate of nations and economies and the future of humanity. You know, you peer into the crystal ball, you see the whole future unfurled. And, you know, it really scared me. I said, "Oh my god, what are the implications of that? Is this the right place to be developing it? What's going to happen next? What do we do with this? Who do we tell about this?" And maybe 10% of the people around me had exactly the same reaction, and it was often the smartest ones. 90% of the people around me, because GPT-2's capabilities in any particular area were not that impressive, right? That it could translate languages and summarize a document and write a few lines of code here and there. It was the broadness and the generality. But people would look at it and they would say, in any particular area, this doesn't seem so smart. And so many people just shrugged and they just dismissed it.
But we just announced that, you know, we're going to be developing drugs for, you know, neglected rare diseases. And you get the usual comments from kind of the jaded, you know, biotech world people: "Ah, tech people have done this before. The models aren't, you know, here are all the limitations of the models." And I just look at it and I say, it's the same thing over and over again. The limitations of the models, you know, now, six months from now, the things that seemed hard, they're going to be easy. 12 months from now, the things that seemed impossible, the models are going to be doing them regularly. And so it's everything we saw with GPT-2, we're seeing it over and over again in wider and wider swaths of the world. And the world is gradually finally being convinced.
I think you were quite polite about it, but I mean, I remember at the time there were some like very well-respected, you know, computer scientists and like AI and ML leaders who were like basically like bullied some of the researchers who worked on it on Twitter and were like, "This is stupid, like we don't know why you guys are, you know, even talking, this model is so incapable." And it was like you were sort of a quack if you believed this was like a really significant moment.
And that's all been memory-holed. Like all the researchers who like, you know, were like, "Oh, this is pseudoscience. It doesn't make sense." Like, all of them are applying for jobs at like the LLM companies. And like, you know, we never say it out loud to any particular one, but like, you know, we look at them and they're like, we remember what you were saying in 2019. You have no credibility on this matter.
So, Daniela, you were vice president of safety back then at OpenAI. And when you hit GPT-2, I assume you're like Dario, somebody who's almost freaked out about it. Did it freak you out about safety too?
There wasn't something about GPT-2, the model itself, that made us, you know, concerned, right? We didn't look at that model and say, "Wow, we think this model has like cybersecurity implications or something," right? The model itself was in some ways kind of laughably wrong about a lot of things, right? It was still a very, I would say, harmless model. But I think the thing that Dario mentioned that I also felt was a sense of like, okay, the difference between GPT-1 and GPT-2, just like extrapolate that out, right? Like there was something about seeing it's almost like if you'd never seen a child before and you'd only seen adults and you were kind of like, I don't know what this little thing is that's walking around. Like I don't know if I should, you know, imagine that it's going to be, you know, capable one day. And then suddenly it starts to walk. You're like, "Wow, huh, I bet this thing is going to be capable of doing a lot of the things that a grown-up could do one day." Like maybe this is where this is going.
