Daniela Amodei, Co-Founder and President of Anthropic: Building AI the Right Way

Thank you so much for having me.

Oh my god. First of all, so nice of you to use the word plan in there. I don't know that I necessarily would have described it that way at any point in time, but I really think, you know, this is sort of a story that I hear from a lot of people who either wind up, you know, founding a company or just doing something a little bit unexpected or unconventional with their life. It was following what was most interesting to me at the time and the intersection of what was I good at, what was I interested in, and what was going to have a big impact in the world. So for me, I think coming out of college that looked like, by the way, I graduated in 2009, which was like not the most fun year to be a graduate. You were like, I have a literature degree and no skills. Who will hire me? But at the time, I felt this very strong pull towards wanting to make the world better. I think that was always a sort of defining feature of both me and Dario from a young age. And where that started out for me was in international development working in global health. And I think my desire there was really to figure out like how do we build a world that is, you know, fair where everybody has access to basic things like food and water and medicine. And even though that's, you know, not what I directly work on now, I think that early grounding sort of gave me this foundation for thinking about how do you do good in the world, right? How do you build something that is of consequence and has a real purpose behind where you're spending, you know, 50 or 60 of your hours per week, right? What you're working on. And so it was kind of a winding journey from there. I worked on Capitol Hill after that. I worked on a campaign. And eventually I ended up coming back to Silicon Valley. I am originally from San Francisco and started working at Stripe, which was this tiny company that at the time like nobody had heard of. My friends on Capitol Hill were like, 'You're leaving to go do what? Payments.' Like now it looks like a great decision, but at the time it's about 40 people. And then from there things really just snowballed into working at OpenAI and then co-founding Anthropic.

So, you know, it's really interesting. I think in some ways I really think of myself as a generalist, right? That if you were sort of look through my background, you would be like, what is this lady actually good at, right? She doesn't have a law degree. She's not a computer scientist. But I think this concept that the ability to be curious and learn across a lot of disciplines, to have a strong foundation of wanting to have impact regardless of the area that you're working on, I think that's an underrated quality. And I see it a lot in the people that we hire at Anthropic and the really talented people that I've worked with in the technology industry more broadly. People who are curious and smart and they want to learn and they want to be helpful. That is like the first of all the description of like every role except for like engineer at a startup. But I think for me it was always very interest and impact driven. So, I was like, 'Okay, there's something that feels really wrong to me about the fact that, you know, I was born in America. I had access to all of the sort of basic things that we take for granted in life.' Some people around the world just weren't, they just, that's just not where they were born. That's not what they were born into. How do I make that more fair? Like, how do we make that better as a broader community of the world? And from there, I was like, I'm just not having the level of impact that I want to. I need some skills. And so I went and I worked on a campaign and I was like, 'Wow, a small number of people that work really hard who are young and driven can really change the world.' Not that surprising that eventually that led me to Silicon Valley where you're like, 'You can do that.' But it turns out that at a startup you have a lot more money and it's a lot easier going than the kind of 80 hours a week of working on a campaign. But I think those kind of core qualities around, you know, really following your passion because you just want to do more when you care about the thing that you're working on either intellectually or from an implicit sense of meaning.

So I think I was well trained on two dimensions. I think the first is I had already spent almost six years at Stripe and so I had worked with a lot of engineers. Obviously research and engineering are different but I think there was some overlap and some grounding. The second is that, you know, I grew up with a very talented technology-oriented physicist who's my sibling and obviously my co-founder at Anthropic among five others who are also all engineers or researchers. But I think the number one thing I would say is both of those experiences just instilled in me this sense to just not be afraid of technology, right? It was ultimately it's a set of skills that are really highly prized I think but is something that anybody can learn and the sort of basics behind it. I think the terminology and the jargon can feel overwhelming at first but if you just ask enough questions and if you have people who are kind enough to be patient with you which I had in my life I was very lucky for that, I just kept asking questions until I felt like I could understand it. And I think, you know, the second part of it was also just like knowing my lane and their lane. So, there were a lot of things that the researchers, I was like, I probably couldn't have trained GPT, I certainly couldn't have trained GPT-2 or GPT-3, but I brought things to the table that they didn't know how to do as well. And so, I really think understanding what your comparative advantage is and knowing how you fit into the broader ecosystem. That takes a lot of skills that are kind of interpersonal. Curiosity I think is a very inherent skill, but one that you can learn and train. And I think all of those sort of put together really put me in this position to be able to be successful in that type of role.

So, there were seven of us that originally left, me and Dario and our five co-founders, and then a number of people that kind of came over shortly thereafter. And I think for us it really came down to just the focus on what it was we really wanted the ultimate impact of the technology to be. And I think in our own, you know, very different ways. All seven of us are people that have a lot of integrity. We're people who care a lot about the impact of what it is that we build. And I think eventually it felt like it was just easier for us to sort of create the type of vision that we saw outside of the company we were in and in a new company. And I frequently say because I know it'll shock you, I'm actually asked this question not infrequently. And we really, I think, were running towards something versus running away from something. Like we had this vision in our heads of wanting to create an organization where the values that mattered to us around safety, around responsibility, were the kind of forefront of what we were doing. That's why we chose to incorporate as a public benefit corporation. That took a while to figure out what was the right kind of form factor to express. Look, we are going to be a commercial entity. We think there's going to be a lot of economic value that's going to be created by artificial intelligence, but it's really important to us that we do this the right way. And I think that was something that kind of united the seven of us. We had all worked on a combination of both capabilities and safety and policy work when we were at OpenAI and it felt like it was just easier to kind of create this structure in a new form.

So, you know, I think of myself as just extremely lucky. Like the seven of us, I think, are a very special group on a couple of dimensions. I think the first thing I would say is like the interpersonal relationships, I know this is going to sound not that surprising, matter a lot more than you think. Like how do you guys have conflict together? So for example, Dario and I have been fighting and getting over it for almost 40 years because he's my brother and like I used to steal his toys, right? So like we know how to work through conflict together and like there's no question like we will love each other at the end of that no matter what. With our co-founders, it's like I've known Jared for like 15 years. I've known Chris for 15 years. Tom and Sam were roommates. Jared and Sam worked together at Stanford when they were in their PhD program. So, there was this kind of long history. Both Dario and I actually managed all of the other co-founders and like I think they had either reported to one of us or both of us. I think the majority had reported to both of us at OpenAI. So, we already had this kind of existing structure and framework of like we knew how to give each other feedback. We knew how to work together. We understood like who we were as people. And I think the other really important thing is just making sure you have a very strong sense of what it is you're trying to do and that that picture is the same. Like if you locked yourself and your co-founder in another room and you wrote down or drew a picture of like what it is you're trying to build, you're not going to walk out and like one has drawn a unicorn and the other has drawn a platypus, right? Like that's the type of situation where you think you're doing the same thing, but I think it just doesn't end well. And I think for us, this kind of vision of what it is we wanted to build, in some ways we were lucky because we had been in an environment where we're like, 'Ooh, it's not quite this. We want to do this other thing,' but we were like in the same zone already. So, we're sort of pre-selected for that level of interest. But I think really being able to like pressure test to the degree that you can, like instead of like starting a company together, like go on vacation together. Just like see how that goes, share a room with them. Be like, how did that go? And if you're like, 'Man, all I want to do is spend more time with you.' Great. If you're like, 'Really, I'm going to need a vacation to recover from my vacation.' You just might, it might be the wrong choice.

Yeah, this is a great question and I think this is a term that's gotten like almost a little bit overloaded in the past few years because it's sort of a catch-all that I think has been reached for in a lot of contexts. But I think to us the highest level of framing is just taking a form of like radical responsibility for the technology that we're developing and the analog that we often use or that we often point to is social media companies which like it's so in vogue to like crap all over them in public now. So I'm going to do that. But basically, you know, like if you imagine going back in time, I actually think developers who created these technology companies, they weren't like, 'I'm setting out to cause a pandemic of eating disorders for teenage girls,' right? That was not their intention, but they were like, 'What are the metrics that I'm trying to optimize for, right? I'm trying to build a company. I would like to see rapid scale growth. Like, let's just build towards that.' And there wasn't this sort of, at the time it just wasn't necessary because we'd never seen something on the level of scale that we have seen today, how quickly these companies grew, how many people adopt the technology so quickly. But if you could imagine, you could go back in time and say like, 'Wow, you're starting Facebook or Instagram or Snapchat or Twitter and you're like, what if I really tried to think through all of the ways that this could go wrong? Like what if I could think about what all the unintended externalities are and like really just kind of tried in advance to like prevent some of those from happening.' And it's a little bit unfair because in AI we've had this whole generation of technologies before us where they've like gotten to mess up and we've gotten to be like, 'Haha, we're not going to do that thing again.' But that is a huge privilege, right? We're able to say, 'Okay, you guys made this mistake. We are not going to make that mistake this time, right? We're going to be careful and we're going to say how do we make sure to think about the other things that might not go wrong because we understand the technology better.' How can we imagine a world where like everything goes right but also a world where everything goes wrong and I think for us safety means like all of the big stuff. So preventing chemical and biological weapons from being developed using our technology which by the way they could have the potential to do. But also like cyber, right, cyber warfare. We've been in the news a lot lately about our decision to not, you know, release our Mythos class model because of the potential for cyber warfare. There's also a lot of work that happens around things like user wellness, child safety, a lot of misinformation, election integrity work. This is not something new that we invented. We've been able to stand on the shoulders of previous safety and security teams who worked on this at some of the other most consequential technology companies in history and say how do we learn from you, right, how do we do this better.

We're asked this question a lot too and I think in general the two don't come into conflict as much as you would expect. So what I would say is most businesses in particular, which is by the way the majority of our revenue comes from businesses, are not looking to have models that are unsafe, right? They're not like, 'Wow, we would love for Claude to hallucinate more or it would be great if Claude just produced like harmful outputs when you're asking it a question.' And so I think until like pretty recently these things were just like 100% in alignment, right? You were like it's actually really good for business to be safe because businesses are correctly, they're risk-averse, right? They're like we don't want AI technologies that are going to be super unpredictable or unreliable. That said, I think we're now entering an era where the capabilities of the models are developing so rapidly that the tension is about time. So it's not necessarily the case that the models can't do amazing things. It's just we don't fully understand at this stage and I think it will be more the case going forward like how serious are the risks? What are all of the risks and how do we help mitigate them? That sometimes means that we take slightly unusual actions like we did with Project Glass Wing and say this new class of model, it would be great if we could just release this to all of our customers. They all would love to use it, but we're just not confident enough yet. Like, we need a little more time to do some of the work to make the models safer to use. But that's uncomfortable, right? It's uncomfortable to say that to your customers, right? They're like, 'Look, we all believe in cyber defense, but like I really want access to that model.' And I think this is the place where we just come back to the mission, right? We say, 'Okay, we understand that desire. We want to get this technology to you as quickly as possible, but it is irresponsible of us to release it until we are confident that all of the patching that needs to be done has been done.'

I think this is actually a very complicated question. My sense is that this probably will not surprise people here. AI is going to change what types of jobs are available and what types of jobs people do. So today I think there are jobs that exist that did not exist five years ago because of AI. I also think there will be some jobs that will not exist in the future because of AI. But what we have seen so far today is that according to our economic index where we study like how are people actually using artificial intelligence technology right now it mostly looks like complementary skills, right? So you have artificial intelligence as an enabler of work, you don't have it as a replacer of work, except in a very vanishingly small number of cases which is mostly customer service. Sorry if you have to email Comcast, it will never be a human again, probably. But I don't know that that was actually different five years ago. So, I think in reality what I expect is like there will be a number of types of work that will feel a lot like they rhyme with a job that exists today, but they're not necessarily the same as a job that exists today. And I think we just don't know the shape of what all of that is, right? Today, I think the thing that's most talked about is coding, right? Software developers. I always, you know, in business meetings, right, people will say to me like we're talking shop about Claude and then like two-thirds of the way through the conversation like a CEO will kind of conspiratorially lean across the table and say, 'You know, my daughter is a sophomore at Stanford, like what should she study, right? Like she was going to be a CS major. Should she not major in computer science?' And I think, you know, the truth is we don't know. But my guess is software developers will still exist, but like they won't write as much code, right? Like a lot of what software developers do is much bigger than just hands-on keyboard. They're talking to product managers. They're working closely with customers. And I think the percentage of that work is going to expand. And I think the sort of things that can be more easily done by AI that will contract, but my sense is that is going to create a very different scope of what's possible.

So I think there's a few things here. I think the first one is we need to start and lead from a place of like humility and not knowing the answers but doing the research. And I think at Anthropic something that we've always aimed to do is be as kind of radically transparent about all of this as we possibly can be. Right? We've always said, look, we don't have all the answers. We do need to study this so that we're able to tell people what we see coming. And I think, you know, sometimes fairly people can say like, 'Wow, you guys are just, this is like a lot of negativity, right? You're like, here's what we think might happen in the future.' But I think it's more important that we start the conversation sooner because we don't want people to be caught off guard, right? We're publishing the economic index to say here's how people are using artificial intelligence today because we want people to have an understanding of like where do we think this is going. So I think that's step one is we have to actually all agree on what reality is to the degree that's possible. I think step two is, you know, we have to be creative and experimental at many different layers. So, how do we have artificial intelligence really be something that is a grounding and a unifying force for people outside of just like, 'Oh, I'm using it at my kind of job,' right? Which is really important. But I think in some ways we need to sort of be rethinking the paradigm of this connection between work and meaning and like social life, like all of these things I think are going to look very different in the future and we need to practice. And then I think the third one which is really outside of the realm of what a technology company can do alone is this is going to become a social and political issue. People are going to care if it feels like their jobs are being displaced by AI. People already care about this, right? It comes up in polling. People are like, 'I have anxiety about what artificial intelligence is going to mean for my future, for my kids' future,' right? That CEO that's leaning across the table from me, it's not just them. And so I think there is a broader discussion that needs to happen at many different levels of government with civil society with universities by the way to say like what does this mean, what is the type of world that we want to be able to build where artificial intelligence is capable of doing many of the things that humans do today.

You know, I think this is a really really great call out. It's so interesting because it feels like, you know, certainly to us at Anthropic and I'm sure at Stanford too, it's like the only thing people want to talk about is AI. Certainly, the only thing people want to talk to me about is AI, but that's probably a me problem. But I think that feeling, you're totally right that even in other parts of America, it's not something that people are comfortable with yet on the whole and it's not even something that people like necessarily know how to use with very high fluency, right? So, you read these like really astounding, super impressive numbers about how many people are using AI tools. But there's a demographic component to that, right? So it's generally people who are college educated, not uniquely. It's more men than women. There's racial demographics involved. There's wealth demographics involved. And if you look around the world, it's really not equally distributed. And so I think what's interesting is that you pair that with some other data that we've collected, which is that people in developing countries are much more optimistic about AI than people in higher income countries. So the global south is almost universally like, 'Wow, this is a huge opportunity for us, right? This is the moment where perhaps we could have an equalizing force that will make things more fair.' But I think in the US and in Europe and in parts of Asia, people have a lot more anxiety. They're like, 'I like things the way they are. I don't want AI to come in and disrupt that. That doesn't sound as good to me.' What do we do with this information? I have no idea. But I think it's really interesting to say like there are different questions of access and adoption around the technology and I think we are actually still very early in the game and that's the

Thing that I think can be missed in Silicon Valley in our bubble that we're like it's already everybody who's a software engineer is like I'm using cloud code I'm using codeex that is not the vast majority of developers in the world at large. And so I think the race is still like we're at like the gun just went off to start the race. And I think there's a lot of opportunity to still positively shape how this technology is going to be used and developed, what access looks like, and just what the values that are baked into it are going to ultimately be.

Yeah. We did this very large qualitative survey at Anthropic. I think it's the largest qualitative study ever done that we know about. We talked to 81,000 people about their use of artificial intelligence. And so some of them were cloud users, some of them were users of other AI tools. And what was interesting is, you know, people have a lot of different feelings about AI, right? And again, you know, cuts differently depending on where you are and what you do. But some people are like, I've never been able, you know, it enabled me to do things that I never thought I could do, right? I think for myself as an example, like I didn't think I could build a website, right? And now using Cloud, I'm like, 'Oh man, that's so easy.' Like I just click a couple buttons and Cloud like built a website for me. What that would have taken me probably a year if I tried to do it by myself and it would not have been a very good website. There's some people though who express a feeling of there's not like a specific term for it, but I think there might be or there might be in another language one day. Like I don't engage my brain because I don't have to.

So, it's not the same feeling as like scrolling on your phone, but it's like I could have reached for this idea. I could have thought through it, but it was so much easier to not do it and to just trust what the AI tool was giving me. And I think this is the source I actually believe of a lot of the anxiety around AI is humans like to I think have an inherent desire to learn, to be curious, to want to expand the aperture of things that they know about. And AI in some ways enables that, but if used incorrectly can sort of disable that, right? It's like I've done this sometimes, right? And I was like, 'Oh, I could like look this up and figure it out myself, but I'll just ask an AI tool and then I'll blindly trust that what it says is correct.' It's not always correct, by the way. Sometimes Claude is wrong. Heretical to say, but a fact. And I think the anxiety there is around how do we actually set some guard rails in place so that it's not impossible to do that, but you actually have to really be trying to do it, right? Like I think some of the work that we do with universities is maybe an interesting microcosm for this. We have this concept of learning mode. Maybe some of you even use it. I don't know if we're at the GSP yet, but faculty and professors and students like one version of this is like you put your homework in chat GPT. I'm going to use that one instead. And you're like, 'Haha, it just gave me the answer.' There's a word for that. It's called cheating, right? And you're like, 'That was great.' There's another version where you use Claude in learning mode and you're like, 'I'm stuck, right? I'm trying to write this essay and I like there's something about the format that doesn't feel right to me.' And Claude is this sort of patient tutor. It's almost like you have an individualized professor who like knows you and understands like what you most want to learn and why this class is important to you. It's like, 'Let me help get you unstuck, right? Do you want to go back and read this section together? Could we talk through this?' I think that's the version where like these tools can make you smarter. They can make you expand the set of things that you think you can learn. And then I think there's the version that's just turn your brain off. And I'm hoping that as an industry we're going to choose to do the second versus the first.

So I have my own views on this which is I think like we talked about a lot of specific like task oriented things right like I'm a financial analyst or I'm a developer or I'm a copy editor. Those jobs are going to change a lot and a lot of that work I think will be able to be done by AI tools. But I think ultimately there's this very real phenomenon which is that humans like to be with other humans. We like to spend time together with each other. We like to learn from each other. We like to be creative. We like to spend time understanding the other person. And we're social creatures, right? And I think I'm imagining that in a world where AI is able to do a lot more of the productive day-to-day work that we do, those skills are going to become a lot more important and much more prized because ultimately like if you're in a work environment and you're like, well, I could just ask Claude to like write a bunch of code for me. You are going to choose to talk to the developer that's going to explain to you like why something broke or why we chose to build, you know, a tool the way that we did. And I think sort of expanding this outside of the realm of just a technology industry, the example I often use is in medicine, right? Today, we hire doctors who are really good diagnosticians. We're like, 'Hey, can you tell me what's wrong with me? I don't feel well.' And you're like basically paying this doctor to say like, 'Here's a set of things that could be wrong with you. This is the one that's the most likely. Let me run some tests.' Guess what? AI is going to get really good at doing that. But the thing that an AI tool can't do is actually like look at you and examine you and also like help understand how you're feeling and help you feel better. Right? There's a reasonable body of medical literature that indicates that people have a good relationship with their doctor. They just like their doctor have better clinical outcomes than people who do not like their doctor. Like that's really hard to explain, but like what's probably going on? Probably the doctor like tries a little bit harder to understand what's wrong with you. Maybe they run a set of tests that were unexpected. And I think those skills, right, that bedside manner is going to be like five times more important in a world where you're not trying to cram that into one of seven things that you're looking for to make a doctor qualified to treat you.