The AI Tsunami is Here & Society Isn't Ready | Dario Amodei x Nikhil Kamath | People by WTF

Many other companies come here as themselves a consumer company and they see India as a market, right? A place to obtain consumers. We actually see things a little bit differently.

Yeah, so I was actually originally a biologist. I did my undergrad in physics, my PhD in biophysics, and I wanted to understand biological systems so that I could cure disease. The thing I noticed about studying biology was its incredible complexity. For example, if you look at the protein mass spec work that I did, trying to find protein biomarkers, it's just really incredible how much complexity there is. You have a given protein, the RNA gets spliced in a whole bunch of different ways depending on where it is in the cell, then it gets post-translationally modified, phosphorylated, complexed with a whole bunch of other proteins, and I was starting to despair that it was too complicated for humans to understand. And then as I was doing this work on biology, I noticed a lot of the early work around AlexNet, which is one of the first neural nets, almost 15 years ago now. And I said wow, AI is actually starting to work. It has some things in common with how the human brain works but has the potential to be larger and scale better and learn tasks like biology. Maybe this is ultimately going to be the solution to solving our problems of biology. So I went to work with Andrew Ng at Baidu. Then I was at Google for a year. Then I joined OpenAI a few months after it started and basically led all of research there for several years. But then eventually, myself and a few other of the employees just kind of had our own vision for how we wanted to make AI and what we wanted the company to stand for. And so we went off and founded Anthropic.

Yeah. You know, I would say, my conviction and the conviction of my co-founders when we founded Anthropic were two of them. And I think one we were starting to convince OpenAI of, the other I didn't feel that we were convincing of. So the first was the conviction in the scaling laws and the idea that if you scale up models, you give them more data, more compute, again there are a few modifications like RL but not really very much, it's pretty close to pure scaling, you find incredible increases in performance. And I was finding that in like 2019 with GPT-2 when we just first saw the first glimmers of the scaling laws. And of course there were a lot of folks inside and outside who didn't believe it at all and we really made the case to leadership like this is important, this is going to be a big deal and I think they were kind of starting to believe us and ultimately went in that direction. And there was a second conviction I had which is look, if these models are going to be kind of general cognitive agents, general cognitive tools that match the capability of the human brain, we better get this right. The economic implications are going to be enormous. The geopolitical implications are going to be enormous. The safety implications are going to be enormous. It's going to transform how the world works. And so we need to do it in the right way. And I think despite a lot of kind of language verbiage about doing it in the right way, I was for a variety of reasons just not convinced that at the institution that I was at there was a real and serious conviction to do it in the right way. And so my view is always, don't argue with someone else's vision. Don't try to get someone to do things the way you want to. If you have a strong vision and you share that vision with a few other people, you should just go off and do your own thing and then you're responsible for your own mistakes. You don't have to answer for anyone else's. And maybe your vision works out, maybe it doesn't, but at least it's yours.

Well, that Yeah. Yeah. We succeeded.

It's like if you want a chemical reaction to produce oxygen or start a fire or something like that, you need different ingredients and if you don't have enough of one ingredient the reaction stops. But if you put ingredients together in proportion you get your explosion or your fire or whatever. And for AI those ingredients are data, compute, the size of the AI model. And so the scaling laws just tell you that if you put in the ingredients to the chemical reaction, the ingredients of data and model size, that what you get out is intelligence. Intelligence is the product of a chemical reaction.

Intelligence as measured by the ability to translate language or the ability to write code or the ability to answer questions correctly about a story. Basically any cognitive task we can think of, any task that exists in text or in images, any task that you can do on a computer.

Yeah, you know, I would say well I mean for example 5 years ago a computer you could not ask a computer a question and have it write a one-page essay on that question. You could not ask a computer to implement a feature in code and have it implement that feature in code. None of those things were possible. You could not generate an image. You could not generate a video. You could not analyze a video. You know, I could get one of those videos of like a monkey juggling or something and say what's going on in this video? How many times did the ball change hands? And right now you could get Claude or another AI model to give you an answer on that. And 5 years ago, none of those things were possible.

Well, you know what I would say is five years ago, you could Google and there might be a website that would tell you a little bit about this, right? But you're just looking up some text that exists on the web, right? Maybe it's not about how to get a monkey to juggle. Maybe it's about how to get a seal to juggle. It's not quite exactly the same thing because maybe exactly the same thing doesn't exist. But as we see when people use these models, you can ask and you can actually get an intelligent response. You can ask a specific question and have the model write one page about it or you can give it a hypothetical. What if I had the monkey juggle clubs instead of balls or what if I did this thing and that information doesn't exist anywhere, whereas the model is able to kind of think for itself and come up with an answer on its own. So it's something totally new. It's not just matching some of the text that exists on the internet.

You know, I was originally an academic and I thought that I might become a professor. I got my PhD. I went all the way to being a postdoc at Stanford Medical School and I was aiming to become a professor. So if I had become a professor I would have done that. But as I mentioned I got interested in AI and to work in AI required a lot of computational resources and that was mostly happening in industry. So that took me off the academic path and into industry and of course ultimately through several steps led me to start a company. But sometimes I think I'm still like a professor at heart.

Well, I mean, first I would say a couple of things. I think there's a lot of folks who are relevant in different ways, right? Even within industry, there's the different layers of the stack. There's the folks who make chips. There's the folks even earlier who make semiconductor manufacturing equipment. There's the folks who make models like us. And then there are other players who make models. There's the folks who make kind of applications after the models. And then there's a bunch of other folks who have a say. There's governments, there's civil society. So my hope isn't that there's just one tiny set of people that's relevant. I think we're trying to broaden the set of people who are relevant and turn it into a broader conversation. But I think at the same time your question is a fair one and one way I could interpret it is like there's a certain randomness to how a few people end up leading these companies that grow so fast and it seems like in the near future will power so much of the economy. And I've said openly publicly, not for the first time, that I'm at least somewhat uncomfortable with the amount of concentration of power that's happening here. I would say almost overnight, almost by accident. And we think about that in a bunch of ways. One is we have an unusual governance structure, something called the long-term benefit trust. It's a body that ultimately appoints the majority of the board members for Anthropic and is made up of financially disinterested individuals. So that's some check on what one single person is doing and then I think as always the government should play some role here. I've been an advocate of proactive although sensible regulation that doesn't slow down the technology, sensible regulation of the technology because I think the people should have a say, governments and the people who elect them should have a say in how this goes. So I actually think of a lot of what I'm trying to do as kind of trying to preserve a balance of power against the natural grain of this technology.

So, I would put it in the following way. The philosophy of Anthropic from the beginning has been that we try not to make too many promises and we try to keep the ones that we make. So we set ourselves up as a for-profit but public benefit corporation with this LTBT governance and we've maintained that. We've said that our goal is to stay on the frontier of the technology but to work on the safety and security aspects of the technology. We've pioneered the science of interpretability. We've pioneered the science of alignment. I don't know if you saw but we recently released a constitution for Claude, the ability to align models in line with the constitution. And we've done a bunch of policy advocacy and warning about risks, right? Warning about risks is not in our commercial interest, right? Like people can come up with conspiracy theories, but I will tell you saying that the models we build could be dangerous. Whatever people might say, that's not an effective marketing strategy and that's not the reason that we do it. And speaking up on when we disagree even with the US administration on policy matters, right? We've spoken up, right? We're willing to say, we disagree on this issue, like we've said that there should be regulation of AI when all the other companies and the administration have said there shouldn't be regulation of AI. And so that's both, the regulation of AI holds us back commercially as a company, even though I think it's the right thing to do. And it's difficult to go against the government and the other companies and say this. We're really sticking our neck out. So we've taken a number of actions that I see as really putting our money where our mouth is here. I can't speak for the other companies. It's again, it's quite possible that some people say these things and they don't really mean them, but I wouldn't look at what people say. I would look at what people do.

I don't agree with that at all. The regulation we've advocated for, for example, SB 53 in California, exempted everyone who makes under $500 million a year in revenue, right? SB 53 was a transparency law which basically requires companies to show the safety and security tests that they've run. And it exempts all companies under 500 million in revenue. So it really only applies to Anthropic and three or four other companies. So it only applies to the companies that have the resources and everything that we've advocated for here, not just SB 53, but all the proposals that we've made, the ones that we've made in the past and the ones that we plan to make in the future have this character. We're constraining ourselves and a very small number of additional companies. We're not, people who say that need to look at the actual content of what we're proposing because it doesn't match that idea at all.

Yeah, I actually wouldn't agree with the question. I don't think I've had a shift in perspective. I think the positive side and the negative side are always something that I've held in my head. And if you look at the history of the things that I've said, I mean, I've been talking about risks for a very long time. I've been talking about benefits for a very long time. It turns out that actually it takes me a while to write one of these essays. Both of these, it's taken me like, I spent for each one I spent about a year having a kind of vague vision of the essay in my head and like trying to write it but like not fully succeeding at writing it. And then in either case I had to be on vacation or somewhere where I could think where the day-to-day business of running the company didn't occupy me. And then I was finally able to kind of write the essay. So all of that is to say, I started thinking about what would be in Adolescence of Technology almost the instant I finished Machines of Loving Grace because I was like, oh, I want to inspire people with the good vision, but I also want to warn people with what can go wrong. And so it just took me a year to write it. But really, both visions were in my head. And I think they're both possible. They're two different visions of the future. And obviously, I want to get the Machines of Loving Grace one, right? I want to solve all the problems and have the positive vision, but it's not a shift in perspective. It's me just finding the time to write the light and then the dark.

You know, I would say overall I'm about where I was before. I've not gotten more positive nor more negative. There may be some places where I've gotten more optimistic or things have gone better than expected. There may be places where I'm more pessimistic and where things have gone worse than expected, but on average they sort of cancel each other out. I would say I feel very good about how things have gone with areas like interpretability. Interpretability is the science of seeing inside these neural nets, as a human would, as we would scan a human brain with an MRI or a neural probe. I've been amazed at what we've been able to find. We've been able to find neurons that correspond to very specific concepts, neural circuits that correspond to keep track of how to do rhymes in poetry. And so we're starting to understand what these models do, right? We just train them in this kind of emergent way as you would build a snowflake. But now we're starting to be able to look inside and understand them. I'm also very encouraged by some of the work on alignment and constitutions. Making sure that models behave in the way that we want and expect them to. I think that's going pretty well. I felt pretty positive about that. I think I felt maybe have been a bit disappointed or felt a bit more negative about some of the things that are more like in the public awareness and the actions of wider society. It is surprising to me that we are in my view so close to these models reaching the level of human intelligence and yet there doesn't seem to be a wider recognition in society of what's about to happen. It's as if this tsunami is coming at us and you know it's so close we can see it on the horizon and yet people are coming up with these explanations for oh it's not actually a tsunami, that's just a trick of the light. And I think along with that there hasn't been a public awareness of the risks and therefore our governments haven't acted to address the risk. There's even an ideology that we should just try to accelerate as fast as possible which I understand the benefits of the technology, I wrote Machines of Loving Grace. But I think there hasn't been an appropriate realization of the risk of the technology and there certainly hasn't been action. So I would say that the technical work on controlling the AI systems has gone maybe a little better than I expected and kind of the societal awareness has gone maybe a little worse than I expected. So I'm about where I was a few years ago.

We even have an optimized Claude for financial services. I don't know if you've tried that but we even have that.

Yeah. You know, one of my co-founders, he was writing this diary with his thoughts and his fears. And he fed it into Claude and he asked Claude to comment on it and Claude said, here are some other fears you might have that you haven't written down. And Claude ended up being mostly right about those. So it really gave this eerie sense of like the model knows you, the model knows you super well, that from a relatively small amount of information it can learn a lot about you and come to know you fairly well. And like most things with the technology, right, we talked about the Machines of Loving Grace and Adolescence of Technology, on one hand something that knows you really well can be a sort of angel on your shoulder that helps to guide your life and make you a better version of yourself and that's the version we can aim for. Of course something that knows you really well can use what it knows about you to exploit you or manipulate you on behalf of some agenda or sell your data to someone else. I mean this is one reason we just don't like the idea of using ads, right? Because you're not paying for the product, you're the product and in this case the product then would be this model that knows you super well and could use that in all kinds of nefarious ways. So we need to make sure we take the positive road here and not the negative road.

Yeah. I mean, you know, do you have to build mail and chat and Yeah. Yeah. I don't think we need to build all of those things. My thought would be, it's going to be a mixture of things we make ourselves and integrating into others, right? Like we can integrate Claude into Google Docs. We can integrate Claude into Google Sheets, like we have external connectors there, we're starting to do that with co-work, same for Microsoft Office, same for other tools. So I think we do whatever is easiest and fastest to do, we integrate into the existing tools. Now it might turn out at some point that the existing tools aren't enough and we have kind of a different vision, we might want to slice things differently, right? Maybe traditional email doesn't make sense or traditional spreadsheets don't make sense given what you can do in AI. So I don't exclude that we could chop up products in a different way, but we're happy to use the ecosystem that exists and work with anyone else, right? In many ways, we're a platform company. We allow many people to build on us, even though we sometimes also build things ourselves.

No, I don't really agree with that. I would again go back to the idea that you need to judge us by the actions that we take. I think the company has taken a number of actions over its time that show that it's really serious about these commitments. So back in 2022, we had an early version of Claude, Claude 1. This was before ChatGPT and we chose not to release this because we were worried that it would kick off an arms race and not give us enough time to build these systems safely, right? It was kind of a one-time overhang like we could see the power of the models. A couple other companies could see the power of the models and so we decided not to do that and that's public, that's well documented and then we waited until someone else did and then we're like okay the arms race has kicked off so now we can release our model but probably the world gained a few months. Now that was very commercially expensive, we probably ceded the lead on consumer AI because of that. We've advocated on chip policy in ways that have made some of the chip companies who are suppliers very angry at us. Voicing our disagreement with the administration on AI policy and AI regulation on some matters. Anyone who thinks we benefit from being the only ones to do that, it's really hard to come up with a picture where that's the case. You look at any one of these and okay, fine. But you put enough of them together and I don't know. I just ask you to judge us by our actions.

But I'm not saying AI is bad, right? We just talked about the two sides of it. My view isn't that AI is bad. That's not my view at all. My view is that the market will deliver a lot of really great things about AI, that it's good to build AI, but that there are dangers of AI and that we need to steer AI in the right direction. We're steering this car, we're steering it towards a good place, but also there are trees, there are potholes, and so what we need to do is we need to steer away from the trees and the potholes. We might need to occasionally slow down a bit probably temporarily in order to make sure that we steer in the right direction. That isn't like, the analogy wouldn't be a rich person saying capitalism is bad. It would be like if a rich person said capitalism is a force for good but the economy needs to be leavened. It needs to be moderated, right? We need to deal with problems like pollution, we need to deal with problems like inequality and then capitalism can be good. If we don't deal with those things, then capitalism might be bad. And so that is more analogous to the position that I have here.

So, this is one of these mysterious questions that we really don't have any kind of answer to. We don't know what human consciousness is and therefore we don't know if AIs have it.

So, I suspect that it's an emergent property of systems that are complicated enough that kind of reflect on their own decisions, that it's something that emerges from complex enough systems. And so I do think when our AI systems get advanced enough, I suspect they'll have something that resembles what we would call consciousness or moral significance. I do think it'll happen at some point. It may not be the same as human consciousness. It may be different in how it works because the modalities are different because the things it's learned are different. But having studied the brain and the way it's wired together, the models are different in some ways, but I don't think they're different in the fundamental ways that matter. So I am someone who does suspect that at some point, even if I don't think they are today, I suspect that at some point the models will, we would indeed say under most definitions that we would endorse that the models will be conscious.

I don't think consciousness necessarily needs to mean anything mystical. There's just some property of being aware of your own existence and feeling things, being able to take in a lot of information and reflect on that information, to feel a certain way and to notice yourself noticing something. I think that we can tell self-evidently from our own experience that those properties and experiences exist. What their basis is, whether it's entirely materialistic or there's something more mystical going on, I think is obviously very hard to know and ultimately not relevant to these questions. What does seem relevant to me is that because we can observe our own experience, these are properties of human brains. I suspect that these models we are building, as they get more sophisticated, are becoming enough like human brains that they will have some of the same properties. That is my guess as to what will happen. We've taken various interventions with the models. We've given the models what we call an 'I quit this job' button, basically where we've given the model the ability to terminate its conversations by saying 'I don't want to be involved in the conversation.' Models do that when they have to deal with particularly violent or brutal content. It usually only happens in very extreme cases.

This is my second time in India. I visited in October. The last time I came here, I met with all the major Indian IT and conglomerates, the usual ones you would think of. We're beginning to work with most or all of them. One of the things I said is, look, Anthropic is an enterprise company. Its job is to serve other consumers. Many other companies come here as themselves a consumer company and they see India as a market, a place to obtain consumers. We actually see things a little bit differently. We want to work with companies in India to provide our tools to them to help them build those tools and help them do their job better. If we work with a company here, they know the Indian market better. They're better at doing what they do, whether that's consulting or systems integration or building IT tools. They're going to be better at that than we are, particularly for the Indian market. Our hope is that we can add AI to what they do and enhance what they do. There's a lot of worry that AI could replace SaaS or all of these things, but my view is if we do this in the right way, if we work with all these companies, then AI can enhance what they're doing, enhance their connection to the market, their go-to-market abilities, and their specific know-how.