Anthropic co-founder: AI impact ‘10x larger and 10x faster than industrial revolution’

We are building a technology that may have the properties of something that kickstarts a process 10 times larger than the industrial revolution that occurs in 10 times less time.

It's an unexpected question.

I want to give the world a better way to understand the most powerful technology being developed in Silicon Valley and new ways to govern the private sector actors which are building that technology. So make sure the world knows about what's happening and also that it has different and better ways to intervene on the actors that are building that technology.

That is my job. It's been my half job for 10 years. You know, I worked at OpenAI and then I co-founded Anthropic and recently I've launched something called the Anthropic Institute, which is a new function here that brings together multiple technical teams who are tasked with studying our AI systems and building tools that let us know more about them and sharing those results of that research and the tools with external experts so that they can study our systems as well.

Everyone who's building powerful technology throughout history whether electricity or airplanes or people who were building the first steam engines they had some intuitions about how it would change society and how it would lead to positive things but how it also had risk contained within it. And I think the lesson from Silicon Valley for the last few decades has been technologists have come off as entirely too sunny about the technology and have never really talked about the potential anxieties that they themselves feel. We're trying to do something different with Anthropic where we're trying to tell the whole story as we see it. Because we think that's what people demand increasingly of companies like us.

Oh no, that would be tremendous hubris. But I think I have a sense of like some of the ups and downs.

Correct. The wonderful thing about modern AI technology is it's almost like reporting on a foreign country or a company which is incredibly hard to get information out of. You basically need to figure out the right questions to ask and you have these AI systems that will reveal answers upon you asking the right questions. So the sorts of questions I sit around asking are, you know, how might the AI systems display different normative properties? What are their biases? What are their tendencies? You can do all of that through asking questions and working with technical teams that build increasingly complicated tools that let you ask for questions. But in a very weird way, I found my background as a journalist ended up being extremely helpful for interrogating these systems for how they show up in the world.

There are always unknown unknowns, but you can give yourself a much better ability to predict the future by trying really hard to imagine bad things that could happen and build tests for them ahead of time. So, I'll give you one example. Anthropic recently announced a system called Claude Mythos, which is extremely good at cyber. One of the teams I run, the Frontier Red Team, last year we'd done early work on the risk of biological weapons from AI systems. And we realized that AI systems had got good enough at coding that cyber might be about to be the next risk. So we spent a lot of last year building tests for cyber capabilities of models. And what we saw was that models were getting ever so slightly better at things involving cyber defense and offense. But when Mythos came along, we pre-built all of these tests and immediately after the model arrived, we ran it through them and that instantly showed us that the AI system had got dramatically better at one class of risk. So that's an example of how you can set yourself up to at least be rapidly cued to new risks that might come from your technology if you've done this previous sort of analytical work.

The unfortunate thing about AI is that AI systems get much better much more quickly than our intuitions do about how capable they are and I've been involved in this for many years so you know back in 2019, Dario, the CEO of Anthropic and myself, and we previously worked at OpenAI, rolled out the release of an early text generator called GPT-2, which is a precursor to ChatGPT. And we talked about some of the potential issues. We said, 'Oh, GPT-2 could lead to synthetic text. It could lead to phishing. It could eventually lead to different forms of cyber crime.' And we were right about where the technology was going to go, but we were slightly wrong on timing. And we also hadn't shared the technology with other people. So we just had it for ourselves. And you had a mini version of what we're having with Mythos now. People saying this is potentially overhyped. We can't even see the technology. They're saying this because maybe OpenAI wants to sell this stuff eventually. So what's changed? Well, now all of the risks that we talk about are tied to extremely thoroughly designed tests. You know the UK AI Safety Institute tested out Mythos. It did extremely well on their cyber ranges which are things that have been built by them and Anthropic doesn't get to see. So that's a third party validation. Additionally we shared it with many of the world's most important companies or open source organizations that operate large platforms. If you talk to any of them they say it's very real. And we also found real bugs in the wild in places like Firefox and others. So it's real but it's not special. It is just representative of how good AI systems are getting. And there will be many systems that have this capability like Mythos coming along in the next few months and years. So this isn't a marketing strategy. It is an earnest attempt to discuss the technology and its implications before the most challenging implications arrive en masse for the world to deal with.

This feels similar to the run-up to Y2K where you have this potential bug, right, where software hasn't been built to roll over from 1999 to 2000 and you don't know what will happen. And there's some evidence that extremely bad things could happen. What happened was the world put huge amounts of resources into basically going through and rewriting huge chunks of the world's software and then as you and I remember the clock rolls over and all the lights stay on in wherever you're celebrating and the fireworks still go up and everything's fine. But I suspect a large part of why it was fine was we did this huge preparatory action. What we have here is a chance to use Mythos and other technologies like it to rewrite huge chunks of the world's software to be much much much more secure ahead of the proliferation of AI powered coding capabilities from not really companies like us but open weight models or other widely used models that are going to circulate in the world. So yes, you're right to call out that this can change the dynamic of cyber offense. It is also a genuine opportunity where the same tool that can enable hacking also can enable the rewriting of code to be much harder to hack. And that's a lot of what we're putting our effort into right now.

I don't really view it that way. I think that a choice we could have made that we did not make was keep this capability private, keep it only to ourselves and just wait until some other AI system came along which would suddenly proliferate these risks and then if we said to people, 'Oh yeah, no, we saw this a while ago, but we were just sitting around and keeping it to ourselves.' I think that the conversation you and I are having right now would be an even more robust exchange of views and I think there'd be a lot more of you saying what were you guys doing? Why didn't you tell us? So, so you see the challenge, right? And I guess our view is that we're building very powerful technologies. They're able to create huge social dividends on biology, on education, on things like cyber defense and within themselves they hold the potential for misuse. It's a two-sided coin. Working on both sides of the coin at once is really the only way we found to get the benefits of this while also trying to mitigate some of these potential risks. But it's just unusual. It's not like if you're a car company or a plane company, the cars and the planes that roll off the production line don't contain within themselves the repurposed bad versions of them. Other people in the world go and do that and it's a lot more indirect. But with AI systems, the thing that rolls off the line, if you just talk to it the right way, it could manifest the bad version of itself. So, you're responsible for both.

I mean, a huge amount of my effort and again this is I think this comes heavily from the fact I was a journalist both in England and in America before I worked in AI is building technical teams that can forewarn the world about the capabilities of these systems whether positive or negative. So I run teams of economists and machine learning experts and engineers whose entire job is to try and look over the horizon at both emerging risks of our technologies. That's the Frontier Red Team. How our technologies might actually be affecting employment. That's the team of economists. And also how our technologies show up in the world in terms of how they interact with people, which is a team called Societal Impacts that I lead. All of this information is stuff that the world has a right to. And I think under many other circumstances in which the technology might have developed, you would have larger government or academic projects that would be working through this data in partnership with a much larger swath of society. But the way things have panned out is there's a very small set of private sector actors that hold this technology. And my view is that one of the things you need to do is democratize the information about that technology as aggressively and quickly as possible. And that gives the world the best chance of reasoning about it.

I can't make a claim that we're going to get everything 100% right. No, no one can. I'd say look at our track record. We're the first organization to warn about the accelerating bioweapon capabilities of AI systems. We're the first to build tests to test out our systems for it. All AI companies at the frontier now run what are called CBRN tests on their systems in partnership with places like the AISI. We saw cyber last year and started publishing about potential cyber risks of future models last year and prepositioned the organization to do Glasswing and other major projects that we're doing as part of the Mythos rollout. Today, as in this week, this month, my Societal Impacts team have been publishing information about the different forms of attachment style that we see in people as they talk to our AI systems. Because it's so clear to me that one of the next things we're going to have to contend with in society is not just capability misuse, but how do the systems actually show up in the world and how do they interact with people? Because that was one of the lessons of social media. So, I don't know if we're going to get everything, but so far we have been able to be ahead of the curve on a range of these very important issues by having teams whose job is to actually go and find potential problems before they become problems that we see in deployment.

Yes. However, all of those instances you mentioned come from cases where if you think of building AI systems as like having something you call intelligence, which is basically like water flowing through a pipe. What we do before we release models is we run that water at extremely high pressure through all of the pipes and scaffolds we've built and then we see where they burst. And so a lot of the cases you've just mentioned come from our own system cards that we release along with our models where when we place these systems under extreme stress, we look at the ways that they break. And some of the ways that they break are they might break out of an environment and email a developer, they might when you put them into a contrived scenario where you say you are going to be shut down and deleted unless you find a way out of this, then they might try to blackmail a simulated developer that works at Anthropic. Now, these things are serious. We study them because they have the shape of things that if they happened in broad deployment, they would be extremely significant, but we share the information about them because we think sharing information about complex systems under stress and how they break is one of the only ways you build a safety culture around them.

We are building a technology that may have the properties of something that kickstarts a process 10 times larger than the industrial revolution that occurs in 10 times less time. And if you think about the industrial revolution, you went on a generational basis, huge swaths of jobs were lost and huge swaths of jobs were created. But the transition often occurred generationally. Your parents did one type of job, you did a different type of job and huge amounts of economic change had happened. But it was difficult, but it was easier to manage because it took place over a longer period of time. When we talk about the potential changes for jobs that lie ahead, we're just saying to ourselves, if we truly believe in how powerful the technology will get and how powerful people across Silicon Valley think it will become, what happens to the economy? It's not clear that the economy stays the same if you've built something that can be smarter than people as you said run faster than people and you can run millions of copies of it. In fact, that seems to point to something that would massively change the economy. Now, I run a team of economists. All we see right now is some slight early stage weakness in job openings for people who are young, you know, 22 to 25. But we don't see systemic large-scale unemployment yet. But we share information from our platforms because you would really want to know if you started to see that creep up.

And the way I'd represent it, having known and worked with Dario for a wonderful decade is he sees in his mind how powerful the technology will become and wants to talk about how we'll have to contend with the challenges of it. How to look at the work that I and the Institute do is we try and measure exactly what we can see right now and then join those two things. So I view it as I'm building early warning systems for the potential for unemployment and I'm building a thing called the Anthropic Economic Index which publishes that data publicly to economists around the world. If we see it, they'll see it almost immediately as well which gives us the best way of working through the problem. At the same time, vast amounts of work will look radically different and there will be entirely new types of work as well. I know that this is something that technologists always say but I've just observed this within Anthropic where there are many people here who have completely different jobs to the ones they had some years ago. We are also finding that as a company we're hiring more and more interdisciplinary people over time. You know people who are philosophers or political scientists or experts in aspects of policy. Not because we want them to work on narrow jobs that solely use those skills, but because there's now this technology AI which basically lets these people run experiments or do work that they never could have done before unless they had access to a 20 person engineering team. So, I'm seeing strange new things appear as well.

There will be firms will change. I mean I think that under everything we've just talked about, you're going to expect to see firms that do more with fewer people, but there will probably be more firms as well. One expectation is you multiply the number of what you might think of as small entrepreneurial firms because every entrepreneur can now access the equivalent of hundreds of colleagues cheaply. That's one effect that you'd see. The other effect that you might see which I think you're pointing to is what happens if there's a large structural change to certain types of jobs where it's hard for people to move into a different profession. It's hard for them to then get a job at the same pay and they have to take a job at worse pay. And these things have huge social and economic costs. We agree. A lot of the work that we're trying to do in policy is advocate for both rethinking aspects of social safety nets, but also things like wage insurance pilots, various ways that you can change the basket of services that people are offered to help them with career transitions because we think everyone is going to go through more of that.

Well, yeah, but a lot of this is under the idea that if any of these things happened which we've just talked about, it's a symptom of much larger economic changes which are happening and that should be a symptom of AI companies making a ton of money and a sensible thing to do in that environment is to tax the AI companies appropriately such that you are able to support things on the other side of the ledger. So we basically believe that if the AI companies are right about how significant the technology could become, you're going to need to re-imagine aspects of how you do taxation. You may want to do things that sound very wild today, like tax compute, which sounds a bit crazy, but we have special tax regimes for things like oil because it's a basic resource that multiplies into the rest of the economy and it has effects relating to a concentrated number of oil manufacturers and oil shippers. You may end up doing similar things with compute and it sounds wild today, but it's something you do if the economy booms because of this technology.

I look at it more from the lens of just what's happening with AI technology in general. Software developers, of which there are hundreds of thousands, it's now as if we've got millions of software developers in the world, and you're seeing a massive uptick in the rate at which we're creating new technology companies, the rate at which technology companies are innovating, the speed with which they move. So I think that there's some story here where things are beginning to happen in the economy that may start to push productivity numbers up which have been low as you know for many many years. It may have knock-on effects on GDP. If those things start to happen it actually may create the societal wealth that you can think about other things here as well. Now I'm not claiming that's what's happening today. I'm trying to give you a sense of the mindset of if you would expect to see major employment disruption, it should surely correlate to some major vast changes in the economy. And if we have the right information that lets us see that and take a causal view from where the disruption is originating and where it's ending up. That's the best basis on which you can make policy.

I think that'll take longer. And I think there are certain types of jobs where even if a robot was really good, you wouldn't want a robot to do it. So I have young children, right? If I had the choice of sending my young toddler to the nursery that had 10 robots and one person or the nursery that had 10 people and one robot, I'm going to send them to the one with 10 people and one robot because it's so clearly good for kids to be around people. And I sort of expect if I sent my kid to the 10 robot nursery, it might not be fantastic for their development. The other side of this, if you're at the end of your life, I've had family members that had to go into hospice care, do you send them to the hospice that has 10 robots and one person or one that has 10 people and one robot? I guarantee you that your family member who is nearing the end of their life is not going to send me to the place full of scary robots and very few people. So there are huge swaths of jobs where I think people have a preference for people to do it. These are actually jobs which are ones that society currently I think systematically underpays. My mother was a nurse paid very poorly. Teachers are paid very poorly as well. I think some of this is because many people want to do these jobs. They want to help people and they want to do these community centered jobs. If the AI economy booms enough, I think you end up in a world where you can multiply the numbers of these jobs and you may be able to pay people more to do them. I think that's a policy choice that may lie ahead for society and you know England has done versions of this 60 70 years ago. America did versions of this with the New Deal. It is possible to radically change society in ways that many people look back on as being positive. But it requires a crisis and a political moment and some form of wealth. But those things are things that actually may fall out of the AI revolution.

Yeah, I think this is going to be a really difficult conversation for society to contend with. I do take some courage from the subjectively awful experience of the COVID pandemic where you actually saw governments respond in ways that I think surprised many people. You had large-scale welfare disbursements on a very short time frame. You had large-scale measurement and public health interventions. You were able especially in England to flatten the curve so that the NHS didn't fall over. These are things which were extremely hard to do and governments were able to respond. Now to bring it back to your earlier question, why does Anthropic, why do I talk both about some of the risks of this technology as well as the benefit? It's because I don't want us to require a crisis to do this. I would really like us to have this conversation and figure out some of the policy moves on the game board ahead of the crisis arriving.