Welcome everybody and welcome to those of you joining us on live stream to this conversation that I have to say I have been looking forward to for months. I was lucky enough to moderate a conversation between Dario Amodei and Demis Hassabis last year in Paris, which I'm afraid got most attention for the fact that you two were squashed on a very small love seat while I sat on an enormous sofa, which was probably my screw-up. But I said at that point that this was for me like chairing a conversation between the Beatles and the Rolling Stones, and you have not had a conversation on stage since. So this is the sequel, the bands get together again. I'm delighted, you need no introduction. The title of our conversation is 'The Day After AGI', which I think is perhaps slightly getting ahead of ourselves because we should probably talk about how quickly and easily we will get there, and I want to do a bit of an update on that and then talk about the consequences. So firstly on the timeline, Dario, you last year in Paris said we'll have a model that can do everything a human could do at the level of a Nobel laureate across many fields by 2026-27. We're in 2026. Do you still stand by that timeline?
So, you know, it's always hard to know exactly when something will happen, but I don't think that's going to turn out to be that far off. The mechanism whereby I imagined it would happen is that we would make models that were good at coding and good at AI research, and we would use that to produce the next generation of model and speed it up to create a loop that would increase the speed of model development. We are now, in terms of the models that write code, I have engineers within Anthropic who say I don't write any code anymore. I just let the model write the code. I edit it. I do the things around it. I think we might be 6 to 12 months away from when the model is doing most, maybe all of what SWEs do end to end. And then it's a question of how fast does that loop close. Not every part of that loop is something that can be sped up by AI, right? There's chips, there's manufacturer of chips, there's training time for the model. So, you know, I think there's a lot of uncertainty. It's easy to see how this could take a few years. It's very hard for me to see how it could take longer than that. But if I had to guess, I would guess that this goes faster than people imagine, and that that key element of code and increasingly research going faster than we imagine, that's going to be the key driver. It's really hard to predict again how much that exponential is going to speed us up, but something fast is going to happen.
So you, Demis, were a little more cautious last year. You said a 50% chance of a system that can exhibit all the cognitive capabilities humans can by the end of the decade. Clearly in coding, as Dario says, it's been remarkable. What is your sense of, do you stand by your prediction and what's changed in the past year?
Yeah, look, I think I'm still on the same kind of timeline. I think there has been remarkable progress, but I think some areas of engineering work, coding or mathematics, are a little bit easier to see how they would be automated, partly because they're verifiable what the output is. Some areas of natural science are much harder to do than that. You won't necessarily know if the chemical compound you've built or this prediction about physics is correct. It may be you may have to test it experimentally and that will all take longer. So I also think there are some missing capabilities at the moment in terms of not just solving existing conjectures or existing problems, but actually coming up with the question in the first place or coming up with the theory or the hypothesis. I think that's much, much harder and I think that's the highest level of scientific creativity and it's not clear. I think we will have those systems, so I don't think it's impossible, but I think there may be one or two missing ingredients. It remains to be seen how, first of all, can this self-improvement loop that we're all working on actually close without human in the loop. I think there are also risks to that kind of system by the way, which we should discuss and I'm sure we will. But that could speed things up if that kind of system does work.
We'll get to the risks in a minute, but one other change I think of the past year has been a kind of change in the pecking order of the race, if you will. This time a year ago, we just had the DeepSeek moment and everyone was incredibly excited about what happened there. And there was still a sense that Google DeepMind was kind of lagging OpenAI. I would say that now it's looking quite different. I mean, they've declared code red, right? It's been quite a year. So talk me through what specifically you've been surprised by and how well you've done this year and whether you think, and then I'm going to ask you about the lineup.
Well, look, I think we were, I was always very confident we would get back to sort of the top of the leaderboards and the state-of-the-art type of models across the board because I think we've always had like the deepest and broadest research bench and it was about kind of marshalling that all together and getting the intensity and focus and the kind of startup mentality back to the whole organization and it's been a lot of work. But I think we're, and we're still a lot of work to do, but I think you can start seeing the progress that's been made in both the models with Gemini 3 but also on the product side with Gemini app getting increasing market share. So I feel like we're making great progress but there's a ton more work to do and you know we're bringing to bear Google DeepMind's kind of like the engine room of Google where we're getting used to shipping our models more quickly into the product surfaces.
One question for you, Dario, on this aspect of it because you've just, you're in the process of a new round at an extraordinary valuation too. But you are unlike Demis, a let's call it an independent model maker and there is I think an increasing concern that the independent model makers will not be able to continue for long enough until you get to where the revenues come in. It's made very openly about OpenAI, but talk me through how you think about that and then we'll get to the AGI itself.
Yeah, I mean, you know, I think how we think about that is, as we've built better and better models, there's been a kind of exponential relationship, not only between how much compute you put into the model and how cognitively capable it is, but between how cognitively capable it is and how much revenue it's able to generate. So, our revenues grown 10x in the last three years from 0 to 100 million in 2023, 100 million to a billion in 2024, and 1 billion to 10 billion in 2025. And so those revenue numbers, you know, I don't know if that curve will literally continue. It would be crazy if it did. But those numbers are starting to get not too far from the scale of the largest companies in the world. So there's always uncertainty. You know, we're trying to bootstrap this from nothing. It's a crazy thing, but I have confidence that if we're able to produce the best models in the things that we focus on, then I think things will go well. And you know, I will generally say, I think it's been a good year for both Google and Anthropic. And I think the thing we actually have in common is that they're both kind of companies that are, or the research part of the company that are kind of led by researchers who focus on the models, who focus on solving important problems in the world, right? Who have these kind of hard scientific problems as a north star and I think those are the kind of companies that are going to succeed going forward and you know I think we share that between us very much.
I'm going to resist the temptation to ask you what will happen to the companies that are not led by researchers, because I know you won't answer it. But let's then go on to the predictions area now and this we are supposed to be talking about the day after AI, but let's talk about closing the loop, the odds that you will get models that will close the loop and be able to power themselves if you will, because that's really the crux for the winner takes all threshold approach. Do you still believe that we are likely to see that or is this going to be much more of a normal technology where followers and catch-up can compete?
Well, look, I definitely don't think it's going to be a normal technology. So I mean there are aspects already that as Dario mentioned that it's already helping with our coding and some aspects of research. The full closing of the loop though I think is an unknown. I mean I think it's possible to do, you may need AGI itself to be able to do that in some domains again where there's more messiness around them. It's not so easy to verify your answer very quickly. There's kind of NP-hard domains. So as soon as you start getting more and more, I also include by the way for AGI physical AI, robotics, working, all of these kind of things and then you've got hardware in the loop that may limit how fast the self-improvement systems can work. But I think in coding and mathematics and these kind of areas I can definitely see that working and then the question is more a theoretical one is what is the limit of engineering and maths to solve the natural sciences.
Dario, you last year, I think it was last year that you published 'Machines of Loving Grace' which was a very upbeat essay about the potential that you were going to see unfold and you were talking about a genius of data at country data center. I'm told that you are working on an update to this, a new essay, so you know, wait for it guys, it's not out yet but it is coming out. But perhaps you can give us a sneak preview of what a year later your big take is going to be.
Yes. So, you know, my take has not changed. It has always been my view that AI is going to be incredibly powerful. I think Demis and I kind of agree on that. It's just a question of exactly when. And because it's incredibly powerful, it will do all these wonderful things like the ones I talked about in 'Machines of Loving Grace'. It will help us cure cancer. It may help us to eradicate tropical diseases. It will help us understand the universe. But there are these immense and grave risks that, you know, not that we can't address them. I'm not a doomer, but that we need to think about them and we need to address them. And I wrote 'Machines of Loving Grace' first. I'd love to give a sophisticated reason why I wrote that first, but it was just that the positive essay was easier and more fun to write than the negative essay. So, you know, I finally spent some time on vacation and I was able to write an essay about the risks. Even when I'm writing about the risks, I try, you know, I'm like an optimistic person, right? So, even as I'm writing about these risks, I wrote about it in a way that was like, how do we overcome these risks? How do we have a battle plan to fight them? And the way I framed it was, you know, there's this scene from Carl Sagan's 'Contact', the movie version of it, where they kind of discover alien life and this international panel that's interviewing people to be humanity's representative to meet the alien. And one of the questions they ask one of the candidates is, you know, if you could ask the aliens any one question, what would it be? And one of the characters says, 'I would ask, how did you do it? How did you manage to get through this technological adolescence without destroying yourselves? How did you make it through?' And ever since I saw it, it was like 20 years ago, I think I saw that movie, it's kind of stuck with me. And that's the frame that I used, which is that we are knocking on the door of these incredible capabilities, right? The ability to build basically machines out of sand, right? I think it was inevitable that the instant we started working with fire. But how we handle it is not inevitable. And so I think the next few years we're going to be dealing with, you know, how do we keep these systems under control that are highly autonomous and smarter than any human? How do we make sure that individuals don't misuse them? Right? I have worries about things like bioterrorism. How do we make sure that nation states don't misuse them and that's why I've been so concerned about the CCP, other authoritarian governments. What are the economic impacts? Right? I've talked about labor displacement a lot and what haven't we thought of which in many cases may be the hardest thing to deal with at all. So, you know, I'm thinking through how to address those risks. For each of these, it's a mixture of things that we individually need to do as leaders of the companies and that we can do working together. And then there's going to need to be some role for wider societal institutions like the government in addressing all of these. But, you know, I just feel this urgency that every day, there's all kinds of crazy stuff going on in the outside world, outside AI, right? But my view is this is happening so fast and is such a crisis we should be devoting almost all of our effort to thinking about how to get through this.
So I can't decide whether I'm more surprised that you a) take a vacation, b) when you take a vacation you think about the risks of AI, and c) that your essay is framed in terms of are we going to get through the technological adolescence of this technology without destroying ourselves. So, my head is slightly spinning, but you then, and I can't wait to read it, but you mentioned several areas that can guide the rest of our conversation. Let's start with jobs, because you actually have been very outspoken about that and I think you said that half of entry-level white collar jobs could be gone within the next one to five years. But I'm going to turn to you, Demis, because so far we haven't actually seen any discernable impact on the labor market. Yes, unemployment has ticked up in the US, but all of the kind of economic studies I've looked at and that we've written about suggest that this is overhiring post pandemic, that it's really not AI-driven. If anything, people are hiring to build out AI capability. Do you think that this will be, as economists have always argued that it's not a lump of labor fallacy, that actually there will be new jobs created because so far the evidence seems to suggest that.
Yeah, I mean, I think in the near term that is what will happen. The kind of normal evolution when a breakthrough technology arrives. So some jobs will get disrupted but I think new, even more valuable, perhaps more meaningful jobs will get created. I think we're going to see this year the beginnings of maybe impacting the junior level, entry level, kind of jobs, internships, this type of thing. And I think there is some evidence, I can feel that ourselves, maybe like a slowdown in hiring in that. But I think that can be more than compensated by the fact there are these amazing creative tools out there pretty much available for everyone almost for free that if I was to talk to a class of undergrads right now I would be telling them to get really unbelievably proficient with these tools. I think to the extent that even those of us building it, we're so busy building it, it's hard to have also time to really explore the almost the capability overhang even today's models and products have let alone tomorrow's and I think that can be maybe better than a traditional internship would have been in terms of you sort of leapfrogging yourself to be useful in a profession. So I think that's what I see happening probably in the next 5 years. Maybe we again slightly differ on time scales on that but I think what happens after AGI arrives that's a different question because I think really we would be in uncharted territory at that point.
Do you think it's going to take longer than you thought last year when you said half of all white collars?
I have about the same view. I actually agree with you and with Demis that at the time I made the comment there was no impact on the labor market. I wasn't saying there was an impact on the labor market at that moment. Now I think maybe we're starting to see just the little beginnings of it, you know, in software and coding. I even see it within Anthropic where, you know, I can look forward to a time where on the more junior end and then on the more intermediate end, we actually need less and not more people. And you know, we're thinking about how to deal with that within Anthropic in a sensible way. One to five years as of six months ago, I would stick with that. You know, if you kind of connect this to what I said before, which is we might have AI that's better than humans at everything in maybe one to two years, maybe a little longer than that. Those don't seem to line up. The reason is that there's this lag and there's this replacement thing, right? I know the labor market is adaptable, right? Just like 80% of people used to do farming, farming got automated and then they became factory workers and then knowledge workers. So there is some level of adaptability here as well, right? We should be economically sophisticated about how the labor market works but my worry is as this exponential keeps compounding and I don't think it's going to take that long, again somewhere between a year and five years, it will overwhelm our ability to adapt. I think I may be saying the same thing Demis is just factored out of that difference we have about timelines which I think ultimately comes down to how fast you close the loop on that.
How much confidence do you have that governments get the scale of this and are beginning to think about what policy responses they need to have?
I don't think that there's anywhere near enough work going on about this. I'm constantly surprised even when I meet economists at places like this that they're not more professional economist professors thinking about what happens and not just sort of on the way to AGI but even if we get all the technical things right that Dario was talking about and the job displacement is one question we're worried about the economics of that but maybe there are ways to distribute this new productivity, this new wealth more fairly. I don't know if we have the right institutions to do that but that's what should happen at that point there should be, you know, we may be in a post-scarcity world but then there are even the things that keep me up, I recognize there are even bigger questions than that at that point to do with meaning and purpose and a lot of the things that we get from our jobs, not just economically. That's one question, but I think that may be easier to solve strangely than what happens to the human condition and humanity as a whole. And I think I'm also optimistic we'll come up with new answers there. We do a lot of things today from extreme sports to art that aren't necessarily directly to do with economic gain. So I think we will find meaning and maybe there'll be even more sophisticated versions of those activities. Plus I think we'll be exploring the stars. So there'll be all of that to factor in as well for in terms of purpose. But I think it's really worth thinking now even on my timelines of like 5 to 10 years away. That isn't a lot of time before this comes.
How big do you think is the risk of a popular backlash against AI that will somehow kind of cause governments to do what from your perspective might be stupid things? Because I'm just thinking back to the era of globalization in the 1990s when there was indeed some displacement of jobs. Governments didn't do enough. The public backlash was such that we've ended up sort of where we are now. Do you think that there is a risk that there will be a growing antipathy towards what you are doing and your companies in the kind of body politic?
I think there's definitely a risk. I think that's kind of reasonable. There's fear and there's worries about these things like jobs and livelihoods. I think there's a couple of things that, I mean it's going to be very complicated the next few years I think geopolitically but also the various factors here like we want to and we're trying to do this with AlphaFold and our science work and Isomorphic, our spinout company, solve all disease, cure diseases, come up with new energy sources. I think as a society it's clear we'd want that. I think maybe the balance of what the industry is doing is not enough balance towards those types of activities. I think we should have a lot more examples, I know Dario agrees with me, of like AlphaFold-like things that help sort of unequivocal good in the world and I think actually it's incumbent on the industry and all of us leading players to show that more, demonstrate that, not just talk about it but demonstrate that. But then it's going to come with these other attendant disruptions and but I don't, I think the other issue is the geopolitical competition. There's obviously competition between the companies but also US and China primarily. So unless there's an international cooperation or understanding around this, which I think would be good actually in terms of things like minimum safety standards for deployment. I think Dario would agree on that as well. I think it's vitally needed. This technology is going to be cross-border. It's going to affect everyone. It's going to affect all of humanity. Actually 'Contact' is one of my favorite films as well. So funny enough, I didn't realize it was yours too, Dario. But I think those kind of things need to be worked through. And if we can maybe it would be good to have a slightly slower pace than we're currently predicting even my timelines so that we can get this right society but that would require some coordination.
I prefer your timelines. Yes, absolutely. I'll concede. But Dario, let's turn to this now because one thing since we last spoke in Paris, the geopolitical environment has if anything, I don't know, complicated, mad, crazy, whatever phrase you want to use. Secondly, the US has a very different approach now towards China. It's a much more, it's a kind of no holds bar, go as fast as we can, but then sell chips to China. And that is, so you've got a different attitude towards the United States. You've got a very strange relationship between the United States and Europe right now geopolitically against that. I mean I hear you talk about it would be nice to have a CERN-like organization, I mean it's a million years from where we are from the real world. So in the real world have the geopolitical risks increased and what if anything do you think should be done about that and the administration seems to be doing the opposite of what you were suggesting?
Yeah, I mean, look, you know, we're just trying to do the best we can to, you know, we're just one company and we're trying to operate in the environment that exists.
No matter how crazy it is. But, you know, I think at least my policy recommendations haven't changed. Not selling chips is one of the biggest things we can do to make sure that we have the time to handle this. You know, I said before, I prefer Demis' timeline. I wish we had 5 to 10 years. So it's possible he's just right and I'm just wrong, but assume I'm right and it can be done in one to two years. Why can't we slow down to Demis' timeline?
Well, you could just slow down. Well, no. The reason we can't do that is because we have geopolitical adversaries building the same technology at a similar pace. It's very hard to have an enforceable agreement where they slow down and we slow down. And so if we can just not sell the chips, then this isn't a question of competition between the US and China. This is a question of competition between me and Demis, which I'm very confident that we can work out.
And what do you make of the logic of the administration, which as I understand it is we need to sell them chips because we need to bind them into US supply chains?
So, you know, I think it's a question not just of timescale but of the significance of the technology, right? If this was telecom or something, then all this stuff about proliferating the US stack and wanting to build our chips around the world to make sure that these random countries in different parts of the world build data centers that have Nvidia chips instead of Huawei chips. You know, I think of this more as like, it's a decision: are we going to sell nuclear weapons to North Korea because that produces some profit for Boeing? Where we can say, okay, yeah, these cases were made by Boeing, like the US is winning. Like, this is great. I just, you know, that analogy should just make clear how I see this trade-off that I just don't think it makes sense. And we've done a lot of more aggressive stuff toward China and other players that I think is much less effective than this one measure.
