As you said, a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing. Then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world. So icy roads do exist.
Yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
The obvious question is, does that pretend AI is becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this.
What are practical steps people should take to be prepared? I mean ordinary citizens, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
While he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. To back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs and stuff is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what, like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
To be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing. Then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world. So icy roads do exist.
Yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
The obvious question is, does that pretend AI is becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this.
What are practical steps people should take to be prepared? I mean ordinary citizens, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
While he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. To back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs and stuff is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what, like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
To be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing. Then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world. So icy roads do exist.
Yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
The obvious question is, does that pretend AI is becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this.
What are practical steps people should take to be prepared? I mean ordinary citizens, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
While he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. To back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs and stuff is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what, like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
To be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing. Then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world. So icy roads do exist.
Yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
The obvious question is, does that pretend AI is becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this.
What are practical steps people should take to be prepared? I mean ordinary citizens, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
While he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. To back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs and stuff is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what, like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
To be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing. Then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world. So icy roads do exist.
Yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
The obvious question is, does that pretend AI is becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this.
What are practical steps people should take to be prepared? I mean ordinary citizens, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
While he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. To back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs and stuff is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what, like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
To be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing. Then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world. So icy roads do exist.
Yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
The obvious question is, does that pretend AI is becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this.
What are practical steps people should take to be prepared? I mean ordinary citizens, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
While he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. To back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs and stuff is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what, like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
To be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing. Then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world. So right do exist.
I think yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
The obvious question is, does that pretend AI is becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept.
The concept we have a couple people who are actually working on this topic, you know, as crazy as it sounds, of you know, do AI systems have morally significant feelings? I would guess that they don't right now, but you know, when you run an AI company, you know, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this. What are practical steps people should take to be prepared? I mean ordinary citizens, me, lawmakers, what do you advise?
Yeah, you know, lawmakers I guess really. Well, let's take them one by one. I think for ordinary citizens, it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. There's a better world, at least in the short term, at least for now, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For the lawmakers, I would say really, really, really be aware of these issues, and we're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying taxes on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
And while he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. But just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student, and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. So it's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough, and so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest, because a lot of the messages we hear from at least publicly from some AI CEOs and stuff is a little bit more calming, saying like these agents are going to be great in your life, and yes there may be problems, but writ large this is a fantastic thing.
Yeah, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say most jobs will change more slowly than most people think. And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
So well, let me just say because you've previously in the past described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. So I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact just on what? Like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracy work, and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our society, our social contract, won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what is that? I mean, that freaked me out when I read that. What does that mean?
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, and you do that so you can see if you really stress test the thing, you can make the crash dummy blow up. So this isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
So icy rules do exist. Yeah, yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems. I mean, does it pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, you know, as crazy as it sounds, of do AI systems have morally significant feelings? I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this. What are practical steps people should take to be prepared? I mean ordinary citizens, me, lawmakers, what do you advise?
Yeah, you know, particularly lawmakers, I guess really.
Yeah, well, let's take them one by one. I think for ordinary citizens, it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. There's a better world, at least in the short term, at least for now, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For the lawmakers, I would say really, really, really be aware of these issues, and we're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying taxes on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
And while he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. But just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student, and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. So it's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough, and so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest, because a lot of the messages we hear from at least publicly from some AI CEOs and stuff is a little bit more calming, saying like these agents are going to be great in your life, and yes there may be problems, but writ large this is a fantastic thing.
Yeah, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say most jobs will change more slowly than most people think. And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
So well, let me just say because you've previously in the past described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. So I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact just on what? Like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracy work, and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our society, our social contract, won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what is that? I mean, that freaked me out when I read that. What does that mean?
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, and you do that so you can see if you really stress test the thing, you can make the crash dummy blow up. So this isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
So icy rules do exist. I think yeah, yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems. I mean, does it pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, you know, as crazy as it sounds, of do AI systems have morally significant feelings? I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this. What are practical steps people should take to be prepared? I mean ordinary citizens, me, lawmakers, what do you advise?
Yeah, you know, particularly lawmakers, I guess really.
Yeah, well, let's take them one by one. I think for ordinary citizens, it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. There's a better world, at least in the short term, at least for now, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For the lawmakers, I would say really, really, really be aware of these issues, and we're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
And while he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. But just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student, and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. So it's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough, and so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest, because a lot of the messages we hear from at least publicly from some AI CEOs and stuff is a little bit more calming, saying like these agents are going to be great in your life, and yes there may be problems, but writ large this is a fantastic thing.
Yeah, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say most jobs will change more slowly than most people think. And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
So well, let me just say because you've previously in the past described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. So I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact just on what? Like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracy work, and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our society, our social contract, won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what is that? I mean, that freaked me out when I read that. What does that mean?
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, and you do that so you can see if you really stress test the thing, you can make the crash dummy blow up. So this isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
So icy rules do exist. I think yeah, yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems. I mean, does it pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, you know, as crazy as it sounds, of do AI systems have morally significant feelings? I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this. What are practical steps people should take to be prepared? I mean ordinary citizens, me, lawmakers, what do you advise?
Yeah, you know, particularly lawmakers, I guess really.
Yeah, well, let's take them one by one. I think for ordinary citizens, it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. There's a better world, at least in the short term, at least for now, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For the lawmakers, I would say really, really, really be aware of these issues, and we're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
And while he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. But just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student, and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. So it's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough, and so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest, because a lot of the messages we hear from at least publicly from some AI CEOs and stuff is a little bit more calming, saying like these agents are going to be great in your life, and yes there may be problems, but writ large this is a fantastic thing.
Yeah, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say most jobs will change more slowly than most people think. And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
So well, let me just say because you've previously in the past described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. So I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact just on what? Like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracy work, and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our society, our social contract, won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what is that? I mean, that freaked me out when I read that. What does that mean?
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, and you do that so you can see if you really stress test the thing, you can make the crash dummy blow up. So this isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
So icy rules do exist. I think yeah, yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems. I mean, does it pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, you know, as crazy as it sounds, of do AI systems have morally significant feelings? I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this. What are practical steps people should take to be prepared? I mean ordinary citizens, me, lawmakers, what do you advise?
Yeah, you know, particularly lawmakers, I guess really.
Yeah, well, let's take them one by one. I think for ordinary citizens, it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. There's a better world, at least in the short term, at least for now, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For the lawmakers, I would say really, really, really be aware of these issues, and we're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying taxes on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
And while he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. But just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student, and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. So it's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough, and so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest, because a lot of the messages we hear from at least publicly from some AI CEOs and stuff is a little bit more calming, saying like these agents are going to be great in your life, and yes there may be problems, but writ large this is a fantastic thing.
Yeah, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say most jobs will change more slowly than most people think. And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
So well, let me just say because you've previously in the past described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. So I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact just on what? Like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracy work, and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our society, our social contract, won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what is that? I mean, that freaked me out when I read that. What does that mean?
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, and you do that so you can see if you really stress test the thing, you can make the crash dummy blow up. So this isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
So icy rules do exist. I think yeah, yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems. I mean, does it pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, you know, as crazy as it sounds, of do AI systems have morally significant feelings? I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this. What are practical steps people should take to be prepared? I mean ordinary citizens, me, lawmakers, what do you advise?
Yeah, you know, particularly lawmakers, I guess really.
Yeah, well, let's take them one by one. I think for ordinary citizens, it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. There's a better world, at least in the short term, at least for now, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For the lawmakers, I would say really, really, really be aware of these issues, and we're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
And while he says AI can lead to incredible advancements like medical breakthroughs and boost the economy, he believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. But just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student, and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. So it's hard to estimate exactly what the impact would be. There's always this question of adaptation, and these technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough, and so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest, because a lot of the messages we hear from at least publicly from some AI CEOs and stuff is a little bit more calming, saying like these agents are going to be great in your life, and yes there may be problems, but writ large this is a fantastic thing.
Yeah, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say most jobs will change more slowly than most people think. And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
So well, let me just say because you've previously in the past described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. So I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact just on what? Like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracy work, and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our society, our social contract, won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what is that? I mean, that freaked me out when I read that. What does that mean?
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, and you do that so you can see if you really stress test the thing, you can make the crash dummy blow up. So this isn't something that the model does in practice in actual usage, but the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
So icy rules do exist. I think yeah, yeah, no, but you want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems. I mean, does it pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, you know, as crazy as it sounds, of do AI systems have morally significant feelings? I would guess that they don't right now, but when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this. What are practical steps people should take to be prepared? I mean ordinary citizens, me, lawmakers, what do you advise?
Yeah, you know, particularly lawmakers, I guess really.
Yeah, well, let's take them one by one. I think for ordinary citizens, it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. There's a better world, at least in the short term, at least for now, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For the lawmakers, I would say really, really, really be aware of these issues, and we're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
He says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. So it's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger and it's broader and it's moving faster than anything has before. So compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know, people like Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily, I would think, in your best interest. A lot of the messages we hear from at least publicly from some AI CEOs and stuff is a little bit more calming, saying like these agents are going to be great in your life, and yes there may be problems, but writ large this is a fantastic thing.
Yeah, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position that I'm building this technology while also expressing concerns about it. The reason I'm doing both of those things is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are as you mentioned six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to just read it word for word. He said, 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job, people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Well, let me just say, because you've previously in the past described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. So I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact just on what? Like I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? And I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracies work, and it's harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. And that's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what is that? I mean, that freaked me out when I read that. What does that mean?
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. If you were to make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage. But the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
Yeah, exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
I mean, does it, the obvious question is, does that portend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic. As crazy as it sounds, do AI systems have morally significant feelings? I would guess that they don't right now. But when you run an AI company, these crazy seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy sounding things like this.
What are practical steps people should take to be prepared? I mean ordinary citizens, me, lawmakers, what do you advise?
Yeah, well let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, for now, we should take it bit by bit, where everyone learns to use AI better. That speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
A couple years ago you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast. Yes, people will adapt, but they may not adapt fast enough, so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody. You know all the names we know: Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that others haven't as much. I think someone needs to say it. I recognize it's always a difficult balance. I'm aware of my position: I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago. But nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job where people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify, and also the impact on what? I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. We need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. That's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? What does that mean? I mean, that freaked me out when I read that.
Just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. To make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage. But the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
Exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
Does that pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy-seeming questions are things that you study. I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy-sounding things like this.
What are practical steps people should take to be prepared? Ordinary citizens, me, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better. That speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
He says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white-collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white-collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast. Yes, people will adapt, but they may not adapt fast enough, so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody. You know all the names we know: Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that others haven't as much. I think someone needs to say it. I recognize it's always a difficult balance. I'm aware of my position: I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago. But nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job where people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify, and also the impact on what? I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. We need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. That's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? What does that mean? I mean, that freaked me out when I read that.
Just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. To make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage. But the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
Exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
Does that pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy-seeming questions are things that you study. I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy-sounding things like this.
What are practical steps people should take to be prepared? Ordinary citizens, me, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better. That speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
He says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white-collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white-collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast. Yes, people will adapt, but they may not adapt fast enough, so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody. You know all the names we know: Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that others haven't as much. I think someone needs to say it. I recognize it's always a difficult balance. I'm aware of my position: I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago. But nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job where people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify, and also the impact on what? I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's harder to make democracies work and it's harder to prevent concentration of power. We need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contract won't work. That's why I think it's important.
During testing, your company's latest chatbot Claude 4 was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? What does that mean? I mean, that freaked me out when I read that.
Just to be clear and to put this in context, this was a behavior that emerged during kind of extreme testing of the model. To make an analogy to cars, this is like you deliberately put the car on the iciest road possible, you mess with the brakes, you do that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something that the model does in practice in actual usage. But the reason we stress test the models this way is that's the best way to test for and prevent problems in the real world.
Exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world. So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
Does that pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy-seeming questions are things that you study. I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy-sounding things like this.
What are practical steps people should take to be prepared? Ordinary citizens, me, lawmakers, what do you advise?
Let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better. That speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
He says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white-collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white-collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast. Yes, people will adapt, but they may not adapt fast enough, so there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody. You know all the names we know: Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
I think the reason I'm raising the alarm is that others haven't as much. I think someone needs to say it. I recognize it's always a difficult balance. I'm aware of my position: I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology right now. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped doing it tomorrow, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago. But nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say lamp lighters was actually a job where people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that AI, if we succeed at it, can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify, and also the impact on what? I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
A little bit more calming, saying like, you know, these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
Yeah, I mean, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position, that I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say, lamp lighters was actually a job—people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that if we succeed, AI can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Well, let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what—I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracies work and harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contracts won't work. And that's why I think it's important.
During testing, your company's latest chatbot, Claude 4, was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during extreme testing of the model. To make an analogy to cars, this is like deliberately putting the car on the iciest road possible, messing with the brakes, doing that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something the model does in practice in actual usage. But the reason we stress test the models this way is that it's the best way to test for and prevent problems in the real world.
Yeah, no, exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
Does that pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy-seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy-sounding things like this.
What are practical steps people should take to be prepared? I mean, ordinary citizens, lawmakers, what do you advise?
Yeah, particularly lawmakers, I guess. Well, let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really, really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
Well, he says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know—Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying like, you know, these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
Yeah, I mean, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position, that I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say, lamp lighters was actually a job—people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that if we succeed, AI can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Well, let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what—I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracies work and harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contracts won't work. And that's why I think it's important.
During testing, your company's latest chatbot, Claude 4, was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during extreme testing of the model. To make an analogy to cars, this is like deliberately putting the car on the iciest road possible, messing with the brakes, doing that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something the model does in practice in actual usage. But the reason we stress test the models this way is that it's the best way to test for and prevent problems in the real world.
Yeah, no, exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
Does that pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy-seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy-sounding things like this.
What are practical steps people should take to be prepared? I mean, ordinary citizens, lawmakers, what do you advise?
Yeah, particularly lawmakers, I guess. Well, let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really, really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
Well, he says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know—Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying like, you know, these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
Yeah, I mean, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position, that I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say, lamp lighters was actually a job—people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that if we succeed, AI can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Well, let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what—I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracies work and harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contracts won't work. And that's why I think it's important.
During testing, your company's latest chatbot, Claude 4, was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during extreme testing of the model. To make an analogy to cars, this is like deliberately putting the car on the iciest road possible, messing with the brakes, doing that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something the model does in practice in actual usage. But the reason we stress test the models this way is that it's the best way to test for and prevent problems in the real world.
Yeah, no, exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
Does that pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy-seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy-sounding things like this.
What are practical steps people should take to be prepared? I mean, ordinary citizens, lawmakers, what do you advise?
Yeah, particularly lawmakers, I guess. Well, let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really, really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
Well, he says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know—Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying like, you know, these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
Yeah, I mean, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position, that I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say, lamp lighters was actually a job—people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that if we succeed, AI can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Well, let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what—I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracies work and harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contracts won't work. And that's why I think it's important.
During testing, your company's latest chatbot, Claude 4, was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during extreme testing of the model. To make an analogy to cars, this is like deliberately putting the car on the iciest road possible, messing with the brakes, doing that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something the model does in practice in actual usage. But the reason we stress test the models this way is that it's the best way to test for and prevent problems in the real world.
Yeah, no, exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
Does that pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy-seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy-sounding things like this.
What are practical steps people should take to be prepared? I mean, ordinary citizens, lawmakers, what do you advise?
Yeah, particularly lawmakers, I guess. Well, let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really, really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
Well, he says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know—Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying like, you know, these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
Yeah, I mean, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position, that I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say, 'Most jobs will change more slowly than most people think.' And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say, lamp lighters was actually a job—people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that if we succeed, AI can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.
Well, let me just say, because you've previously described a future where cancer is cured, the economy grows at 10% a year, the budget is balanced, and 20% of people don't have jobs.
That's exactly what I was getting to. I agree with all the positive potential. I think that isn't wrong. But I think the quote you just flashed is maybe too optimistic, too sanguine, about the ability for people to adapt. People have adapted to past technological changes, but I'll say again, everyone I've talked to has said this technological change looks different. It looks faster, it looks harder to adapt to, it's broader. The pace of progress keeps catching people off guard. I don't know exactly how fast the job concerns are going to come. I don't know how fast people are going to adapt. It's possible it'll all be okay, but I think that's too sanguine an approach. I think we do need to be raising the alarm. I think we do need to be concerned about it. I think policymakers do need to worry about it. If they do worry and they do act, then maybe we can prevent it, but we're not going to prevent it just by saying everything's going to be okay.
Do we as a society even understand the potential inequalities that this may amplify? And also the impact on what—I have little kids, I have a 5-year-old and a 3-year-old. What do they grow up aspiring to if machines can do pretty much everything better? What does it do to initiative or drive or striving? I don't know the answer to that, but it concerns me.
Let's take those questions one by one. Those do concern me, but I do have thoughts on them. In terms of inequality, I'm worried about this. There's an inherent social contract in democracy where ultimately the ordinary person has a certain amount of leverage because they're contributing to the economy. If that leverage goes away, then it's hard to make democracies work and harder to prevent concentration of power. So we need to make sure that the ordinary person maintains economic leverage and has a way to make a living, or our social contracts won't work. And that's why I think it's important.
During testing, your company's latest chatbot, Claude 4, was capable of what was described as extreme blackmail. Specifically, I understand it threatened to reveal an engineer's extramarital affair when it was told it would be taken offline and replaced. This was a simulation. Were you surprised by that? And what does that mean? I mean, that freaked me out when I read that.
Yeah, so just to be clear and to put this in context, this was a behavior that emerged during extreme testing of the model. To make an analogy to cars, this is like deliberately putting the car on the iciest road possible, messing with the brakes, doing that so you can see if you really stress test the thing, then you can make the crash dummy blow up. This isn't something the model does in practice in actual usage. But the reason we stress test the models this way is that it's the best way to test for and prevent problems in the real world.
Yeah, no, exactly. You want to turn every adversarial condition up to the max all at once, and it's only by testing in that way that you get the model to not do that in the real world.
So I wasn't surprised by these behaviors, but this is an example of how we have to be very careful in how we take control of AI systems.
Does that pretend AI becoming self-aware, and could that lead to dire consequences?
I certainly don't exclude the concept. We have a couple people who are actually working on this topic, as crazy as it sounds, of whether AI systems have morally significant feelings. I would guess that they don't right now, but when you run an AI company, these crazy-seeming questions are things that you study. So I would say this is probably not the case now, but again, the field is advancing so fast that I don't think you can exclude even crazy-sounding things like this.
What are practical steps people should take to be prepared? I mean, ordinary citizens, lawmakers, what do you advise?
Yeah, particularly lawmakers, I guess. Well, let's take them one by one. For ordinary citizens, I think it's very important to learn to use AI, learn to understand where the technology is going. If you're not blindsided, you have a much better chance of adapting. At least in the short term, we should take it bit by bit, where everyone learns to use AI better, and that speeds up the adaptation that is definitely going to happen eventually, but it'll be less painful if it happens quickly. For lawmakers, I would say really, really be aware of these issues. We're trying very hard to produce data on the economic impact, not taking off the table some fairly radical notions. I wouldn't exclude the notion of levying a tax on AI companies. If AI creates huge total wealth, a lot of that will by default go to the AI companies.
Well, he says AI can lead to incredible advancements like medical breakthroughs and boost the economy. He believes it could also lead to half of entry-level white collar jobs disappearing and 10 to 20% unemployment in the next 1 to 5 years. It's something that's been talked about within tech circles for a while, but rarely so starkly and so publicly. I spoke to him just before air about that and other implications he thinks AI may have on society. Dario, you've said that AI could wipe out half of all entry-level white collar jobs and spike unemployment to 10 to 20%. How soon might that happen?
Well, first of all, thanks for having me on the show. Just to back up a little bit, I've been building AI for over a decade, and I think maybe the most salient feature of the technology and what is driving all of this is how fast the technology is getting better. A couple years ago, you could say that AI models were maybe as good as a smart high school student. I would say that now they're as good as a smart college student and sort of reaching past that. I really worry, particularly at the entry level, that the AI models are very much at the center of what an entry-level human worker would do. It's hard to estimate exactly what the impact would be. There's always this question of adaptation. These technology changes have happened before, but I think what is striking to me about this AI boom is that it's bigger, broader, and moving faster than anything has before. Compared to previous technology changes, I'm a little bit more worried about the labor impact simply because it's happening so fast that yes, people will adapt, but they may not adapt fast enough. So there may be an adjustment period.
You are running an incredibly important company in AI, and you know this better than anybody, or as well as all the names we know—Sam Altman and others who are working, Elon Musk and AI. Why are you raising the alarm? Because it's not necessarily in your best interest. A lot of the messages we hear from some AI CEOs publicly is a little bit more calming, saying like, you know, these agents are going to be great in your life, and yes, there may be problems, but writ large, this is a fantastic thing.
Yeah, I mean, I think the reason I'm raising the alarm is that I think others haven't as much, and I think someone needs to say it. To be clear, I recognize it's always a difficult balance. I'm aware of my position, that I'm building this technology while also expressing concerns about it. The reason I'm doing both is, one, I think the benefits are massive, and we need to find a way to achieve the benefits and mitigate or prevent the harms. The second thing I would say is, look, there are six or seven companies in the US building this technology. If we stopped doing it tomorrow, the rest would continue. If all of us somehow stopped, then China would just beat us, and I don't think China winning in this technology helps anyone or makes the situation any better.
I do want to read something that Sam Altman, your former boss at OpenAI, said in September. He was saying that AI can cause significant change in labor markets, good and bad. He went on to say that most jobs will change more slowly than most people think. And then he said this, and I want to read it word for word: 'Many of the jobs we do today would have looked like trifling wastes of time to people a few hundred years ago, but nobody is looking back at the past wishing they were a lamp lighter. If a lamp lighter could see the world today, he would think the prosperity all around him was unimaginable. And if we could fast forward a hundred years from today, the prosperity all around us would feel just as unimaginable.' For younger people, I should say, lamp lighters was actually a job—people went around lighting lamps on the streets. I think a lot of people may not even know what a lamp lighter is. But do you think Sam Altman is wrong?
Yeah, so I think there are some things I agree with about that, and then there are some things that I think are too optimistic. I definitely agree that if we succeed, AI can grow the pie greatly. I wouldn't be surprised if economic growth is much higher than it is today as a macroscopic phenomenon, that AI allows society as a whole to be more productive.