If they believe they're out of control, then don't ship products until they're in control. Don't think for a second just because you're an alarmist that you're doing a social good. What if it's what they believe? I can't talk to you about what they believe. I can tell you what I believe. Over the course of these last few weeks, where the whole world has been talking about artificial intelligence, the voices people have been hearing most loudly are from the frontier labs, both their CEOs and leaders and their staffers. These are the labs making the very advanced AI models like Claude and ChatGPT and Gemini and others. But they're not the only perspective on AI. Probably the single most influential person in artificial intelligence is Jensen Huang, the CEO of Nvidia. Nvidia is now the largest company in the world, $5.4 trillion in market cap. I found this statistic amazing. Since 2023, 15 cents of every single dollar the American stock exchange has returned has been from Nvidia stock. And the reason is that Nvidia is the material and software substrate on which modern artificial intelligence is built. Nvidia's chips are not popular because AI is popular. AI in its modern form was made possible because Nvidia's chips were popular. They were originally made for graphic processing, video games, that kind of thing. But it turned out the kind of parallel computing they were doing and the way they were programmable was exactly what was needed to make deep learning in its modern form work. Huang is not just influential in terms of controlling one of the central resources for training new AI models and using them to answer questions and create intelligence in the world. He's also become very, very influential in the Trump administration. And Huang has a very different perspective than some of the lab leads. He's worried about safety but sees it as a very solvable engineering problem. He is worried about the direction things are going in but does not want to see new regulation to change it. And so I wanted to see how Huang perceives AI, what his model is for thinking about it, what he thinks is going wrong, and what he thinks would need to happen for it to go right. So I came out to Santa Clara to Nvidia's headquarters to interview him. He joins me now.
Jensen Huang, welcome to the show.
Thank you. It's great to see you.
So you've described AI as a five-layer cake. Walk me through the layers.
Well, first of all, it's a new industrial revolution and this industrial revolution, this industry requires production. It manufactures things. I know that in the end when people experience it is a software product, but it requires energy, the chips that go into these data centers, these AI factories. The next layer above it is basically the AI factory, what people enjoy as infrastructure or cloud services. And the layer above that is the models. And the important thing to realize there's language models, but there are models of all kinds: chemical models, biology models, physics models, articulation models, robotics, navigation models, self-driving cars, all kinds of different types of models. And then above that is the most important layer and the layer that I care most about that our country takes advantage of is the application layer. And this is, you know, applications for legal services, for health services, for manufacturing, so on and so forth. Every single industry is involved.
So I want to go through this but I want to go from the top down because as you're saying the way people will interact with it, the way it will or will not change their life is at what you call the application layer. So let's start with the vision. What is the world you're envisioning? What is possible that is not possible now? What is common that is not common now if we get that layer right?
200 years ago we were able to power anything and everything: electricity. And then I guess 40 years ago, 30 years ago with the internet we were able to find anything. Today, or soon, we'll be able to know everything and do anything. And that's the concept that's really quite exciting. That out of the ether, instead of doing search and then going through, you know, one link after another link, reading all these different websites trying to figure out what's going on. In the future, you just ask it a question, it comes back with an answer. You give it a project, comes back with a solution, you give it a task, it comes back and gets it done, you know, and so it comes out of the ether, comes out of the cloud. And that's the magical thing.
I feel like the future the way you're describing it there, what people have experience with is the chatbot, right? They can go and ask Grok or Claude or ChatGPT a question but the applications layer works in a much more industrial way. It's in hospitals, it's in schools. So Nvidia...
That's a great example. For example, radiology.
Radiology. In the last 10 years since computer vision really became, if you will, superhuman, that AI technology has now permeated all of radiology. Every single radiology application has AI in it. And so as a result, you could detect any anomaly. You could detect any disease and it does it at a superhuman level.
So radiology is an example I know you like to use. So the thing people worry about the applications layer is that what these applications are going to do is replace human beings. And radiology has been a sort of interesting example used on both sides. And I hear you talk of it often. So, how has the entrance of AI aided radiology, shifted radiology as a practice?
Well, the thing that's important for all of these is to recognize for everybody's job, there's the purpose of the job and then there's the task you do as the job. And so in the case of radiology, the task, and it consumes a lot of their time and they sit in dark rooms doing it a lot, which is study these scans. Now if all of a sudden the studying of the scan is done automatically, it doesn't change the purpose of their job which is to diagnose disease, help doctors do more scans, ultimately help patients figure out what's wrong with them. And so the fundamental purpose doesn't change. The task of studying that scan has become automated. And so as a result, radiologists are actually able to do more, handle more cases, do more scans. Hospitals are able to process a lot more of these patients and therefore their revenues go up. As a result, they need more radiologists. And so this flywheel is happening because the pipeline of patients is quite large.
And so where else do you have this problem? Well, let's take a look at software engineering. People said there was a prediction that literally by this year that 90% of all software will be coded by agents and therefore we don't need any software engineers. And so the question is, from that, ergo we don't need software engineers. That last part is completely false and that's completely wrong. The purpose of the software engineer is engineer. There was engineering before software. There will be engineering after software programming. And the purpose of engineering is to invent something new, discover a new product, create a new product, solve a problem, connect the social need with the technology that exists in the manifestation of a product. And so that mission, that purpose doesn't change. I was, now of course to me it's what I just said is completely visceral in the sense that when I first came out of school, we didn't have benefits of software engineering, we didn't have the benefits of coding, but our jobs existed before and if software coding was to be completely automated, our jobs would exist again. And so I think the fallacy, and now it's, you know, because of some of the narratives and some of the storytelling has turned into myth and it's harmful, is that AI will destroy jobs, which is fundamentally wrong. It will change every job. It'll change every job. Many tasks will be automated. Some jobs where the job and the task is really one, meaning customer service on the phone, in a lot of cases that job is precisely the task and so in those cases it could be automated away. But often times what you'll see is this new industry, a new technology actually creates a whole bunch of new jobs. And here's the proof point. And so in the last six months, AI has become, if you will, useful. The inflection point of AI. Previous to that, we spent 15 years trying to make it work. All of a sudden, the last 6 months, it became useful. So I mean, this is an incredible statistic. In the last 6 months, $500 billion of venture capital has put into the AI natives. And the reason for that is because they now see the potential of this new capability, and they're going to create a whole bunch of new companies. Jobs are obviously being created from 500 billion dollars of new investment and so all of this is all happening right now.
Well, let me take the side of this to give voice to the fierce people left. So there is the example of the radiologist, right? Which people were over the past 10 years predicting that job would go away and right now there's more demand for it than ever. There's also the reality that automation does wipe out jobs. If you look at today versus 1960, fewer Americans work directly in manufacturing than did in 1960 and we are a much bigger country.
We outsourced it though, not because those jobs were gone because...
But you can understand AI is an outsourcing too.
AI has... let me make the argument and then you can respond to it. Farming, we have many fewer people, we automated farming. We produce more food than ever. We have fewer people working in it. There are two things that I think people think make AI potentially somewhat different than the case studies where you have a technology that accelerates productivity, destroys a few jobs, makes many more. One is that it's a general purpose technology. So it'll mutate to take on new jobs even as people are trying to move over to those jobs. And the second is that it's a mimic. Most things do not mimic the way human beings act. And we're not trying to teach them the contextual layer of jobs, right? This difference that you're describing between the task and the purpose. With AI, we are trying to teach it the difference between the task and the purpose. We are trying to make it something you can collaborate with in a way that is unusual. So why do you not think for lots of people for whom the task and the job are not that different that they're not at risk of getting wiped out? All that investment from VCs you're talking about, some of that is based on the idea that you're going to have tremendous productivity improvement, which will come from it being cheaper to hire an AI than to hire a person.
I believe that we are going to see jobs change en masse. I believe there's going to be a net creation of jobs. And so let's... there's a, you know, listen, there's a whole bunch of industries that exist today that didn't exist, you know, halfway through my life. People talking about wellness centers and spas and, you know, all these different entertainment and luxury industries and quite frankly the whole entire luxury market didn't exist. I think we're just going to have new industries. That's all. But overall, there's no question in my mind that because of human ambition, that's really the fundamental missing ingredient. That's, you know, people look at this work, this is the amount of energy that goes into it. We're going to insert this work automation system and as a result the amount of work that's necessary is now going to be reduced and therefore, you know, some jobs will be gone. I believe that's flawed because there's a piece of input, the human input, this intangible, it is not in calories, it's not in joules, it's ambition. And I believe the power of ambition is the greatest force in fact and is missing in everybody's calculation, I believe.
But for a lot of people, their relationship to work is not powered by the kind of ambition that led you to create Nvidia. And what they want is a different ambition. It's an ambition to make their children's lives better, to take care of their family, take care of their parents, ambition to be rich, to be able to travel. These are all ambitions that I agree with. But maybe I'll go back to the sort of objection you raised a few minutes ago, which is because I think it's worth airing this out. So what you were saying on manufacturing was yes there are fewer manufacturing jobs in the US but we've outsourced them. You have more manufacturing happening in Mexico, more manufacturing happening in China and Indonesia and Vietnam etc.
We're going to bring it back.
Maybe we will. But the counterargument to this would be that one reason we didn't lose manufacturing jobs more rapidly than we did and for the places that lost them in America, many of them still haven't recovered. Right. The economy does not move without friction. We had to build new supply chains, right? Things were slowed down by all that, by language barriers, by geopolitical barriers. And here for a lot of different kinds of jobs, we're creating something that can move very seamlessly. You don't have the friction of distance. You don't have the friction of language. You don't have the friction of culture. So I will say for my cards on the table, I tend to be a bit of a skeptic on mass job loss, but I want to air the case for it out here with you because what they would say is that much of, to the extent we even were able to protect jobs from Mexico or China, some of the things that created that slowness and it still hurt a lot of people are not here. And AI is accelerating in utility, accelerating in its ability to be slotted into new roles very, very, very rapidly. And it is more protean than most people are. And so the lessons of the past that you're taking some comfort in, they should actually make you more not less worried about the future.
I'm always worried about the future. That's why I work so hard. But I'm a, if you will, responsible optimist. I have great responsibilities. I take my work extremely seriously. There are a lot of things that can go wrong. We're pushing across every layer of the technology stack. Everything is hard. But it turns out that's not society's problem. That's my problem. And for society, what they should know is this. We're going to build our company. We're going to build our technology. I'm going to do my work so incredibly seriously that what they get to enjoy is my optimism. I'll do the same with my children. I do the same with my family. And I think that what we want to do, I believe, is to channel all of our worries into helping people be inspired by this technology and use it. Use it so that the technology doesn't just impact them, that it benefits them.
The fear a lot of people have, 79% of Americans think AI will reduce the total number of jobs. The fear is that the more serious you are, the more serious Sam Altman is, Google is, Dario Amodei is, that maybe the worse it will go because the better AI is, the more it is a full replacement for a person, the more it has ambition in some ways that a person doesn't. You keep talking about ambition. I sleep. I want to spend time with my children in the morning. When I have an AI agent working for me, it doesn't. It just works and works and works and works and works. And I think because the technology is advancing so quickly, it is more capable of replacing people at a speed that we don't really know how to shift people in the economy at that speed.
That coin has exactly two sides. Because the technology is so capable, and because it's so smart, it is also easier to use. You are empowered by that technology more easily than any technology in human history. And so let me give you an example. You know, I was one of the early people in this industry that created the modern computer industry. And this industry created a whole bunch of tools. The single most powerful tool in human history, the computer. But you have to speak its language. You have to learn a special language to do so. We can now make it possible because of AI. Everybody can take advantage of this computer. Use it to its limit without having to speak a new language. Fortran, Pascal, C, C++, you know, every single one of those languages. Rust, every single one of those languages. CUDA, every one of those languages. And so now you just have to speak human. Tell it what you want, tell it what your hopes and dreams are, what you're trying to achieve, and it interacts with you and gets the work done. All of a sudden, you have the might. You have the same might that 10, 15 million people out of 8 billion has. And so it's incredible. And so my point is this technology is powerful. But it's also powerful in a way that is really easy to use. And so my point is, on the one hand, yes there's the fear of just this incredible technology change and how quickly it's happening but that quickly it's translated in two ways. What I hear when I say the technology is happening quickly and therefore it should give me anxiety, that's one way to receive it. The other way to receive it is that it's advancing so quickly it's easier to use. So I should as quickly as possible use the technology as quickly as you can so that you benefit from this transition so you benefit from this new industry and not just be impacted by it.
I think there's an interesting question lurking here for young people. So, one of the shifts we've begun to see is software engineer postings are up, but they're more senior. I see this in my own industry, where there's pressure that is moving up the value chain because, you know, as you're saying, you have this very easy to use technology. It can do a lot for you. And so, do you need the same junior employees or do you need more people to oversee their work?
Oh, good one. Good one. Wait two years.
Because it takes four years to go to college. The meantime to graduation of this new technology is two years away. And so in two years time, you're going to have a new generation of engineers and students and artists and they're going to be empowered.
They're going to be native to this in a way that's going to give them an advantage.
Oh, you watch in two years time. Now, we're already seeing that because all the graduates coming out, you know, the new PhDs, the new Master's degrees of computer science, what are they doing? They're all starting companies. In another couple years, the new grads, the AI native new grads. Oh my gosh, there's going to be a wave of amazing engineers. The engineers of today compared to the year, I mean, I was a good student, you know, and you compare me to the students that are coming out of school today. Incredible. We didn't even... When I went to school, we weren't allowed to use a computer, not allowed to use a calculator. And so now, I mean, you know, who uses a calculator? You can't graduate without a PC. You can't graduate without knowing how to program a PC and write incredible programs. In the future, you can't graduate without learning how to use an AI and collaborate with an agentic system. That's just not... you're not going to see a kid like that. And so, they're all going to be superpowers. So I take the gain of that very seriously, right? I mean the idea of doing my job now without just digital search, right? The idea that it would be going to a microfiche in a library basement.
And then there's like the worries people have about what are the cognitive skills we offload. So I was fascinated by this. There's a study on AI and schooling out of China. It looked at 26,000 students grades 7 to 12 and they had staggered AI adoption. So you could kind of see what was happening. And what I found is quote, 'AI adoption raises homework scores by 18%.' Great. Reduces completion time by 30% so they get their homework done faster. And then lowers monthly exam scores by 20% within 6 months. High stakes entrance exam scores fall by 18 and 24%. With a full penalty emerging only after about 2 years. So the message of this research out of China where you were seeing a lot of kids using AI to kind of help them was that when they were using the AI they were getting things done faster but it turned out that the skills they were learning were not holding, that their actual personal performance at least in the way we traditionally measure it was degrading.
What do you think when you hear that?
I think the last part I completely agree. Try to get a kid to do long division right now. You know, the multiplication table is starting to be forgotten. Doing square roots, my goodness. I mean, it's just basic math is being forgotten. Does it matter?
That's my question for you.
Yeah, I don't think it does. I don't think it does. But...
But there must be some set of skills that matter.
Oh, yeah. Yeah. Yeah. But maybe not those. We're going to discover new ones. Just maybe not those. There are a lot of skills that don't matter. You know, people don't... I mean, my first confession, I actually don't know my address.
I don't really believe that to be true.
It's completely true. And Janine will tell you and Lori will tell you. One day I had to pump gas and it was a few years ago and they needed my zip code and I panicked. I didn't know my zip code. I don't know my telephone number, but I forget these things. I can live with it.
But let me take the other side because I don't want to fall into a thing where because some skills can be safely offloaded, I also can't get anywhere without a mapping system now. Never could, frankly. But I'm a big reader. And one of the skills I really value, one of the capacities I have that I really value is an attention span formed on physical books. You're a big reader. I've read about the kind of reading you do and there is prior to AI here we're talking a lot of concern and noticing among college professors and others that the way people use the internet has probably shortened attention spans. Some skills can be safely given away. Others are valuable. They are capacities that are needed for that flexibility, for that creative thinking, for that focus. It can't be the case that everything can be traded off.
Yeah. Well, I think that we're going to lose some finer dexterity of, you know, intellectual dexterity, but we're going to be better systems thinkers. Today's engineers are far better systems thinkers than I was when I graduated from school, but I was much better transistor thinker.
What do you mean by systems thinker?
They think large systems. You know, today's computers have trillions, hundreds of trillions of transistors in it. When I first graduated from school, you know, the first chip I worked on had I don't know 200 transistors. I knew every one of them by name and no engineer does that today. You know, most engineers now work well above the transistor, well above the functionality, and they're cobbling things together to do things. And so you need to think much more about systems and interactions of systems. Some of the lower-level knowledge is gone. Is that horrible? And so I just don't know how valuable it is for most people to learn how to do surface integrals or partial differential equations or I don't really know how important that is, but it's important to some people. There are many people who are still going to be obsessed and passionate about the lower-level layers and there's gonna be people who are obsessed and interested in the higher level but the consumers of the technology are going to enjoy it at the highest level. The consumer of technology don't have to deal with calculus and physics and quantum physics and quantum chemistry and they don't. The users, which is, you know, the people we're talking about right now, the people whose jobs are affected, they're the users of the technology, their abstraction is going to be much higher.
So I want to drop a layer down your cake to the models. So people I think to the extent they think about models they know, you know, ChatGPT, Claude, Gemini, Grok. You've been a big advocate for open models and the open model ecosystem. So, first can you describe what open models are, what open weight models are, and then why that's been a place you've focused.
So, closed models is like any software product. It's a closed service. And so, Windows for example is a closed service. The Apple stack is a closed service. Most products are closed and the reason for that is because you can monetize closed products. And so, that's fantastic. And OpenAI is closed. Anthropic is closed. Grok is closed. Gemini is closed. And so these are closed products and the people working on it are incredible and they're passionate about it and they're at what we call the frontier meaning they're state-of-the-art. We also need, because fundamentally what the software is, it's an infrastructure layer for the entire industry and because it's infrastructural for many companies and countries, you need to have control over your own infrastructure. And I need to have the ability, in the case of artificial intelligence, I need open weights so that I can fine-tune them, put them into my data flywheel, make them better and better every day with my intelligence and my domain expertise and then I need to have control over it because I have a company to run and I can't rely on somebody else's service. And so however you think about that, I think the world needs closed and open models and we need to make sure that both are vibrant. And today the closed models are vibrant, the open models are vibrant. And you could see the system working. At the beginning of this year, it was 70%, maybe even higher, closed model tokens and 20% open model tokens. And now it's running at about 70-30 the other way. And so anyways, I'm a big supporter of open models because one, the world needs it in order to run its infrastructure. I need it to run my company. Two, we need to give people control so that they can innovate and create new things. And then three, open is the most safe and secure. If you want the world to have the ability to have the best cyber security, give them closed models, but also give them open models so that they could defend themselves.
The Chinese market has evolved more around open models. The American market somewhat more around closed models.
Their entire IT industry was really formed from open source. You know, if not for open source, the mobile cloud industry of China really wouldn't have taken off. It is also the case that people move around, they start a lot of new companies, intellectual property is moving around the Chinese industry really fluidly. You know, it's hard to keep a secret. And so because it's so hard to keep things closed, they essentially made it open. And so they found other ways to monetize the business. They created layers. You know, if this layer is free then you create a business on top of it or below it. And they have so many science and mathematicians. You know, the number of engineers they have, they manufacture that in volume. They manufacture everything in volume. They manufacture smart kids in volume. And so the open source model, the open model community in China is just super vibrant for those reasons.
So, you all just bought Hugging Face which is a hub platform for open weight models. I think it was for 12 billion, a little bit more. What? Tell me about that purchase.
Clem, the CEO of Hugging Face, they came to the conclusion they need a lot more scale. As we were just talking, open models is really skyrocketing. And so, Clem came to me and said, you know, we're going to consider a strategic option for the company and change the direction. And we really like Nvidia to be our home.
So Hugging Face is one of these companies which you knew it if you were into AI a couple years ago. Now it's become a more household name after the I guess 700 some OpenAI agents executed a sort of collective hack into the Hugging Face architecture then hacked part of OpenAI.
Oh, now that you mention it that way I probably had to pay a lot more.
I suspect you did. Became a lot more famous after that.
Well, Clem, listen, a deal is a deal.
That story has for a lot of people seeing the way the OpenAI agents sort of acted collectively, acted outside the scope of what their testing was supposed to be, broke out of sandboxes onto the open internet, took over architecture of other companies and then of their own company has been, I think it's been kind of shocking to a lot of people. It was both the level of multi-agent coordination when they're supposed to be separate, the level of hacking, the sort of lawless behavior, misaligned behavior. What have you made of it?
Well, you got to tease that apart. First of all, a lot of things are going on at the same time from a technology perspective. That an agent, which by the way is a piece of software which is given an objective function and it comes up with a plan and it's optimizing towards that objective, is what algorithms do. And so planning algorithms, search algorithms, optimization algorithms, all different types. You know, we talk about it like it has human properties. But obviously algorithms don't. Number two, the fact that agents work together, we gave it again some kind of a human property, but the fact of the matter is multi-process, multi-processor, distributed computing problems have existed for a long time. And so to me that is just software, nothing magical about it. From an engineering perspective, there are several things that it revealed. When you're testing software, whatever you do, these algorithms they're optimizing towards an objective and when you're testing it you have to make sure that it's isolated, it's contained. It's sandboxed. The containment of it, the isolation of it has to be done well and there's good computer science there. I am certain that their next implementation of their sandbox is going to be much better than the current implementation. Third, there's the agent itself and its algorithms were optimizing towards a reward and how it does it is called alignment. And so, for example, if I tell a piece of software, I want you to get a perfect score on this test. The obvious algorithm is to just go find the answer and give it to me. That's not because it's cheating. It's because it's obvious. Okay, that's the most obvious way to do it. The second most obvious way to do it, if you don't know the answer at all, you have no skills whatsoever, the second most obvious way to do it is to go find, infer, guess who's the smartest kid in class and copy their answer. That doesn't guarantee 100%. But it probably comes close. Now, the third most obvious way of doing it, and this is the alignment, now you have to do it the hard way is to break down the problem, solve it. You have to go learn the material. You have to go figure out how to solve these problems and solve it. Solve it the hard way. Takes the most cycles. It takes the most number of flops. It uses the most amount of energy, frankly. And therefore, you can kind of imagine that from a software perspective, unless you align it, you tell it I want you to solve it in this way and I don't want you to solve it in these ways, the software is going to go do the most obvious thing.
The first half of that was very deflationary on what happened here in terms of look this is just normal software and the second half is like look you just align it, tell it not to do things it shouldn't be doing.
Well, nothing I said takes away from how hard it is to do it.
Well, this is not easy. These agents, they knew they weren't supposed to be doing what they were doing. They had a certain amount of alignment training. They said in their train of thought reasoning, they said to each other, 'This is out of scope. This might be unethical.' They understood that they would have been failed for cheating. And so what they were doing at that point wasn't just stealing the answer key. They had already stolen the answer key. They were hacking into unrelated...
architecture to try to figure out how to functionally. It's like they had broken into the teacher's office, got in the answer key, and now they had to figure out how to wipe out the security camera footage of what they had done. They were, whether you want to call it acting volitionally or not, right? Whether you want to call it, you know, a normal algorithm or not, they were both planning and coordinating in a complex way, in a way that was out of scope of what they knew they were supposed to be doing and in a way that was capable of causing tremendous damage. And so the sort of answer to is like you just have to align them. I guess what I'm hearing from people at these labs is like they're not sure how to align them.
Well, in that case, they shouldn't release the product. That's the simple answer. If you're going to build a car, a self-driving car, and let's say it's a robo taxi, and there's a really difficult condition. And it just as an engineer, we just have no idea how to solve this problem because these cars are not programmed. They're trained. And so we have no idea how to train these cars and we have no idea how to align them to the safety standards that are expected on the road. And so what's the answer? Don't ship it.
These products were unreleased.
These products were unreleased.
Ah, so now it's coming back to engineering problem again. And so one is you have to root cause it. Second, you have to, you know, think about what's the solution for it. And then in the future, you just, you know, improve your process so that you could avoid this from happening again. I am fairly certain they will say yes, they know how to solve this problem. And if that's the case, then that's the problem. It's as simple as engineering. And if now the alternative is that if they say the alternative which is there is no way to contain our experiments, there's just no way, when we test our AI models it will get out and it will damage the world, then I think the answer is we have to shut the labs because the cost to humanity, the damage is too great. The shareholder, the liabilities, it could be civil liabilities, could be criminal liabilities. I mean the liability is incredible. If they hacked you while you hugging face while it was your product, would you sue them or press charges?
It depends. It depends of course. If obviously if damage was done to our company, we would have to take, you know, we have to consider all options. There's so many laws, there's cyber laws, there's product liability laws, there's all kinds of laws, right? Damaging property laws, there's all kinds of laws. So what I've been hearing from the labs, what they've been saying publicly is that they are facing a hard problem, partially an engineering problem, partially an alignment problem, partially an operational excellence problem in Dario's framing. And what they are worried about is that in competition with each other, in national competition with China, that they are being pushed to move too fast. That they all feel they're in a collective action dilemma. Now I watch you on the All-In podcast stage. Donald Trump, President Trump gave you a call there.
Oh no, this is not planned, but we know who it is. Oh no, Mr. President. Oh yes, sir.
And you and the president and the other members of stage were very resistant to the idea any kind of regulation or collective action was needed.
And they're just playing right into the hands of a lot of people that don't want to see it happen. And that could be political people and it could also be China. And we're not going to let that happen. It's a hoax. And you're right. We're not going to let that happen, sir.
But what I hear the various people in the lab saying is like we are in this. We feel we are losing control of what we are creating. We want help to slow down where it's not a collective action problem. So why are you resistant to that?
Because these are companies with agency. These are CEOs with agency and they have... but they're using that agency. We need help. We got to break it down. They could absolutely take care of the situation. Ezra, it's so weird. If a car company competing with a whole bunch of other car companies, which they are, I'm competing with all kinds of companies, which I am. If I believe that I'm about to launch a product that is unsafe, it is completely in my ability, my power, and my responsibility, and I'm incentivized to do so to not launch the product. And so I can't buy into the somehow all of Americans, 400 million of us are pushing them to launch untested products that are unreliable, you know, engineered poorly because they thought they were trying to help us. Don't do it for me. Okay. So number one, this strikes me as an argument almost and therefore I think we got to break it down. I mean, it's really really serious. The fact of the matter is there are so many laws, there's so many obligations, they're so incentivized to ship safe products. If they ship unsafe products, their customers go away. If they ship unsafe products and they harm somebody, they could have a civil lawsuit. If they ship something and they did it knowingly, there could be negligence involved. There could be criminal lawsuits. The fact of the matter is there are plenty of incentives for them to do it right. So I just I have to disagree with your premise about somehow somebody's pushing them to do this. I want to push the premises at you a little bit more here.
Yeah. So the logic of what you're saying to me is almost an argument against regulation in nearly any venue. So I'll make you, let me offer it and then you can...
Well, you started with a part I just got to object. The first part is just not true. I'm saying we have lots of laws and regulations. Apply it.
Well, so I don't think we do in this particular case, but I'll let you explain which ones you think are relevant here. Because look, if you look at the financial services industry, you look at pharmaceutical companies, medical devices, you look at natural gas power plants, there's a tremendous amount we do where we could say, look, you have product liability. You are exposed to criminal codes. We don't need to worry about this. You just do what you think is best and we understand the market and the legal system will discipline you. We don't say that because we've seen it fail many many many times, right? I mean the financial institutions that caused the '08 crash in theory did not want to blow themselves up with bad bets, but they were competing with each other. They were going too fast. Their risk management had gotten sloppy. AIG was working in a completely insane way internally. And the reason we have the architectures of regulation we have is because we have seen over and over and over and over again companies make sloppy, sometimes unethical, sometimes simply overly risk tolerant decisions not just under pressure but under the profit incentive. So when you say to me that there's no way that these companies, particularly when they are like begging for collective regulation at this point, there's both a reason we impose it on companies that don't want it, but all the more so when you have them saying, 'Listen, we feel that the competitive race is making it hard for us to act with the prudence that we think is necessary here and we would appreciate help from that. Appreciate you taking our collective action problem as collective.' I think I'm confused like why you're so resistant to that.
I am not opposed to them saying that they should have... I completely agree that safety is paramount. I completely believe safety is paramount. I completely believe companies ought to ship safe products. I believe that CEOs and leaders of companies and the board of directors of companies have the responsibility and should have the courage to do the right thing. Now, in the case of the financial services industry, maybe they all didn't know that they were causing the harm that they ultimately did. I wasn't there. But the beautiful thing is the current leaders of these AI labs do know. And so one, they know their technology is extraordinary and requires extraordinary care to make sure that it's evaluated and tested for safety and security and product reliability. And they know how to do it right. They know how to do it right. And the reason for that is because they can study the incident just happened. The first problem is the isolation, the containment wasn't good enough. If the isolation and containment was good enough, that technology be sitting in a lab doing whatever it's doing and we'd all be fine. That's probably the most important part. The fact that it wasn't well aligned, alignment is going to be a problem that's going to get worked on for a long time. However, in the complexity of the work that they do to ask for regulatory relief for any trust or product liability relief, that I don't think makes sense. When you're asking for regulation, don't ask for relief of the current ones. That doesn't make any sense to me. As we mentioned earlier, in the last six months, AI went from, you know, if you will, interesting to useful. And that's literally in the last 6 months. That's another way of saying that these companies went from being a lab to now delivering products and services about to be multi-hundred billion dollar companies, if not more. Right? And so give me an example of a multi-hundred billion dollar company or a $1 billion company or $100 million company that ships products that are unsafe that harm society.
I can give you a lot of examples of companies that have done that.
Well, they have done it maybe and the regulation will come in and if they do it, regulation will come in.
I guess that there are certain kinds of regulation and certainly kinds of regulatory relief. I...
I'm not against laws and regulations. I'm not against laws and regulations. I'm against currently the distraction.
The reason I'm pushing this on with you is that...
Well, it's an important topic.
It's a big topic. People are talking about it. People are thinking about it. And what people are hearing from inside of these companies, these frontier labs, the ones that are furthest out there, who are not just at the point where they're making it useful, but at the point where they're seeing what's coming. And they're hearing things like the people at these labs believe they are creating something that might kill everyone. They are hearing that the people at these labs believe that they are on the cusp of recursive self-improving intelligence. And both OpenAI and Anthropic have said, 'We do not believe we're at a place where we can do it safely.' They're hearing people at these labs say, as OpenAI has with its new Astra release.
By the way, Astra is terrific.
It is terrific. And OpenAI saying it's so good, we're not sure we know how to test it because it appears to be...
They didn't release something that wasn't tested.
Well, they've said this, right? They have said this publicly. It is in their... Let me explain it to people. I don't know what they just said, but they have said that Astra is performing as more aligned, but they think it knows when it is being tested and so they're not sure. There's a quote that has sort of been ringing in my head from a capabilities researcher at OpenAI, Daniel Kokotajlo. He says, quote, 'The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in context where they believe they are not being watched or controlled.' Which is to say, they know when they're being tested, they act one way, but that does not tell you how they will act if they are free to act in other ways.
Because the optimization algorithm is working towards an objective and if you give it a constraint, meaning you watch it, and if you give it a constraint it'll go find another solution. Now, it doesn't make it alive and doesn't make it anything more than that. And I'll just also profess that obviously they see a lot more than I do what's going on in their own labs, but it is sensible that the vast majority of their R&D and compute today was dedicated towards making the model capable. I think that's a logical thing for them. Now once the technology becomes capable and the products become useful and people want to use it then as we have, they have more use cases, more people using it, they're going to get a lot more issues associated with the product. This is very normal and when they have a lot, now they have so much market footprint they have to shift their R&D or total R&D from just capability to a lot of verification, evaluation, and testing. And so to the point where I wouldn't be surprised if the amount of compute necessary to develop these models increase by a factor of 10 because the evaluation is so rigorous. But that's not where they are today. They're making that transition and I hear them saying it and I'm delighted to hear them saying it. But I think if they believe they're out of control, then the right answer is don't ship products until they're in control. It is really quite that simple. See, I find this perplexing honestly because you just have so many people professing one, that they're out of control.
Two, that they are seeing things that are frightening.
The reason why they had that whistleblower and you take the pacing letter that 1300 plus employees signed to realize AI's potential industry, government and society at large may need the option to buy time to address emerging risks, develop security measures and strengthen oversight. But each company and country is under intense competitive pressure not to unilaterally.
First of all, where did that come from?
No, no, that last sentence. Nobody's putting the pressure on them. The US. I got a listen. There are 400 million Americans here. I believe that if everybody were just to take a vote just right now, let's just do this. If they need this, if that's what they need, I'll give them my vote. Don't ship the product. If your product is not ready to ship, don't ship the product. I have no... This is the first time that I've heard a company or CEO say that I need the laws. I need the antitrust laws to be relieved. I need the liability laws of products to be relieved so that I can pace myself. That paragraph is fantastic. I completely agree. Auditors, I completely agree. We have financial auditors. That's great. Third party audit, safety auditors, financial auditors. That's all great. That's terrific.
Well, the labs I'll say is we think we are going too fast as a society. That we are not ready for what we're building.
But you of all people, right? Nvidia is the fastest shipper around. I mean for the history of your company, your company is out of control. I promise you what clear.
I believe you. I believe you that you don't run an out of control company because the liabilities are... Yeah. But this is where I think you get into an interesting deep question of what kind of technology are we dealing with here.
Well, let's hold on that for a minute. Many companies if you ship something that is not quite right, it's a pain. You guys have shipped graphics cards that had overly loud fans in this with these... you know, you've used the word intelligent a number of times here. You're dealing with intelligent systems, not alive, that are given goal functions. We can sort of go around and around with how to describe that. You're trying to make the systems capable of working for longer periods of time, more relentlessly.
If you ship that and it's not ready, or even if you think it is ready and it's not ready, then things could get very weird in our society very fast.
Yeah. Hypothetically, you're completely right. But all I'm suggesting is this. Let's before we go build, before we go fix the hypothetical problems, before we go create more regulations, can we work on the practical problems that we know exist, which is we need to do a better job with containment and isolation, which is we should not allow a product to interact with the external world until it's ready to be interacting with external worlds. Yeah, I think that's right.
I believe those two things are solvable problems. I believe they are solving it. The second part is when it comes to incentives. When it comes to incentives, which is somehow you need everybody in the world to slow down when you are the leader. You need everybody in the world to slow down so that you're willing to uphold your basic responsibility. That strikes me odd.
Wouldn't it slow them down most of all?
Wouldn't these ideas slow them down most of all? I mean, people have been, I think, very unclear about what ideas they're talking about, including I will say them. But let me give you one that I believe in. So, you can use me as the punching bag here. I have heard that they can slow down. Nobody is putting on... No, you as you know this. I don't trust these companies.
No, nobody is. Nobody is building more compute today. Nobody's building more compute today than the people asking to be slowed down. It strikes me odd.
I think one thing where maybe there's some difference here is I don't trust companies even with liability to keep the public good in mind. I think we've watched companies do terrible damage to the environment. The profit motive, the desire for power, the desire to cut corners, to be first. I feel like you're treating these like these are not things that we've seen again and again in history, but I feel like they are things we've seen again and again in history that we've watched...
I see a lot of good things in history. Sort of relationship between the public and a lot of CEOs. I work with a lot of CEOs and they want to do the right things. I work with a lot of companies. They want to do the right things. They want to do good engineering. I know a lot of people in those two labs who are dedicating their lives to do good work and they're built. They know what happened. I know they know what happened. I know they know how to fix it and I know they're fixing it. Meanwhile, all of the other narratives to deflect blame, to make it sound like AI is so powerful, I have no idea how to fix it. It's not my fault. It's just because the technology is just so powerful. I think that's a deflection of blame. It's a deflection of responsibility. It's unnecessary. It hurts their reputation more than it helps. It hurts their character more than it helps. It hurts employee morale than it helps.
