Episode 43: Jensen Huang on Generative Computing, Re-industrialization, & Physical AI

Very happy to be here.

We're in a new computing paradigm. Artificial intelligence, if you break it down to the basics of computing, the way we used to do computing is essentially retrieval-based computing. We would pre-record information, take a picture, record a video, write a story, do a brief, so on and so forth, and we put it online. That's why you would put it in a data center, a center of data. And then based on whatever retrieval method you use, if you're shopping, it's probably a recommender system. If you're doing search, it recommended a list and you point at, you click at something. Or if you use YouTube, it just feeds you whatever it feeds you. And so then once you select, it would go and retrieve that file for you. And if you're a shopper, then it's very likely that based on your previous preferences and whatever context it is, where you are and from what other link did you come from, right, all the cookies you left behind, it might use that information to recommend one of maybe 16 versions of ads that's already created for you. It's all retrieval-based computing. So far, all of the way we largely use computing, we create content and we retrieve content. Well, the way that we do computing now with artificial intelligence is based on your context, based on your request, based on your intentions. It would then generate from some initial seed of information. It could be grounded in some fact. It could be grounded in a piece of report and it generates information for you for the very first time. And every single time it's, you know, largely different. And so it's generative computing. It used to be retrieval-based, now it's generative. The benefit of course is that every time you use computing based on your context, based on the changes of ground truth, based on your intention, intelligence will be applied and provide you the information that best suits you. And so that makes sense. Okay, that's the big breakthrough. And the reason for that, we need so much more computation instead of a bunch of storage. We need a lot of computers. That's the fundamental difference. Now of course the big breakthrough is because of that we can now, we have to learn how to perceive information of all kinds. It could be text, it could be video, it could be images. We have to reason about what it is that you're asking me, the context and your intent. And then I have to plan to answer your question accordingly. It could be providing you an answer or providing you a summary or maybe write you a brief based on all the things that I've researched or even do some shopping for you. And so we went from retrieval-based computing to generative and on top of that we now made it possible for us to do tasks.

The first time I saw a chatbot was shortly after Nvidia built Megatron. Before the first large language model was announced, Nvidia announced with Microsoft the first large language model. It was called Megatron. And Megatron was about 400, 500 billion parameters. The technology that we invented to make it possible is pre-training the models and together between us and Microsoft we pre-trained a really large model. At the time, this is probably late '91, mid '91, late '91, okay? So this was before ChatGPT. And the thing that was great about it of course was it was able to memorize a lot of knowledge but the thing that we didn't invent that OpenAI invented to make ChatGPT useful is the concept of alignment, reinforcement learning human feedback. And as a result, you know, you give it a prompt, the Megatron, you give Megatron a prompt, it would generate just a, it would start spewing off all kinds of things that it was encoded in its memory and it was just nonsense and it wasn't very useful. And we were waiting for another great invention which was reinforcement by human. But the first time I saw GPT I was mind-blown. I was so happy to see it.

Yesterday I was using it to finish my shareholder letter. And so I'll outline something and then I'll say go off and read everything that I've already read, all the things I've already written, all the keynotes that I've already done and based on this outline just populate it with things that I've already said, just give me a basic framework. And then from that I might take it and refine that or, that wasn't very good, I'll rewrite it altogether. And then I would give it to the AI again to ask it to make it tone consistent or highlight some things that it could do a better job on. Writing is so hard. I hate writing but you know with AI it was tolerable.

We build the computing infrastructure for modern AI. And so if describing modern AI is the new computing paradigm, then we are the computing infrastructure, the computing fabric that makes that possible. A computer consists of the chips, but the systems, the system software, the algorithms where it's computing. It could be in a data center. It could be on prem. It could be inside a factory. It could be in a base station like the work that we do with telecommunications. It could be in a car. And so it doesn't matter really where computing is done but we create the computing infrastructure necessary to do that. Each one of these different applications also have different algorithms and our job is to translate the intention of the science to best fit into our architecture. And so NVIDIA is really this collection of software middleware, essentially the operating systems of these domains that makes it possible for people to do all this computing. And then we connect all of that to the world's ecosystem of partners whether it's Eli Lilly for drug discovery or Caterpillar in energy generation and construction and heavy equipment. And you know, the number of companies we work with in the United States is basically everybody.

AI is a new computing paradigm. It is also a new industry altogether. And the reason for that is because the way that AI operates is so fundamentally different than the way computers operate. It no longer stores the information exactly as you described it and retrieve it from storage from a disc drive or SSD. We now generate it and the generation output is numbers and those numbers are essentially tokens. We call them tokens. No different than, well, tokens, just floating-point numbers or some number. And we take those numbers and we reformulate them into the output that you desire. And the output you desire could be text, it could be video, could be images, it could be sound. You know, kind of like Tang, you know, it's in powder form and we add water to it and turns into orange juice. And so we reformulate it in such a way that you can use it. Well, that token generation process requires really large computers. And so the first, and computers need energy. And so the way that this industry has formulated is one, energy on the bottom, whole bunch of chips and systems, computers. And these computers are fairly gigantic. I mean, you know, they're the size of football fields. And there's a lot of them. And so the next layer up from that is we call infrastructure. Infrastructure is land, power, and shell, but also the software for cloud services. And then on top of that is models. And on top of that is really the most important part for the United States and every single country is the adoption of the technology. And this is in fact the layer that I'm most concerned about. There's one area that really worthy of us spending time to talk about is making sure that we are mindful about on the one hand making sure we have proper guardrails and keep people safe in the application of the technology but to ensure that United States is at the front, at the pioneering front of applying the technology because it drives so much productivity and prosperity and technology and as well as economic leadership. We can't afford for another country to leave us behind. We were the front runners of applying technology in the last industrial revolution. We need to be careful not to be the last in this industrial revolution. And so the five-layer cake starts with energy, chips, infrastructure, models, and applications. We of course think about large language models which is the things that we can engage easiest the most. But remember AI can represent information of any kind. And some of the most important information we represent isn't language and numbers, but biology, chemicals, physics, articulation, animatronics. All of these types of information is represented by these tokens. And it's really important to recognize that when we think about AI, don't just think about the chatbot and the services we're talking about. There's enormous industries that are adjacent to it that don't get the voice that some of the AI companies get, but they're super important to the future of our nation. And so the AI models and the applications on top.

Well, that's a really important question. I think first of all you got to take a step back and ask ourselves do we want to reindustrialize the United States. Do we want to bring back to the United States this entire sector of the economy? This entire sector of labor that we consider essential to having a properly shaped society and a properly shaped economy. We've become a nation and an economy where unless you get a four-year degree, unless you get a master or PhD, you're going to get left behind. And that's unfortunate and unnecessary. I think we can all agree that that's unfortunate, unnecessary. We have to re-industrialize this country. For the first time in a generation, we have a market force that's incredibly powerful to drive the reindustrialization of our country. AI will cause us to create several plants. The first plant is chip plants. We're the largest AI company in the world today. We committed half a trillion dollars of consumption so that we can bring the supply chain from the east into the west, back to the west so that we can build chip plants and packaging plants, computer plants so that we can build all of the manufacturing necessary for Nvidia's AI supercomputers to be built here and used here. So the first plants are chips. The second plants are associated with the computers themselves. And the third, putting these computers into AI factories. Altogether we're talking about trillions of dollars of manufacturing, high-skilled labor jobs. We're going to create enormous amounts of manufacturing opportunity here in the United States. That I think is the first thing we have to confront. Do we want to be that country or not? Do we want to be that kind of society or not? Do we want to be that economy or not? Now, if we decide the answer to that is yes, then we obviously need energy because you need energy to transform atoms in one form to atoms in another form. You guys understand what I'm talking about? You have to break covalent bonds and you have to make covalent bonds. And in order to do that, enormous amount of energy is necessary to change the phase of matter. And so the whole concept of manufacturing is about energy. And so we need energy in this country in order for us to re-industrialize. And we are manufacturing. You ask me a question, I'm sorry. And what is the state of our capabilities? Clearly we are behind because of all of the policies we've had in the past. All the concerns about climate change caused us to of course underinvest in energy. Now we want to be conscious about the environment. Everything has to be balanced. And so now the question is what do we do going forward? The first thing that we can do is we can modernize the grid. We can make the grid more efficient. We overprovision in the grid. As you know, we have to make sure that our grid can take care of our society and all of our infrastructure on 12 of the worst days of the year. The rest of the time is overprovisioned by enormous amounts. And so the question is what kind of service level agreements can we come up with so that the power utilities can provide excess energy when they can and during the times when they can't, now we have to provide for subsequent backup energy. The backup energy can come from solar. It could come from nuclear, you know, all kinds of sustainable energy in the future. But we have an opportunity right now with this incredible market-driven force to take advantage of this opportunity to one, to make sure that United States become a manufacturing nation again, create enormous amounts of manufacturing labors and jobs and high-skilled and high-paying jobs. And then the second is to use this opportunity to enhance the energy system of our country.

There's a lot of conversation yesterday we've had here and in town about the state of Agentic AI, the successes, the failures and what's happening right now. You know, the big breakthrough from large language models to chatbots was reinforcement learning human feedback. The large breakthrough from large language models to agentic systems is a system called harnesses. The model itself advanced no doubt, the pre-training is improved. The reinforcement learning is improved. Teaching it how to use reason better and tool use, all of that improved. However, the big breakthrough is the concept of harnessing these large language models so that they have connection to ground truth, so that they could do research, that they could use the web browser, so that they could reason and have memory and improve itself and communicate with others. And so these agents in the last six months made enormous breakthroughs. Almost all of them perform incredibly well. We're big fans of Codex. Codex and Claude Code both are incredibly good. Codex 5.5 just came out, huge breakthrough, big leap, and Claude Code obviously incredibly good. The thing that we can now do almost, the vast majority of software tasks are now completely automated, meaning that we don't have to do the programming ourselves. The one interesting observation which was the prediction was because of agents coming out all software engineering jobs will be gone, that the first thing you should do is don't, whatever you do, plan on being a lot of things in the future but don't be a software engineer. Well it turns out the number of software engineers we're hiring is increasing. Every company is increasing. The number of software engineering jobs is increasing. And the reason for that is this. It's a very big idea. This also happened in radiology. 10 years ago somebody predicted that the first job that's going to go, the first industry that's going to go is radiologists, that job is gone forever. And the reason for that is computer vision was going to completely transform studying these images. The prediction was 100% right, AI has now permeated every aspect of radiology. The only thing that was wrong was the prediction. Radiologists are in short supply. And so the same thing is happening in software engineering. The reason for that is this. In our jobs, the task that we do, in the case of software engineering, the task is programming. You could argue it's kind of a fancy version of typing. And so the task is programming, coding. However, the skill, the purpose of the job is not programming. The purpose of the job is not coding. The purpose of the job is innovate, solve problems, connecting with collaborators, find problems that exist and solve it. Find problems that nobody's even expressed. It's called innovation. Connecting unrelated things, creating something new. That's the purpose of software engineering. And so our engineers, their purpose in life is to innovate, solve problems, move the company forward. It includes coding, but coding is not their job. Coding is their task that they, you know, some of them do in their jobs. AI is creating jobs. Anybody who is saying that AI is wiping out jobs is scaring people and is scaring people out of precisely the jobs that I need. The one thing that I hate for us to do is to tell all of the young people don't be software engineers because it turns out I need them. And hospitals don't tell people, AI researchers should not tell people to stop being radiologists because humanity needs radiologists. The radiologist's purpose in life is to diagnose disease. Reading scans is a task that they do in service of diagnosing disease. So if you separate the purpose from the task in a job, it'll help you think through it better.