Jensen Huang on Vision, Risk, and the GPU | Only In America

My parents had nothing. Sold everything. Came to America. That was the beginning. Along the way, the jobs I had, the schools I went to, the opportunities I had, and here we are, Nvidia being the most consequential technology company in the world. This can't possibly happen anywhere else.

I'm in uniform.

Nice to see you.

Welcome.

What do you think about our new building?

This was completely built in a simulation first.

This building is designed so that we make maximum use of this incredible space called California. And so we decided, let's use the space for the beautiful sunlight and the weather. And this entire space is very energy efficient. So we use the maximum amount of sunlight and that's why there are light wells on the roof.

And so notice where the triangles are, the light wells are only where it needs to be.

And the roof looks flat to us from here, but is undulated to track the sun. So it lets sun in just enough but doesn't let any more sunlight in than is needed. Otherwise, we'll have to remove the heat.

Completely efficient. We simulated this building every single hour day, through a simulation of the entire year and we moved everything until it was perfect. Perfectly harmonious mathematically.

That's an engineer trying to be an architect.

And the undulation, this one creates a baseball cap. And the reason for that is because you could see during the afternoon when the sun's about to set, it comes right into the building.

And so I sloped the walls to move the offices further back in. And so this is the baseball cap of the building and it fits right into the beautiful design.

I've got to be a little wiki wonky.

Something's got to be a little off.

Yep.

Uh, no. You're going to be good.

I tell people Madam Secretary was a while ago. Dr. Rice was my father and Professor Rice is with my students. So I think that leaves us with Connie.

I was born in Taiwan. When I was five, my father got a job in Thailand to help start an oil refinery. And so we moved to Thailand. We were there for about four years. In 1973 there was a coup as it happens every now and then. And my parents thought it was unsafe for us to be there. My older brother was one year older than me. He was 10. I was nine. My parents wanted to send us out as soon as possible. And so they got our uncle who lives in Tacoma, Washington, to take us for a little while. My first impression was I'd never stood in a house with carpets before. And it was the strangest feeling. I felt like I was walking on my bed with my shoes on and just everything, from cereal and the morning television, Speed Racer, and in the afternoon, Partridge Family, and all the candy, the Snickers bars, everything was like I couldn't imagine what this amazing country was and everything was so beautiful, the cars, everything was just incredible. And we were in Tacoma, Washington for about three months. And then they sent us to the most affordable, accessible boarding school in America because my parents couldn't afford much. A wonderful school. They welcomed students of all backgrounds. Many from difficult homes, some international kids and that's how we were able to come. They sponsored us and allowed us to come to the United States and stay in that boarding school and I was there for about two years in Oneida, Kentucky.

It's just one little dot. I think when I was there it was probably population 600. It's probably population 600 now. Beautiful, incredible. I had a wonderful, wonderful childhood there. It was very difficult and very scary coming to the United States by yourself. But it's amazing what my older brother did. Frankly, I just followed him around. He was 10 years old. Could you imagine a 10-year-old bringing along a 9-year-old, traveled all the way from Thailand, never been to the United States, landed in Tacoma, Washington and went to Kentucky, but laid over in Chicago.

The Chicago airport is gigantic. And so we had to go find a connecting flight and a 10-year-old did all that by himself. My older brother's incredible.

You know, 1973 United States, in Kentucky there were still biases and the school had never seen this Chinese kid before, right? Wonderful kids there, but none of them ever seen a Chinese before and so there were all of the things that comes along with being a stranger in a town where nobody's ever seen someone like you before. And so there's a lot of biases and of course things like that, but kids are kids. And so for us at the time, the United States was amazing and I was part of the swim team and I was part of the soccer team and the food was interesting, sausage and gravy. Who wouldn't like that? Hamburgers and after swim meet, the coach just took us to the most amazing restaurant in the world and the food came in boxes and the menu was all lit up and we sat in a restaurant that seemed like a spaceship and it was McDonald's.

And so I think it's really all about expectations in life. When you come from an even more difficult circumstance, that's the thing about immigrants, is that when you come to America, you came because of choice, you wanted to be here. My dad sent us here but in fact when I was four years old he had the opportunity to come to be trained in the United States and turned out the place he came to was New York. Could you imagine to come to the United States and it's New York City from Taiwan in the 60s. And so he always said that he wanted his family to eventually come to America to come to this incredible place. So the thing that's really great about immigrants is you came with very little expectation. You came with great hopes and dreams. You appreciate everything. You come with a sense of gratitude for where you are and the new opportunities.

I was a good athlete. Yeah, I was a good athlete. And so we met our parents back in Tacoma, Washington. They had nothing and literally they put all their belongings on the plane. So they had nothing and they came with, you know, literally the suitcase. I mean that story, the immigrant story, and they saved their way to build a life for their kids. My mom was a maid at a Catholic school. And my father was an engineer. They saved everything they had. My dad bought a green truck, a van, and there were no seats in it, so he put a carpet in the back and put some milk crates. We got into that green van and he drove us all the way from Oregon down to LA so that we could go see Disneyland. And that was the one vacation of our family.

I know I must be Denny's best ambassador.

No, I love Denny's so much. It never occurred to me to be able to afford or go anywhere for college and so it was always just my plan that I would grow up in Oregon and I loved math. I loved science and when you're in high school and you love math and science you're only going to have three other friends who also love math and science. And so the two or three friends of ours, we were all in a club, we were in a math club. We were also in the science club, right? And then we were in the computer club all together. And so afterwards, after clubbing, we would go play arcade games, play pinball or go play arcade games. And that was growing up. And my best friend, Dean Verhiden, he said, you know, I'm just going to go to Oregon State University. My brother went there, my parents went there, and I said, that sounds great. Oregon State had a good engineering program. That was just good happenstance. I loved the school. And it turned out that there was another kid there, Lori. She was one of three girls in a class of 250 other boys, right? And she was, you know, I was the youngest kid in school. I was only 16. And she was a year and a half older than me and I was determined to go meet her. And so I statistically weeded out everybody by getting myself maneuvered into her lab class. So now I've reduced the competition from 250 to four.

That's very clever. That was very strategic even when I was a kid. And I knew odds and then I gave her the ultimate pickup line. I asked her if she wanted to see my homework.

I cinched it and so we've been together ever since.

Yeah, we met. We've been in college.

I always wanted to go to a great school. You know, I always imagined getting a masters, but I never imagined that I could afford it. And so, during Oregon State, the recruiters came from Silicon Valley. And so I took a job at AMD and that was a really good choice because the work was really interesting. I had really great colleagues and they had a program where you could go to Stanford and work at AMD and they paid everything. And so when they were pitching the offer to me, I said, hang on a second. I can work at AMD. You're going to pay me a great salary and you're going to pay me to go to Stanford at the same time. And they said, yep. I said, that sounds like a dream come true. So I took that job. And then of course a year later, Lori graduated. We got married and then life started. So I took some classes, took some time off, took some classes, took some time off and altogether it must have taken me some eight years to graduate. I'm probably the longest running student at Stanford. Nobody paid Stanford more. I just for the education and along the way, Spencer Madison came and along the way I founded Nvidia and literally during my entire tenure, during that time of being at Stanford, my life happened.

People talk about Silicon Valley and the ecosystem and so forth. But you just used a really great term. It is really kind of a big stew, is it? And you meet very interesting people and they change your thinking about something and that leads to something. It sounds like Stanford was a place like that for you.

And it was really cool.

It was at the beginning of the PC revolution. It was the beginning of really Moore's law, the beginning of the CPU era, this incredible technology engine that really changed everything. All of that in Silicon Valley was all about CPUs. Everything was about general purpose computing. Everything was about CPUs. Everything was about Moore's law. Everything was about the PC. And there were two simultaneous ideas that Chris Malachowsky and I were considering. And one idea of course is every application that's interesting and meaningful, can it be run on a CPU? And we believed that there were so many interesting problems to solve whether it's real-time computer graphics which is at the time one of the hardest problems to do in computer science or simulation or other things beyond that, a normal computer like a CPU can't possibly be the right format. And so we imagined that there would be a way to accelerate the CPU, offload the work that is not suitable for general purpose things. It's kind of like in your house, you only have one tool in the kitchen. Or you go into your garage, there's only one tool in your garage. You come to your company, there's only one tool. You know, there's the right tool for the right job. And we believed that there's another tool that could augment the CPU and make that computer essentially a supercomputer. And so that was one idea. The second idea was how do we even get this company started? The problem is the CPU at this point, general purpose computing as of today is probably about 64 years old. And so the nature of the computer industry is that if you create an architecture that produces benefits to applications, the install base goes up, the sales goes up. Applications use more of it. This positive virtual cycle would cause the two to reinforce itself. And so in fact over the course of the last 64 years so many applications are built on top of the CPU. How do you even cause the application developers to consider another architecture? That chicken or egg problem is incredibly hard to solve and has never been solved. In fact, aside from Nvidia today really everything else runs on CPUs, right? And so the idea that we would get this new architecture to be adopted by developers was incredibly hard. And so the question is what's the first application? And so which brought Nvidia to its second idea which was maybe the first application is computer graphics. But the problem is computer graphics was such a small market at the time. It was just Silicon Graphics.

And Silicon Graphics was a large company but by the standards of computer architecture is very very small. And so the question is how do we find an application that would on the one hand need this architecture on the other hand be sufficiently high volume to cause this architecture to proliferate.

And so we thought maybe 3D graphics for video games. Computer games are not only fun, but they perhaps more than any other application push the edge of computing power. Most PC upgrades for consumers are driven not by the grown-ups in the house, but by the kids who want a more powerful gaming machine.

That's exactly right. The GeForce 3 is the latest generation graphics processor from Nvidia. So, this product actually includes a 57 million transistor graphics processor. To put that in perspective that's more transistors than a Pentium 4 plus a Pentium 3 put together.

And it was, I still remember explaining it, everybody's going yeah but you know this and that, this requires this be solved and have multiple chicken and eggs and yet Silicon Valley right here, Sand Hill Road would finance me and Sutter Hill would be investors in this company and that we would be able to attract the brightest minds, the brightest computer scientists in the world to come work here.

But we were just determined that on first principled basis, the general purpose computer cannot possibly be the only computing platform, right? And that there were too many interesting problems that we could solve if we were to introduce this new idea.

That's exactly right. For the first time, the graphics processor is as programmable as the CPU. We have an instruction set just like the Intel processors have an instruction set that you can program Microsoft Word or Windows with. We now have an instruction set that game programmers can program to create special effects on our processor.

For 30 years.

Yeah.

Computer graphics is basically a simulation of the world. It's a simulation problem. In a lot of ways, artificial intelligence is a simulation of the mind, simulation of the brain. And so the computation of simulation can be done not completely but largely in parallel. And so the architecture, the processor that you want to use for simulation versus the architecture used for task execution, a recipe is step one, step two, step three. In the case of simulation, the world is happening concurrently and in parallel all at the same time right now, right? And so on that first principle, you would think that simulation of the world, whether it's quantum, Newtonian or otherwise, should be something that is a different architecture than recipe execution and instruction execution. And so that's the big, big idea. Now the question then is how do you manifest that idea in technology? Number one. Number two, how do you go find application for it? And so we found of course the first application, computer graphics. The second application was seismic processing or inverse physics, CT reconstruction, ultrasound, seismic, very similar problems. The next problem we found was molecular dynamics, Newtonian physics. And you know on and on and on and we just kept finding one problem after another. And then one day some researchers, one at Stanford, Andrew Ng, Jeff Hinton at the University of Toronto, Yann LeCun over at New York University, they were all trying to solve for a similar problem which is deep learning and they reached out to us and I, you know, being alert, I realized that this is a problem that we could really make a contribution to and I was happy to help. And because of the work that we did together, it achieved a level of computer vision capability that no one had ever imagined. And because of its success it triggered even further introspection. Why does it work? What else can it do? How far can it go? What is the implication to computer science? What is the implication to the whole industry that we're built on? And so step by step by step, we broke everything down to its first principles. And then we rebuilt back where Nvidia is going to be in that world. How do we apply all of the techniques that we know? How do we navigate step by step so that on the one hand we can pursue this unknown future? And so all of this is about reasoning, vision, strategy, discipline, patience, belief.

We suffered our way here.

You know, we suffered every single step of the way. We suffered our way here because nobody believed in it, right? So, we had the benefit of building all of this for a decade before anybody even paid attention. The hard part of course is that you're endeavoring something that has no positive feedback, right?

How do you stay motivated and how did your employees stay motivated? Because there was this period of time where Nvidia was not on the tip of the tongue of anybody here in the valley. All kinds of other things were going on.

First of all we have to believe in what we're doing. We have to go back to our core values as you know in almost everything great that's done. You have to go back to your core foundations. You have to demonstrate that you are determined to go pursue that. That you see that future in your mind's eye even though nobody else can. You see that future in your mind's eye. You have to tell the story so that everybody else could see it in their mind's eye. And you have to believe it yourself.

It has to first start from what are your core beliefs? For what reason do you believe this future will happen? You could say that about America. You could say that about Stanford. You could say that about Nvidia. America has core foundations. You know, we spoke a little bit about it. The American dream has a core foundation. There's a reason why it exists. There's a purpose that drives it. There's a promise of it. There are pillars that keeps it up. When you see the world from my lens, when you see America from my lens and when you see America now today as I travel around the world and see America from the lens of everywhere I've been and the countries that aspire to be us, the industries that would aspire to want to be part of our industry, you know, it is just genuinely a miracle.

Exactly. We have a word for that. We call it freedom.

That's right.

I would always retreat back to first principles. I would always reason from first principles when I'm uncertain about the future when things are moving too fast. You always go back to first principles. What are my core values? What makes me great today? What do I aspire for? And to go back to that, I would say to young people that it's possible to have multiple feelings at the same time. It's possible to be grateful for everything that you have, to be unsatisfied with where we are at, and to have aspirations for greatness. You're allowed to have all of those feelings at exactly the same time. Doing technology change, doing world change is the only opportunity for greatness. Status quo, you know, it's really hard to make a difference. And so I would deeply dive into the capabilities of artificial intelligence and use it in every possible way. And it's not just about mastery of science, but it's about mastery of using artificial intelligence in my field of science. Your purpose, if you decide to go into medicine, is to care for people. It's different than studying a radiology scan. Your purpose as an engineer is to either solve a known problem or discover problems that have never been solved, that are worthy to be solved. There's a fundamental difference between the tasks that we do in our job versus the purpose of our job. Everybody's purpose, every job's purpose surprisingly consists of tasks but not defined by the tasks.

Yeah.

Cautious optimist.

Yeah, intelligence is foundational to every aspect of society, every aspect of industry, everything we endeavor. In almost everything that we do, the fundamental ingredient is intelligence. Now, of course, we have to be cautious. We have to be cautious so that we advance the technology as quickly as we can so that it works as we promised. So that it's functional as we expect. So that it doesn't produce intelligence that sounds like it's intelligence, but it's not. It's flawed.

I want my car to function as promised. AI needs to function as promised.

I think that one of the challenges is to define what is AI. That is probably one of the most, there's a lot of different ways to think about the technology but one way to think about it is like a five layer cake and we have to win every single layer. The first layer is just energy, land, power, and shell. First layer. Second layer is the chips layer where I'm at. The third layer is the infrastructure layer which is kind of like cloud services. The next layer above that is the AI model layer. This is where everybody talks.

But it's not the only layer. Everything about that layer is very important. But ultimately the most important layer to our nation is the layer above that which is the application layer.

Right.

Somebody comes here from Taiwan, they're nine, they end up in Kentucky of all places.

They somehow go to Oregon State because their friend goes to Oregon State and the family's there. And then they go to Stanford because somebody will pay for it. You meet a collection of people who are kind of intellectual geniuses, but they're a push on your own.

I'm the bus boy at Denny's that met up with Chris and Curtis, the two geniuses. I mean, that's how the story goes.

I think that it's a chain of extremely low probability events that leads to Nvidia.

And without the tailwind that America provides, America provides tailwind, not headwind, you know, it provides tailwind. Laws and rules that are understandable and that you can count on, a business environment, an industrial environment with people playing by the rules that you can understand and count on. You know, when everybody's playing by the rules because the rules and laws are for good reason, then at least you could find where there are segments of the market that are underserved and that you can rely on that. You can rely on the fact that you can create something great and take it to market in a way that serves a market that has a demand and that it wouldn't be foreclosed on you, randomly, arbitrarily, unknowingly, you know, unpredictably. Those things that entrepreneurs rely on, you know, are alive and well here.

Very similar. Yeah. Right. You have the same feelings. Yeah, I am certain my feelings about Nvidia and my constant desperation to do better is exactly the same feelings my parents have to secure a living for their family. They have nothing else to rely on, nothing to fall back to. I can't imagine another place where this is possible.

You know, this is genuinely, Nvidia genuinely is an only in America story in a lot of ways. I'm an only in America story. This is in one lifetime. I'm not talking about a fifth generation thing. I'm not talking about third generation thing. This all in one body in one generation with parents that gave up everything to be here with no way to fall back. And they sacrificed their whole life to build a life for their children so that we could have more opportunities than they did and for a country to create the opportunities for me and all the resources, the systems, the institution, the foundation that makes a company like Nvidia to be possible. I am the embodiment of the American dream.