Hope you had a great coffee break. Let's cut to the chase.
Jim, let's rewind the clock to 1982. That's your early days at DEC. There is a story that you spent nearly an hour in a heated technical debate before realizing the person you were arguing with was Gordon Bell, CTO, the president and co-founder of the company.
I mean, it's unfathomable to me that you were not aware who the president was. It's like someone at Harvard University saying, 'Wait, who is Alan Garber?' So what really happened there? And is that true?
Well, it's almost true. So, first, I'm an electrical engineer. So I didn't actually know much about computers when I joined DEC, as we called it. People say Digital. I think that was a rebrand. And I was working in Florida, and a friend of mine said, 'You should really work at DEC.' And I thought, 'Great.' So I got the VAX 780 manual and 11/70 manual, and I read it on the plane on the way to the interview, and I thought it was really interesting. And I grilled everybody in the interview loop because I had lots of questions. And they said they hired me because they really wondered what would happen, not because I had any skills whatsoever. So it was kind of a funny thing, and so I didn't know a lot about digital, as we say now.
And then we were doing this thing called hierarchical design, and we bought a new CAD tool to do that, and we were just starting to use it. And Gordon Bell had a lot of opinions about it. So this guy came in this room, and we had these workstations, and he started telling me how we should use it, and I started telling him why it didn't make any sense, and we went back and forth for an hour, which is true.
And then he walked out, and somebody said, 'Do you know who that is?' And I was like, 'No.' But he was wrong about almost everything. [laughter] 'He's Gordon Bell.' And I said, 'Who?' So that part's true. And then I got to know him later. He's a really smart guy, really fantastic person.
I love it. So, like, you got a job.
Okay, great. So, like, how did that shape how you handle technical disagreements in Tenstorrent today?
So my first project at DEC was the VAX 8800, and our architect was Bob Stewart, MIT grad. Great guy, really smart, but he often steamrolled everybody. And like people would say, 'I want to do this or this,' and he would immediately point out what the best way to do it was and how he would do it and expect us to do it. And that was causing a lot of stress, and we had a lot of young engineers at the time. DEC was hiring a lot of college grads, and so our manager brought in a creativity consultant, which is really a Bob consultant. And one of the lessons was before you disagreed with somebody or said no, you had to acknowledge it three times. So you'd say, 'That's a really interesting idea.' Or as Bob would say, 'I've never thought of that before.' And he would restate what you did in a way so you could hear him, which also got him thinking. And so every once in a meeting, we'd start to argue, and Bob would say, 'Three times.' [laughter]
And creativity is like the most important thing in design. And if you shut it down,
you don't get what you want.
That's right. So then leaders have to have a trade-off because you need a goal. You need to go, and sometimes you know everything about how to get there.
You have to give the right prompt.
But you have to enable people to come up with it, and that's a trick.
So Jim, during the coffee break, one faculty member, the School of Engineering and Applied Sciences, pulled me aside, and he said, 'Is that Jim Keller?' I said, 'Yeah.' He said, 'How did you get him here? How?' And I said, 'Well, Harry called him.' [laughter]
And so, Jim, you are really a superstar in this world. And so, thank you for coming and sharing the wisdom with us. So, first of all, quick show of hands. How many of you have an iPhone?
Well, 97% in the room, maybe 98%. You should all give a thank you to Jim. So, Jim, [applause] he is the man behind Apple's... and I have my notes because there's so much about him. Apple's A4, A5, A6, and A7 chips.
Right. So those are the ancestors of the silicon that's powering the phone in your pocket today. So you joined Apple in 2008, and from, you know, the fact that it was through the acquisition of the small startup, P.A. Semi, right?
And so tell us, what was the transition going from a 150-person startup to the Apple world of perfection of scale, and what did you learn there?
Well, first of all, you know, the Apple I joined was a pretty small company. They had shipped the first iPhone, which was a great product and a terrible computer chip, and they were just finishing the second chip. And I was involved in design reviews of that. And the Apple chip team was 50 people, and we mostly outsourced the implementation to Samsung. And I joined the platform architecture group, and I worked for Mike Colbert, who was, I think, you know, the de facto CTO of Apple. He worked for Steve Jobs for 25 years. And so our group was responsible to architect the computer chips and some of the systems we built, and then think ahead a bit. So I was one of those guys.
And Steve had this idea that to make the best product, you had to have the best technology. To do that, you had to have the best talent, and he was really driven about that. But he also wanted the teams to be really small and independent and go fast. And so when I was there, we grew from about 50 people to 400, and we slowly brought in more of the design, what they call the design of the IP, in-house, and that took about a four-year transition. And the big thing we did is in the last chip, which we called the H6 for some reason, Steve bumped the number publicly, H6 was A7. We told him we're going to put a supercomputer in a phone, and he's like, 'That's the best idea. Let's do it.' And the software guys were terrified because the power of the chip was way too big for a phone, obviously. So you had to turn it on and off really fast and manage it really carefully. It was really a challenge, but it reset the bar for phones
to be a fast computer, not just a small one.
That's impressive. So fast forward to 2012. People often call you the father of AMD Zen processors. You like to say that you're just the uncle. I'd love to understand that difference. But anyway, at AMD you led one of the most remarkable turnarounds in chip history with the Zen architecture, which completely changed AMD's trajectory. [snorts] And so the Zen chips turned AMD from an underdog into a heavyweight again. So for all of us, when you look back, what was the hardest part about leading the transformation at AMD? Was it the technical challenge, the cultural one, or something else?
Yeah, it was almost all cultural. So when I joined, I was an architect, and I knew if I went as architect, I'd have to argue with the VP about who did what. So I lobbied strong, and Mark Papermaster is the CTO at the time and a friend of mine. He took the bet because at Apple I managed one person, and it wasn't going great. So I went into AMD and I managed 500, and I hired a consultant who's a brilliant guy, Vancout Ralph, to figure out how to manage a team. And he had me read about 10 management books, and I quickly found out two things. One is if you read management books and you use them, if they're good, it mostly works. Like, not everything, [laughter] right? And then the other is I had read 10 more management books than anybody at AMD.
And then as an architect, like I visualize how people work together, kind of the way I visualize how computer subsystems work together. And people are pretty straightforward. There's inputs, functions, and outputs, and you have to organize them creatively, and it actually worked pretty good. My worry was AMD was going bankrupt. Obviously, they laid off a lot of people, and when I got there, there wouldn't be any good people. It turns out they actually had a really interesting culture. They were obviously the underdog, but they liked working together. They liked working on stuff they had the freedom to do, and they kind of stuck it out. And the actual quality of the engineers there was actually not bad, but it was horribly organized.
And we had leaders who were managers first. And I reorganized the team to be led by the much stronger technical people, and we let some management... and then we went like crazy. It was really fun.
Great. So after turning around AMD, you left and joined Tesla in 2016, where you led the Autopilot hardware engineering, helped shape Tesla's in-house silicon strategy, and worked closely with Elon Musk's fast-moving culture. You once described working at Tesla as building a rocket whilst already in flight. What was that ride like, and how did that experience influence your approach in Tenstorrent today?
So it's really interesting. So Elon expected us to really understand cars in the factory. So I started building a team of chip designers, and we had this meeting because Model S yield was terrible. We had 200 cars in the backyard, you know, with missing wire harnesses and parts and broken stuff. And so he was like, you know, Monday morning, 7 a.m., everybody's there. And my software guys and chip designers showed up, and like, we don't know what to do. And the answer was, well, let's figure it out. And we started debugging the factory software and tracking and logistics and fixed the wire harness, the wire harness delivery. And everybody got this hands-on feel, like I knew the whole factory. Like I could, you could say, 'Meet me at, you know, body-in-white or wire harness testing or, you know, this or that,' and I knew where it was. And we went in there, we fixed... So they update the software in the car as they're building it on Wi-Fi, and the Wi-Fi had holes in it, so some cars wouldn't get their software updated. There were so many weird problems, and we just started fixing them all
while we were designing the Autopilot chip. Now, on the Autopilot chip, chip design is one of the few things Elon's not an expert at, at least in, you know, as he sees it. And he gave us a lot of space there on mechanical engineering, a whole bunch of stuff. He's actually really good. And so we had more space there, but we told him we'd build a chip and drive a car in 18 months, which we did.
I often wish we'd told him 24 because that would have been a lot easier. But yeah, it was sort of amazing that the hands-on approach. And then Elon likes two things: you have a real goal, like we're going to save the planet, we're going to build electric cars, we're going to accelerate the transition to sustainable energy. I can still say it, right? And then, what are you doing this week?
And he didn't really care about the middle part, you know, are you aligned to the goal? What are you doing this week? It was a wild management strategy because most companies, they focus on the middle: roadmaps and plans and Gantt charts.
Do you think it was a good strategy or not?
Well, it's working for Tesla. It's amazing. So I like the immediacy. Lots of software programs, they do like sprints, like two-week sprints. That's pretty interesting. Chip design takes longer, but we've actually, I've incorporated some of that in how do we iterate faster with real achievable goals that you can see.
Got it. So Jim, I've noticed that pattern. So once the rocket is cruising smoothly, you are already prepping for the next launch. When Apple's iPhone chips matured around A6, A7, you left. When AMD's turnaround was done, you left. When Tesla's FSD chip taped out, the same story. It seems like what excites you most is the takeoff, the climb from chaos to clarity. So most people love cruising, right? So when things are working, you seem allergic to cruising. Am I right?
Yes. [laughter] No. So I like missions. Like at Apple, a friend of mine recruited me. I said, 'What are we going to do?' He goes, 'Can't tell you.' It was a secret. I went through three locked doors to get to my office. He said, 'I promise you it'll be great.' And I knew about the iPhone, and there was a vague plan for the iPad, and that was sort of a surprise. And I was always amazed at how well Apple worked despite not being anything like any company that I ever worked at. And I decided I was there to help and learn, which I learned a lot. AMD, they were obviously going bankrupt. I had a lot of friends there. They had hired people from Digital at some point. So I knew a lot was going on, and I thought, well, this would be amazing. Let's see if we can fix it.
Like Tesla was a project, and Elon's an episodic guy. So big push for Autopilot chip, and then go slow, and then a big push. And when you do chip design, you basically build a system to build chips, and then you start cranking them out every year or two. Like you have to have a roadmap, the infrastructure to do that. And it was clear to me at Tesla we weren't going to do like another Autopilot chip every year.
Even though I thought we should.
So some of it's the mission, some of it's, you know, when the project's done, it's done.
At Apple, actually, Steve Jobs passed away, and then my boss passed away.
And I knew it was going to be a different company.
Like your catchphrase is 'fix it'.
Yeah.
Good. So let's fast forward to 2020. You joined Tenstorrent and later became the CEO. I still remember the first time we met in person. Bob had scheduled our meeting in Santa Clara, and I flew down from Boston to Santa Clara. And the day before our meeting, change of plans: meet Jim in San Francisco. So I followed the address and found myself outside of a, I have to say, like shady-looking place. [laughter] Seriously, right next to a mechanical shop. There's a rusty, like heavy security door. Looked very suspicious.
And I rang for quite a while without getting any response. For a moment, I thought I'm going to be abducted. [laughter] Then someone buzzed me in, and you gave me a crash course on how to build computers for AI. Then I realized, oh, that is the real Jim Keller.
So, speaking of Tenstorrent, from your perspective, how is Tenstorrent different from, you know, the big name NVIDIA? What do you see as key advantages and limitations of each approach?
Yeah. So like first, technically, NVIDIA is a GPU company, right? And GPUs were designed to run small programs on lots of pixels. And then they invested in a software stack called CUDA for HPC computing, and it actually paid off for AI. I don't think they ever made a lot of money on HPC, but the AI worked really well. And then they've extended it with what's called a tensor processor. So AI is basically math on tensors, which you could call, you know, that's what the physicists call tensors. It's actually matrices, multi-dimensional matrices. And so they're a GPU to run pixel programs with tensor extensions. Tenstorrent is a native tensor processor,
and more like the Google TPU is several large tensor processors. Tenstorrent's design is more smaller tensor processors. There's a technical reference. I think it's easier to fill small processors, but there's a coordination layer that we had to write. I think it's more power efficient. NVIDIA's roadmap is slowly adding features that Tenstorrent's had for years, but obviously they're super successful. So there's one thing, which is a tensor processor design, which I think is more efficient and better aligned to the AI problem than a GPU.
And then we did a couple things that are different. We open-sourced our software stack top to bottom, which was great. We get so much comments and feedback on it, and most of the current products are all proprietary. They say this is open or that's open, but it's not open. Like their math libraries are encrypted. And then we also license our IP. So we talked to lots of people who are worried about very large companies having proprietary technology so they can't innovate or own their own destiny. And so our business is we design a full-stack AI computer. We open-source the software, but we also license the IP and the chips if people want to build their own solutions, which I think is novel.
Got it. Thanks. So I'm going to share a part of an email from a Harvard professor, David Brooks, that I think you will enjoy. By the way, Tenstorrent recently donated a Tenstorrent QuietBox to our lab, and we have several students starting to use it for a project in my graduate course this semester. It is great.
It is great. [laughter] So first, we make our chip, and it goes on a PCIe card in a server, and we shipped it to some developers. And one of my friends said he had to turn it off after dinner because it was too loud. So we made another one that's water-cooled. We call it the QuietBox. And I want to make the fastest AI workstation you can make that plugs in a wall outlet. So that's 1500 watts. It's a lot of power, but it's a real AI computer development system.
And then for students, so RISC-V CPU architecture was designed by Dave Patterson, Krste Asanović at Berkeley, and they open-sourced it, and it's amazing. So we build our CPU and AI based on RISC-V technology. And when I go to universities, they all use RISC-V now. Like it took over the world. Like Intel sells PC chips and ARM sells mobile chips, and RISC-V is the CPU for everybody, which is really cool. So we open-source our AI technology. We're going to release our AI reference model. And then I've talked to professors who say their students want a computer where they can do anything they want, including power management, low-level firmware, drivers, and see the whole stack. So we're now delivering to universities this platform, which is cool. And it's a better learning platform because you can do anything you want at any level. It's pretty simple. So
Yeah, just for the audience, so you are doing the AI chip to take on NVIDIA, and you are having the RISC-V CPU chip to take on ARM, effectively.
Well, first of all, the AI and CPU business is really big. So a lot of startups go, 'I'm going to take on NVIDIA.' Really? It's a $4 trillion company that's built a castle with a moat and a [laughter] you know, air defense system and everything.
Yeah. So we're in this big transformation, right? And you know, like, and we've seen this before. It's what you call Bell's Law or Gordon Bell: mainframes, minicomputers, workstation, PC, mobile. AI is really anomalous because it's the first new computing paradigm that uses more power and it's bigger than the last one. So it's kind of a wild anomaly. But usually in the transition, like IBM didn't win minicomputers and Digital didn't win workstations, right? So there's lots of room. And then if you look at the distribution of applications, like in the US especially, it's all NVIDIA, OpenAI, X, Amazon, you know, the big guys. There's a really long tail,
and those customers are really underserved. And I talk to them a lot. Like, 'I want open-source software. I want to control my business. I want to be able to buy a chip for five years and not be cut off. I don't want to be on allocation. You know, I want to be able to do innovation. I want to be able to write new kinds of software models,' right? So that business is really big. You know, it's not a hundred billion today, but it's tens of billions of dollars. And when you're in a big transition, you know, things keep changing. Everybody says, has anybody seen an article that says LLMs have hit their limit? Like, right, like every five minutes, you know, AI has stopped. It's amazing. But we just started.
Right. There's going to be so many new models and so many new ideas, and so I want to be part of that, like that innovation place
where people are doing something different.
So I don't have to compete with ARM and NVIDIA. I don't really care about them very much.
Sure. So I'm curious in the way that you're actually thinking, Jim. So you often, you know, in your talks and when you're writing, you often hint that chip design is a mix of science, heart, and intuition.
So how do you personally balance those three when you're working on something new?
Yeah, it's, well, computer design is like different than quantum physics or something because at the base, like Boolean logic, it's really simple. Like, you know, and when you count 0, 1, 1, 0, 1, 1, like it's amazing, right? Like at one abstraction layer. Now transistors are very complicated, but when you build a computer system, you start with really simple abstractions and build it up into systems that are unbelievably complicated. You know, modern chips have a hundred billion transistors on them. And one of the things I, you know, I tell my team and I've given lectures on this, like we're not getting smarter. Does everybody agree about that? But we are building way more complicated systems.
