Excited to be here with Jensen. There has been a ton of advancements in AI and agents over the past year, but a few months in particular, I feel. We've seen a lot of these advancements come in the form of better performance, but at the same time, we've also seen that openness and control and trust in a lot of these models and systems around them has become more and more important. And so the first thing I want to start with is how and why are you guys at NVIDIA investing in an open agent ecosystem and stack?
First, before I answer, I want to congratulate you for all the work that you do. In fact, if you look at the last six months, we could both agree that, although we've been working in AI for 15 years, the last six months changed everything. And so, all of the technology, all the large language model advances, all the scaling, all of the breakthroughs, all the omni models, multimodality stuff, all that stuff is fantastic. But in the end, it was the last six months where everything came together, and now finally AI is useful. And when AI is useful, every company in the world, every enterprise in the world wants to get their hands on it. And now the question is how? And this is where LangChain comes in. And you always had a vision that the large language model was the essential ingredient, the essential enabling technology, but in order to turn it into a useful product, you have to surround it with what is now known as a harness.
So much more. And back in the old days, we used LangChain to help us turn a large language model into a promptable API. And we used LangChain to build our RAGs, and we used LangChain to step by step, which led to today's agents. And really what happened in the last six months, the big breakthrough are these agentic systems that are grounded on info, grounded on knowledge, that can use tools to do search and has memory that it manages, and it has safeguards, and has the ability to iterate until it gets the job done. But it ultimately needed some models that have reached a level of capability where everything comes together into that flashpoint, and that's where Claude Code really brought the imagination of agentic systems. OpenClaw, of course, was a big deal, and all the work that you did with, what, Deep Agents, and we use that ourselves, and all of that kind of came together and bam, here we are with agentic systems. The reason why we do it, we've dedicated ourselves for many years to build open systems. And the reason for that is because ultimately AI is a fundamental technology. It can only be useful if applied in a whole bunch of different use cases. Now, of course, the first use case is just language and cognitive intelligence, and that's very important, of course. We imagine a world where scientists and digital biologists and designers and roboticists and students and researchers, enterprise IT, all of us could use agentic systems, AIs, to solve domain-specific problems. And many of the problems that we want to solve, either we have specialized domain knowledge that is just simply not available outside that we have to embed into, imbue into our AI, or it's because we believe that AI becomes ultimately great, becomes a super agent when we put it into a flywheel where we use it, it gets smarter, it becomes more useful. We use it even more, it gets even smarter. Kind of like us, kind of like humans, learns over time. Learns over time. And so we imagine this future where AI has a foundation, and the work that Anthropic and OpenAI and Google's doing is all fantastic. But there's specialized AIs and domain-specific AIs and proprietary AIs that people want to build, and we want to enable that world to happen.
Maybe digging into that for a second on this topic of specialization. How exactly do you think it's best to specialize these systems? Is it going to be purely the model? Is it going to be the harness as well, the context outside? What goes into the specialization?
The specialization starts with you need to have intelligence that's good enough, and this is why we worked on Nemotron and really love the fact that you're part of the founding team of Nemotron Coalition. We made Nemotron Ultra pretty incredible. Now, Nemotron Ultra is a great model as a start, but it becomes an incredible model when you put the LangChain framework around it, the LangChain harness around it, so that you ground it on information that is domain-specific. And a person, an intelligent person, becomes super useful when we give them access to particularly important information. And so access to information is important. Putting it into a flywheel where maybe you're even training the model, post-training the model inside the LangChain harness against a harness so that the model becomes good at applying the harness around it. What you want it to do for that task. What you want it to do. Yeah. And so I think that this moment has now arrived, but we need an open harnessing system that we can build ourselves, that we can apply and then, of course, improve against over time.
I love what you said about the model being good enough. I feel like that threshold was crossed, I don't know, maybe a year ago by some of the frontier models, six months ago by some of the open weight models. You talked about Nemotron 3 Ultra. We've done a lot of work with that to make that really good in Deep Agents. Some of the things we did is tweak the harness to make it best for this model, because we found that different models need different prompts, different tools. And with that tweaking, we managed to get Nemotron 3 Ultra in Deep Agents to, we have an internal benchmark, and it managed to get up to eighty-six percent on that.
Claude Opus, for comparison, is at eighty-seven. You've got DeepSeek and one of the Minimax models at eighty-two, eighty-three down there. So we're starting to see that some of the more recent open weight models are really getting to frontier performance.
I know. I am so proud. But it is so incredible.
But it is so incredible. One of the just as important things is it's ten times as cheap as Opus.
Yeah. And I think open weight models are starting to really strike a good balance between performance and cost. So I'd be curious how you see this cost part changing the equation for builders.
The benefit of cost is it comes in a couple of different ways. I happen to think that when you have cost-effective intelligence, people just use more of it. When you have a cost-effective agent, then you can iterate across a larger search space. And as a result, the answer could actually be better. And in the case of Nemotron, it's cost-effective because it's so fast. It's so computationally efficient. When it's computationally efficient, it could explore larger spaces. And it's no different than when somebody can think fast, you could explore more space. When you can try things more quickly, you can find a better answer. And so this is the incredible benefit of Nemotron 3 Ultra inside the LangChain framework and the LangChain harness inside Deep Agents. It could think so quickly, it could explore so quickly, it could iterate so quickly and efficiently that it's going to find better answers. And so I'm just really excited that we created a model that was near the frontier. But adapting the environment around Nemotron, you made it deliver frontier capabilities. Now, insight for humans, it's the same. Of course, we like to hire the smartest people in the world, but beyond that, we also give them access to tools, we give them access to information, and we also create the world around them so that we allow them, enable them to create the conditions for them to achieve their full potential. And so you adjust the environment, not just the model. And this is where LangChain came in.
What you said about using more of the intelligence as it's cheaper and faster, we see that to be so true. I think one of the things that I like to think I'm AI forward. One of the things that I've underestimated is just the demand for intelligence and for tokens and how big and massive that market is, and I think that's become especially true recently. With these models getting good and being really fast and really cheap, how should we think about using frontier models? Should we just use these open source models all the time? Is there a time and place for both?
The frontier models are getting better all the time, and I fully expect the frontier models to be unbelievably good. And they still have a long runway of improving the models. The scaling laws, of course, are going to sustain. Their harnesses are improving all the time. Their technology for dealing with memory, whether it's working memory or long-term memory, is advancing incredibly quickly. The compaction technologies, all of the advancements in retrieval-augmented generation and knowledge graphs, and there's still a lot of incredible advances that are being implemented into these frontier model APIs. The way I think about it is, frankly, I always start all of my work starting with the frontier. Okay. And the reason for that is because it's useful. I know what's the potential. It costs a little bit more money, but my time to getting the work done is fast. However, you know, over time, I find that I want to add sub-agents to them. I want to connect sub-agents that are super agents at certain skills. And so we have optimization problems inside our company that relate to supply chain. Maybe it's related to chip design optimization, floor planning optimization. And these optimization problems are insanely hard. And so you're not going to just have a general AI go off and crunch on it and think that you're going to find a great answer. So we create super sub-agents, and these super sub-agents we would create with Deep Agents, LangChain Deep Agents with Nemotron 3 inside, and we'll even connect them to specialized tools. And that thing is built for one job. That super agent is not trying to book me travel appointments. It's just trying to optimize our supply chain. And in that case, I really do need to have LangChain. I really do need to have Nemotron 3 Ultra, and I connect it to a lot of proprietary knowledge and proprietary skills. I've got a whole team who's just dedicated to refining that. Now, I think that defines a company. A company is really about a collection of a whole bunch of these super proprietary, super important workflows. And now we can have LangChain with Deep Agent and Ultra, Nemotron 3 inside, and it gives them all of the control they need, super efficient access to incredible tools. That's the future.
Do you have any advice for enterprises if they're following your practice of starting with the frontier and then starting to specialize? When should they think about specializing? What are some triggers that you look for that as soon as it gets good enough?
So I would start with Claude Code and Codex and use it for as long as I can. And frankly, for a lot of things you never have to replace, because they're getting better all the time, and they're on a trajectory that's going to bring capabilities insanely fast. And so I think that in the future, just like companies are today, we have employees that we hire because their domain specialization and the refinement of the work and the work process and all of their life learnings here in the company is just too valuable. But we also hire consultants, and we license external tools, and we outsource work to other people, and so on and so forth. I think this is the future for AI, and are we going to continue to use frontier models? Absolutely, and tons of it. But are we also going to create specialized super agents with LangChain and Nemotron 3 Ultra that in fact arguably could be your crown jewels? And the answer is absolutely true.
I think even for the consultants that you bring in, just like when you bring in a consultant, you need to get them up to speed in your organization and give them context on the organization, how things work, what tools do they need that have access to data that's only inside your organization. And so I think one of the things we've seen is as enterprises start to adopt AI, there's all of these kind of systems that they have to build around them in order to make the agentic systems as a whole trustworthy and safe and proper kind of governance. I'm curious, how do you see...
And just to add on that, just today, most companies are built on business processes. Yeah. In the future, most companies will be built on harnesses. And so the idea, LangChain would just become the tool that creates the operating system for the company, and everybody will be using LangChain to create their specialized harness, which represents a workflow of the past. And now this harness inside that workflow becomes autonomous, agentic, much more efficient.
I think we see that these things are... There's the harness, there's the model, and then there's all the context around it, and all of these can be optimized at different points in time. And so the work that we did with Nemotron 3, I think was a great example of doing some pretty high ROI things around the harness, changing the prompt, changing the tools. One of the things we're looking forward to is experimenting with post-training Nemotron. It takes a little bit more time, but I think it really raises the ceiling of what this overall system can do.