Welcome to Interrupt, everyone. Isn't this awesome? This is crazy. Yeah, this is the first time we've done this in New York. So, we've done this in SF the past two years, but we're bringing it here for a full day. We've got an awesome lineup. We'll be announcing a lot of awesome stuff, and we've got a ton of great fireside chats with folks in the field, and then a lot of great talks from people that are building agents and practitioners. So, a lot to hopefully learn from.
So, I want to talk a little bit about the state of AI now and where we think it's going. But before diving into that, I just want to do a brief kind of like tour over the past few years, because it seems like things move so fast, but it's only like 3 years ago was when all of this kind of like really kicked off. Actually, a little bit before that was when LangChain launched, but in 2023, we were building prototypes. In 2024, we were shipping some of these simple applications to production, but they were still like single LLM calls. 2025, we started exploring agents. And really this year, we've started seeing that agents have transformed the application layer. And so, a lot of the talks that you'll hear from today are companies that are doing exactly that. They're taking these LLMs and they're really building them into differentiated experiences. So it's not just the model that they're providing to their customers. It's everything around the model.
And so we're seeing that these domain-specific agents, they're not just wrappers. They're providing real value to folks. And people are having to build a lot of things around the model to properly take advantage of that. And so, calling out a few. So Rogo, we'll do a fireside chat with them later today. They had a great blog where they wrote about how they built a specialized harness with model routing to provide a domain-specific agent for finance. Harvey, who we heard from at some previous Interrupts, has been talking a lot about how they've fine-tuned models for specialized tasks to get the best performance on legal domain. And then JPMorgan Chase, we'll hear from them later today. A little bit of a different use case, internal use case, but customizing the harness around the model to provide a lot of internal advantages for the agents that they're building as part of their Jarvis assembly line.
So, all of these are examples of companies taking the model, building a lot of stuff around it to make it truly theirs, and the phrase that has been going around to really exemplify all of this is owning your intelligence. When we talk about owning your intelligence, we talk about all of these examples. Basically building a lot of domain-specific things in and around the models to make it truly best for the domain that you're trying to tackle, and really owning your intelligence. And we think there's a few reasons why this has become a phrase that's top of mind for so many people these days. First, token costs are rising a lot. People are spending more and more on AI, and so as they do so, they want to make sure that they're getting the proper ROI, and being able to control and monitor token costs is really important. And so owning your intelligence in that regard is a key factor. Second, we see open models becoming more and more popular. So this provides not only a really cheap alternative in order to own the token costs more, but it also lets you customize things more. And so as these base open-weight models get better and better, you can start to post-train them. You can start to use them in specialized ways, and that just makes it easier than ever to really own your intelligence.
And then the last thing that I think has become pretty clear over the past year is that the models themselves are becoming somewhat commoditized, and it's all of this stuff around the model, in particular the data, whether that is actual kind of like context you're providing or some of the domain-specific paths that you're baking into the harness. This is becoming the differentiator to provide a lot of value for folks. And so because of all of these factors, we're seeing people want to own their intelligence more and more. And that's how we think about our mission as a company. It's always been this way. LangChain got started a month before ChatGPT, but really took off after, when people saw that came out and they were like, "I want that, but for my industry, for my vertical." And then a big value prop of LangChain has always been kind of like model optionality. So from the beginning, people were building these domain-specific things that were naturally kind of like decoupled from the model. And so this mission of kind of like owning intelligence has been something we've thought a lot about, and we think there's three core pillars to this. And this is what we help folks with.
So the first core pillar is open and controllable harness. And so there's two kind of like key parts to this. One, we think the harness needs to be model neutral. So you shouldn't be coupled to a particular model. This is both offensive and defensive. When the best new model comes out, whether it's from OpenAI or Anthropic or TypeSafe with some of their Jev models, you want to be able to switch your harness to that as quickly as possible. And so your harness needs to be able to adapt to the best new model. It's also defensive. You don't want to get locked into a single model provider so that they can raise rates and things like that. The other part of this open and controllable harness is again what we talked about in terms of making these agents like really domain-specific. All of that is about bringing specialized context in particular ways to the model, to bring the right context at the right point in time, and you need to really control what the harness does, and what that control looks like could look different for different scenarios. So we've had LangGraph for a while, which is an open source framework for really creating these directed graphs, and that's a really controllable way of bringing this harness. We also have Deep Agents now, and I'll talk about that later, but that's another kind of like open source model harness. We have a concept of middleware in there, which is a different way of injecting like decisions to make and things to do into this core agentic loop. And so regardless of what it is, you need a way to control really in real detail what's going into the model.
The second core pillar that we think makes up owning your intelligence is compounding, specifically compounding the intelligence that you get from when people use your system. So when you launch an agent, that is far from having a successful agent. You need to learn from how people use the agent, what the agent does, see where it makes mistakes, and being able to compound that agent's intelligence, whether manually, by seeing errors and updating the code or updating the prompt, or more automatically, whether that's with memory or prompt optimization things. This compounding loop is really important to build into your systems and to own. If you can define what better and what best looks like for your domain better than anyone else, you will be able to build the best agent for that domain. And so having eval... intelligence is a really good way to build these domain-specific agents.
And then lastly, governed. And so this is really crucial for internal agents. We're seeing a ton of companies transform how they do internal operations. But when you have a lot of agents running around, you need to make sure that they're properly governed. You need to make sure that you know how many exist, that you can control costs, that you can handle auth reliably. So different people accessing it only have access to what they should have access to. And so we see that governance is becoming an increasingly big topic around owning your intelligence. And so this is how we think about the three core pillars of owning intelligence. And we think there's a bunch of different ways that you can get there. And so calling three out at a high level: people, process, and technology. And we're going to talk most about technology during the rest of the keynote, but I want to touch really briefly on people and process, because those are also needed to transform your organization into an organization that can own its intelligence. So touching really briefly on people, this is roughly how we see a lot of organizations being built. There's a ring one of like core platform engineers. These provide a lot of tools for other people to build agents. When we talk about governance, when we talk about this kind of like eval compounding loop, these provide the platform that other agent building teams internally can use to build and govern and compound their agents. Outside of that, we see agent engineers. So, this is a great term to describe the weird hybrid mix of data scientist and engineer and machine learning engineer that makes up the people who are driving these agents. And it's a new skill set, and we see a lot of people from different disciplines coming in to be agent engineers, consuming things from the core platform team. And then lastly, there are a ton of subject matter experts inside and outside the company who are often the best people to decide what good looks like for an agent. And so having a platform that can touch those non-developers at the edges is super important. And so people, and kind of like setting up your organization to have this type of model, is what we see people doing.
We also see people either implicitly or explicitly adopting a kind of like new process for building agents. As I mentioned before, it's very iterative. You first launch an agent, it's not going to be good. You need to iterate. And so there's this life cycle, this loop that people are adopting in order to really get agents to be good enough. Roughly speaking, it's build, test, deploy, monitor, and then crucially, when you take those monitor learnings and you bring them back into the build step. And of course, you want to do all of this in a governed way. And we'll come back to this a little bit later, but people are doing this type of process all the time to improve their agents.
And then I want to talk about the technology. And when we think about the technology, and the technology that we help build and provide, there's really like two core parts to it. So one is the runtime. And this is how we got started, with the open source LangChain libraries, and continue to do open source things with LangGraph and Deep Agents. And then of course we want to build agents that don't just run locally on your laptop but run in the cloud and can serve everyone inside your org, or the customers that you're building for. And so there's a deployment aspect to this runtime as well. And then there's this observability and eval part, which is really crucial for the compounding layer. And so being able to see what agents are doing and measure how good they are behaving is really important. So these are the two kind of like core pillars of things that we build. And recently we've also focused on kind of like a third part, which is an intelligence layer that sits above them. So now when we think about LangSmith, our commercial platform, we think about it serving the runtime aspect, serving the observability and eval aspect, and then building in intelligence across the platform to really help people utilize these components and speed up that process, that agent development life cycle that I talked about earlier. And so, for the next few minutes, I want to walk through these kind of like three core parts of our platform and talk about some of the cool things that we're doing and where we think the future's going.
So, I want to start with runtime. And so, what does the runtime for kind of like an agent look like? When we think of an agent, we generally think of there being three components. There's the business logic. So, this is things like instructions, tools, skills, hooks. These are things that you guys have to provide when you're building your domain-specific agents. What skills for a financial agent might look like are very different than what skills for a legal agent might look like. And so this is the business logic that everyone out here will provide to the agents. Then there's the harness. And so the harness is what takes the model, takes a lot of this domain-specific context, and basically wires it all up so that the model can see the right context at the right point in time. This can be a very kind of like custom harness that you might build in LangGraph, or it can be a more kind of like general purpose harness, which we're seeing more and more of. And so that's why we built Deep Agents. And so Deep Agents is our newest open source library. We launched it about a year ago. It's an off-the-shelf agent harness. It learns a lot from what coding agents do. So it has an execution environment with a file system and sandbox. It does a lot of context management in ways that coding agents do it, with compaction, but it's built so that you can build these domain-specific agents. And then lastly, there's the infrastructure. And so if you take this agent and it's running locally on your laptop, great. You want to bring it to the cloud. There's all of these components that you're going to need to add in so that it can run remotely. You'll want a runtime in a server that has durable execution. You'll want some way to access tools securely. You'll want some way to run this code that it writes. And we see a lot of agents writing code even if they're not coding agents. You'll need to access models. And so there's all this infrastructure that goes into it as well.
And so these are a lot of the components that we've built into LangSmith in pieces. So we have LangSmith Deployment, that's really around the agent runtime. We have LangSmith Sandboxes for running code, in beta. We have an LLM Gateway in LangSmith, and I'll talk about that, but that's for access to models. And then a tool server is something that we're working on for easy access to tools. And so there's these different components that make up the infrastructure. And one of the things that we launched in beta at last Interrupt, but I want to spotlight real quick, is Managed Deep Agents. So Managed Deep Agents basically takes this harness and the managed infrastructure and combines it into one. And so now when you're building an agent, you just provide your business logic, and then you get the harness and managed infrastructure together. And so that's a lot easier to get started with than having to go business logic to harness to assembling all these three or four or five different pieces of infrastructure. And so we're really, really excited about the Managed Deep Agents platform as a way to accelerate time to value and speed to kind of like production for a lot of these agents.
There's a few things that we're announcing today as part of a new Managed Deep Agents release that we think will help even more with this. So first, one of the hardest parts of building these, especially internal kind of like company agents, is auth, and making sure that the right people can access and use the agent, but also that when they use the agent, if the agent connects to Notion or Jira, it's connecting either on behalf of the user or as the agent's kind of like fixed set of identity. And what the right answer is might depend, and so we're introducing a lot of different primitives in Managed Deep Agents to facilitate this type of connections. So first is connections. These are basically tools, when you have a tool server, which is how Managed Deep Agents connects to tools. You can specify how you want your end users to auth to those connections. So we're using auth there. Related to that, we have this idea of identity for agents. This just defines the agent to be either kind of like user permissioned or service permissioned on a whole. And then third is channels. So a big part of building these agents is exposing them to people. Slack is a massive one. When you have agents in Slack, how do you deal with auth in like a shared channel? If I message it in a shared public channel and my coworker messages it, whose auth is used for which runs? A lot of that we're baking into off-the-shelf channels in Managed Deep Agents to hopefully make the auth problem a lot easier.
Second, we're adding more things around memory into Managed Deep Agents. And so, Managed Deep Agents has always had a concept of agent-level memory, but we're now adding in user-level memory. So this means that it can remember things specific to the individual users that it interacts with. This is really powerful for customizing agents. And then third, we're adding in built-in tools. So one of the most common things that agents need to do is search the web. And so we're adding in a built-in tool via Parallel so that agents can access the web. And so now you no longer have to go get a separate API key for that. You can just use your LangSmith API key, use it with Managed Deep Agents, and get access to the Parallel web search MCP. And so putting this all together, Managed Deep Agents, we think, is the easiest way to go from the business logic to an agent that's actually running in production. We've been working with a number of early design partners on this, and we've seen really good results, especially for just getting an agent up and running and having it be like this company harness, and that's how we think about Managed Deep Agents, is this company harness that makes it really easy to get started building and sharing agents inside an org...
...that I want to spotlight is decision models. And so over the weekend I, as I'm sure many of you did, played around with Jev. So it launched over the weekend. It's a really interesting new type of model. And so what exactly is Jev? It's a decision model. What does that mean? It basically means that it makes decisions. You send it a state. You send it a list of questions. It can then score those questions. And there's different things that you can have it score on. So it can either be like a boolean or an actual score or a categorical choice, but it basically makes decisions. It can't generate text, but it can make decisions. And so even though it's more limited, we think this is really interesting. We think there are a bunch of different types of models and a bunch of different types of agent harnesses that will be built. So we think absolutely there will be these frontier models that are driving these really complex harnesses, and that's what you're going to use in like Claude Code and Codex and Deep Agents. These take advantage of these really frontier models. But there's also a ton of other decisions that need to be made. And that's where we think a decision model can be really helpful. And maybe these are small things that actually go along with the frontier model, or maybe they empower the LangGraph cognitive architectures and routing between different things. And so calling out a few of the things that we think these decision models are really useful. One is eval. So either running after the fact or in the loop. If you have an agent doing work, if you can score it quickly according to some criteria as it's running, that's really powerful, because then you can catch mistakes in real time. Two, guardrails. So flagging user input really fast. So I was talking to some people yesterday, and we were talking about guardrails. One of the downsides of guardrails is that it always adds latency. If now we can add them in a really low latency way, that's really cool and powerful. And so we're excited about it for that. And then lastly, model routing as well. Choosing which agents or which models or which skills or tools to load. We think things like Jev will be really, really useful for that. So you can access Jev through the LangSmith LLM Gateway. You can also access an open source decision model that we're hosting there called SemIf. There's been a massive explosion in these open source decision models since Jev launched. We think this is a new type of model that there will absolutely be a lot of options for, and we're really, really excited to power and host a lot of these through our gateway.
Speaking really quickly of our gateway, this is the last thing in the runtime that I want to spotlight. So, we launched LangSmith LLM Gateway in beta about a month or two ago. There's two main things that I want to highlight. One, we try to make it so that the gateway can be used in all of the LLM applications that you're building, whether they are customer-facing or internal. So, specifically, we normalize everything through OpenAI and actually Anthropic message format. So you can access it there. But we also allow straight pass-through in case there are new features that come out in these APIs and you just want to take advantage of. And then we also have a bunch of coding agent integrations, so Codex and Claude Code and Cursor. Again, we think you should be routing all of your traffic, for governance and cost control reasons. We think you should be routing all of it through a single LLM gateway. And so that means it needs to integrate with every application that's using AI. And so we want to make it really easy to plug into all of those. And then the main kind of like features that we're focusing on: one is cost controls, having cost visibility, but also setting spend limits for different teams, different users, different API keys. Two is rate limiting. So making sure that there's not a single application that's really throttling or taking up a lot of usage there. And then third is fallbacks. So you can fall back to different model providers when you get an error. And then one of the things that we'll be launching soon is actually more stateful fallbacks. So if you notice that requests are failing and Anthropic might be down, you don't even try them for the future. You just switch over to OpenAI for the next 10 minutes or 30 minutes, and make it really configurable in that way. And so we want to make LangSmith LLM Gateway the most kind of like secure and reliable way to monitor and govern and fall back on different LLM providers when you're using them. So that's everything that we've got going on in the runtime space.
I now want to talk about observability and eval. This is a space that we think is crucial to really power that compounding loop, and where we've done a ton of stuff over the past year, but especially over the past month. So, we think traces are really the core of this loop. When you think about this loop, this process that I mentioned, we think traces being the system of record and showing what the agent does at all steps are really, really important to be able to power this kind of like learning loop. When we've talked about traces previously, we've talked about basically runs, traces, and threads. And so a run is like a single LLM call. A trace is like a single invocation of an agent. And then a thread is a series of traces, basically. So when you have a multi-turn interaction. So these are all really useful primitives, but it's become pretty clear over the past year or so that there's a more useful standard for how to represent what agents do. And this is what we're calling trajectories. And so a lot of the agents these days, whether they're coding agents or first-party agents, run in a similar way where they basically run an LLM in a loop and compile this list of messages. This list of messages is a standard format that can be used in a lot of really interesting ways. And so today, one of the things that we're excited to launch is LangSmith Trajectories, a way to standardly interact with these trajectories and use them in a bunch of places. So let's take a look.
So what do trajectories do? We think there's two main benefits to them. One, they make debugging a lot faster. So if you've used LangSmith before and you've had a really complex trace, ones that have thousands of kind of like iterations, you might have noticed that it's pretty hard to parse through all of that. So the LangSmith Trajectories in the UI have a much more user-friendly interface for viewing what's going on. They'll look basically as if you're using like a coding agent or a chat agent. We basically condense and display tool calls in the same way that those do. So you can easily kind of like scan through them. One big benefit of this is simpler annotation. So now that you can see what's going on, you can leave feedback on them much more easily. A third big benefit that we think is this actually makes it much easier for post-training models. So when you post-train models, you commonly have these trajectories that are in this kind of like standard format. And so we do a lot of work in the back end to parse these really complicated kind of like trace formats that come in from all these different SDKs and different coding agents and expose them in a standard trajectory format. This can be used really, really easily in fine-tuning.
When we talk about fine-tuning, there's generally two pieces. There's the data, and this will come from a lot of the traces and from a lot of the trajectories, and then there's the inference and training infrastructure. And so we want to make it as easy as possible to bring these two things together, which is why we're also announcing LangSmith Fine-Tuning today. So what does this mean? So you have all your data in LangSmith. These are all the traces, which now we're porting into this trajectory format, which is a much more condensed and standard representation of what's going on. You can then use the smithtune CLI, which is a CLI that we're launching today to basically select traces, filter down to relevant ones, pre-process those traces in a way that makes them even more fine-tune ready, then pass them into Fireworks or Baseten, which is the training and inference infrastructure to do the actual training. We can then evaluate the models from there, and as the output, you get an optimized model. And so we think this is a really powerful way to get started fine-tuning, whether it's on agents or classification tasks or guardrails or anything like that. We want to help you all take the most advantage of all the data that's in LangSmith. And so we think trajectories power a lot of this.
Now, where we're storing these trajectories is SmithDB. We launched this at Interrupt a few months ago. This is a database that we built in house, and it's really good for agent workloads. It's a lot faster to ingest and store and query a lot of these workloads. We've made trajectories show up really nicely in them. So for example, I was talking about what they might look like earlier. This is a screenshot. You can see that we condense a lot of the long tool calls that are going on, but you can click into those, expand those. You can click into an LLM call and jump directly into that. So we try to make it look as nice as possible, but we know there's a ton of different ways to actually visualize the data that you're working with. And so that's why today we're also launching LangSmith Custom Apps. Let's watch a video on that.
So, LangSmith Custom Apps you can basically think of as Lovable, but for your data that's already on LangSmith. So, you can chat with it. It will create a completely custom app. We actually open sourced our design system so that the coding agent that we use under the hood can leverage it, and it comes built in with all the permissions for sharing with your team and respecting the workspace or RBAC or ABAC boundaries that you've set up. And so we think this is a really powerful way to bring humans into the loop and inspect data. So that's what we're launching for observability and eval.
The last thing I want to talk about is intelligence. So I mentioned earlier that there is this process, this life cycle for building agents. You build an agent, you then test it, deploy it, monitor. One of the things that we launched previously at Interrupt is LangSmith Engine. So LangSmith Engine is an engine that sits on top of this whole loop. Specifically, it sits on top of your traces in a tracing project. It runs in the background, and it basically does all the things that an AI engineer would do. It looks at issues. It tries to cluster them. It creates this like issue board. It tries to fix it with code. It tries to add an evaluator, add things to a dataset. And so this is something we've been working on for the past six months or so, and we've been working on a lot of improvements, and so today we're packaging up a lot of those and announcing Engine v2. So let's take a quick look at a video, and then I'm going to walk through a few of the exciting things in here after that.
So I want to call out a few of the things in there. First, I want to call out that Engine can now test its fixes proactively. So what exactly does this mean? So this integrates really nicely with LangSmith Deployment, which we talked about in the runtime. So specifically, Engine now will spot issues in traces. It will spin up a preview branch on your deployment. It will confirm that that issue is in fact an issue. It will then make a fix, spin up a preview branch for that fix, run it against that to confirm that it's fixed, and then basically present it as an issue that you can go in and see all that evidence that that issue actually was real and that the fix now fixes it. So again, this integrates right now with LangSmith Deployment, and that's how we do it there. Another thing that I want to call out is red teaming. So previously, Engine only identified issues that existed in traces that people were sending to it, that were happening in the wild. With red teaming, Engine can now simulate different issues and then see what happens. So specifically, Engine will, and again this integrates with LangSmith Deployment for now, which is how we know how to call it. So LangSmith Engine will basically have access to the deployment. It will spin up different issues, different hypotheses, run these against the deployment, identify which ones produce bad results, where bad could be it learned something that it shouldn't have. It produces a bad result. It is off-brand in some way. And then it will flag any issues to you. So, this is really good for identifying places where your agent fails. It's also really good for creating an initial eval dataset. So, we've heard for a while that creating evals is really hard and really painful. And so, this helps automate a lot of that. This can absolutely be used before you even ship your agent into production and before those issues even occur.
Another thing I want to call out is the performance improvements that the team has made. So, we've worked really hard on making it both better at finding issues, better at fixing issues, and then also making it a lot cheaper as well. And so, a lot of these are packaged up into Engine v2, which is live as of now. Engine has already scanned over 70 million traces and detected over 21,000 issues. It's used by a lot of the fastest growing AI companies out there to really speed up that iteration cycle. That iteration cycle is everything when you're building agents. And so anything we can do to help speed that up, we will absolutely do. And the last thing I want to announce is that Engine is now also available in self-hosted with bring your own key. And so this will be rolling out in V17 of self-hosted, which should be out in a week or two. So we're excited to bring this to even more of the customers that we work with.
So this is everything that we've launched and how we think about the technology that we build. We think there's this runtime, and we think about this as the open source but also deployments, and more and more we're seeing that these managed agent runtimes are becoming really powerful for going to production. We have observability and eval. We think a lot about how we can make this more agent native with trajectories, how we can make this more customizable with Custom Apps, and how we can help you use this data to tune models to better serve your workloads. And then lastly, we have this intelligence layer that sits above, and we want to make Engine do a lot of the things that agent engineers and AI engineers are doing now. And so the scanning of traces and the red teaming are just the start. All of this we want to build to help you guys build domain-specific agents and really own your intelligence. Again, whether that is agents that you're building and putting in products or agents that you're building to transform the internal operations of your company. And so you'll hear from a lot of speakers and panelists later today about different ways to do that, different techniques, different examples of what that looks like. We're really excited to be doing Interrupt in New York. We're really thankful for all of you guys to be joining us. I think we're going to have a fantastic day, and so I'm going to step off for a little bit, and I'd like to welcome up Jake, who will be our MC for the day. Thank you guys all for coming.
