Context Engineering Our Way to Long-Horizon Agents: LangChain’s Harrison Chase

People use traces from the start to just tell what's going on under the hood. And it's way more impactful in agents than in single LLM applications because in single LLM applications, you get some bad response from the LLM. You know exactly what your prompt is. You know exactly what the context that goes in is because that's determined by code and then you get something out. In agents, they're running and repeating and so you don't actually know what the context at step 14 will be because there's 13 steps before that that could pull arbitrary things in. So like what exactly is everything's context engineering? Context engineering is such a good term. I wish I came up with that term. Like it actually really describes like everything we've done at LangChain without knowing that that term existed. But like traces just like tell you what's in your context and that's so important.

I am really excited as well. Thank you guys for having me back. It's quite an honor. I'll tell my mom again that I'm back on the line.

Yes, that was a great term. You guys wrote a great article.

I mean, I agree that they're starting to finally work. I think like the idea of running an LLM in a loop and just having it go was always the idea of agents from the start. AutoGPT was basically this and this is why it took off and captured so many people's imagination because it was just an LLM running a loop completely deciding what to do. The issue is the models weren't really good enough and the scaffolding and harnesses around them weren't really good enough and I think the models got better. We learned more about what makes a good harness over the past few years and now they start to like really really work and you see this in coding first and I think that's the domain where they're taking off the most and that's spreading to other domains but you can give a task to an agent and you still need to communicate to it what you want it to do and it needs to have the right tools and all of that but it can actually operate for longer and longer periods of time and so that yeah the long horizons kind of like framing of it I think is really really apt and really really good.

So, coding is the place where there's the most. I think that's the one that I probably use. Yeah, that's the one that I use the most. Adjacent to that, I think like really good ones are AI SREs. So, Traversal I think is a Sequoia company and they have an AI SRE that operates over longer time horizons. Research in general, and I'd call like AI SRE kind of research, like they're taking an incident and they're going and digging through logs, like research in general is a really really good task because it ends up producing like a first draft of something and the issue with agents is they aren't like reliable to 99% of reliability but they can do a ton of work and more and more work over longer time horizons. So if you can find these framings where they run for a long period of time but produce like a first draft of something, those to me are like the killer applications of long horizon agents right now. So like coding, coding is an example of that. Like coding you usually put up a PR, you don't directly push to prod unless you're vibe coding which is also starting to get better and better. AI SREs usually surface it to a human who comes in and then reviews it. Report generation, you don't send it out to all of your followers right away. You look at it, you edit it. It creates like a first draft of something. So we see this in finance a bunch. This is a huge research opportunity. Customer support. We see a lot of things pivoting from kind of like the initial customer support was like first-line response, like someone messages you, you just respond really quickly and there's still that and that's going great but now there's examples, K is a great example of this, where it's like humans and AI working together. When the first line fails you escalate to a human, you don't just have the human handle it, you have this long horizon agent run in the background, produce a report of everything that happened and then hand it off to the human agent there. Agent starts to get confusing in customer support. So I think the killer use case of all of these is places where you have like this first draft type of concept.

Yeah. So and I'll maybe bring in like framework as well because I think early on, I mean that's how we describe LangChain as, that's what LangChain is, it's an agent framework and now we have Deep Agents which I'd call an agent harness and we get asked about what's the difference. So a model is obviously the LLM, tokens, messages in, messages out. The framework would be abstractions around that, so making it easy to switch between models, adding abstractions for other things like tools and vector stores and memory and things like that, but pretty like unopinionated about what goes in there. More values are more like batteries included. So when we talk about Deep Agents we're talking about we actually give it a planning tool by default. So it has a tool that comes built into the harness. That's pretty opinionated that like this is the right way to do things. We do compaction. So you have these long horizon agents. They're running for long periods of time. Context windows are larger but they're still not infinite. And so at some point you need to compact that. How do you do that? There's a lot of research going on there right now. One of the other sets of tools that we and a lot of people are giving to these agents are tools for interacting with the file system whether directly or via bash and this is it's kind of tough to separate from the models because the models are being trained on a lot of this data as well. And so there's this kind of evolution between like I don't know if we could have known that like these file system based harnesses are the best thing at like if we go back two years ago I don't think we could have known that because models weren't really being trained on that as much as they are now and so they're kind of evolving together. So I think it's like a combination of things. It's the models absolutely are getting better. Reasoning models are helping a lot. But it's also the fact that we're figuring out all these primitives around compaction and planning and these file system tools being really useful. And so I do think it's a combination of both.

Yeah, I think that's right. Yeah, so we build Deep Agents on top of LangGraph. It's one particular kind of like LangGraph instance. It's very opinionated. It's more general purpose. And so I think early on we talked about general purpose architectures and more specific architectures. And what we've seen is that a lot of the specificity for tasks previously that might have been in LangGraph because you needed to put more structure on the models. Now that specificity is moving into the tools and the instructions. So there's still the same level of complexity. It's just in natural language. And so prompting and editing those prompts and maybe automatically updating those is becoming a part, but the harness is remaining a little bit more fixed.

I think a lot of the companies that are doing the best harness engineering are coding companies. Honestly, I think that's the place where it's taken off a bunch. I mean you look at Claude Code. I would argue a big reason for the popularity of Claude Code is the harness itself.

I don't know. So the next company I was going to mention is Factory which is another coding company and I think you look at the harness they've done there. AMP is another coding company has a really good harness. I think there's pros and cons. There definitely is some aspect of the harness being tied to a model. And maybe not just a specific model but a family of models. So like all Claude models, like Anthropic fine-tunes on some specific tools. OpenAI fine-tunes on different ones. So like I think probably when we were doing this last time we maybe talked about how prompts need to be different for one model versus another. Harnesses also need to be slightly different for one family of things versus the other. But there are similarities. All of them use the file system in some sense. So I think this is I actually don't know the answer to that. It's a really interesting thing. We see that a lot of the coding companies, everyone who's building a coding company is basically building their own harness right now. Yeah. And there's all these leaderboards and you can see it's actually kind of interesting if you go to Terminal-Bench 2, which I think is probably one of the more popular coding benchmarks right now. You can actually see they have like the agent harness and then the model and so you can see the variation in performance and Claude Code is not at the top of that. So there's differences but I think it doesn't necessarily mean that the model labs are better at it. It just means that you have to understand how the models work and people who look at what makes a harness tick around the model can get some performance gains there.

I think part of it is definitely understanding like what tools the models trained on. So I think OpenAI trains really heavily on Bash. I think Anthropic has an explicit kind of like file editing tools and so I think leaning into that is part of it. Compaction is becoming more and more of a thing. So especially as you start doing longer horizon tasks like you start to fill up the context window and so what do you do there is a really big question and there's a bunch of strategies for kind of like approaching that. I'd argue that's part of a harness. I mean, so all of these harnesses also this is where like skills and MCPs and sub-agents start to come into play as well and you can use those in like different ways and I don't think a ton of like skills or sub-agents are trained into the models yet like those are still pretty new. Yeah. And so like one of the things that we see in our harness is like when you have a sub-agent the main model needs to communicate with it like well it needs to give it all the appropriate information. It needs to let the sub-agent know that it needs to like give it its final response out. So like we would see some failure modes where the sub-agent because basically what happens is you kick off the sub-agent and then only the final response is passed back to the main agent. And so we'd see some failure modes where the sub-agent would do a bunch of work and then it would be basically like look at my work above and then you know pass that back to the main agent and it can't see it and it's like what are you talking about? And so like that type of like prompting to get these pieces to work together is a big part of it. So like skills, sub-agents, MCP, there are like prompts in all of these harnesses that make them work well or don't make them work well. And they're hundreds of lines long if you look at some of the ones that are out there.

So I think there's maybe like three eras I would say. I'd say like early on and this is when LangChain was just started like these were still the raw like text in text out like not even chat-based models. And so they didn't have any of the tool calling. They didn't have any content blocks, any reasoning at all. They were really just like really really basic. And so the things that people were doing were mostly like single prompts or like chains. And it wasn't even possible to do anything like that complicated. Then a lot of the model labs started training in a lot of like the tool calling into the models and they got really good at kind or they tried to make them good at like thinking and planning and they still weren't they still weren't good yet. They sort of weren't good as they are today. But they were good enough to like decide what to do. And this is where like the custom cognitive architectures would come in more into play because you'd ask it explicitly like what do I do here? But it was like a very like point in time and then you go down this branch and then like what do I do here? And maybe there's a loop and there started to be some loops but it's still a little bit more kind of like scaffolding around it. And then there was an inflection point and I don't know where exactly that was. I would say I think we noticed it probably in like June July of this year where we saw Claude Code taking off, Deep Research taking off, Manus taking off and these all use the same architecture under the hood of just the LLM running in the loop but like cleverly like a lot of harness is context engineering like everything around compaction, context engineering, sub-agents, context, skills, context engineering. So we basically saw them using the same core algorithm but making just like improvements on context engineering and we're like oh that's interesting that's pretty different than before. And so that's when we started working on Deep Agents. I think for a lot of people in the coding community, I think probably like Opus 4.5 was when they started to like really feel this. It might have also just coincided with winter break when everyone went home and started using Claude Code and realized how good it was. But I think like around like November, December like I think there's has been this like at least I sense a pretty big like vibe shift in like people just like yeah you throw hard problems at these things and you get long horizon agents. And so I don't know whether it was early 2025 or late 2025, but at some point the models got good enough and that's when we moved from like scaffolds to harnesses.