I already am really curious about something after the United talk. The absolutely fire line was: the repo grows, but the context window is limited. Like that tends to be like you can vibe out something really special and pretty impressive right away, but then when you get into enterprise software, lots of connecting systems, hundreds of tools, you start to like come right up against that context limit. So, first thing I want to know actually is how does LangChain build LangChain and LangSmith?
It's a good question. I don't think we're probably not the model kind of like AI adopting company, honestly. Like practically speaking, like what we do is we give people a choice of their coding agent. We're pretty liberal with that. We have most people using either Claude Code, Codex, or Cursor, or we have OpenSWE, which is our internal kind of like background style Devin agent. And so, it's some combination of that.
We have a few kind of like we have agent.md files, a few skills. The best skill we have actually is maintained by our security lead. I think it's called like Langster skill. It's supposed to prevent bad things from happening in our repo. We experimented with a few AI code review tools. We actually ultimately ended up building our own. So, OpenSWE review that we'll probably open source at some point as well. This is just recently that we ended up turning off with some other ones and building our own there.
Yeah, I'd say maybe like the best thing we do is we have this like OpenSWE that sits in a Slack channel, and then people just message it and ask it to build things. And so, this leads to I think we tweeted out a doc of documentation PRs, like basically doubling as soon as we implemented that because people before didn't want to like pull down the repo, make a change, push it up. But it's so easy to say in Slack, like, 'Hey, this needs to be better documented.' Or this should change, this shouldn't say X, it should say Y. And so, that's probably the most unique thing we have.
Nice. I figure it's always interesting to hear how, you know, cutting edge Valley tech companies are building, especially since you have your own coding agent now that you've open sourced as well. Where I wanted to take tonight because we're in Chicago is how agents are affecting enterprise. So not just building software, but I think like really agents. There's, you know, we all use agents as builders to write software, but then real businesses are now using agents not just as like customer support tools or things like that. They're like living inside of systems, changing things, updating records. And so I'm curious like is there an agent that like is your favorite that you know is built on LangChain?
Oh, that's a good question. I mean, my favorite agents are probably the most creative ones. It's probably not inside an enterprise, but I remember meeting someone at Interrupt, which is our annual conference about a month ago, and they were building an agent that sat inside of a video game and act as like an NPC that you could interact with and I thought that was super cool. So those are my favorite agents.
You said in your it was an agentic operating model white paper. So I love that paper. If you guys haven't seen it, I highly recommend go download it, read it. Don't just put it into Claude and let it summarize it. Like actually read the paper. But it's worth the read. Like all these enterprises have gone through so many transformations. We did, you know, an agile transformation. We did a cloud transformation and now we're doing an AI and agentic transformation. There's all the software side of the AI transformation, but what's the organizational change that you see companies needing to like adopt?
Yeah, I think it actually stems closely to the software side as well, or at least they stem from the same thing. Like the reason we need new pieces of software is that fundamentally agents are a new type of software. They're non-deterministic and they're driven by LLMs. And so the same thing affects kind of like the people and process side of things as well. So people need to understand what non-determinism means and understand how hey, you might run the agent twice and it might produce different results and why is that and what should you look for?
Agents in terms of that maybe gets into building agents. In terms of just like debugging agents, like how do you debug agents? What is the context window even mean? How does that affect things? How can you change things to affect that? In terms of using agents, hallucinations exist and you need to not have it be completely autonomous. You want to have humans in the loop at some very kind of like key points and that's a lot of UX design in terms of thinking about these agents as well.
And so, we see that there's, you know, throughout this life cycle, there's new things that generally get inserted. I'd say most of them have to do with kind of like the quality of the agent. And so, I think building evals and thinking about evals up front is a big part of that. And then likewise, just like there's like SREs for applications in production, there is this new style of like AI reliability engineer that's looking at these agents in production and is it actually doing what users want or users using it in new and kind of like interesting ways.
And so, I think it's fun and easy to get swept up in kind of like all these agent changes, but it's important to remember like what's actually changing at the end of the day. And I think the biggest thing is now these LLMs are part of the applications you're building and that means new tech, but that also means new skills and new processes.
So, I started software development at a little company called ThoughtWorks doing test driven development. And actually the root of test driven development being pulled through into agentic where you start with those evals and you just like define what this agent should be doing with a handful of examples just seems like such a nice way to begin at least.
There's a good I don't know who's multiple people have said this, but like evals are like the new PRD. Like it's basically defining the spec of what the agent should do. How do you want to handle what scenarios do you even want it to handle? How do you want it to handle those scenarios? And I think there's a lot of I think it's hard to even enumerate that and then it's hard to create evals that properly kind of like capture that as well. It's a real skill, but I think just like I think it'll become a crucial part of building agents.
Yeah, and they're like they evolve with the agent over time as well. So you're like you might start with just a small handful cuz your agent only does one thing and then you add a few and you add a few.
And it's really annoying because agents evolve fast as well. So we have an agent that we just launched in our product called LangSmith Engine. So basically LangSmith is our observability platform. You trace all your agents there. We have an agent that sits in LangSmith, looks at it, runs in the background, runs like on a schedule every 6 hours, looks at all these traces, tries to identify issues and propose fixes. It can propose code changes or adding evals or adding data sets.
So we're writing evals for this. And we want to create issue bench. This is a benchmark of issues and how well this agent can identify it. A is actually quite hard to create this. We've been working on this like two or three weeks already internally. And now by the time like we still haven't finished and the agent's already changed. Like it's already doing way more things than we thought it would initially. And so it's hopefully like I think as it becomes this is still a very new agent for us and so hopefully as it becomes more mature there's less kind of like volatility in some of these evals which we do see.
But yeah, it's you got to keep on evolving them and agents evolve fast and evals need to evolve fast.
100%. I do love that you say this is really hard, it's taking two to three weeks and we haven't finished. Like [laughter] not a world of software that existed six months ago. That's wonderful though.
Yeah, it's such a fast moving world and we get to do it. I actually one of the things is like
that's actually incredible because when a bank you're building an agent at a bank and a bank has a six-month change review process or a quarterly release process and you're in this like rapid speed of this is hard, and we're not done in 3 weeks. Like, how do you reconcile those two things? And how is a bank or an airline or an insurance company going to stay on top of agents?
It's a good question. I mean, I think there's like a few I think there's a few of kind of like factors that play here. So, one like we're still very much in the very, very early innings of our agent. Like, one of the things that we recommend is that you ship early and then iterate quickly. And so, we're trying to dog food our own advice and I think we built the beta version of this in like or we built the like very beta version of this in a month.
We dog food it with customers for like 2 to 3 weeks. We launched at Interrupt, which is about a month ago, so it's been out since then. So, it's still pretty early on and we're constantly kind of like evolving it. And I do think we have a luxury that you may not have inside of enterprises. Like, I think we there's less red tape for us to kind of there's less organizational kind of like red tape for us to run through. And so, shipping quickly and iterating quickly after that is easier said than done, but I think we're trying to do that as much as possible.
I think that was like a big part of that Agile transformation, like moving away from waterfall development and where, you know, you have these huge release gates and you get a little bit more iterative and a little bit more iterative. And then we've gotten into the cloud, so our infrastructure is more dynamic now. So, this is just this next phase, what it feels like, of pushing our companies to release faster. And there's definitely risk that you have to put around it. I like the United guys in their talk, they say like, you know, sometimes you're dealing with payments data or PII, and so maybe we slow that part down, but this one other agent over here can move super fast.
I think there's also UX design things you can do. So, the first version of our agent, which we didn't ship, actually opened PRs directly in a repo. And I actually had a really hard time understanding the UX of that agent. And then at that point we had two people on the team, Ben and Palash, who were pushing this agent forward and they've done a terrific job with that.
And when it was opening PRs, to be honest, like I didn't get the agent at that point. I was like, I don't want an agent opening up PRs. It's going to like we have so many AI-generated PRs on LangChain repos. I was like, I do not want more of those. And then we changed the UX to be more similar to if people have used Codex Security, basically their product where you point it at your codebase, it runs in the background, it then shows you issues, you go in, and there's a button and you can see the changes that it would make, and then you click that button and opens a PR.
And so we changed it to that. And I think that's a really good example of, again, practicing what you preach, which is basically like find these places where you put a human in the where you can do a bunch of work, but put a human in the loop at the end, so that they're still reviewing it, they're still in control. And so I think that was a small UX change that we made that, you know, changed it from being something that I was struggling kind of like to justify shipping quickly. To now it's like, okay, we ship it fast. The worst thing that happens is you just don't look at this tab in LangSmith. Like nothing bad happens. And so I think we limited the blast radius a little bit.
I like that. And I really like the engine product. Like it suggested some things for our little Richie robot outside, and we've been using it on other stuff as well. I like the UX thing is actually really interesting. So, we have the luxury of building agents for dozens and dozens of customers and enterprises. We see a lot of different patterns. And I think everybody naturally gravitates to a chatbot, like because that's what they use for AI. But the most impactful applications are actually when we've deleted the chat window, and the AI just kind of runs in the background, and it almost understands what the user should have been asking. They don't need to understand a magic incantation of words to call the right tools and all these other things. Are you guys seeing that kind of stuff, too? How do deep agents fit into that? I know you used to talk about ambient agents a lot.
100% we're seeing this. Like we've added a few agents in our product. I think Engine is the third one. So, first I think first technically we launched Insights, which was kind of like a similar thing, but it didn't take actions. It just showed you kind of like information. And then after that, we launched Polly, I think now renamed to like LangSmith Chat or something like that, which is just a chatbot. And then we launched Engine, and I think Engine in terms of UX, we've heard much better feedback and we think it's much more impactful for the reasons you described.
It runs in the background, so you don't have to kick it off. So, like knowing when to kick it off is actually like a cognitive [clears throat] burden that we want to overcome. And then Insights was great. It would show people interesting things, but like what do you do next? That was like how can we close the loop more? How can we go further? Because it's great to have these insights, but you want to action on those insights at the end of the day.
And so, yeah, I think we wrote a blog post over a year ago at this point called kind of like ambient agents, which is basically agents running either like on a cron job or triggered by events basically in the background. I totally think that's the future of where things are going. If you think about an enterprise in particular, there's a ton of processes that are happening driven by humans, and a lot of these if you think about an AI-powered enterprise, you don't want them fully autonomous.
And so, human in a loop becomes really important here. How do you surface to the user what the agent is doing, and at what points does it need to step in and approve things? But you do want these processes running in the background that just do things. Like if I think about agents sometimes the same way as like a smart intern that you have at your company. You want the intern to learn to do things over a longer period of time. If you're constantly just like slacking with them, it's exhausting. Like it doesn't give you leverage. And so, I think background and ambient agents gives you way more leverage.
100%. We've done a handful of agents where actually like in early in the engagement, we delete the chatbot. So, like working with an insurance company, they had a chatbot spec- especially for their like claims agents, human agents. And instead of having the no one used the thing. And so, instead of having the claims AI agent be its own little chat, we pulled it into the ticketing system that they already used, integrated with that. The AI agent would, you know, look through the tickets, look for ones that it knew it could handle, pick them up, add information to them.
We did the same for Let Us Entertain You here in Chicago with a handful of agents there. Went for a bank, delete the chatbot, get in there and solve the issues. Kind of the agent can know what the people want based on the context. You have that focus framework. There's context around that can trigger an event and then just like shows up for the person versus being asked.
I think one of the trickiest things with ambient agents is the like UX for how you expose either what it does or the work it did or when it gets stuck. I'm curious for those things like did you end up building your own UX to see the agents or did you integrate that with like an existing platform?
So, that on the technical side, we use LangSmith for all of that. And actually, LangSmith has been a really powerful like governance tool as well. So, there's a big question is always when an ambient agent exists, where does accountability lie? So, if someone prompts a chatbot and it does something, it's really easy to say, 'Oh, we'll forward those credentials through and it's that person's responsibility.' But, if there's just something doing stuff, especially if there's write action to take, but even read action could like, you know, break past a permissions boundary or something like that.
We use LangSmith for a lot of that. And so, LangSmith, you know, with evals, with online evals, taking sample data, you can start to have alerts understand if these things are doing what they should be doing. And then, for the user specifically, we meet them where they are. So, for the bank, it's, you know, wanting to meet the finance managers in their inbox rather than in a chat window. For the claims agents, we want to meet them in the claims management tool that they already had.
And I think that's actually why like OpenClaw and Hermes agent were so popular is like they were just the interface was right in what we all use. It's in WhatsApp, it's in Telegram, it's in Discord. Deep agents existed everywhere before then, but they were right there in the tools we already use, so it just felt more integrated into our lives.
I had actually not thought of LangSmith being more powerful for ambient agents because of that, but that's good. I'm going to steal that.
Good. Please do. Please do. I'm going to steal the focus framework from them because it applies really well to my company.
[laughter]
Who writes the evals? I think is one of the really hard questions.
Yeah, it is. And I think it's getting harder as evals increase in complexity. So I think if you think about kind of like writing evals for a single shot LLM call. I think what we saw is that there was a big push to have product managers, domain experts, subject matter experts involved in that. Whether it's by writing There's different types of evals as well.
So one type that we saw was people would write out like the ground truth. Another type we call kind of like criteria eval where it's harder to define the ground truth, but you can write out some criteria like this should mention X or this should not mention Y or something like that. And so we have the annotation queues in LangSmith where we saw a bunch of people sending runs and then domain experts would come in and label those and those would get brought into data sets.
I think it's becoming trickier for a lot of these like longer running stateful agents. So one of the things that I've actually been playing around with in my spare time is there's an open source framework for running evals called Harbor. Which if people are familiar with Terminal Bench 2, which is a really good coding benchmark, this is the framework that uses. And so we've done a bunch of integrations with Harbor and so that you can run deep agents in it. Everything gets logged to LangSmith. You can use LangSmith sandboxes to run a bunch in parallel.
But the tricky thing is you basically have to create like a little simulated world for these agents to run in. And you have to do that because these agents are now like really complex. Sometimes they're like interacting with stateful systems. And so you need to mock those out. You can't have like So we use this for our issue bench thing for LangSmith Engine. We don't want to hit production LangSmith servers, so we mock out a fake LangSmith server.
We have to purposefully create simulations where we know that it is bad or where we know that it is good ahead of time because that's the best way to get verifiable kind of like evals. And that's why it's taking us to do it. Like it's a hard task. And I think like I think there's you know and I don't know what this will look like in the future because I do think it's domain experts that are the people that are best qualified to say kind of like what is right, what is wrong.
But if you have to create this like mock simulated world, like that's a pretty tough engineering challenge as well. And so I think it's actually yeah I guess I think it's like slightly easier. I think yeah evals are I don't want to say easy but easier for kind of like single shot things. But for agents, they're really really hard.
Especially when you start getting like you know multi-turn tool calls, that kind of stuff, failures, that
100%. I mean, we find ourselves like the you know in the race of the enterprise, a software team would often like consult a subject matter expert. So if you're building a bag handling app at United, which I had the privilege of doing in a past life, we had like baggage handlers and people who ran those people like ran the belts and all that kind of stuff consult with us on what the app should feel, look, and act like.
Now I actually see them becoming more and more integrated in the team. We can prototype so much faster. So instead of like using like archaeology to extract information out of them, we can actually like build a prototype be like, 'Does this make sense?' And then that turns into an eval. And then that lives there. One of the things that happens though is these evals are compounding in cost. And eventually they become a little bit eye-popping to a CFO.
So like we used to run you know you could run a JUnit test suite and it cost zero effectively. So what is how do you balance the cost of evals to determine like the testing of deterministic software around them and that kind of stuff?
There's different types of evals and you run them at different stages. So we have some more like sanity check evals that we run for Engine that are smaller things like does it make the first tool call correctly? Like they're basically single pass kind of like unit test for in specific situations. And then we have issue bench which is our long end-to-end thing. Yeah, I don't like seeing how much it costs to run on that.
And I think like you just run them at different periods. So like we see people doing there's also like these are all automated evals but there's also a big kind of like manual QA process for a lot of these apps as well. And so a lot of companies base like you know it depends a bit on your release process but if you're rolling a big change out you might do the heavy benchmark plus manual QA and have people go through. If you're doing smaller changes you might run these small kind of like smoke unit tests. There's basically a gradient that you kind of build up.
So you just rebuild the testing pyramid.
Yeah.
It all comes full circle. I mean like the resource we had before was time. Right? And so we would run the integration tests infrequently and we'd run the unit test frequently because integration were expensive in time. Now they're expensive in time and money which makes CFOs care a lot more.
I'm thinking about like conversations I'm having when we're out talking to customers or potential customers and one of the things that comes up all the time is in the cost space or the FinOps space as well like how do you decide what model to use and how do you make sure you know this open source model is as good as this you know Opus model or something like that. So do you guys benchmark all your models? What's your process there?
Yeah, I think and by the way actually this ties back to something you asked earlier which I forgot to talk about which is like basically the change management of a lot of these systems. Like yes, you want to iterate quicker on the underlying system, but another thing that we have happening that didn't exist in software is new models come out and that's or models might get deprecated and that's outside of your control. You have to be able to update the agents not just because you're learning about what makes the agent better, but because there's a new model or the old models get deprecated and so you need to basically have these evals in place to ensure those get rolled out smoothly.
In terms of what we see in terms of models, I mean we see I think over the past few months we've started to see a much bigger focus on cost. I think adoption is finally getting there basically. I think you had Uber's CFO came out and said they blew through all of their tokens for their coding agents in the first four months of the year. And you know they're an AI forward organization and so probably not all organizations are like this, but we're like this internally.
We yeah, we absolutely are starting to look at our coding agent costs and evaluating open source models, evaluating what the heck we do when Anthropic gets rid of their $200 max plan because it's subsidizing so much use. And I'm very bullish on open source models. I think GLM 5.2 came out last week. And from all I would say I actually was using it literally today.
And the reason I'm talking about coding agents is because honestly I think like that's where a lot of the usage is. I would bet enterprises are spending more on coding agents than they are on first party agents that they provide for the most part. There's obviously some exceptions, but I do think the same thing will happen with first party agents as well. We see it happening with Engine. At the end of the day Engine's a coding agent that runs.
And so, we use different models for sub-agents. So, we use cheaper models for sub-agents that we spin up more frequently. We're definitely benchmarking kind of like open-source models versus Haiku. This is also why it's really important to have kind of like a model-neutral harness. It's not just that you'll want to switch the main driver from Opus to GPT, it's that you'll want this system to be an amalgamation of different models for different needs at different points in time.
And so, generally we see kind of like a pretty smart main driver with some sub-agents that are smaller and faster and cheaper in quality. And so, yeah, I think being able to mix and match models at different points in your agents really important.
When we do this, we take a very scientific eval-driven approach, essentially. So most of the stuff we end up building is not a coding agent, actually. It's a use case agent. There's a workflow that exists inside of a company. They think that that workflow can be automated in some way that couldn't with traditional software. You build the evals for that, and then you prove essentially that there is like efficacy in the LLM. So, you take the most expensive, smartest model that exists, and you just say, 'Cool, we prove that this can be done.' And now we have a benchmark.
And then we start to look for the intersection of like cost and reliability that is acceptable on that agent. And of course, agents can have many models. So, you're looking inside of each component on tuning that, but you need those like unit-level evals to do that. And then you need the kind of integration smoke-level eval to make sure the whole thing did, you know, each unit worked, but the whole thing doesn't kind of thing. But you can kind of start rationing down and ratcheting down.
And we're seeing a lot more interest in open-source models, and a lot more interest in or just turning the model down from like an Opus or a GPT-5.5 to something faster. Even if it's not cost, the latency starts to become an issue, especially in these like kind of active agents where you're talking to them.
This is why Austin and Focus are the best. Like that is such a good answer and that's exactly the right thing to do. And so yeah, I mean if you guys haven't already if people if you want that to be done at your company, talk to Austin and Focus.
Thank you.
[laughter]
And are you seeing people like fine-tuning models?
We're starting to see more conversations around it. Yes, but not in production yet. A lot of these enterprises have really sophisticated data teams and so they're treating older more traditional like regression models as an integration point for like a tool call, but they're not fine-tuning open-source like Gemma or something like that yet. They're talking about it.
Yeah, we're mostly seeing that in some of the more like AI natives that are running at really high scale because I think when you have that, you A have data, you B have a lot of cost you need to manage, and C you found a workflow that's clearly valuable and people are willing to pay for that. And so we're starting to see more of that in the AI natives. We're thinking a bunch about that and like how you can use traces and LangSmith to do that. So if anyone here is interested in exploring that, would love to chat about that.
I mean the fine-tuning stuff is going to come. And before I started Focus, I ran engineering for a division of Braintree. And we had an incredibly sophisticated data team doing risk management. And so as those people move out of Hadoop and into how do they fine-tune an LLM? Like that's going to come and those people are there, the knowledge is there, and the data is there. And so it's just going to get pulled in, I think.
I think one thing I didn't touch on that does come up in a lot of sales calls is authentication and authorization. Like that is a really tough question. You have an ambient agent, you have a human kicked off agent. What patterns are you seeing come out of the AI natives space? What are your enterprise customers doing when it comes to like how do you authenticate an agent and get that all the way through tools and back out again?
Yeah, there's generally two patterns that we've seen and we've built these both into Fleet, which is our no code offering. It's easier to do it in Fleet than in the open source because open source is so open you can do whatever you want. In Fleet we have more control and so that's where we put a lot of our best practices. But there's two patterns. One is called kind of like on behalf of. So when a user is interacting with an agent the agent will basically act on behalf of the user.
So if it's interacting with Slack it will use my usually we use MCP and MCP uses OAuth 2 as their credentialization broker. So it will use OAuth 2 or use my OAuth token to connect to Slack. It will only see what I can see and then pass it back. The other model we see is we call them claws but it's basically just like a fixed identity for the agent.
And I do think OpenClaw started to popularize this more because you start to think of the agent as its own person. I know multiple companies talked about making it easy for OpenClaw accounts to set up their own accounts. Because if I want to give OpenClaw access to Box for whatever reason, I might not
necessarily want to give it my Box account, but I want it to have a Box account. And so you can create a seat there almost for the agent. I do think there are interesting middle grounds that are not super solved by either of these. So you might want an agent that has my credentials but only a subset of them. And that fine-grain access is not super well supported by a lot of providers these days.
So one of the things we do in Fleet is, sure, you can give it access to the Notion MCP server, but you can maybe remove the write tool so it only has read. That doesn't perfectly solve it because I might want to say, hey, only give it access to these pages in Notion, and that's not solved. So it's not a perfect solution. And then the other thing you can do is you also just have kind of like authentication around all of these agents. So if you have a Claude-style agent that has access to a fixed set of credentials, you might not want everyone in the company to have access to that. And so we've built that into Fleet in terms of having just like a control over who can access it, who can run it, who can edit it.
There is like a security model that you kind of have to hammer into your security teams, which is like sometimes agents act as a user inside of systems and they get provisioned user credentials. And then you just need really, really good tracing, auditability, observability of what's happening. And you need all the agent observability that like we love out of LangSmith. And then you need traditional observability as well to look over like the tool call that called the other system that called six other systems, pulled data from a database, and then like work its way back out.
We built LangSmith observability initially to help with just like making the agents more accurate and hallucinate less. But it's a fantastic like audit log for everything that happens. And we see a number of people using it that way.
It is a primary like conversation point when we go into enterprises and they're like, 'How do we know what they're doing?' Like a LangSmith demo is like, this is your governance tool. It is your audit log. This is your data retention. If a compliance officer or regulator comes and says, 'Why did this agent do this?' like the high-cardinality querying of the agent stuff lets you essentially pull exactly what happened and be like, 'This is what happened and why.'
There's a great story I love to tell of a data scientist at one of the big banks in New York. And they have to go through a model risk management process when they're trying to publish anything in AI. And it's scheduled for 30 minutes, but it usually always goes over. And she just rolled up, showed them LangSmith for the tracing of everything that happened, showed them evals as well so she could say like, hey, these are the scenarios that we tested, this is how it does, and she finished it in record time.
Amazing. It's so funny like these things just happen, you know, trends come around. Life repeats itself. At HCSC in a past life, I remember sitting in a room with their security officer, head of software engineering, and a few other people and we were trying to explain how like filling out a 100-question Excel document for every change was like super redundant because they now had like Jenkins set up, they had Git, they had all these other things. And with CI and Git, you had every question there and you couldn't lie. Like because in the Excel document, people kind of like, well, that one, yeah, we didn't touch that. So it was like this and this is just happening again. And I think LangSmith is a really great place. Like the visualization it gives people is incredible. I think like the last question I want to talk about because I actually like struggle with this and it's why I love LangChain. So we bet focused on LangChain hard. And the reason we did that is because it's the same reason like I love like any framework. Like I like going, I like abstracting what I think is like trivial or eventually going to be something that I want to swap. And so where do you see differentiation coming in like the frontier model space over time? Is it going to be the harness? Is it like LangChain is going to commoditize all of them and it's just like everybody should have started there, but they'll eventually be there kind of thing?
Like how will we differentiate the...
Yeah. Yeah, and what matters for these frontier labs? Like why are they pushing their agent SDKs so hard? That kind of stuff.
