Harrison, welcome. It's good to meet you in real life. We've chatted a few times on the networks, just randomly history-wise. So Harrison slid into my DMs and that's how we got together. You're like, 'I have some problems. Can you solve it?' And I believe we solved it, and hence...
Like all good relationships on Twitter. Yeah, exactly. They start on Twitter.
So, to start off this conversation, a lot of things changed, I feel, from our point of view, from December till quite recently in the market, and you have insights deeper than others. So, what happened? Tell us.
Yeah, I think the models got really good and the harnesses got really good, basically. So we started paying more attention. So there's this idea of an agent that's basically a model running in a loop. And I think this has always been the north star for agents. When LangChain launched like three years ago, this was one of the things we had in there. AutoGPT, if you remember that.
Yeah, that was like three years ago. And that took off. And in many senses, I'd say like OpenClaw is an interesting reflection. Yeah. It's like a future version of it, but it's all the same core idea. It's just the agents running in a loop calling tools. And I think they started to get really good at this about a year ago, which I don't think as many people noticed as they should have, probably because they tried to run them in a loop and call tools and they didn't work.
And then they were like...
And then they were like, yeah. And this is why we built things like LangGraph to help structure those things in more regimented ways. But then they started to work. This is when Claude Code came out about a year ago. Manus came out about a year ago. A lot of the deep research stuff came out about a year ago. And those all use the same running in a loop, but they also use this, I would argue, a very similar harness in terms of they have planning tools, they have file systems, a lot of them have code execution.
And so I think this is really when harness engineering started to become more and more a thing. We started to pay attention in like June, July and released our version of the harness called Deep Agents around then as well. And then I think we as an industry kept on building these harnesses and making them better and better. And then Opus came out sometime in November and that really, because all these harnesses, the main exciting part of this harness is it lets the model do more and more things. It gives the model more and more control.
Yeah, more control over what to do. There's less hardcoded things, and so the better that the models get, the more they can do, and the more they can do that reliably. And then they start to really work. And then I think what happened is everyone went on Christmas vacation and started playing around with them, and then they realized that it started to work.
Yeah. So you've written a bunch about deep agents and stuff, and there's an article that you wrote which I liked a lot, which is like, why didn't I write that article? Which is when we started Daytona as this sandbox vendor, we essentially pushed away from running agents inside the sandbox because we felt that that is production versus development, outer versus inner loops stuff. And so we always thought the agent would sit outside and then you'd call it like a tool, like a human turns on your computer. But now, especially with Claude Code and all these others where the agent sits inside of these things. And so you had this Twitter article, I believe, which went into this.
Yeah, I mean, and I'd love to get your take on this because I'm asking everyone what they see, because basically we see people running sandboxes in two different ways. It's about a 50-50 split. And it's funny, when I wrote this Twitter article, I had people on both sides telling me I was stupid for thinking the other side was even a possibility.
That's why you have a lot of viewership on this. It's great.
But like the two things we see are like, one, you stand up your sandbox and then you install the agent in there and you run it inside of there. Or you stand up your sandbox, you stand up your agent separately and you have the agent call it as a tool. And those are the two patterns we see. There's pros and cons to both of them. I think practically what we've seen is that if you're starting from something like Claude Code, which is very much meant to be run kind of like locally, then the first pattern of you just stand up a sandbox and install Claude Code makes a bunch of sense. If you're coming at it from kind of like fresh eyes or from an existing agent and you want to run it in a sandbox most of the time there, then we see people actually standing up the sandbox separately and connecting it as a tool. But it really is like a 50-50 split. I don't know. Are you seeing anything?
Yeah. So we see both on our side, and the interesting part is, as a provider sandbox, you have to have different tools for each of the use cases, which is also an interesting thing that you have to do. And the way I've understood it, you articulated it as well, is I feel that people are used to using Claude Code on their local machine, and so how to get an agent to do something well, the easiest thing is fire off a sandbox and put it inside. You don't have to think about it. It's kind of redundant, but it is an easy way to get it up and running. I believe personally, and this is not the view of the company, but me as a person, is that it's better to have it outside because then you can spin up multiple sandboxes, you can fork these things, you can have knowledge of state of these things across, versus if your agent is inside a sandbox and, for example, you fork it, it actually doesn't know it's a clone, right? It is the same thing. So we're seeing both, but that's just how I think about that.
I think I'm probably aligned with you. I think the separation of the agent's execution environment and the sandbox's execution environment, I think that's a nice separation to have. But we'll see.
How does that affect your tooling or your products around that, depending on how people decide what to do?
So we want to support both. So Deep Agents, which is our agent harness, I think you can pretty trivially run any of these things inside of a sandbox by just spinning up and installing. But one of the pretty unique things that we have in Deep Agents compared to like the Claude Agent SDK or something like that is it has this sandbox abstraction that you can basically pass into Deep Agent. So we have Daytona as a provider, and basically you pass it in there, and all of the code execution that the agent does gets executed inside the sandbox. And so we have a few different primitives for that and a few different hooks, and we have a few different providers, including like a local provider, which is basically just use the local file system.
And so that's something where we've tried to support that a lot and spent a lot more time there because we do think that's a pretty valid deployment option. And I think that's pretty a lot of the other harnesses we see out there are coming. It's kind of funny, like I'd say a harness is somewhere in between a coding agent and a framework. And we started with a framework, LangChain, and kind of went up towards Deep Agents. And then you have Claude Code, which is a coding agent, and went down. And so I think a lot of these code agents don't have that naturally kind of built in because they're so built to be run kind of like globally, locally.
Yeah, that's exactly because it's the easiest way to use them. And so what do you see as like, in you guys as sort of like integrate with all these tools, and I talked earlier with Parag, is like the infrastructure layer is like sandboxes, web search obviously because he does that, file systems, we have Aaron later on talking from Box as well, these things. What are the other things? Like what do you guys see that people use? Are there anything there?
Yeah. I mean, there's the models, obviously. One of the really interesting things we're paying attention to, by the way, I saw Lynn on earlier from Fireworks, is open source models. Like I think practically, if you want to use one of these harnesses to their full ability, you probably need an OpenAI or Anthropic or maybe a Google model at the moment, but we are very, very excited for open source models. So the models, one, this harness that connects everything, I think is becoming a bigger and bigger piece of the puzzle. And obviously there were a ton of frameworks before, but I think a harness is more powerful. It's more batteries included. I think you're much more likely to like... I think the main value of a framework is abstractions, which can be good or can be bad. The main value of a harness is in a lot of the batteries that are included, and I think that's more naturally valuable. So I think this is a big piece. I think things around like memory are really important. And so this includes both just like short-term conversational memory, but also long-term procedural memory, semantic memory, which is also very similar to RAG. So there's some sort of database data structure that's needed to support this memory. I think that's a big piece. Sandboxes are a big piece because we see code execution being a critical part of a lot of agents, not just coding agents. And then there's a long list of tools. Web search is definitely probably at the top of that list, I would say. Yeah, there's such a long tail, but like the biggest individual tool providers that we see, I think have always been kind of like web search ones. I mean, we think observability and eval are a core part of that stack, kind of like orthogonal. They make sure that the agent's actually working as intended.
I think those are what I would say are the standard parts right now. I think in the future, you know, I think there's a lot of buzz around kind of like AI payments, for example.
Like what Stripe launched recently.
Yeah, that's something I think is still early, but like we're excited to see where that goes.
Authentication, anything for the agents or whatnot?
Yeah, good question. I mean, so that ties into kind of like the long list of tools. We're actually thinking a bunch about this. We see kind of like two different patterns there. One is like agents that authenticate on behalf of a user. So it uses kind of like, so if I use it and you use it, we'll get different results because it uses, you know, my Slack credentials.
But another type of agent is kind of like an autonomous or like Claw-style agent. And this is like, I think modeled after OpenClaw, where OpenClaw is, they basically have their own identities. They've got their own memories. They've got their own credentials. Those could be the same as their creator's credentials or they could be a service account. And so I think this is a separate paradigm. I think there's two different paradigms of agents and auth around them. And so yeah, I think auth is becoming an interesting kind of differentiator for these two different types.
Tell me about as much as you can about memory, because that's where I'm personally the lowest. Like you mentioned three types of memory. Are there like standardizations around that? Do you guys have internal memory for the agents that you guys create? Or like tell me more about that.
Memory is like so new, or it's not even new. It's not new. It's just like...
Unsolved. Yeah. We've been thinking about memory for like two years. I think we still have like pretty low-level primitives because I think there are ideas about memory, but I don't know if like a higher-level primitive right now makes sense. So like, you know, there's short-term memory, there's long-term memory. Short-term memory is what happens within a conversation that's large. You know, there's some interesting questions there around like compaction when the conversation gets really long. But that's kind of like the main open question there. Other than that, that's relatively solved, I'd say. And then for long-term memory, and a lot of my learnings here, I've learned so much more about human psychology in the past two years than ever before because of this. Yeah. So there's three types of memory for humans: episodic, procedural, and semantic.
So semantic is basically searching facts. And so semantic search would be a great way to do this. Yeah. Right. And I think we know how to do that. The interesting question here is like, how do those facts get put into the data?
That was my question. Yeah. What, you can do some search, but against what and where is that? Yeah.
So I think we see, so one, like I do think there's a difference between knowledge and memory. So like knowledge would be like, hey, let's ingest like, I don't know, all these documents and put them into a database and great, like semantic search over that is solved. Memory, I think you kind of like learn on the fly almost. And so I think there's two ways to kind of like update memory, not just for semantic memory, for all these memories, but there's two ways. One is to basically have the agent do it as it's running. So the agent, as it's running, if you say like, I like ice cream, the agent could call a tool that like puts that into a database.
Yeah. And it does it as it's running. The other way would be to run basically a cron job that runs once a night or once whatever we want and say, hey, look over all previous interactions and like, you know, store stuff.
This is what we do when we sleep, by the way. Yeah.
Yeah. Sleep time compute is a great name. The folks at Letta came up with that, are basically doing that. And so yeah, so yeah, semantic memory would probably be putting them in some sort of database that supports semantic search. Episodic memory is easier. That's just like previous interactions. And so the way you can do that is you can just give the agent a tool to look at its previous conversations. It can just look it up. Yeah.
I think the most interesting is procedural type of memory, which is like what the agent should actually do, like its core instructions. And so to me, this is things like the system prompt and the skills it has and maybe even the tools it has. And so this again, there's the question of how does this get updated? Does it get updated by the agent like editing those things as it goes or in the sleep? But like that's to me the most interesting type of memory because I think that's really useful for like, I think a lot of the value of agents is having it do things for you. And a lot of those things are oftentimes repeated. Like I have an email agent, it looks at every single email.
