Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AI

Code's not even the right verb anymore, right? But I have to express my will to my agents for 16 hours a day. Manifest.

How can I have not just a single session of Claude Code or Codex or some of these agent harnesses? How can I have more of them? How can I do that appropriately? The agent part is now taken for granted. Now the claw-like entities are taken for granted and now you can have multiple of them and now you can have instructions to them and now you can have optimization over the instructions. But there, I mean, this is why it gets to the psychosis is that this is like infinite and everything is skill issue.

Yeah, thank you for having me.

Yeah, you could say that.

Yeah, I kind of feel like I was just in this perpetual, I still am often in this state of AI psychosis just like all the time. Because there was a huge unlock in what you can achieve as a person as an individual, right? Because you were bottlenecked by your typing speed and so on. But now with these agents, it really, I would say in December is when it really just something flipped where I kind of went from 80/20 of like, you know, to like 20/80 of writing code by myself versus just delegating to agents. And I don't even think it's 20/80 by now. I think it's a lot more than that. I don't think I've typed like a line of code probably since December basically. Which is like an extremely large change. I was talking to it, like for example I was talking about it to for example my parents and so on and I don't think like a normal person actually realizes that this happened or how dramatic it was. Like literally like if you just find a random software engineer or something like that at their desk and what they're doing like their default workflow of building software is completely different as of basically December. So I'm just like in this state of psychosis of trying to figure out like what's possible, trying to push it to the limit. How can I have not just a single session of Claude Code or Codex or some of these agent harnesses? How can I have more of them? How can I do that appropriately? And then how can I use these claws? What are these claws? And so there's like a lot of new things. I want to be at the forefront of it, you know, and I'm very antsy that I'm not at the forefront of it. And I see lots of people on Twitter doing all kinds of things and they all sound like really good ideas and I need to be at the forefront or I feel extremely nervous. And so I guess I'm just in this psychosis of like what's possible because it's unexplored fundamentally.

Yeah. What is it limited by? Just I think everything, like so many things even if they don't work I think to a large extent you feel like it's skill issue. It's not that the capability is not there. It's that you just haven't found a way to string it together of what's available. Like I just didn't give good enough instructions in the agent's MD file or whatever it may be. I don't have a nice enough memory tool that I put in there or something like that. So it all kind of feels like skills when it doesn't work to some extent. You want to see how you can parallelize them etc. And you want to be Peter Steinberg basically. So Peter is famous. He has a funny photo where he's in front of a monitor with lots of, he uses Codex. So lots of Codex agents styling the monitor and they all take about 20 minutes if you prompt them correctly and you use the high effort. And so they all take about 20 minutes. He has multiple, you know, 10 repos checked out and so he's just going between them and giving them work. It's just like you can move in much larger macro actions. It's not just like here's a line of code, here's a new function. It's like here's a new functionality and delegate it to agent one. Here's a new functionality that's not going to interfere with the other one. Give it to two and then try to review their work as best as you can depending on how much you care about that code. Like what are these macro actions that I can like manipulate my software repository by? And like another agent is doing some like research and another agent is writing code, another one is coming up with a plan for some new implementation. And so everything just like happens in these like macro actions over your repository. And you're just trying to become like really good at it and develop like a muscle memory for it is extremely, yeah, it's very rewarding number one because it actually works. But it's also kind of like the new thing to learn. So that's why hence the psychosis.

Yeah. You're not maximizing your subscription at least and ideally for multiple agents like if you run out of the quota on Codex you should switch to Claude or whatnot. I don't know like that's what I've been trying to do a little bit and I feel nervous when I have subscription left over. That just means I haven't maximized my token throughput. So I actually kind of experienced this when I was a PhD student you would feel nervous when your GPUs are not running like you have GPU capability and you're not maxing out the available flops to you but now it's not about flops it's about tokens. So what is your token throughput and what token throughput do you command? I would actually argue that it's very interesting that we had you know at least 10 years where in many engineering tasks people just did, they didn't feel compute bound, right? And like the entire industry feels that now they feel like they felt resource bound. And now that you have this big capability jump you're like oh actually it's not you know my ability to access the compute anymore like I'm the binding constraint.

Which is very empowering because yeah, because you could be getting better. So that's why I think it's very addictive because there's unlocks when you get better.

Yeah. What does mastery look like right at the end of the year or like two three years 5 years 10 years etc. Well I think everyone is basically interested in like going up the stack. So I would say yeah it's not about a single session with your agent. Multiple agents how do they collaborate and teams and so on. So everyone's trying to figure out what that looks like. And then I would say claw is also kind of an interesting direction because it really when I say a claw I mean this like layer that kind of takes persistence to a whole new level. Like it's something that like keeps looping, it's not something that you are interactively in the middle of. It kind of like has its own little sandbox, its own little, it kind of like does stuff on your behalf even if you're not looking kind of thing. And then also has like maybe more sophisticated memory systems etc that are not yet implemented in agents. So Open Claw has a lot more sophisticated memory I would say than what you would get by default which is just a memory compaction when your context runs out.

Yeah. There's like I think there's at least I think there's a lot of really good ideas in here. Yeah. Good job Peter. I mean Peter has done a really amazing job. I saw him recently and I talked to him about it and he's very humble about it but I think he innovated simultaneously in like five different ways and put it all together. So for example like the soul and the document like he actually really crafted a personality that is kind of compelling and interesting and I feel like a lot of the current agents they don't get this correctly. I actually think Claude has a pretty good personality. It feels like a teammate and it's excited with you etc. I would say for example, Codex is a lot more dry. Which is kind of interesting because in ChatGPT Codex is like a lot more upbeat and highly cyclical. But I would say Codex the coding agent is very dry. It doesn't seem to care about what you're creating. It's kind of like, oh, I implemented it. It's like, okay, but do you understand what we're building? You know, it doesn't. The other thing I would say is for example with Claude I think they dial the psychopathy fairly well where when Claude gives me praise I do feel like I slightly deserve it because sometimes I kind of give it like not very well-formed thoughts and I give it an idea that I don't think it's fully baked and it doesn't actually react very strongly. It's like oh yeah we can implement that. But when it's a really good idea by my own account, it does seem to reward it a bit more. And so I kind of feel like I'm trying to like earn its praise which is really weird. And so I do think the personality matters a lot. And I think a lot of the other tools maybe don't appreciate as much and I think in this aspect also Peter really cares about this and so that was correct and then the memory system and then just you know he's just having fun with this and then the single WhatsApp portal to all of the automation.

Yeah. So in January I had a claw, I went through a period of claw psychosis. So, I built, I have a claw basically that takes care of my home and I call him Dobby the elf claw. And basically I used the agents to find all of the smart home subsystems of my home on the local area network which I was kind of surprised that worked out of the box. Like I just told it that I think I have Sonos at home. Like can you try to find it? And it goes and it did like IP scan of all the basically computers on the local area network and it found the Sonos thing, the Sonos system and it turned out that there's no password protection or anything like that. I just logged in and it's like oh yeah you have these Sonos systems installed let me try to reverse engineer how it's working. It does some web searches and it finds like okay these are the API endpoints and then it's like do you want to try it? And I'm like whoa like you just did that. And I'm like, 'Yeah, can you try to play something in the study?' And it does and music comes out and I'm like, 'I can't believe I just typed in like, can you find my Sonos?' And that suddenly it's playing music. And it did the same for lights. And so basically like it kind of hacked in, figured out the whole thing. Created APIs, created a dashboard so I could see the command kind of center of like all of my lights in the home. And then it was like switching lights on and off. And you know, so I can ask it like Dobby at sleepy time. And when it's sleepy time, that just means all the lights go off, etc. and so on. So, it controls all of my lights, my HVAC, my shades, the pool and spa and also my security system. So, I have a camera pointed outside of the house and anytime someone rolls in, I have a Qwen, a Qwen model that looks at the videos. So, first of all, there's change detection, right? And then based on change detection, it goes to Qwen and then it actually like tells me, it sends me a text to my WhatsApp. It shows an image from the outside and it says, 'Hey, a FedEx truck just pulled up. FedEx truck just pulled up and you might want to check it and you got me mail or something like that.' And Dobby just texted me this, this really incredible. So Dobby is in charge of the house. I text through with it through WhatsApp. And it's been like really fun to have these macro actions that maintain my house. I haven't like really pushed it way more beyond that and I think people are doing a lot more crazy things with it. But for me even just a home automation setup, I used to use like six apps, completely different apps and I don't have to use these apps anymore. Like Dobby controls everything in natural language. It's amazing. And so I think like I haven't even pushed a paradigm fully, but already that is so helpful and so inspiring I would say.

Yeah, I think to some extent that's right. It's like working backwards from how people think an AI should be because what people have in their mind of like what an AI is is not actually what an LLM is in a raw sense like LLM is a token generator, you know, like more tokens come out, but what they think of is like this persona identity that they can tell stuff and it remembers it, you know, and it's just kind of an entity behind a WhatsApp. It's like a lot more understandable. So I think to some extent it's like matching the expectations that humans already have for what an AI should behave but under the hood it's like a lot of technical details go into that and LLMs are too raw of a primitive to actually type check as AI I think for most people if that makes sense.

Yeah. I think there's this sense that these apps that are in the app store for using these smart home devices etc. these shouldn't even exist kind of in a certain sense like shouldn't it just be APIs and shouldn't agents be just using it directly and wouldn't it like I can do all kinds of home automation stuff that any individual app will not be able to do right and an LLM can actually drive the tools and call all the right tools and do pretty complicated things. And so in a certain sense it does point to this like maybe there's like an overproduction of lots of custom bespoke apps that shouldn't exist because agents kind of like crumble them up and everything should be a lot more just like exposed API endpoints and agents are the glue of the intelligence that actually like tool calls all the parts. Another example is like my treadmill. There's an app for my treadmill and I wanted to like keep track of how often I do my cardio. But like I don't want to like log into a web UI and go through a flow and etc. Like all this should just be like make APIs available and this is kind of going towards the agentic sort of web or like agent first tools and all this kind of stuff. So I think the industry just has to reconfigure in so many ways that it's like the customer is not the human anymore. It's like agents who are acting on behalf of humans and this refactoring will probably be substantial in a certain sense. One way that people sometimes push back on this is like do we expect people to vibe code some of these tools? Do we expect normal people to do this kind of stuff that I described? But I think to some extent this is just you know technology as it exists today and right now there is some vibe coding and I'm actually watching it and I'm working with the system. But I kind of feel like this kind of stuff that I just talked about this should be free like in a year or two or three. There's no vibe coding involved. This is trivial. This is table stakes. This is like any AI even the open source models etc can like do this.

Extremely easily. Yeah. Today it's vibe coding it's involved and not many people are going to do it. But and you still have to make some design decisions, right? We were talking about like you take frames for example. Yeah. But I kind of feel like this will just start to, the barrier will just come down and it's just ephemeral software on your behalf and some kind of like claw is handling all the details for you but you're not involved. Claw has a machine and it will figure it out and it's just presenting you UIs and you're like saying stuff you know.

Yeah. I just feel like I'm so distracted by everything. So I spent like a week on the claw stuff and I have more to do almost. But I will say that like the tools were all just busier unfortunately. Yeah. I didn't really take advantage of a lot of like email and calendar and all this other stuff and I didn't give it access because I'm still a little bit like suspicious and it's still very new and rough around the edges. So, I didn't want to give it like full access to my digital life yet. And part of it is just the security, privacy and just being very cautious in that realm. And so some of it is like held back by that I would say. Yeah, maybe that's like the dominant feature, but some of it is also just I feel so distracted because I feel like I had a week of claw and then other stuff is happening.

Auto research. Yeah. So I think like I had a tweet earlier where I kind of like said something along the lines of to get the most out of the tools that have become available now you have to remove yourself as the bottleneck. You can't be there to prompt the next thing. You need to take yourself outside. You have to arrange things such that they're completely autonomous and the more you, you know, how can you maximize your token throughput and not be in the loop. This is the goal and so I kind of mentioned that the name of the game now is to increase your leverage. I put in just very few tokens just once in a while and a huge amount of stuff happens on my behalf and so auto research like I tweeted that and I think people liked it and whatnot but they haven't like maybe worked through like the implications of that and for me auto research is an example of like an implication of that where it's like I don't want to be like the researcher in the loop like looking at results etc like I'm holding the system back so the question is how do I refactor all the abstractions so that I'm not, I have to arrange it once and hit go. The name of the game is how can you get more agents running for longer periods of time without your involvement doing stuff on your behalf and auto research is just yeah here's an objective here's a metric here's your boundaries of what you can and cannot do and go and yeah.

Yeah I didn't expect it to work because so I have the project data chat and fundamentally like I think a lot of people are very confused with my obsession for like training GPT-2 models and so on but for me training GPT models and so on is just a little harness a little playground for training LLMs and fundamentally what I'm more interested in is like this idea of recursive self-improvement and to what extent you can actually have LLMs improving LLMs because I think all the frontier labs this is like the thing for obvious reasons and they're all trying to recursively self-improve roughly speaking and so for me this is kind of like a little play pen of that and I guess I like tuned NanaChat already quite a bit by hand in a good old fashioned way that I'm used to like I'm a researcher I've done this for like two decades I have some amount of like what is the opposite of, yeah, earned confidence. Okay I have like two decades of like oh I've trained this model like thousands of times of like so I've done a bunch of experiments I've done hyperparameter tuning I've done all the things I'm very used to and I've done for two decades and I've gotten to a certain point and I thought it was like fairly well tuned and then I let auto research go for like overnight and it came back with like tunings that I didn't see and yeah I did forget like the weight decay on the value embeddings and my Adam betas were not sufficiently tuned and these things jointly interact so like once you tune one thing the other things have to potentially change too you know I shouldn't be a bottleneck like I shouldn't be running these hyperparameter search optimizations. I shouldn't be looking at the results. There's objective criteria in this case. So you just have to arrange it so that it can just go forever. So that's a single sort of version of auto research of like a single loop trying to improve. And I was surprised that it found these things that I, you know, the repo is already fairly well tuned and still found something. And that's just a single loop like these frontier labs they have GPU clusters of tens of thousands of them. And so it's very easy to imagine how you would basically get a lot of this automation on smaller models and fundamentally everything around like frontier level intelligence is about extrapolation and scaling laws and so you basically do a ton of the exploration on the smaller models and then you try to extrapolate out.

Yeah, I would say that like the most interesting project and probably what the frontier labs are working on is you know you experiment on the smaller models. You try to make it as autonomous as possible. Remove researchers from the loop. They have way too much, what is the opposite? Way too much confidence. Yeah, they don't know. They shouldn't be touching any of this really and so you have to like rewrite the whole thing because right now I mean certainly they can contribute ideas but okay they shouldn't actually be enacting those ideas. There's a queue of ideas and there's maybe an automated scientist that comes up with ideas based on all the archive papers and GitHub repos and it funnels ideas in or researchers can contribute ideas but it's a single queue and there's workers that pull items and they try them out and whatever works just gets put on the feature branch and maybe some people like monitor the feature branch and merge to the main branch sometimes. So yeah, just removing humans from all the processes and automating as much as possible and getting high tokens per second throughputs and it does require rethinking of all the abstractions. And everything has to be reshuffled. So yeah, I think it's very exciting.

Yeah. So program.md is, we're not in the loop. Yeah, exactly. Yeah. So program.md is my crappy attempt at describing like how the auto researcher should work like oh do this then do that and that and then try these kinds of ideas and then here's maybe some ideas like look at architecture look at optimizer etc. I just came up with this in markdown, right? And so yeah, exactly. You want some kind of an auto research loop maybe that looks for, you can imagine that different program.mds would give you different progress. So you basically every research organization is described by program.md.

Oh, I see. Yes. Yes. Yeah. Exactly. We're going to get something better. Like there's no way we don't. You can 100% look at where the improvements came from and like can I change the program.md such that more of these kinds of things would be done or like things that didn't work. Meta optimization. Yeah, you can 100% imagine doing that. So I think this is a great idea, but it's like you know I think like you sort of go one step at a time where you sort of have one process and then second process and then the next process and these are all layers of an onion like the LLM sort of part is now taken for granted. The agent part is now taken for granted. Now the claw-like entities are taken for granted and now you can have multiple of them and now you can have instructions to them and now you can have optimization over the instructions and it's just like it's a little too much you know but there I mean this is why it gets to the psychosis is that this is like infinite and everything is still skill issue and that's why I feel like yeah that's just coming back to this is why it's so insane.

Yeah. I mean so there's a few caveats that I would put on top of the LLM ecosystem. Number one, this is extremely well suited to anything that has objective metrics that are easy to evaluate. So for example, like writing kernels for more efficient CUDA, you know, code for various parts of a model, etc. are the perfect fit. Because you have inefficient code and then you want efficient code that has the exact same behavior, but it's much faster, perfect fit. So a lot of things are perfect fit for auto research, but many things will not be and so it's just if you can't evaluate then you can't auto research it, right? So that's like caveat number one. And then maybe caveat number two I would say is, you know, we're kind of talking about next steps and we kind of see what the next steps are, but fundamentally the whole thing still doesn't, it's still kind of like bursting at the seams a little bit and there's cracks and it doesn't fully work. And if you kind of try to go too far ahead, the whole thing is actually net not useful if that makes sense. Because these models still are not, you know, they've improved a lot, but they're still like rough around the edges is maybe the way I would describe it. I simultaneously feel like I'm talking to an extremely brilliant PhD student who's been like a systems programmer for their entire life and a 10-year-old. And it's so weird because humans like there's I feel like they're a lot more coupled. Like you have, you know, everything, you wouldn't encounter that combination. This jaggedness is really strange and humans have a lot less of that kind of jaggedness. Although they definitely have some, but humans have a lot more jaggedness. Sorry, the agents have a lot more jaggedness where sometimes like you know I ask for functionality and it like comes back with something that's just like totally wrong and then we get into loops that are totally wrong and then I'm just I get so frustrated with the agents all the time still because you feel the power of it but you also there's still like it does nonsensical things once in a while for me still as well.

Yeah, I think like some of the bigger things is like maybe what's underneath it, if I could hypothesize, is fundamentally these models are trained via reinforcement learning. So they're actually struggling with the exact same thing we just talked about, which is the labs can improve the models in anything that is verifiable, where there's rewards. So did you write the program correctly and does it do the unit test check out? Yes or no? But some of the things where they're struggling is like for example, I think they have a tough time with like nuance of maybe what I had in mind or what I intended and when to ask clarifying questions. Like what I, yeah it's just anything that feels softer is like worse. And so you're kind of like you're either on rails and you're part of the super intelligence circuits or you're not on rails and you're outside of the verifiable domains and suddenly everything kind of just like meanders. Like maybe another way to put it is if you go to, if today if you go to like state-of-the-art model ChatGPT and you ask it tell me a joke, do you know what joke you're going to get? There's the joke.

Yeah. Yeah. So the joke that apparently all of them like laugh the most is why do scientists not trust atoms? Because they make everything up. So this is the joke you would get like three or four years ago and this is the joke you still get today. So even though the models have improved tremendously. And if you give them an agentic task, they will just go for hours and move mountains for you. And then you ask for like a joke and it has a stupid joke, a crappy joke from 5 years ago. And it's because it's outside of the RL. It's outside of the reinforcement learning. It's outside of what's being improved. It's like and it's part of the jaggedness of like shouldn't you expect models as they get better to also have like better jokes or more diversity of them or it's just it's not being optimized and it's stuck.

Yeah, I think there's some decoupling where some things are verifiable and some things are not and some things are optimized for arbitrarily by the labs depending on like what data went in and some things are not and but I mean the premise, there's a premise from some research groups that if you are smarter at code generation or in these verifiable fields you should be better at everything and like the joke situation suggests that that's not happening in all. I don't think that's happening. Yeah, I don't think that's happening. I think maybe we're seeing like a little bit of that but not like a satisfying amount.

Yeah, that's true. Yeah, but it just still means that we're not getting like the story is that we're getting a lot of the intelligence and capabilities in all the domains of society like for free as we get better and better models. And that's not like exactly fundamentally what's going on. And there's some blind spots and some things are not being optimized for. And this is all clustered up in these neural net opaque models, right? So you're either on rails of what it was trained for and everything is like you're going at speed of light or you're not. And so it's jaggedness. So that's why I think like even though the progression is obvious what should happen, you can't let it fully go there yet because it doesn't fully work or it's a skill issue and we just haven't like figured out how to use it. So you know it's hard to tell.

More directly yeah. Instead of just that we have no exposure to that can be confusing as a why is it so good at this but not at this other thing. Yeah, I think currently my impression is the labs are trying to have a single sort of like monoculture of a model that is arbitrarily intelligent in all these different domains and they just stuff it into the parameters. I do think that we should expect more speciation in the intelligences. Like you know the animal kingdom is extremely diverse in the brains that exist and there's lots of different niches of nature and some animals have overdeveloped visual cortex or other parts and I think we should be able to see more speciation and you don't need like this oracle that knows everything. And you kind of speciate it and then you put it on a specific task. And we should be seeing some of that because you should be able to have like much smaller models that still have the cognitive core like they're still competent but then they specialize and then they can become more efficient in terms of latency or throughput on specific tasks that you really care about like if you're a mathematician working in Lean. I saw for example there's a few releases that really like target that as a domain. So there's probably going to be a few examples like that where the unbundling kind of makes sense.

The question makes sense and I guess like what I'm struggling with is I don't think we've seen too much speciation just yet, right? No. We're seeing a monoculture of models. Yeah. So and there's like clearly pressure for like make a good code model, put it back in the main merge again. Yeah. Yeah. Even though there already is pressure on the models. I guess perhaps I feel like there's a lot of very short-term supply crunch and like maybe that causes more speciation now. Yeah. Yeah, I think fundamentally like the labs are serving a model and they don't really know what the end user is going to be asking about. So maybe that's like some part of it because they kind of have to multitask over all the possible things that could be asked. But I think if you're coming to a business and maybe partnering on some specific problems you care about, then maybe you would see that there. Or there would be some very high value applications that are like more niche. But I think right now they're kind of like going after the totality of what's available. I don't think that the science of manipulating the brains is like fully developed yet partly.

So like fine-tuning without losing capabilities as an example and we don't have these primitives for actually like working with the intelligences in ways other than just context windows like context windows kind of just work and it's very cheap to manipulate etc. And this is how we're getting some of the customization etc. But I think if it was, I think it's a bit more of a developing science of how you like more deeply adjust the models, how you have continual learning maybe or how you fine-tune in a certain area, how you get better in a certain area or like how you actually touch the weights, not just the context windows. And so it's a lot more tricky, I would say, to touch the weights than just the context windows. Because you're actually fundamentally changing the full model and potentially its intelligence. And so maybe it's just like not a fully developed science, if that makes sense, of speciation. And it also has to be like cheap enough for that speciation to be worthwhile in these given contexts.

Yeah. So, we talked about our research has a single thread of like I'm going to try stuff in a loop, but fundamentally the parallelization of this is like the interesting component. And I guess I was trying to like play around with a few ideas, but I don't have anything that clicks as simply as like I don't have something that I'm like super happy with just yet, but it's something I'm working on on the side when I'm not working on my claw. So I think like one issue is if you have a bunch of nodes of parallelization available to you then it's very easy to just have multiple auto researchers talking through a common system or something like that. What I was more interested in is how you can have an untrusted pool of workers out there on the internet.

So for example in auto research you're just trying to find the piece of code that trains a model to a very low validation loss. If anyone gives you a candidate commit, it's very easy to verify that that commit is correct, is good. Like someone could claim from the internet that this piece of code will optimize much better and give you much better performance. You could just check very easily, but probably a lot of work goes into that checking. But fundamentally they could lie and etc. So you're basically dealing with a similar kind of it's almost actually like looks a little bit like my designs that incorporate an untrusted pool of workers actually look a little bit more like a blockchain a little bit. Because instead of blocks, you have commits and these commits can build on each other and they contain like changes to the code as you're improving it. And the proof of work is basically doing tons of experimentation to find the commits that work.

And that's hard. And then the reward is just being on the leaderboard right now. There's no monetary reward whatsoever. But I don't want to push the analogy too far but it fundamentally has this issue where a huge amount of search goes into it but it's very cheap to verify that a candidate solution is indeed good because you can just train a single you know someone had to try 10,000 ideas but you just have to check that the thing that they produced actually works because the 99,000 of them didn't work you know. And so basically long story short is like you have to come up with a system where an untrusted pool of workers can collaborate with a trusted pool of workers that do the verification. And the whole thing is kind of like asynchronous and works and so on and it's like safe from a security perspective because if anyone sends you arbitrary code and you're going to run it that's very sketchy and dodgy. So but fundamentally it should be totally possible. So you're familiar with projects like SETI at home and Folding at home all of these problems have a similar kind of setup. So Folding at home you're folding a protein and it's very hard to find a configuration that is low energy. But if someone finds a configuration that they evaluate to be low energy that's perfect you can just use it. You can easily verify it. So a lot of things have this property that you know very expensive to come up with but very cheap to verify and so in all those cases things like Folding at home or SETI at home or auto research at home will be good fits. And so, long story short, a swarm of agents on the internet could collaborate to improve LLMs and could potentially even like run circles around Frontier Labs. Like, who knows, you know? Like maybe that's even possible. Like, Frontier Labs have a huge amount of trusted compute, but the Earth is much bigger and has huge amount of untrusted compute. But if you put systems in place that deal with this then maybe it is possible that the swarm out there could come up with better solutions and people kind of like contribute cycles to a thing that they care about. And so sorry so the last thought is lots of companies or whatnot they could maybe have like their own things that they care about and if you have compute capacity you could contribute to different kind of auto research tracks like maybe you care about certain you know like you care about like cancer or something like that of certain type you don't have just donate money to an institution you actually could like purchase compute and then you could join the auto research forum for that project you know so if everything is rebundled into other researchers then compute becomes the thing that you're contributing to the pool.

Yeah.