Sierra CXO Summit: Lessons Learned at the Frontier with Anthropic Labs’ Mike Krieger

Yeah, for sure. Hi everyone, good to be here. It's kind of funny initially the idea of a lab sort of within a lab. And this is our really third take on a labs team. I think part of our mantra in labs is also be like constantly evolving the structure itself. So V1 was 2024. I just started at Anthropic. All of product and product engineering was about 30 people. And the models weren't good yet. So it wasn't like they were really accelerating our ability to code. And I kept having this experience where we'd have some interesting idea and then I'd go talk to the product teams and they're like, 'That sounds good except we need to get all this like enterprise readiness things done like site stability like there's just not enough people to handle both Claude AI and the API.' So we thought, 'Okay, it's valuable to then partition a group that's not thinking about next week or next month but really 3 months out.'

And we sort of seeded it with a couple of internal transfers but we really went out and hired either former founders, just people that demonstrated a lot of sort of initiative and drive. And then we ran a less formal process than we run now but basically a process where they were pitching ideas working on them for you know a couple of weeks and then we'd come together and see how they were doing. And I think the role that I saw myself play and then Ben Mann who was sort of the Anthropic co-founder that really got labs off the ground was really helping people to be what we call like even more AGI pilled. So as a really concrete example, we hired Boris Turney who had actually been in Instagram as well although like right after me so we didn't really overlap. And he, we were you know open with him like what kind of things you want to work on here at labs? And he's like, 'Well, you know, I think the models are getting good enough where I think that they can do code linting. Like that could be really useful like people are writing code.' Again, this is 2024 it's like the age of like autocomplete still for code, right? And then maybe we could have the Claude like read the code and see if it's good or not. And Ben to his eternal credit was like, 'I don't think that's AGI filled enough. Like the models are going to get better. Like they were terrible a year ago. They're like getting better now. They're going to keep rising on this. Like what if the models just broke all the code and we pushed them on it and that's what eventually became first what was called Claude CLI internally and then we shipped as Claude code. And we've just run that process over and over again and found good results. But then a funny thing happened which is our model is products graduate with their sort of team into the organization. We did a lot of studying around like what labs how labs teams go sideways. Like anti-patterns like it's like a retirement community for your engineers that are burned out elsewhere like not what we wanted. Or it's like the innovation factory and then like that innovation dies when it joins the main team. So we have this model where every person basically working on a product goes into the sort of receiving team. So we did that for Claude code. We did that for the model context protocol. We did that for our Claude browser extension. And we looked around and we had like two people left in labs. And we're like, 'Okay, maybe labs has served its purpose. Like we've done all this innovation. We've graduated things. Rather than restocking the cupboard, maybe it's time to sort of pause labs.' And we did that for a good year. So end of 2024, maybe like beginning of 25 to end of 25 we basically had no labs team. We had a prototyping team but it was very small. But then near the end of last year, we sort of felt the same pull of even though the teams have grown, you know, work expands to fill the space that the head kind of allotted to it. So all the teams we'd have interesting ideas and say like, 'That sounds great, but we're really busy doing these three or four things.' So we sort of restarted that at the beginning of this year, got that going again. And I think now we run an even sort of more formalized process where every initiative every two weeks we go through a meeting. We used to call it pivot or persevere but we got some feedback that was a little like extreme. Then we went through like four other names and now it's called Miz on Plus for some reason. But basically we have a Miz on Plus meeting with every single one of the teams, but the purpose is the same. It's deciding, is this initiative still learning what it needs to be learning? Did you learn something useful the last 2 weeks? If not, is it time to pivot? Is it time to wind down that initiative? And if it is working, how do we learn even more the next 2 weeks? And that process has been working really well. So, you know, even maybe especially at a company that is accelerated by Claude to a large degree in all of our product areas, it's still valuable to have the team that is not thinking about anything on the road map and is thinking about, all right, what happens if we push this particular thing to the extreme and the models aren't ready for it yet, but maybe they will be a few months from now.

Yeah, it is, it's a term we experience internally. I've been humbled by model progress. I'll give you a really concrete example. I built this Labs only thing that we never shipped called Hatch, which the whole idea was you gave Claude or gave Hatch a pretty high-level thing like, I want an iPhone app that will help people meditate, or I need an internal tool to help manage inventory for my company. And it would go and at the time again, like only March, you know, it's like only 6 or 7 months ago. But at the time if you gave that prompt to a Claude code, it could build a pretty good V1 with a bunch of gaps. You'd be like, oh, but this doesn't save anywhere and Claude be like, oh, I didn't realize you want me to save the data. I thought you just wanted the thing. Or you know, if you build a mobile app, you'd be like it'd be like, I'm done. And you would run it the first time on your device and you'd say, wow, like this button looks like it's completely in the wrong place. So, like verification and completeness and sort of perseverance were like attributes that Claude wasn't good at yet. So, the kind of premise on Hatch was like, we are going to apply the same techniques that GAN networks do or adversarial networks do, but to building products. So, there would be a builder agent, it would build the product, and then a bunch of verifier agents would sort of adversarially review it, and so on and so forth. And it like actually worked extremely well, and there are still things that we built internally using Hatch that are still in use today at Anthropic, which is cool. It means it worked well. But as we talked to other companies about commercializing it, we realized like a whole new surface didn't really make sense. And you really needed to catch people right at the way it was built because it wasn't that flexible. It was powerful but not flexible. You'd have to catch people right before they were about to build something, which is just like a hard timing thing. So, we decided to sort of wind down the project. We upstreamed a bunch of things into other product areas, and kind of let it be. A couple of weeks ago, I tried, this is when we were getting, I think it was when we were getting Fable 5.1 out the door. I tried the same projects that I'd done in Hatch. And Hatch took like weeks of really careful prompt engineering and harnessing and like this whole thing in Fable like out of the box, no extra hardness, no even instructions to like be really thorough in your verification. The only thing I did use was dynamic workflows, which was the one piece we upstreamed from Hatch. But anyway, just with that, it totally outperformed the thing that we had spent months on, you know, kind of perfecting earlier on. So, that is the thing I have to keep learning, which is, you know, one, you can't hold your product shapes too strongly. You have to hold them lightly cuz they may just go away, over time. And two, if the entire sort of value prop of even a labs bed is like, 'Oh, we did some really, really careful harness engineering.' Like, that's fine, and it actually might help push the model. Like, the way we think about it is there's what the model's capable of, and then there's what the model's capable of with like additional sort of scaffolding and harnessing and really careful prompting, and then that plus like some human interventions at the right place. And the idea is like basically capabilities are increasing to the point that's going to consume both of those. And you can capture some value in the meantime there, but be aware that it is temporary.

Yeah, I think a lot of the things I think about, if the theme from going like from beginning of the year to now is really being able to be like sort of thorough and autonomous and really run for hours. It's interesting like earlier in the year I would get on stage announcing a new model and I'd say something like one of our customers like worked with this model like to do 6 hours of work. And at the time that was true but also like end of one like it was like basically Rakuten who has like an amazing harness engineering team and they like really like extracted all of that. Not really most customers. We're now like many people are getting hours of work out of Claude. So like if that's like the piece that happened before I think like the couple that are happening now that we're looking at a lot. One is proactivity. So the models aren't great yet at being like proactive in helpful ways. Like even you know if you set it up to like watch your email or your slack like it'll still be like Mike, don't forget this thing. You're like oh you don't realize I already did this thing. It's like a you know or yeah somebody asked me this but it's actually not that important or urgent like it's okay. I have an air quality monitor in our house and it like alerts you if it like the air quality rises and it's like nine out of 10 times it's like oh you were cooking in the kitchen and forgot to turn on the hood. But like it's really interesting watching Claude be like Mike, there's something really wrong in the kitchen and I'm like or like then like three days later like something keeps happening in the kitchen. I can't tell what it is. Like well it's probably you know something like that. So like nailing proactivity is like a model judgment and taste question I think we still need. I think computer use has come a long way but still needs like some additional leaps both in latency and cost to really like have their breakthrough moment. I feel like it's having more of that breakthrough moment with things like instinct and muse where people are realizing that it can be increasingly useful. So we're thinking about that as well. Claude's ability to interact with into the real world I think is also very interesting. We opened something up called the model hard hardware standard a couple weeks ago around how do you get these different pieces of lab machinery to actually talk to each other and do it with Claude. Claude can actually do a lot of it even without the standard but that kind of helps sort of like if that's like the piece that happened before I think like the couple that are happening now that we're looking at a lot. One is proactivity. So the models aren't great yet at being like proactive in helpful ways. Like even you know if you set it up to like watch your some of these things as well. And then I think the last piece that I'm thinking a lot about is also like Claude's like I think the death of SaaS has been greatly exaggerated. People still want their data in places that are secure and well thought through and like have like providence and they're like, you know, have somebody you can call if that like 2:00 in the morning it goes offline. But I think the interface layer gets a lot sort of fuzzier and you're seeing companies like Salesforce kind of embrace that and say like, 'Great, we're happy to be like a system of record and like the connective tissue of all this and we're going to open up the ability to actually have other services like Claude be kind of full-fledged apps writing into that.' That's a trend that I'm really interested in. I've been interested in that for years, but it's like now coming true.

Yeah, I think the thing that I found really resonant with Dario's essay is that there's pieces that are like very clearly like frontier lab sort of responsibilities like interpretability for example and like having observers etc. But there's also pieces in there around operational excellence that I think are equally true and when I like they're as true now as they were before except now probably more people at your companies are builders, the software's going to built more quickly. There's definitely less code review I imagine. Like Anthropic we have agents review a lot of code, but honestly humans don't for most of the code that's getting written because the volume of it is just way beyond any human's ability to read it. So that means like having really really good primitives around sort of security and boundaries around agents become like extremely important. I think one thing we could do better is also even talking externally about how we do some of this as well. But one example like we now have I think every Cloud Code session by default and I think every co-work one as well run through our auto mode classifier. And that was like this piece around being really practical with security, which is, you know, the most secure agent is the one that does nothing at all, but it's also useless, right? And then like, okay, the next most secure one is the one that asks you before doing literally everything. And then it reminds you of like that Windows version where you're like you'll be like, Windows wants to like do something like full screen model and of course people are just like, yes, just do the thing. And I think people had fallen into that same behavior where they either had Cloud in like dangerously skip permissions mode, which is clearly not a good idea, you're just going to run it without any checks, or, you know, most enterprises had it pretty locked down and it was like, yes, you can read this file, Cloud. Yes, yes, yes. And it annoyed us, so we built this auto mode classifier where you have like a pretty robust set of classification that happens and then it only asks for you really specific cases. And that was an example of something we built internally for our own use cases, but now have actually available to like every customer. Because of that like desire to make like the external piece more available.

Well, that's an interesting one. I think maybe tied to the second piece, which is like, what are the internal demands that you are getting and are going to increasingly get to actually unlock value? Because the other alternative is to be so locked down that you'll actually be like left behind and the other companies will outcompete you. So what are those demands that are coming and how can you set yourself up to safely enable those things? Whether it's, you know, having a good vendor stack in terms of like agent observability, whether it's having good controls on folks' laptops as well. Like I don't have a quick way to definitely gone from like the actually pretty locked down even when I started to like now like very like they're very thoughtful routes about who has access to which piece and like, yes, it does cost friction and then put your minds on making that friction be as reduced as possible rather than sort of make like just risk accepting it and not having there. So, I think those are some of the pieces. I think the next frontier is very clearly going to be agents transacting on your employees or your company's behalf, which sounds very scary, but I think we'll be here before we know it. I think it'll become sort of a necessary piece. So, what are those approval checks? Like what are the places where humans still need to make the call? You know, on an individual level many more people than maybe you would have expected at the gate are pretty comfortable giving like Muse and Instinct their credit card being like, 'Go buy stuff for me on the internet.' But then you see something like Stripe Link, which is a really nice like, 'Okay, people are going to want agents to buy stuff for them. Can we make it in a way that's actually like, you know, officially blessed and has an approval flow, etc.' Like I think sort of like Stripe Link for all of agent actions is maybe a good thing to aspire to.

Yeah, I mean I think a huge part of why I went to work at Anthropic was at the time I had a four-year-old and a two-year-old. Now I have a seven-year-old and a five-year-old. And being like, 'Their world is going to be so sort of imbued with AI in all these different places. Like if I can have like some small part in nudging things in a positive direction on both like how well AI goes overall, but also really specifically about like what is it like to actually interact with these different things.' And I think none of the products got that totally right yet, but like things to aspire to are like be able to meet people where they are in terms of like what they're ready for in the moment. Like just the other day I was like at the end of a long day, talking to like we tag really extensively internally at Anthropic. And like I had a particular tag that was like, 'Mike, there's like eight decisions waiting for you.' And I was just like, 'I can't right now.' Like I just need So, like having that managing attention, but managing sort of like that emotional relationship is really useful and interesting. It was great. It was like, great. You know what? This is the actual decision that I need from you right now. I can like go forward with like reasonable defaults here and we'll talk about it tomorrow. You know, I think that piece is one. Mental health obviously like an enormous problem not just in this country but beyond. Like how can AI powered products be a force for help there rather than like ones that sort of help like or create more spirals in people I think is a big open question that like we're thinking a lot about. We put a lot of safeguards into the products. But I think there's like reducing harm but then there's also the doing like right by people and doing better that I think we could do more of. And then the third one is around people's skills and how they relate to their own sort of sense of work where, you know, we saw it internally at Anthropic. It's really interesting being there because you get to kind of see things first there. Like I used to pride myself on writing really good code and like being a good like sort of steward of it. And I've now had to reassign like where I think I'd value which is definitely not that anymore. It is still in like some system architecture pieces that's still valuable. But it's actually much more in having the taste to decide what to even go work on. Like I was spinning up a new labs project this week and I didn't really touch any I didn't have quite write any code for the first 48 hours cuz I think I was like before yes, it could do the whole thing in a day. But what are we building and like who are we building for and what are the tradeoffs and what are like these pieces that are going to relate? So I think that human aspect of like helping people realize that they are going to have to probably redefine their own like work value often but I embrace that rather than be like, okay, well, now that means I can't do anything anymore.