This is the Neo Show where the minds shaping tomorrow speak today. Brought to you by Neo, the browser redefining what AI can do for you. Let's get started. Welcome to the show. So, this is the Neo Show. We have a fantastic guest. I would like him to introduce a little bit, but Neo is really about AI and then people working on AI. Neo is an AI native browser that allows people to get the latest experience when they are interacting with their digital universe. We're still early, but you know, we already get a lot of users and the user feedback. We look forward to continuing our journey along this journey. Every week we invite a fantastic speaker to do a Neo show because Neo is about AI, is about AI native, and then we want to talk about how other companies, how other products are using AI. So welcome to the show. Why don't you give a minute to talk about you and then your product, and then we will dive right into it.
Sure. Yeah. I'm Dan Lawrence, the co-founder and CEO at Chain Guard. We're about four years old and we're working on open-source security. So open source software is, you know, free software put on the internet by people that mostly write it in their spare time. And somehow over the last 40 years it's gone from like a small niche kind of cult community to underpinning pretty much all modern technology. Most estimates say somewhere between 90 and 98% of all the code and everything you're running from your credit card transactions at the bank to probably your Neo browser are open source software. It's great. It's so much code we can all reuse and build on for free. But the security implications start to get pretty scary when you realize modern systems rely on volunteer code from strangers on the internet. So that's the problem we're hoping to solve. AI is, you know, we were a pre-AI company. You know, we started before ChatGPT. But AI has made a lot of the things that we've been trying to build possible lately. It's a huge longtail problem where there's, you know, tens of thousands or hundreds of thousands of different open source packages that comprise these and you can only get so far with manual review and manual work and handwritten automation. So, AI has enabled us to, you know, scale to things that we knew we were going to have to figure out someday, but didn't quite know how we were going to get there.
Yeah. You know, the problem you're solving resonates with me very well, right? You know, the SolarWinds thing happened what about three years ago, right? Was a huge huge thing, right? You know, I was the head of the AI running AI machine learning at Zscaler, right? You know, one of the largest cybersecurity firms. We were like, right, you know, at the front line dealing with that kind of things. And you know, totally right, you know, vast majority of software or, you know, a very good component, a vast majority of the components of any modern software is probably, you know, relying on open source. That's very fair to say. And you know, whether we have a secure software hinges on the open-source components security. So that's a really, really interesting problem to solve.
Yeah. SolarWinds is one of those moments where everyone remembers where they were when the news came out.
Yes. So then the reason, you know, you and I are on the show right now is a few weeks ago when we discussed this, you know, what you are doing, I realized that one, to your point, you started as a pre-AI, well, AI has been there for 50 years, but you know, a lot of people define a ChatGPT moment to be the AI moment, so let's use that maybe, you know, not artificial definition for a second. So you started a company pre-AI. Now we are kind of three years into AI. You know, you and I discussed this briefly and I would love to have you elaborate and then talk to our audience about what is it like, right? You know, kind of a transition from the pre-AI to AI and then what did you do? What's the implication? I'm pretty sure there's still plenty of things you wanted to do further. So, give us that picture.
Yeah. Yeah. The pre-ChatGPT moment kind of timing. It's funny in a lot of ways, but I mean that was the big wakeup call the whole industry had. But ChatGPT wasn't actually the first consumer AI product. Copilot from Microsoft, you know, a lot of people consider as, you know, the first real consumer product where they're using LLMs to do tab completion and editors. So people knew pretty early on that AI was going to help in software development. And people been using stuff like Copilot for years and they kind of forgot that it's the same technology that's behind ChatGPT. But yeah, the implications of AI on software, I'd say in particular the last six months have dramatically sped up. We've been trying to use it internally for, you know, the last 18 to 24 months and by use it I mean more than just Copilot auto completions and automate more core parts of our system and for most of the last year we just weren't getting better results than if we, you know, had people doing it. We invested a lot and we spent a lot of time trying different ways to plug LLMs in and RAG engines and vector search databases and, you know, kind of all the cool things you would see people playing around with on Hacker News but we were just never able to get good results until...
Meaning that you were not able to make the product substantially better with this, you know, AI technology. Okay.
Yeah. Yeah. And I mean the way our product works it's not like directly exposed to users. We didn't bolt on like a, you know, ask Chain Guard kind of like chatbot like a lot of people did to our website or anything but it was more like we have a lot of core processes and automation that power the work that we do to build and secure this open source software. You know we call it our software factory. And it's a lot of automation that our team has written and then a lot of manual work that we have to do for the thing that automation can't handle. It's a, you know, a business of edge cases in a lot of ways. And those edge cases were the things we were trying to get AI to help us solve before we would kick something out for human review. And we've never structured this in a way where we expect AI to be perfect. We still have humans in the loop. But it was just, you know, can we get fewer humans in the loop or can we get more automation before we hand things to a human to review and it was just that the results were garbage. People are having to spend more time, you know, reviewing these results and fixing things than if they had just written it themselves for the longest time. And then the start of this year I think March or April some of the new models came out and all of a sudden things like Claude, Claude Code and, you know, the 3.5 series of models all of a sudden started, you know, pushing things above that bar. And then Claude 4 came out, Opus 4 and 4.1 now with Sonnet and the bigger context windows and I think especially for our business I've been kind of looking back and I think part of it is the model strength but I think part of it is the ecosystem around the models now. The models have gotten a lot better at ingesting the relevant data that they need to do things in kind of an agentic workflow. The areas we were seeing results were on, you know, software written 20 years ago where all the error codes and bugs and messages were on Reddit forums and posts that made their way into the training data sets for the initial models. But when we took it to how do you patch a CVE that just came out yesterday and it's kind of post-training window, they just hallucinated and guessed and made things up and made things worse for us. But they've gotten a lot better at that tool calling, fetching context, you know, agentic retrieval, knowing when to go out to the internet, when to and where to look. And then accessing more things from our own database and our own logs and all the past times we've done certain things. So it's a very, very fast-moving space, I guess I'll say. And just trying to stay on top of it is kind of the most important thing.
I think what I'm hearing from you is, you know, you wanted to use AI, you know, gen AI, and because of the model performance wasn't, you know, good enough, so you were, you almost wasted time, right? Because, you know, trying to use it, it's not good. I'm pretty sure engineers complaining about it.
Yeah, all last year it was like half and half. One day I would wake up and be like, why are we wasting all this time with AI? We know how to do this work but we have, you know, half of a team playing with AI. And then the next day I'd see some new demo on Hacker News and be like, why aren't we only doing AI? And it was just kind of this frustrating back and forth all last year.
Right. Right. Right. And then starting from March, April this year, you know, starting from the Claude model upgrade, you see that you started seeing not just it works but also you're excited about it and by the way it's not just you but also your team, right.
Yeah. I think behind the scene a lot has happened on the model obviously every week every month but I think one of the major breakthroughs was the reasoning model o1 from OpenAI and of course, you know, the Anthropic opened the world.
Yeah, they improved it and then that's why, you know, we have been always, you know, hearing about this agent for the last three years but I would say agents was, the agent is mostly a joke.
It's just a for loop, yeah, it was just a for loop around an LLM for the longest...
Yeah, yeah, yeah, until, you know, this reasoning model is a reality. So I think there is a lot of things behind the scene happen and by the way...
Stuff like, yeah, structured protocols for injecting other data I think have been huge.
Yeah, and I think what you observed is not unique. The Cursor CEO has been talking about this all along that hey, they wanted to do Cursor but they knew the performance wasn't good but they had the faith that model performance will get to certain level. And boom, right, you know, everyone knows that, you know, the last one year how, you know, the kind of the growth they had. Speaking of that, do you use Cursor or their competitors or what do you guys use?
I'm fully, I mean the team uses a lot of different things but I think probably 80% of it is on Claude Code now. Yeah, it's, I never myself, I have Cursor installed. I never myself was a huge fan of the new IDE kind of model. I like my IDE. I'm one of those people where I don't want to change it. And so the Claude Code UX of just running in a terminal alongside your existing IDE has been my favorite. I've tried Codex from OpenAI and the Gemini CLI from Google and stuff and they're getting better fast, but Claude has definitely had the head start there.
Right. You know, cool. I'm not a spokesperson for Augment. Nothing to do with it, you know, other than a user of it. You may try out the Augment because it seems to be able to deal with large code base well because you guys are dealing with open source. I mean a lot of that is a, you know, huge source code base. So yeah, you know, just a...
Yeah. Yeah. We definitely have some unique challenges where it's not one giant codebase we have. It's a whole bunch of small to medium to large code bases that are all very different from each other.
Yeah. Yeah. I mean in my case, right, you know, some of the open source code is like 400,000 files in sort of one repo, right, so that's huge. So cool, so basically you kind of started seeing benefit. Can you sort of elaborate more about the benefit you are seeing, like the excitement, why you're excited now, right? You talk about the transition, the evolution, tell us what's going on now.
Yeah, you know, I would play around, you know, carve out a Sunday afternoon every couple weeks, a few times a month, all last year and just play with whatever the hot new tools were, you know, Cursor and all the new agentic CLIs. I remember BabyAGI in the beginning was the first one where it was, you know, those Python programs and they would just run it in a for loop and, you know, it was cool and every once in a while you'd get lucky and it would do something that would shock you, but then you try something else and it would just completely hallucinate and go away. And, you know, I would never really get that far. And then, yeah, the first couple releases of Claude were the ones that like, oh wow, like, okay, I tried one thing and it worked. All right, let me try another one. Still worked. Wow. Okay, let me try something even harder. And it was like, oh, wow. This is, you know, actually real now. And I think, you know, with an engineering team, we have about 100 engineers somewhere in that range. And, you know, a lot of people are excited and stuff, too. But a lot of people, I think, had fatigue from all last year when all last year, you see these crazy headlines about results and bench scores and try stuff on real world workloads and you wouldn't get the same results. And so, there's a lot of fatigue of like, oh, why is this time different? And, you know, showing actual results, I think, is kind of what gets teams like that on board. We don't have a mandate. You know, I've seen other CEOs like Coinbase and stuff like this kind of forcing things top down or, you know, the Shopify memo. But engineers want to move fast. They want to play with new tools. So, just showing that it's allowed, that you can do it securely, and that you can get great results, kind of all you need to get a team on board.
Let me ask two questions. One is, what's the state today? Like everyone bought into it or at this point you bought into it, a lot of people bought in. I mean still there is people who somewhat skeptical about it but you are just confident they will get over the hump, like what's the current state? I'm actually curious about the evolution still with all the hindsight, what would you have done differently? For instance, right, hey, you know, instead of everyone start trying, I only get two people trying out until they are successful, the rest hide the rest of people, something like that, I don't know, but just the window with all hindsight, what would you have done differently?
Yeah, I mean I see pretty much the vast majority are on board now using these tools every day. I've talked to other companies like how do you measure it? Is there some uplift in productivity you're seeing? Do you see PRs is faster? We haven't really figured anything like that out, but I think the real test I look at is how many people would be upset now if we told them they couldn't use AI anymore. And I think it'd be pitchforks from probably 95% of the engineers if we took it away. So I think that's the main metric we're looking at today.
By the way, this is my nightmare test as well because you know, if you tell people you don't have internet, most people would feel like it's a nightmare, right? Unless you have... or something like that. But I would say, you know, even in the US today, vast majority people wouldn't call it a nightmare if you take ChatGPT or the AI. Not yet. So, we're not there yet. But, you know, it's a nightmare test. Okay. So, continue, please.
Yeah. Looking back to do it differently. You know, hindsight's always 20/20 and everything, but like yeah, we probably would have spent more time last year getting ready for this moment. That whole flip-flopping between nothing's working yet, some results, nothing's working.
You would have said, with the hindsight, you would have invested more because I was expecting you to say, 'Hey, maybe we should get smaller number people working on it until, you know, we see the sign of life.' And you know, I'm curious. Can you double click on this a little?
Yeah, I mean I think the hard part is like nobody knew these terminal based agentic editors were going to exist in this form. You know, until the first versions of Claude Code came out, it was all IDE based. And the IDE stuff is good, but I see that as a flow of like making engineers more productive, which is great. Making engineers more productive at what they're doing. They're still in the driver's seat. That's great. The agentic flow though, what I really like about it, the terminal-based agentic flow, is it's almost like an Emacs macro where, yeah, you can use it where you're in the driver's seat watching it make every change, but if you get a decent prompt set up, you can then save that prompt and these things all have headless mode and everything too, and now you can run, you know, you can tune the prompt, get it working on one task, and then do that same task a hundred times across an entire codebase. And I think, you know, if we had known this flow and this style of technology was going to take off and was going to work, I think we would have designed a lot of systems differently to get ready for them when they did start to work.
Interesting. Emacs macro.
You know, this is like 20 years before our, you know, co-host Excel was born. So, yeah, you wouldn't have got it, but resonates with me.
Yeah. It's cool. Yeah. Because you can transition it from that driver seat flow to autopilot mode I guess is the part that I really like.
Right. Okay, so let's just cut to more like a chase of it. What is the AI value? What's the value of AI? Just can you double down like for people who are not in the industry, can you describe, hey, here's the value I'm getting out of AI.
Yeah, I think in short it's you get more leverage out of your team. And, you know, there's all these debates and discussions like is AI going to replace engineers? What's going to happen to engineering salaries in the future? Are people going to have to hire fewer? And no, I don't think so. Like what happens if a resource all of a sudden gets 10 times as productive? You're going to get more of it, not fewer, and they're going to be worth a lot more money. Nobody's going to stop hiring engineers because AI has made them 10 times as productive. You're going to want a lot more of them. Because now each unit you add is 10 times as productive.
Because you have 100x work for them.
Exactly. Yeah. And they can do 100x work that increases the demand for something. It doesn't decrease the demand. And so it's, yeah, that's the simplest way I can explain it across all fields not just engineering. It increases the leverage of your workforce whatever they're doing.
Okay cool. So this is actually very, you know, encouraging, you know, especially for Excel and his generation, you know, because he's, you know, going to face, hey, what do I do next? What I'm going to face when I'm getting to the profession. So I want him to sort of ask you more questions from here on because, you know, this is about the future not just about past and then current.
Yeah, before I do that I'm curious did you get the chance to try Neo?
