By the way, I don't have a psychosis. I have a Claude complex.
Why is everyone making that joke? Wait. Which joke? The psychosis joke.
I thought you were going to be proud of me for saying Claude Complex.
I think I do one pun finally for Tracy, and I was over making that joke.
Well, I was thinking that was a joke. I had to go.
I finally make a pun, and you just jump right over it.
Well, everyone keeps saying that Claude Code is a psychosis for smart people, right? Like, how did that become addictive?
Yeah. All right. But there's a good... Fun also very bro coded, I find.
Hello and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal.
So, Tracy, you're cool. Like, if I like, you know, just start doing this part time as I like, build out my software business. Right? Like you're cool about that.
I was going to say I've been thinking about AI and productivity. And so far your productivity has gone down, Joe. So instead of doing all those things, you're coding your own software.
Except that I'm creating content for the Odd Lots newsletter about coding, and that is productivity.
A creative. Debatable. Debatable.
But but you're cool with that. You're cool with like me. Like, oh, I'm just going to, like, check in part time on Odd Lots when we have a record.
Okay, good, good, good. That's the right answer. I want you to. I want you to be really sad. But like a few other people, you know, I have, like, caught the sort of, like, bug of, like, AI coding, and I'm totally blown away. I like, played with it from the beginning. I started playing around with it last year, but then over the holidays and I've been writing about this in the newsletter suddenly, like, my Twitter feed is like Claude Code, Claude Code, Claude Code. And you just cursor before which I was very impressed by at the time. And so when I got home from vacation, one of the first things I did is like, figure out how to install Claude Code on my computer. And I was like, oh, I am like hooked. And this is actually like, I see why I have my Twitter feed is just like people posting about this.
All right. So I have to say, I have not tried it because I only have a work computer and I can't install new software. And I probably definitely cannot install new software that then makes changes to existing software. I don't think Bloomberg would like that. But I have seen the hype. Lots of people talking about it. Have you seen Claude Code? Have you heard of... Oh, yeah. Yeah, yeah, yeah. So one of the criticisms of Claude Code was that, you know, like, okay, you code, but you still need some background knowledge and coding because, like, you know, the interface is kind of like... And all of that or 1990s. Claude Code apparently, like, goes a step further for, for normal people, including it makes it super, super easy. And the funniest thing is that apparently Claude Code actually coded.
Yeah. So the so this is like really relates to my experience last year and then this year, which is that even last year, like trying to use the AI coding tools, it was an annoying process because there were various things that you had to do in the actual command line of the computer that were like, I didn't, I don't know, command line vernacular. And you have to like install these libraries and stuff. Yeah. So there was this sort of like barrier that existed and but what's the what's really changed in the last year or with the, with Claude Code, which has actually been around for a while and I should have like played with it before is the like because it sits on your computer. It's sort of takes away in deeper access and so when you talk about. It, it actually does the stuff. It does. Is it just like, oh, it's like, oh, we're going to need to install this open source natural language processing library. It just does it automatically. Instead of me trying to like figure out like what are the right keystrokes to pull that in or why is this not going into the right file folder or whatever. And so like, like Claude Code, it's like all, like all of these sort of like little friction, like these technical things like command line use are very rapidly are like dissipating. Yeah. And so that like then you have something like Claude Code where it's just like they know they're taking care of that. And so you get this like user interface that's just like it's just getting easier and friendlier. There's almost no technical frictions at all anymore.
Also, it feels very iterative, like the code is improving upon itself.
Yeah. At this point I think that was one of Claude's main selling points. Well, this is like you've seen like people talk about like, oh, is AGI here? And this is like part of the debate because one of the ideas, I guess, behind AGI is like, well, what happens when you have software that can train itself and so forth? And I don't really know if I buy that, but you do just see, like how fast the iteration cycles are. And I think we want to get into this in part, they're fast because a bunch of people are suddenly getting excited, and then the human provides this sort of like we're sowing the seeds of our own demise because we're so enthusiastically participating in the evolution. But I just like it's suddenly clear, like, oh, this is going to change. I think computing. And the other thing is the code works like it creates code that like this is like there's no bugs. You know, it works.
Did you see speaking of automating yourself? Yeah. See there was a post on Reddit from a lawyer who said he's basically used Claude Code to automate, like, his entire job, and he hasn't told anyone.
I'm not exactly surprised, because the other thing that I experimented with is, and I haven't 100% verified this, but on jobs day last week, I downloaded the full PDF and I just typed into the Claude Code like find the most interesting details and make some charts based on. And it did it in like a couple of minutes. I have no like, ability. Like I've never like built charts myself and or whatever, or like designer or whatever. And I didn't totally confirm yet that the data was all correct, but I'm pretty sure it was because everything I spot, so I didn't. Just that crucial. Detail, I know I didn't. That's why I didn't want to like, oh, like, here's what, here's the here's jobs report and charts. But it might.
But what application did it actually build it in the charts?
I don't know. I just had a like that's the thing. I had a file on my computer at that point.
Yeah, that's the crazy thing, I don't know. And so there was just this image that had a bunch of charts, and my spot check did suggest, like, I didn't see anything off. And people get paid money to, like, build that kind of stuff for, like, analysts and stuff like that. Right.
So this is the other big question. If everyone can build their own software, what actually happens to software? And I was reading something. I forget who it was by, but someone used Claude Code to create. They wanted a website that would basically make them money for doing nothing. And that was the prompt. And did they do? Yeah. So the idea that, the model came up with was you can sell prompts, packages of good prompts and sell them for like 40 bucks and you'll make tons of money. And I was thinking about that, like, okay, it's possible to make money that way. But also, why wouldn't I just use Claude Code to do the same thing? There are many big questions that we as an economy are going to have to think about. And I think my main takeaway is we're going to have to think about this sooner rather than later. But what is Claude Code? Why is everyone so hyped about it? And what is it about this particular piece of software that versus what exists from OpenAI and Gemini and all this stuff? Like, why is this captured everyone's imagination? We really do have the perfect guest because it's someone who, unlike me, has been getting their hands dirty and this stuff for longer. One of the few people that I know who is into LLMs before ChatGPT existed and was actually using them via the API, and was actually talking about their technical capacity to do things like coding even before November of 2022. So truly the perfect guest we're going to be speaking with Noah Brier. He's the co-founder of L'échec, a consultancy that helps big companies deal with AI stuff. So, Noah, thank you so much for coming on, and let's thank you for having me.
What's the deal? How are you like using LLMs before ChatGPT existed? I don't know, I know very few people who were doing that.
I had the good fortune of shutting down a startup in 2022, and so I had a lot of free time on my hands.
And then how are you using it, though? Like how do you look at your like, how did you aware that there was this thing that could be of potential use to.
So my very first thing I was doing was using GitHub Copilot, which at the time was built into VS Code and it was autocomplete inside VS Code. So it was a nice and pretty immediately realized that there were certain coding tasks that it could just handle completely. Anything that was very pattern based. So if you write code, you write a lot of tests. If you write tests, every test kind of follows the same pattern and you want it to follow the same pattern, you're looking for that structure in and over time, because it was looking at your code base, it was able to basically autocomplete it. I also started playing with the GPT-3 API, which had come out, I think that came out in November of 2021, and that was the first time it was publicly available to everybody. And they had a large language model as we know it today, available to them. So I was just testing and building things, and I pretty immediately realized the very first thing I did where it just blew my mind, was I built a web scraper. So I was I was just trying to pull pricing data from a website. And I've done a lot of this in my career. It's maybe the most annoying task you have to do in all of coding, because HTML is the most miserable language to have to parse. And I just had this thing where I took the page, I took the content, I took the text, and I gave it to the AI, and I asked it to give me back the pricing table, and it gave me back the pricing table. And I just thought, I'll never do it the other way again.
That's is that HTML mentioned? Just brought up like memories of me in like the mid-nineties on HTML. Good. Easy to remember that site. Yeah. I wonder if it's still is it still up? That would be wild. Does Claude Code? Does that count as AGI? This seems to be the debate right? Is it AGI?
I try not to wade into what's AGI and what's not. I think my guess on on AGI, for what it's worth, is that it's probably going to be a conversation like the Turing test, where everybody thought it was really, really important. For a really long time, we thought the Turing test was the biggest thing for 70 years or whatever. And then. ChatGPT very clearly passed the Turing test, and now everybody pretends like it's just that they forgot. They pretend that it never mattered. Oh, and so I am kind of guessing that that's going to be what the conversation is like. It's just going to be a sort of forever moving goalpost. Because it turns out that the idea we had for what general intelligence looks like is not quite that. But I also think, you know, the computer scientists and the sort of serious AI researchers would say that much of what's going on inside Claude Code is not the model itself, it's the model paired with a human. And I think that is a pretty important distinction. But I don't know about AGI.
Well, okay, so you were using GPT to code prior to the release of ChatGPT. So therefore coding models have been around a long time. So what is for those who haven't played or played around with it. What is Claude Code? Because again, coding models have been around for a long time. If people maybe have heard of Cursor Copilot or some of these other harnesses, etc., what is Claude Code?
So if we back up first and we go to Copilot because so Copilot was the first sort of commercial application of a large language model. By most accounts. And what Copilot did in its initial instantiation was just. Auto some Microsoft. Product. It's a Microsoft product. So Microsoft owns GitHub, GitHub developer Copilot. It was Microsoft had the partnership with OpenAI. And so they built it in and what it was doing was doing autocomplete. So if you're writing code, a lot of writing code is boilerplate or trying to remember the name of a function. And, you know, the reason StackOverflow existed was because you can never remember the exact name of that function or, or the exact regex that you need to use in order to find and replace something. And so you would go search for it. And they realized that you could just build that into the IDE, your code editor and, and have it autocomplete for you. And it was pretty amazing. Yeah. Then, ChatGPT came out, and even before that, I had built a simple chat bot for myself because I realized that, hey, I could just ask this and instead of going and searching StackOverflow, it was totally capable of answering code questions, and it was capable of writing regex or doing these things. And did it make mistakes? Yes. But like there's famous mistakes on StackOverflow of incorrect regex that now exists in every codebase in the world. And so, you know, there were a lot of us just kind of playing with these things and realizing they were a huge boon. And so I think really the next step is Cursor comes out. And the thing is to realize that Copilot didn't was that it wasn't good enough to have autocomplete. You also needed the Q&A because you have these things that you can't just autocomplete. You want to be able to ask the question and answer it. And then ChatGPT came out and everybody was switching between idea. And then I think really the next big piece is that Claude Code came out, and what Claude Code did that was so remarkable was they took the same set of models, really. And they took them out of the chat bot and they really just gave it some very basic functionality to operate within your machine. Right. And so, you know, if you really look at kind of what exists within Claude Code, you're calling out to a model and they gave it capability around sort of two big things. One is you can read and write files on your computer. And then two is that you can operate Unix. The, the base commands, the bash commands that exist in your environment. And again, because these models were trained on the internet and there's no greater source of information on the internet than how to make the internet, they know how to use Unix commands incredibly well. Right? Because Unix has existed for whatever it is 60 years. And the way these commands were designed, they're all designed to be very, very simple. There's a find command and, you know, there's a thing called grep and it can search to a code base. And Unix has this sort of beautiful way of tying one command to another so you can take the output of one command and send it to another. And they kind of just gave the model access to these 2 or 3 very simple things. And it kind of turned out that it unlocked a whole bunch of functionality that I don't think even the people who built it fully realized. Like one example that I think about a lot is just the challenge you have with all of these AI models is that they're stateless. So every time you talk to ChatGPT, it's sending your entire conversation history back to ChatGPT because it has no saved history of that chat. Right? And that's fine. It's the way it works. It just fact, but it means that, you know, it forgets things. It doesn't know conversation, a conversation and one very easy way to save your state is just write it to a file. And so you give it write access and it can create files. And now all of a sudden you've overcome this like probably the single biggest challenge that exists inside these large language models, which is that they're fundamentally stateless.
So Claude writes itself little like memory notes, right, to to remember the entire context of the conversation. And that's how it solved that problem.
No. So there's sort of two things going on in Claude Code beneath the hood. There's one thing that works exactly like ChatGPT or any of these other ones, which is it's maintaining a conversation history. So every message you send it and every, action, it takes, it's recording to a log. Right. Which is just one big file that's really no different than what ChatGPT can do. Where it gets really interesting, though, is it can also write files that it can then read. So whereas that conversation history is all saved off and eventually that conversation gets too long and it needs to do a thing called compaction. And when it compacts it, it tries to sort of just remember the bits because there the total window is, is, is large.
But I mean, it's like 100,000 tokens. So that's what I mean by memory notes. Right? It compacts the information into the important stuff. So it doesn't retrieve. It does that.
It only does that at the end. Like once it runs out of space, okay. Once it runs out of context window. So it has 200,000 tokens, I think, and 200,000 tokens in rough terms is probably 150,000 words. It says, okay, it's time for me to compact all of this stuff. And so it still saves your whole history on your computer. You still have the entire message. But for that session, it just compacts it down to this, you know, maybe 25,000 token memory. 25. What it was. Yeah.
And is this, like something that was not obvious before as a solution like this compaction. How important is it for this being like, okay, as a human, I can work on this on a project for a long time. Like how much of an unlock was there?
I'm not sure compaction was the okay. I think the compaction functionality is helpful. Okay. The way ChatGPT does it for what it's worth is they don't do compaction, they just forget your messages eventually. So if you're in one chat, eventually your oldest message is going to fall off the back. For coding, that's probably less helpful, but there are trade offs. I have both techniques work. I think fundamentally the thing that is special about Claude Code is not the compaction, it's the it's the ability to write and read files on your computer. Yeah. Which means you can always write off memories.
And then what does that mean? Right off memory.
So you could say, hey, I it's really important that I remember this thing for future sessions. I want to always work this way. So in a code base of mine, I have a set of documentation that explains how I like to do things, and Claude Code makes a mistake. And so the next time I can write a memory, essentially it's written as a thing they call a skill. And you can write it off and you say, hey, whenever you run into this, I want you to operate in this kind of way. And that existing across every session is really a thing you can only do when you can store it as a file. Yeah, it's a thing you can't do in quite the same way when you're operating in this environment where it's just going back and forth to the API. So this access to the file system is one really big piece. And then the second is, is just the Unix commands, I mean computers, every computer program lives on top of the sort of baseline functions. And the way that the designers of Unix built them is really elegant and they're very small. They all do one thing and they're all composable. And in coding terms, composable means they can be they can be chained together. Right. And so you can say, hey, look for files that mention this word and then from those files, I want you to take this second action. And then from the output of that action, I want you to take a third action. And that's just built into Unix. You literally just put a little pipe in between and you just pipe them from one to another and, and that's it. And so you give it access to write these commands, and all of a sudden it gets these sort of second and third order effects that are just incredibly powerful and built over a really long time.
So how much of Claude Code, the way it's different to other models, how much of that was overcoming technological challenges versus like just having a good idea because hearing you describe it, I mean, giving access to a computer seems like kind of obvious. Like, let's, let's just do that.
I don't have a good answer to that. I think that it was kind of just a good idea. Yeah. I think they did some patterns really well. They're clearly incredibly talented, not just engineers, but kind of thinkers about how to structure it. Like, the, the primitives inside Claude Code are just smart. And then the thing that they've done and Boris Cherny, who's the, lead developer on Claude Code at Anthropic, he talks about, latent demand a lot. Right. And latent demand is basically just, hey, look at the ways people are using these systems and then figure out ways to make that a part of the product itself. I think what they've done brilliantly, and this is kind of easy when you have a community of developers who are nerds, who want to go talk about all the ways that they're using these things, is they have. I am amazed at the speed in which, you know, I have a small community of 15 CTOs who all use this stuff religiously. And, you know, when we first started that community, it took them a month to I would see it in the chat, and then a month later, it would get built into Claude Code. And then increasingly like it's like a day later, it feels like they're just they're just listening to it. But I think they're just not only tapped in, but they're really fundamentally, you know, they're they're dog fooding. They use their own products when you, you know, they talk about the productivity engineering, productivity at Anthropic. You know, despite growing at a crazy clip, it continues to go up. And, you know, anybody who's built had to manage large scale pieces of software, large scale codebases knows that's not the norm.
So VS Code and Cursor, these are ideas. Claude Code is not an IDE, it's a CLI. A command line interface. Got it. And the other labs now they also have CLIs. So why are we all talk about Claude Code and AI charging. Because users are called Codex. I don't know what Gemini's is called. I think it's just called the gem. Okay, so. Why are we all talk about Claude Code rather than the other CLIs that kind of have the same thing? Like what is the difference?
I think first and foremost they were first. Okay. So and I think they've they've had a lot more. And you know, from my very personal opinion, I think they've done some things smarter and better as far as the permissioning models. So, you know, one of the really dangerous things is you've got this thing running on your computer. You don't want it to go and delete everything. Right? And, they have a very fine grained permissioning model where you can say, hey, I want to allow this just this one time. I want to always allow it. You know. I always click, always allow. I living on the edge. You can, you can, next time you run it, you can just do a flag that says dangerously skip permissions. And and it'll just, they call it yolo mode. I think I think more fundamentally, though, if I look at at Codex versus Claude Code, I think it's, a difference in philosophy around what you want AI to do. To me, Codex, which is excellent, is very focused on building an agent that you can just give something to and it'll just go do it. So I want to give it that task. I don't want to intercede. I don't want to give it any more feedback. And Claude Code is much more designed to be, kind of a pair programmer. And so, you know, in engineering, pair programming has existed for a while. It's a really weird sort of productivity thing where you put two engineers on the same problem and it turns out that you can get better code and worse, multiply. Yeah. And it sort of makes up for the fact that obviously, you know, you're doubling the staff on it, but because of how many fewer bugs because you have both sets of eyes, it it has seemed to work out for many folks. Most companies don't practice it. But I think Claude Code fundamentally is much more designed in that way. It's a pair programmer. It's, they, you know, whenever I start a project, I start in plan mode. So you start in plan mode, you put together a plan. I really I mean, I spend a lot of time in plan mode. You go through you, it gives you a plan back. It asks you how you feel. You can give it a whole bunch of direction, and then it's only then that it goes off and it goes into it. So, you know, we're working together and I actually have a whole system now that I've designed where, I use a task management system called Linear. So I have Claude Code write tasks off to Linear. And then I've worked, with Claude Code to write a document that helps sort of decide a set of heuristics to decide when you should assign it to Codex versus when you should give it to Claude Code. And so if it's tightly defined enough and simple enough, I just send it off to Codex. And it does it totally independently. And then if it's complicated enough that I think it requires my time and attention, then it it saves it for me, us to do together. And we'll work on it together. And so if it's, it's sort of touching, kind of important enough if it's changing some part of the data model, there's these other kind of, you know, fairly basic set of criteria that I use. And, but that to me is the fundamental distinction. And, you know, I find Claude Code in that way to be just in it sort of fits what I want to do and how I want to work. Much better.
