Yeah, I mean for anyone who doesn't know, I am Oliver Emberton. I run lesswrong.com. I also happen to run this campus. Thank you. Chris, how about you introduce yourself?
My name is Chris Best. I am the founder and CEO of Substack, another online essay platform. Thank you.
Yeah, I really enormous thank you. You know, I've been a decade ago I started getting into, you know, thinking about online essays and seeing kind of like a lot of the internet move towards social media, moving towards Twitter. And Substack was really like in some sense the biggest shift of the internet that I'm just like extremely grateful for. I've just like you have made the internet like long-form internet essay like a mainstay of the internet again as like, you know, WordPress and the blogs were dying, slowly everything moving over to Twitter and Facebook and all that stuff. And I'm just like incredibly grateful to you for that.
Well, thank you. I needed something to read, you know.
Yeah, I mean that's also how I felt when we got started with LessWrong. I was just like, 'Look, man, like if I don't do this, like what am I going to spend my time doing?' You got to get the office ready. Okay, so the topic of today is I think we have both chosen a kind of funny time in history to build long-form internet essay platforms. We I think just both pulled them together just before most of the writing on the internet has become AI-written and most of the traffic of the internet has become AIs reading the traffic on the internet. And I think we both are pretty confused about what this will mean for the future of long-form writing. I mean, I'm also confused what it means for the future of humanity, but we are going to be focusing on specifically the essay and intellectual idea exchange part. So, I guess you kind of come in with like you, you know, when we asked you what would you want to talk about, you're like, let's be confused about the future of the essay along from internet writing. Do you want to give a small introduction of like where you're currently at?
Yeah, I pitched this as noticing that we're confused because I don't feel like I have great answers, but I have what feel like increasingly interesting questions. This has been sort of my experience of the AI moment that we're in. It sort of feels like there's a bunch of fundamental philosophical questions about what we're doing and why and what the value of things are that I actually never knew the answer to, but now I'm suddenly forced to confront the fact that I don't know the answer. It's like all of these philosophical questions suddenly become kind of urgently practical.
Yeah, I guess I was reminded of, you know, a few years ago there was that XKCD that was like, when the spam bots finally write good internet comments, we like won. And like it turns out the spam bots do now write internet comments indistinguishable from people from other people on my platform and we didn't win.
And so clearly it means something must have gone wrong in my models. Like I kind of did think that that XKCD had a point and now I'm like, okay, something was missing in that.
I'll give you the titles of the two Substack posts that I pitched as the title for this. One is by Sasha Chapin. Chapin. Books are subjectivity merging devices, not efficient information transfer devices. And the other is Sam Kriss, if you let AI do your writing, I will come to your house and kill you. Subhead, did you think I wouldn't be able to tell? I can tell. And I think those things kind of sum up sort of this objective experience I've been having going through this period. You know, on Substack we're very pro editorial freedom. You're it's your thing, it's your place. We don't have a lot of rules about what you can put. This includes, you know, you can put AI things, you can put things that are assisted, you can put whatever you want. I'm kind of, you know, let the flowers bloom, see what happens, let people kind of cook, you know, create an incentive system that can reward things that people actually value. And then I've been having this experience where I feel like I'm losing my mind on the internet. It's actually not worse on Substack. On Substack, it's relatively better than other places. But the worst version for me is this like Invasion of the Body Snatchers feeling I have. When somebody that I know, somebody that is smart, somebody whose writing I follow, I start reading Testing, testing. I start reading their tweet and I get like six words in and I'm like, this is just Claude. This is just And it's not just like a little bit Claude, this is like the sloppiest slop that ever slopped.
Yeah. I mean, for me, I think it was for me it was Hacker News. Like sometime in the last 6 months, it's like my experience of reading the front page of Hacker News has just become like kind of a constant feeling of paranoia. Like I see the title of a post and I'm like, 'Oh god, is that AI written?' I click into it, I read the first paragraph, and like I'm just looking out for the tells. And it's like I can't stop myself because I just like I will immediately stop reading when I find it. And then I read the comments and it's like everyone is super excited about it and nobody gets it and I just feel so gaslit. Like, 'Don't you see it? Don't you see that it has no human behind this essay right now?' But okay, but why does it matter? Are we shaking our fists at the sky, at having some strange weird fixation on human meat fingers typing out essays and future generations, my son sitting in the front row, will read his essays and won't care if a machine made it and he'll laugh at us for our backwards ways. Or like, why does it matter?
Yeah. Yeah, I mean, I think I currently have two models that seem both very relevant that are at the core of my understanding of this. One of them there's another post on LessWrong that I really like called LLM generated text is not testimony. And the core idea of the post is that when people write internet essays and I think the post is actually related to the Sasha Chapin essay you just mentioned. Like when people write things on the internet, it's not just the case that they are making statements that you can independently verify. It's that they're speaking from a certain worldview. They're expressing their observations. Like you can notice what terms they use and those terms are going to be shaped by their experience. Like if there's somebody who has worked in economics, somebody who has worked in maybe even fashion, you can tell from the terms that they use about how they view the world. You can tell from the sentence constructions that they have about a lot of different ways in which they are perceiving the world. And that feels to me as a huge fraction of what I get out of internet writing. And the thing that I like is that by reading a lot of someone's writing, I feel like I end up pretty reliably with a shoulder version of them that I can consult and then I have the shoulder version of like I mean for many years Scott Alexander, for many years Eliezer, but also many other authors that I'm a huge fan of. And that feels to me like it doesn't happen when I read the LLM writing. Like there is a single Claude. I have a shoulder Claude, but it's the same Claude across all of them. And so that's one of them. I think the other big thing that is a core point of the essay is that this has been very important on LessWrong. Like when you make a statement in an essay, there is a sense in which I will believe that statement in substantial parts because you put your honor behind it. Like it's like I once learned when I was in Idaho that X happened. And I made these observations. And there's a sense in which you could have made all that up, but you didn't make it up because humans don't just lie to each other. There's a sense in which your statement is testimony. Like I can use it in a social context to hold you accountable to the things that you said. And for AI-written content on LessWrong, it's so common that somebody will submit an AI-written content, like a piece of AI-written content, and there's an implication of their writing, and then I call them out and be like, 'Wait, is this true?' And they respond with like, 'Oh, no, that part Claude wrote. That doesn't quite Actually, I wouldn't quite write that. That's not really quite what I would say.' And I'm like, 'Oh my god. Like, look, I could only make sense of your essay because I understood you as giving testimony in this context about what happened to you and what you believe and what you think. And we lose that attribute of testimony whenever you use Claude to speak for you.
Yeah, it's like if I wanted to ask Claude, I could have asked Claude. And if you want to, just send me the prompt. Don't send me your Claude thing.
Yeah, I think that's kind of the second point. Like the first point, this is AI writing as testimony. That feels very core to my worldview of like I can rely on it. But the second point is also just this idea that if the only thing that you did was to give Claude a prompt, then there is a sense in which the whole complexity of your essay must be capable of being summarized in your prompt plus what Claude would tell me. And so there's a sense in which I'm just reading Claude didn't have that much to work with to write your whole essay.
So how are you handling it on LessWrong? What's your policy for AI writing? I mean, we've been going like [sighs] the central policy, and I thought a lot about this. I had lots of conversations with lots of our core office, and we thought a lot about this. And the thing that we settled on is it is okay for you to use AI writing however you want. You just have to disclose it. And if you do not disclose it, we will just reject your content outright. And luckily, at roughly the same time, Penlight kind of came up onto the scene. And Penlight is the one AI detector that's actually pretty reliable. And so now we just have as our policy whenever you submit any post to LessWrong, we have Pangram run on it. We exclude like we have a special punt type of content block that's an LM content block where you can say what model output is as a tiny annotation. We strip all of those from your post and then we check does the remaining content rank above a point two score on Pangram? If yes, automatic reject. Don't get to go through. And then also separately we have a human review step where we check especially the ones that rank 0.15 or something and we check whether a section of your post is totally AI generated and then we reach out to you and maybe ask you to put it into an LM content block or something. But that is we're very strict about that because that's kind of centrally driven by this idea of AI writing as testimony. Just like it is completely okay for you to add Claude testimony. It's just really important that whenever you say 'I' that is a sentence that you say. And whenever Claude writes anything, Claude can say 'I' but then it needs to be in a content block that indicates that the author of that section is Claude, not you.
And how is that different? Like if I was writing a LessWrong post but I had a human editor, I had my friend look over it. They maybe wrote some of the different sentences. You know, I'm working with them. This is like people when people write books they'll use human editors. Sometimes they'll use ghost writers. Do you think that's fundamentally different?
It's actually this is really interesting because I think in some sense our policy is actually a continuation of a long-term very controversial policy we've had on LessWrong for a long time which is that we don't allow organizational accounts. Like people for years have been like hey can I please have an organizational account? Can Miri have an organizational account? And I've been like no, sorry. Posts are written by a person. Maybe posts are written by multiple people co-authorship is real fine fine fine. But it needs to be a finite set of people that if I go to every single one in a list of offers and I try to get you to defend what is in this essay there's a specific person who's responsible. And ideally it's written in a personal tone that says 'I' instead of 'we'. And so that's been a long-running policy. And I think it's actually very similar. I think a thing that happens when corporations and large organizations release statements is actually very similar in vibe to what happens when Claude writes something. Like I look at it. And what on earth does it mean for Microsoft to release a statement saying 'we believe'? And I'm like nothing, your organizational institution does not have a map of the world that meaningfully corresponds in a way that believing makes sense. Like a CEO can believe something. Two people can believe something. Maybe even you all agreed that you want to say something or you all want to say that you all believe something. But the organization trying to describe beliefs to an organization just like really has always felt off to me. And so in some sense the Claude policy here or the AI policy feels to me like a natural continuation of that.
Very interesting. So if we started to narrow down, if I started this conversation completely philosophically perplexed about what the point of an essay on the internet is. I'm like a lot of people seem to be writing these things. A lot of them seem to be reading them. I don't know why. I don't know what that's for. I don't know what that thing does. If we take your received policy as a guide we could start to rule some things out maybe. For example I might say the point of an essay is to make a good argument. I could string together a series of logic. I could make a bunch of statements that proceed from some premises and go towards a conclusion. And the value of that logical reasoning is one thing that I could be getting from an essay. And I think if that were the point, Claude would be just as good, right?
I mean, this is actually really interesting. In a sense, I think I was much more sympathetic to this until both the mathematics community and the open-source software community ended up in substantial chunks of those communities banning AI-submitted content. And you would be like, 'Come on, it's math. You just submit a proof. The only thing that matters is whether the proof is valid.' But no. Turns out actually a huge chunk of the professional exchange of ideas between mathematicians relies on statements as testimony. You say that you proved X. But a lot of what you're saying with that is 'I verified the proof myself, and I'm putting a decent chunk of my professional reputation behind the fact that this is the case.' And similarly, when you have a program, you're like, 'Hey, I can say I tested it, but a lot of the time, you just want a professional engineer who has some professional reputation to put their testimony that the code that I've written is high quality.' And that's the only way to meaningfully handle complicated open-source contributions. And so, even in these domains where in some sense we think verification is the easiest, like software or mathematics, I actually think it's just the case that you can't purely rely on verification. And so, I think in some sense, if we were in the domain of full formal verification, you could do this. And I think that's actually some of the domain where we have seen the most progress with using AI. Like things like frontier math and domains like that. And we've seen mathematical conjectures fall. But I just think very few parts of human endeavors are structured such that verification is so much cheaper than generation that it just kind of checks out. I think I'm curious from you about the essay. I like the Sam Kriss essay. It was evocative and gave me a lot of feelings. But the other essay I've been thinking about for longer, which is the Sasha Chapin. What was the title again?
Books are a subjectivity merging device or technology, not an information delivery.
Yeah. So, do you briefly want to summarize? And that seems clearly relevant to answering what is the point of an essay.
Yeah, I mean, I think the title kind of says it, right? The thesis is, the reason you read a book is not to get an argument, not to get a bunch of information, not to get a series of facts or sequence of things. It's kind of like you are to some extent deliberately submitting yourself to the perspective of the author. Right? You talked about having the person on your shoulder. You are opening up your mind and allowing yourself to be changed, at least temporarily. At least while you're reading, you're putting on this subjective feeling of being in the mental process of somebody else. And you're letting that into your mind, you're entertaining that voice and thought, and probably realistically that thing is leaving an impression on you. It's changing you in some way. At least at minimum, you're going to be able to model that perspective in the future. And probably at maximum, it's going to actually change your own default thought and how you think. I'm actually not summarizing the essay anymore. I'm extrapolating with my own theories now. So, don't blame Sasha if any of this is stupid. And I think everyone has this experience, right? People you've read many books in your life. If somebody asked you, 'Hey, what's in all those books?' you would probably struggle to give a very good description of it. And yet, the books change you. They've left you different in some way. And so, to some extent, when you read a long-form piece of writing, you're inviting somebody else into you to change yourself.
Interesting. In some sense, it feels to me like that is one thing that AIs could be pretty decent at. And in some sense, it's actually one of the domains where I felt the best, one of the domains that I found the AI systems most useful is that AIs are really good at imitating experts from any domain. And they're much more patient than any guy I can call up on the phone. So, when I'm in a domain like I've had interest over the last few years in understanding Berkeley and California zoning and construction law in a lot of detail.
And indeed, one of the things that has been extremely useful is just that I now, it's not even that the AI is telling me all the relevant facts. It's that I do feel like I can talk to the AI system as kind of an expert in zoning management for hours. And the thing that's most helpful is more like it starts using these terms. It starts using these abstractions of how it even gives an explanation that ties together. So in some sense, it feels to me like because the AI is actually capable of imitating experts in basically arbitrary domains, that seems like actually a really great tool. But I haven't gotten a sense that anyone has successfully curated a large set of AI output that helps me become an expert in any domain. And maybe the answer is, well, we built the wrong platform. ChatGPT will take our lunch in this domain. And I have a less wrong than Substack will help.
I do think I sometimes think about how am I navigating this time? I'm running a company. I'm noticing that all of my philosophical underpinning of what it all means and what we're doing is woefully insufficient to the task. And I think it's helpful in those cases to turn on my stupid brain a little bit. Just noticing what I'm thinking and feeling. And you just mentioned the thing of you can talk to Claude for hours about zoning. You're probably going to something that you're interested in. A feeling that I have is I can do that, too. I think a lot of people can do that. But then, when somebody else sends you something written by Claude, like when I see those boxes, I have the same feeling I have when somebody else tells me about their dream. Does anyone ever try to tell you at great length about something they dreamt? And almost every time, it's just the most boring thing in the world. It's so meaningful to them. It's part of their inner life that has something. But almost, unless you're a brilliant storyteller or have some point to it, hearing about somebody else's dream is kind of the dullest thing in the world. And that's why of those two essays, I kind of found the Sam Kriss one almost more useful and persuasive, because I don't have a good theory of what is the value here, and where is this all going, and are we actually obsolete and clinging onto something. But I definitely recognize the thing that I see there, which is, 'Oh my god, you cannot send me this slop. You cannot send me a thing, I'm reading something and it's just obviously written by the robot.' And it just turns to dust. It turns to ash in my mouth. I can't take it seriously anymore. And in the absence of having a beautiful theory to explain it, I think it's productive to at least follow that instinct as it seems that you have with your platform.
Yeah. I guess that makes me most curious about, do you have any counter example? Do you have an instance in your life where somebody did send you some clearly evident content? Like, maybe it was an employee of yours and be like, here is an analysis of something. What are the best instances of people doing that in the vague domain of long-form writing?
I don't have that in long-form writing. I did admit that I listened a few times to the, I think it was an April Fools' thing, but somebody put out a LessWrong Suno generated set of music about various AIs.
Yeah, that was me. I mean me and my team, yeah. We made the Fuming Shoggoths album.
The Fuming Shoggoths album. I thought that was pretty good. I've been listening to it.
If you haven't checked it out, I would recommend it. Ben my son Ben is sick of me putting it on in the car because he'd rather hear his own music, but
So why is that good? Or why do I think that's good? Maybe I have no taste.
Well, that's impossible. I mean I tried to put my soul into it surprisingly much. I do think maybe one of the things that was true there and I think maybe that points towards the answer. Like when we made the Fuming Shoggoths album, we sampled a ridiculous amount. We had the guys from Suno call us up. I made a post where we announced the album and we were like for every song in this album, we generated like 1,500 to 2,000 songs. And at some point I got used to listening to music at 2.5x speed using my HTML5 video player thing so I could listen to the song generations faster because you can tell if a song is coming together in the first few seconds. And so I would just sit there for literal hours be like, okay, like AGI and the EMH, how the [ __ ] are we going to make a song out of that? Really not prime song material. But if you sample enough, and I do think that is a non-trivial number of bits. 1,500 generations of songs. I do think that's maybe among the single most curated pieces of AI writing on the internet. And also, the Suno guys reached out to us and were like, 'Okay, I think you have'
They told you you have not been a good user. They actually think they gave me immediate access to the private beta program, which was great. Particularly nice first in the private beta program, don't tell them that. But in the private beta program, they don't have all of the copyright restrictions active. And so, when they release new features, I often get to experiment with making all the Suno derivations of the copyrighted songs. I haven't put any of them onto the albums because that would feel bad. But that actually has allowed me to explore a part of AI-generated music that nobody else in the world has got to explore yet.
Okay, but let me put on my bitter pill hat, or my skeptic's hat. And it just sounds to me, if I take a step back and you're like, 'Well, I put my heart and soul into it. I cared about it.' You know, all of these it sounds sort of airy-fuddy. Surely, give it 6 months, give it 12 months, give it 18 months, and somehow I'll stop gagging when I see Hunter Biden post half of his stuff is obviously human and then half of it is like recovery stuff. I'm like, 'You had ChatGPT write this.' But surely that will go away. Surely we'll get to a point where it could make just as impressive a Fuming Shoggoth album or just as impressive as a Lester Young post. What is the reason? Is there any reason why that won't just happen and that everything we are talking about and doing here is going to be sort of like this temporary artifact in this between-times period where it's somehow super intelligent except completely crap at writing essays that don't suck.
Yeah, I do think that it's my leading hypothesis is that what's going on is just that the systems are not quite smart enough and we're picking up on that and they're just not quite pulling things together. And I think I've experienced this the most in coding which I do every day. I do software development and it's just very solidly. I think it was December last year that was the break even point of just like AI code was clearly sloppy and you would write it and you would feel like ah god, actually that just is not at all the appropriate abstraction for the right context. Like it feels like you're just mashing together a bunch of
Random piles of code from the internet that does get the job done, I guess, technically arguably, but you would feel bad. And then like December last year, I would just... I think it was over Christmas break, I was working with the models and I was just like, holy [ __ ], now. These are now writing code that's as good as my code a lot of the time. And then in the three precious days of Fable that I had, I could see it like, oh no, that system is not as good as I am in all respects of software engineering, but it's like, yep, it has just passed a threshold. We're now at a point where it writes most code better than I do. And my honest best guess is we are going to see similar things in AI music generation, in essay writing, and poetry over the next year. I still think you will have these huge testimony issues. You will have these other issues, but in terms of will things still feel like slop? Maybe. Maybe. I'm 50/50. I'm 50/50 that this is the last year where the humans get to write the best... Well, I don't know. The best internet essay can last for a while, but that's my best guess. But leaning into the worlds where I think it's not just raw intelligence, but there's a few other dimensions that are relevant.
It's kind of fun not knowing. It's kind of... [laughter] It's kind of fun, I mean not in the sense that we should preserve that, but it's kind of fun to be in a time where it's like, I have no idea. I don't have... I have no idea what it would even mean to be the better essay. Can I try one? Can I try one, uh, crazy idea on you and see if anything sticks. Yeah. And this is not mine at all. This is something I'm stealing wholesale from Robin Sloan, who came to a Substack debate and there was a debate about, can an AI have taste?
And he made... I'll... Mistakes are mine, but he made the following argument, which is, do you remember move? Was it move 47?
Yeah, it was... Yeah, whatever the the the the god hand move in the game against Lee Sedol. A.
Yeah, as the Go AIs were becoming superhuman, the first there was one, there's a game against the reigning master where it made a move that at first blush looked totally insane. And then after the fact, people sort of realized that it was secretly genius and beautiful and kind of subjectively beautiful and wonderful, and it opened up whole classes of new human play that people then explored, and the standard of human play got much better because they had discovered this thing. Um. And the argument that Robin made was, isn't it interesting to think about? Okay, so this is an example people will cite as something that debate... has Jasmine, who I think is here somewhere, was arguing as an example of an AI having taste. And he said, 'Isn't it interesting that while it was in its self-training mode, its self-play mode, it probably made hundreds or thousands or millions of moves that are just as brilliant, just as beautiful, just as, in every kind of objective sense, just as creative. And yet [snorts] those are kind of fundamentally unknown and fundamentally meaningless.'
Yeah.
I mean, I guess it must be the same true in code. I wonder how many extremely beautiful pieces of code were generated during the last training run of Opus or Fable. Um.
I'm starting to wonder if code might be different than essays.
I think code eventually, you just want it to do the thing. You want... I want something else. It's an instrumental thing. And if it does it in a clever way or beautiful way, so much the better. You can certainly have code that feels like art. But I'm starting to wonder if there's something else we're missing. I don't know what it is. I'm still confused.
