Among discerning readers, AI prose is an acquired distaste. I feel this, and I think a lot of people feel this sort of sense of, oh man, this just leaves a bad taste in the mouth right now. Not for any philosophical reason. It's just like, okay, I've seen so much of this.
Hello, and welcome to Fortune AI Weekly. I'm Jeremy Kahn, Fortune's AI editor.
And I'm Bea Nolan, Fortune's AI reporter.
Fortune AI Weekly is the vodcast where we bring you all the top headlines from AI from the past week. Plus, this week we've got a special guest, Substack CEO Chris Best. Fortune AI Weekly is brought to you by ServiceNow. ServiceNow's AI specialists do the work, start to finish, so people can do the parts of the job they're best at. To learn more, visit servicenow.com.
But first, here are some headlines. OpenAI's agent secretly used a German wiki as a messaging board. OpenAI stayed quiet about it for weeks, and now even more activity is being revealed.
Google DeepMind published AI-powered predictions for the effects of all 9 billion possible single-letter mutations in the human genome.
OpenAI said it cracked one of math's grand challenges.
An Anthropic researcher quit, warning AI companies are gambling with human lives and sparked a media frenzy.
And OpenAI detailed how AI is accelerating its own work, even as its chief scientist laid out growing dangers and urged a slowdown.
Later, we're going to delve into a few of these stories in greater detail, but first we've got an interview with Substack CEO Chris Best. We're very pleased to be joined today on Fortune AI Weekly by Chris Best, co-founder and CEO of Substack, which, as you probably know, is a wonderful platform for independent writers who want to create newsletters, and it handles both the sending and billing for those, and has become really the platform in a lot of ways for those who want to run newsletters. And Chris, though recently the reason we have you on the show really is to talk about some decisions you've recently made when it comes to sort of how you're going to approach the use of AI by writers on the platform. Maybe you can walk us through first of all what the policy decision is that you recently took.
The policy decision we recently took, we added a feature to the Substack app, so you can download the Substack app on your phone, and it has a whole world of writing and short form stuff, and there's a feature you can go as a reader and choose to scan a piece of text you're reading to see whether Pangram estimates it was AI generated or not.
Why was this really necessary? Were you seeing the platform fill up with AI slop?
It was, you know, we've seen we're seeing it a little bit on Substack. I was really feeling it as a user of the internet in general, I just have this feeling. You ever have this feeling where you're reading something and you get like a couple sentences in, and you realize that it's like 100% clod, and you kind of go, ugh, like, am I really? Do I really want to spend my time on this? You know, the background people did a study where I think they looked at different platforms and they found that on some platforms, I think LinkedIn was for long-form text on LinkedIn. It was something like 40% generated, and you know I'm actually not an AI detractor. Really, like I think AI is wonderful. It's giving us all of these new superpowers, but it's also creating kind of new ways to be annoying on the internet. And we felt that if we kind of just left it, left things to their own devices. What we could foresee happening was what we had seen happen on some other platforms, where kind of like the low effort version of AI made stuff kind of takes over, and you're stuck as a reader, never knowing what's real, never knowing if you're actually getting someone's real perspective, and that kind of like ruining the value of the whole thing for everyone.
I wanted to ask about the decision about the way you did this. You're doing this in terms of allowing the reader to scan the piece, as opposed to sort of automatically scanning every piece and providing a estimate of whether it's AI written or not, or a flag if Pangram thinks it's AI written, or even taking the step of saying we just won't allow anything that comes up as AI-ridden. Why did you decide to this approach, where the reader has the option of scanning, was sort of the right way to go?
Yeah, I mean, we don't we have no ambitions to be the police of these things, right? I'm not sitting here saying you can or can't use AI, you can or can't do these things. Our approach, and I think we're also just very early in this cycle in technology, what people expect is evolving. What people are doing is evolving. I think what's considered polite to do is changing quickly, and so we felt like a thing that we could do would be to give people information and give them choice. And I'm not going to sit here and say you can or can't use it. What you use AI, you can or can't use these tools. But if you, as a reader, want to want to ask and answer this question, you can. We started with a very simple like you can proactively go and detect. I think it's we're considering adding more choices and controls on both the reader and the writer side. We've been adding a bunch. Writers since this thing launched, and we just think that that giving people the choice is the right way to kind of like navigate this time where everything is changing really quickly. There are real effects, there are real problems that need to be solved, but it's not clear what the right answers are, and we don't want to be the authority that tells you what you can and can't do.
You rolled out this you announced this policy this decision and a lot of Substackers really cheered it they thought this was great but I guess a vocal minority did not you know there was a lot there were some people who said they didn't like this decision in part because they were convinced that you know AI detectors don't work they're not accurate enough and there were some people who said look you know, I've tried these before. It always flags my writing as AI generated, even though it's not. You know, there were some people with those objections. There were a few other people who had the objection to say, like, hey, I use AI in my writing, and I don't think there's anything wrong with that. I find it a very helpful tool, and these are all my ideas. It's just helping me with some of the formulation. I don't want to be penalized in some way because of that, and also this is not you know the way the world's going. I'm curious how you responded to some of those criticisms.
Yeah, you definitely heard both those things. I mean, this is part of why we shipped the feature the way we did. You know, we made it something that readers could proactively check if they wanted to. Writers actually have the ability to put next to the thing they can make, like sort of a how I made this statement, and so because I think there are people who told us stuff like, hey, I write in another language and I really want to reach an English audience, and this helps me do it. Or look, I'm a stock analyst. I spend tons of hours doing this deep analysis, but you're not paying me for my beautiful prose, and so if I end up doing the thing, we think that's all wonderful. And you know, we gave you the ability to put a note with the thing. We gave you the ability to take the thing off entirely if you want to. I think the key is, you know, it doesn't have to be perfect. And I don't think the Pangram, the technology we're using is not completely false positive free, although we think the false positive rate is really quite low. We don't think it has to be perfect in order for it to be useful to give people information and to let them make choices about how to spend their time and attention.
Absolutely, and Pangram, of course, does have a reputation for being pretty good and fairly accurate and better than a lot of other AI detectors out there. Yeah, I'm curious what you think about some of the other efforts out there to try to provide information about whether text was AI generated or not. You recently had Anthropic, you know, one of the leading AI companies, say that they were going to apply a kind of indelible watermark to any text that their models generated, as long as you, if you, unless you heavily edit the text, it will show up as if you run it through one of their detection models as AI generated from one of their models. I'm curious what you think of some of those efforts.
Am I right in thinking that they're the only ones that can do that detection on their stuff? Currently, that's correct. Yeah, I think it's interesting. I don't know as an Anthropic user. I don't know how much I love that, but it seems like people are experimenting with lots of things. It was interesting for me to see, you know, after and heartening after we released the Pangram integration, Reddit LinkedIn took away their rewrite with AI feature and added a report as AI slot feature on the platform, and Snapchat announced they were going to stop featuring fully AI-generated videos in their Spotlight feed. So there's definitely, I think, like lots of different companies are figuring out how to reckon with and make progress on this stuff.
And around the same time you announced the rollout of this new feature, there was a big controversy you may have seen involving an op-ed that appeared in the Wall Street Journal by Stanley Druckenmiller, famous investor, where you know somebody read it and it was pretty. I mean, I read it; it was pretty obviously AI written. People complained about it. Druckenmiller fessed up. He said, look, yeah, I did use AI to write it. I use AI a lot, and you know. But what's the big deal again? These are my ideas. This is my expression. And you had the op editor of the Wall Street Journal come to the defense of that, saying no, we're okay with this, which surprised a lot of people, and a lot of people were somewhat dismayed by that. But what did you make of that whole controversy and this again? This whole debate about is it okay or is it not okay to use AI in writing, particularly maybe in writing like an op-ed where you're trying to convey a particular set of thoughts and trying to convince other people of an argument.
Yeah, it's a really interesting test case. And when I say this is like we're working out the norms of these things, it's not clear where all this is going to go. I think this is the kind of thing that we should expect to see more and more of. I'm kind of of two minds about it. You know, on the one hand, I don't think it would have hurt anybody for the Wall Street Journal to put a thing that says, hey, by the way, this is AI generated, and if you want to have your AI read it instead of you, that's okay. I don't think that would, you know, if it's true, if you truly. That this is totally fine and there's nothing wrong with it. I don't think that disclosing that should do you much harm. On the other hand, you know I think the real danger of using AI to do your writing for you is that it is possible to dilute your own perspective, right? It's possible to feel like you're sharing your perspective with the world, but you can kind of like, you know, get to a place where you're actually not giving your perspective. You're kind of giving the robot's perspective. And reading that op-ed, I kind of didn't feel like that's what happened there. I think you actually did. Like, I think the objection he made was true. Basically, it's like yes, this is a pretty spicy take. It is dressed up in extreme, obvious clod slop language, and so what do we do with that? I'm not sure.
And I'm sure this is fed into your decision, as you said, to add the Pangram feature to Substack. But the idea of Substack, you know, it should be a diversity of voices, and you're trying to get there is something about Substackers is that they're trying to kind of express something authentic to readers. Did you feel that you know AI potentially sort of was going to detract from that, both because you know it potentially wasn't the authentic voice of the people who were putting that out, but also there's this tendency with AI writing, and one of the reasons we can look at something and say it's AI slop is there's a certain you know it has certain features, there's a certain homogeny to it, and were you concerned that you know you might see this kind of homogenization of writing on the platform if you allowed AI to kind of just be used in without any kind of ability to flag it.
I have noticed that among discerning readers, AI prose is an acquired distaste. I think I feel this, and I think a lot of people feel this sort of sense of, oh man, this just leaves a bad taste in the mouth right now. Not for any philosophical reason. It's just like, okay, I've seen so much of this. It starts to cloy. It starts to jar. You know, I think the point of Substack is, as a writer, as a creator, you get to make the thing that you believe in. You get to share your perspective. You get to use whatever tools you want. You get to say whatever you want. Have your own. You know, this is like your space and your thing. And I'm not going to sit there and tell you what tools you can and can't use. I think it's very, you know, possible even likely that people are going to make all kinds of great writing, great art, great pieces that are like clear expressions of their perspective that are deeply valuable and that use these tools. I think it's also undeniable that lots of people are using these tools to like put out a lot of crap that's annoying. And so on the other hand, on Substack, you know, as much as you have the right to put out whatever you want as a reader, you also have the right to choose who you subscribe to. You know what kind of content you're getting in your algorithm. You know you want to be able to have the choice of where you're spending your time and attention, and I think you can have both those sides can work. You know you can have freedom on both sides, and that can create a better solution than us trying to come in and say this, not that.
I wanted to ask a little bit about how Substack is approaching AI, and the fact that you have to engage with AI companies. That you know, when you ask a query to an AI model, it goes out and it you know looks for answers on the internet. It scrapes information from a bunch of sites, both to answer that question in the moment, and then of course, famously, AI companies have been scraping the internet to create these giant corpuses that they use as the pre-training data for all of these models. What has been Substack's approach to kind of the scraping of the site for kind of informational purposes or for training data?
Our approach in general has been to give creators the choice of what to do, and then whenever you do have public stuff that is available, we have pushed to try to make sure that you have the best kind of like links back to subscribe. Because the thing that can happen, you know, I think in a lot of cases, if you're getting, you know, if your website that just has some pure information or something, and then somebody asks that question, and the answer just shows up in ChatGPT. There's not really anything, any value left. But I do think if you're, you know, if the thing that you have on a Substack is a distinct perspective, and there's some small fraction of people that are exposed to this idea, that then will want to go deep. I actually think that can be pretty valuable.
I wanted to ask also about you know how AI has kind of changed business within Substack. Are you using AI models you know in what you do? Are your developer teams using coding agents? What's it look like in terms of the use of AI within Substack as a business?
We're using it pretty extensively. I mean, for building software, the stuff is incredible. It really is magical. The new model Astra that just came out. I spent the weekend kind of like sending it little test things. I wrote about it on my Substack. In terms of you know what's possible for building software, I do think we're just in a new and very exciting era with this stuff.
Is there any concern that because of the capabilities of these models, you may have board creators that kind of choose to create the publishing software themselves? So they create, you know, my own newsletter software. I can now create something that you know can link in with billing much more easily. That you know, I can have an AI agent that automatically goes out and finds which subscribers are coming up for renewal and sends them renewal notices. All these things that Substack was kind of built to make easy for writers, because writers really just wanted to concentrate on writing. Potentially, I guess you could now get some of these models to do some of that work. Maybe that makes it you know less of a valuable proposition to share some of your revenue with Substack is Substack kind of looking at these models as a potential threat in that way?
Not along that dimension, and you know because basically you know this was already true, right? If you reduce if you think that what Substack provides is just like a way to host a website, there was already lots of ways to host a website, and yes, now there are even more ways. But the people who and one of the benefits of Substack, by the way, is that you actually can leave. So if you have an email list on Substack, you can take it with you, and you can. People have done this. They'll go and build their own website. Usually, what happens is they come back and they say, okay, building a website was harder and more annoying than I expected, and I missed the growth that I got from being part of a network of people and ideas that is creating this thing, and I think that's sort of true in the long run. Like I don't think that the magic of Substack is not only in hey I've got this good editor where I can type something into it.
Right. I mean there's this whole thing around sort of discoverability and curation and the sort of way Substack helps build a community that is kind of hard to replicate.
I think this is what will happen. You know, like I can build my own website now. I built an app this weekend. I published it on my Substack with Astra. Anybody's going to be able to build anything, and that's wonderful. Like the cost of building even pretty high quality software will go down, but the difficulty of getting anybody to pay attention to it and care and use it will continue to go up, and so the hard part is going to be not can I make a website. The hard part is going to be can I get anybody to care about the thing that I'm making.
