Using your model, are you able to quantify like what percentage of the internet at the moment is AI slop?
Oh, how do you... based on what you just read, how'd you get that number?
So a lot of the internet is just like SEO written articles, and like much of that... yeah, it's articles written for search, basically, so that your website comes up more often, search because it's targeting certain keywords. And a lot of that industry has switched over to using AI because then instead of having to pay writers, you could churn out articles for pennies on the dollar. But I think that kind of results in a lot of the internet being AI written. It's also kind of platform dependent. So this is just about 40% from an internet page perspective. About a year and a half ago, we looked at Medium and found that over 50% of newly written Medium articles were AI generated, which was a crazy high number.
Reddit, about it was 7% a year ago, I believe. A little over 10% today.
Hello and welcome to another episode of the Odd Lots podcast. I'm Joe Weisenthal and I'm Tracy Alloway. So, Tracy, you know, you ever come across some writing you can't articulate exactly why, but you're like, I'm pretty sure I wrote this. Does this happen to you?
So full disclosure, I haven't really thought about it that much. Yeah, because the thing is, I probably should think about it more, but there's a lot of bad writing out there, and I've become sort of inured to it. And I also think that, I don't know, trying to figure out whether or not something was generated by AI nowadays, if you actually dedicate a lot of your own time to doing that, that is a huge mental burden.
Truly, especially you and I are in the journalism industry. How many of the pitches do you think that we get from PR right now are being generated by AI? I imagine if you're reading each one of those and trying to figure it out on a daily basis, you know what? I suppose I think about it the most is, someone will respond to a tweet. Yeah. And I'll be like, well, if this is a real person, then maybe this person deserves some engagement and they ask a question or I want to respond. But if this person isn't a bot, then obviously I don't. And that's where I'm like, you know, I don't want to figure it out. I would like to know the answer. You know, I have a controversial view about AI writing, by the way, which is that it's pretty good. I mean, like by and large, when I said this, I think maybe in a recent episode, when you consider the fact that I don't know, the majority of the population, like, doesn't know where to put a comma within the sentence, well, this is my opinion, pretty good.
I mean, yeah, one thing I will say about AI is it never gets the placement of a comma wrong. On some level, it's perfect.
Did you see that? I think it was in the New York Times.
The Times. I kind of hated that.
Well, because I'll tell you why. First of all, it's only five examples. There's not very many to it. Ask the reader, which do you prefer? But I think they were different subjects as well. Yeah. And also I think most people probably treated that as, can you guess which ones are human? Because everyone wants to say they prefer the human. I didn't think it was like a great test nonetheless.
Well, it is. I mean, not only is it often indistinguishable, not often is it often fine writing. Sometimes it can come up with a really remarkable turn of phrase.
Yeah, but I still, by and large, don't like it. You read like a thing, especially a long text that's AI, and it's like, even if you can't articulate it, it's like this feels... it has a certain sickly sweetness to it that is often annoying.
So what I notice about it is it doesn't do style very well. Right? So if you ask it to write something in the style of a writer, if you choose anything other than something really obvious, like Shakespeare, yeah, it really suffers. But the text that it actually outputs is pretty clear.
Yeah. Right. Like for basic understanding. Totally. It's probably better than a lot of what's on the internet.
You know, the real people who are going to have to worry about this are like teachers. Obviously universities and lawyers, student lawyers and maybe law. It's fine. But there are some times I was like, okay, did someone write this or not? And there has to be... it'd be nice if like, we could know the answer.
Well, the other thing that's starting to happen is, have you seen any books out there that actually come with a disclosure or disclaimer that say, this book has been written only by humans?
No, I haven't. I saw that for the first time on a book that we actually read for an All Bots episode. I don't think it's come out yet, but that kind of threw me. Yeah. No, it's more and more anyway, as we enter a world in which the vast majority, if not already, of words written are written by AI. I was going to be interested in this question of whether we know. Anyway, there's this company called Pangram Labs, and they have a little thing and you can pay for it, but also a free service where you can drop like a text in and they'll say the odds that it is written by human or AI, and I'm pretty impressed by it. I like did some samples of my own writing and then I output it. Got them all right. But then I did some like further, like I tried to stump it to see if like... so what I did was I took a piece of AI writing, and then I had it translated into Chinese. Okay. And then I had it translate that into Chinese. So it's like, okay, imagine this is being written by a more formal register. And then I had that translated into Hebrew, and then I had that translated into English. So the original thing is a series of AI telephoned through various translations, and then I put that output back into Pangram, and I got that right. It said it was AI. So even after a series of sort of transformations designed to obfuscate the original style of the piece, to see if, you know, eventually it would emerge in something else. So I was pretty impressed. It seems to work. And, you know, I think that's interesting for a couple of reasons, which is maybe there is something that you can just tell, but two, it sort of worries me because, you know, there have been articles in the US saying like this is written by AI, and I think one of my big fears would be that I write something. I like to use the em dash. I have always been an em dash fan. I love em dashes. That's how people talk. I'm sorry. And then what if it says you wrote this by AI and I'm like, I didn't. And then here is this black box that is suddenly judge, jury and executioner for my career. Potentially. You wrote this with AI, the lab says so. You are now done. Like that worries me. So I think this raises a lot of very interesting questions about this model detection thing. And I want to learn more about how well... there's also a lot of philosophical questions about just what we value in writing.
True as well, because no one's going to yell at you for using spellcheck or correcting like that, right? Like it's kind of crazy to think that reputational risk is going to hinge on whether or not you might have used a platform, a chat platform to, like, do some basic copy editing. Totally.
Well, very happy to say we do, in fact, have the perfect guest. We're going to be speaking with Max Spero. He is the founder and CEO of Pangram Labs, and he can answer all of our questions. So Max, thank you so much for coming on Odd Lots.
How do you know what's right? So someone puts in a piece of text and we'll get into the method in a second. But someone puts in a piece of text and it says human or AI. What makes you believe that? You have a very good track record on this question.
So when we started Pangram, we started by doing the thing we call a human baseline, which is how well can we, like as a human, predict whether something's AI or not? That's the first step at like learning. Is this problem tractable? Is it how hard or easy is it? And I found like, me personally, I was able to get about 90% accuracy. And so we figured an AI model should be able to do it much better than that.
So I have a bunch of methodology questions which we can get into. But just before we get into any of that, why is AI slop bad, in your opinion? Why does it need to be tracked and identified?
I think the problem is it's just so easy to generate. And so like there's... it's very difficult to know like what is the intent behind it. Basically like right now I think we're actually pretty lucky living... we live in a world where the signal to noise ratio on the internet and in our information channels is pretty high. We have pretty high signal to noise, but any bad actor can come in and just flood our information channels with AI slop that looks legitimate. It looks like somebody put actual effort and thought into it, but really, it was just like a single prompt, which could have also been automated.
This is something that I think about a lot, which is that there was a point in time, and maybe still is the point in time, where if you read something that was grammatically correct or the punctuation was strong or the spelling was strong, there was reason to think that the person who wrote it was a person of like a certain seriousness and a certain intelligence behind it. And I think that the issue that you're identifying is that that link is now being severed, so that we can't use these heuristics anymore, such as the strict quality of the prose, to know, in fact, whether this was published by someone who was like a serious actor, intelligent or not. And now you have people inserting typos into their copy. I know that to do that, they are just to establish... yeah. Boyd. Sorry, just to go back to my original question. So you mentioned, okay, you were able to get at 90%, right? But now we've just been used a lot more, and you have people paying for your software, presumably teachers and journalists, etc. Given all of that, getting from 90% to 100%, I mean, if you could make one out of ten, it is clearly an unacceptable error rate. Of course, for a piece of commercial software that could call someone an AI creator. So you have to do a lot better than 90%. Talk to us about like, what you've seen so far in your data since releasing it as commercial software. What makes you believe the software is doing a correct job of allocating between the two categories?
So we've built out really comprehensive evals. Okay. And so our evaluations, there's two kinds of errors. There's a false positive, which is when something is written by a human and we say that it's written by an AI. Okay. And there's a false negative, which is if it was AI written and we don't catch it. And so we track our numbers for both of these, and for human writing, we're actually pretty fortunate. We have like millions and millions of samples so we can get like a very... we can get a false positive number that we have a very high degree of confidence in. And our number right now is about 1 in 10,000. Okay. So if we scan 10,000 documents, on average one will come back as AI when it was actually human.
And what about in the other direction?
False negative, I would say around 99% accuracy. So like around 1% false negative rate. I think this depends a little bit more on like how adversarial the prompting is, how much they're trying to like what I did.
Exactly right. We send it through multiple filtrations to obfuscate the original output.
That would be an example of adversarial prompting. Exactly. But in the general case where we're just looking at straight outputs from AI, it's above 99%.
Okay, okay. So what is your model looking for exactly when it's evaluating a text? Because as we mentioned in the intro, you know, syntax and grammar tends to be pretty good on AI generated copy. The style is sometimes more of an identifier, I would argue to your point, Joe, like sometimes it reads very saccharine and kind of overly earnest in some ways. So what exactly are you focusing on here? What are the tells?
Yeah. So the style and the word choices are definitely part of it. But I think what a lot of people don't realize is they're actually making a lot of decisions when they write a piece of text. So there's, you know, dozens or hundreds of ways to phrase every single phrase. And over the course of 50 or 100 or 200 words, you're making thousands of decisions, actually. And so what we're doing is we're learning the patterns in how like these frontier models make these decisions. And if the vast majority of these decisions line up with how the frontier models are doing it, then it's vanishingly unlikely that this was written by a human. You would have to just happen to make the same exact decisions that the LLM does hundreds of times.
Interesting. Okay, but this is a really important point. So everyone at this point has some feel for like, oh, the AI wrote it. Right. But my understanding is it's not like you don't go in and like hard code if you see a bunch of dashes, this is the thing. These decisions in many cases, I imagine neither you nor the model itself can articulate in English what the decisions are. All you know is that the decision pattern exists. Is this correct?
Okay. Can you explain? So therefore, what does it mean that your model has learned these decision patterns?
So what we're doing on a very broad scale is we're training a deep learning model. So it's a pretty big black box. But it has the base model of a language model. And then instead of predicting the next token, it's predicting whether the text is AI or not. Okay. And what it does... well, how we train it is we train on tens of millions of examples. So it sees millions and millions of human examples. And for each human example, we also show it an AI example. So for example, let's say one of these is a five-star review for Denny's that's 78 words long. Then we'll ask an AI to write a five-star review about Denny's at 78 words long in the style of the first one. And obviously these two will be different. And so our model is able to learn through contrast what is the difference between...
And the important thing, sorry, just to be clear here, is that you and I might not be able to articulate the difference. There will be some difference in maybe the sentence length. There will be some difference in word choice. There'll be some difference in punctuation, syntax, whatever. But you and I wouldn't obviously spot it. However, after millions of examples of these side by side, the model learns what the difference is.
Exactly. I think the best that a human can do is look for some of these like really obvious tells, like ChatGPT loves the like it's not just X, it's Y framing. Earlier models really liked some specific words like tapestry and intricate and delve.
Yeah. Delve. Tapestry. Yeah.
But yeah, I think by training on this, we're able to go much deeper than this and look deeper than the high-level signals at the document-level signals.
So one thing that this kind of reminds me of, and I'm thinking how to phrase this, but it reminds me of, you know, those exercises people used to do where you would take a bunch of different faces and meld them all together and come up with like one, oh yeah, that was attractive. So like, to what extent is this basically a distributional detector in the sense that you're looking for like certain paths that you think AI would choose? Could you get a false positive just from someone who's choosing like the average of the average of the average in a way, to state a particular sentence?
Maybe, yeah. I mean, I think that's... there's a reason our false positive rate is one in 10,000 and not zero. It's because, you know, sometimes we look at the false positive and it's like, oh, it reads exactly like an AI generated review or essay, except that it was written in 2019. So it was probably a human who just happened to find the exact like mode collapsed type of way that like...
I think it's a good way to think about the distribution of writing, or writing as a distribution where like, you know, there's a space of all human writing, and then AI writing is really just like a small point within the space. It's very... no matter how much you prompt it, it doesn't go that far from where it was trained to be.
Yeah. Okay. What's the black box? So I built a little model myself. I built this thing that detects... you can upload text, and it says whether it's more resembling of the written word or the spoken word.
Oh, I saw that. Yeah, yeah.
And I used BERT, which is like one of these things, open source one from Google. What is the core model that you trained on, or is it something... or do you build it yourself? Like talk to us about that.
Our very first model was actually built on BERT. Okay. But future models, we needed to up our capacity hiring that. So basically we were running into capacity limits with our model. It was capping out at a certain false positive, false negative rate. It wasn't learning the deeper signals. So we had to 10x and then 100x the parameter count. So that can learn like really deeply what like how these frontier models...
Right. Have you noticed any interesting differences between how the models... Right. Can you... and actually, is your model trained to identify different models as well as whether or not this is just broadly AI generated?
So we don't specifically train it on different models. We don't see like, hey, this one is Claude 3 and this one is ChatGPT 5. But what we've done, we've done some interpretability work to look at basically the output embeddings of the model and where we find that it actually learns which model the text came from. So you can see like little clusters like this is the Claude cluster and like all the Claude... yeah, cluster around here. And then these are like the DeepSeek and Qwen and then this is like ChatGPT and they all kind of like cluster into different spaces in embedding space. So clearly the model is able to learn what the differences between these frontier models are.
Actually, since you mentioned Qwen, I'm very interested. Is there anything like distinct in terms of how Qwen generates text versus platforms that have been developed in the US?
I think Qwen is unique because it's trained on a lot more Chinese and multilingual tokens than other models. So, you know, I've heard from Chinese friends that it's much better like being conversationally fluent in Chinese. Beyond that, I don't know that I can tell... it would be hard for me to look at a text and say like, I know that's Qwen, but I think somebody who is more familiar with it might be able to.
Let's talk about the sort of some of the philosophical or societal implications of this work. Have you had anyone whose text has been judged to be AI written by Pangram and they're like, I swear to God, this isn't AI. And they like really insist. And what do you think about this? Or what do you do? Or talk to us about that.
I've had a couple times this happened. I think one time, like there have been times where I genuinely believe that, you know, this is just a false positive. We scan... we've scanned hundreds of millions of documents. So like at a certain scale like this will happen. But I also get people who all the time they're just like AI detectors don't work. You know, they're like, it's like a total fraud. And then whatever they're putting out on LinkedIn, it's just 100% AI generated and they're just not there. They're getting called out. And then you look back like farther into their past in their history, like everything they're putting out is AI generated until about like 2020. So like for everyone, if you look historically, there's a lot of like sloppy accounts that are putting out total slop. And you can tell either they like weren't posting as much before. And if you scan back in time, then you see that they were writing human text at some point. So there's a number of accounts out there that basically right around the beginning of 2023, where if you scan the entire corpus of their work, it very clearly shows a switch right around early 2023. Yeah, it really like depends on the account. I think one thing we saw that was interesting was there is a writer for The Guardian that was covering the Winter Olympics and somebody was like, hey, your this article is like total AI slop. Ran it through Pangram, was AI. The Guardian was like, no, of course our writers don't use AI. And then we... so we scanned this single writer's like history. And we found that they really did start picking up AI like mid to late 2024 and were using it more and more in their articles.
