The business model of the last 27 years of the internet is dying. We might be on the cusp of what could be a truly golden age of content creation. What's really changed is actually the consumer behavior. And the reason the consumer behavior has changed is because the models have gotten better. Answer engines are better than search engines. Full stop for consumers. What we need to do is preserve journalism and we need to preserve knowledge creation and we need to preserve storytelling. Most humans make decisions on what small businesses they interact with based on either convenience or emotion. Agents don't care about either of those things. I do worry that left to its own devices, agentic commerce is a massive consolidator. Who's going to be the first salesperson who's sitting in a customer meeting and a customer asks for a feature and they build it in the meeting and release it to production while they're sitting there?
Welcome to Disruptor Confessions. I'm your host, John Ays. Our guest today runs a piece of the internet that a lot of you probably never even think about. And when it works, you don't know it's there. When it breaks, a good portion of the web goes dark. Matthew Prince is the co-founder and CEO of Cloudflare, but he's also one of the sharpest voices on what's actually happening right now. What is AI doing to the open web? I actually heard Matthew speak at the World Economic Forum earlier this year in Davos and he shared a warning of sorts and he said something to the effect of big tech has never been more vulnerable than they are today. And he went on to talk about how AI agents are changing things and in fact when AI agents start to do the buying for us that local businesses that hold our communities together they're actually going to get cut out of the picture completely. And that's the conversation we want to have. So Matthew, with that, welcome to the show.
John, thanks for having me.
Yeah, fantastic. So I guess the first thing, you know, I think of Cloudflare as just this leading, you know, security infrastructure for the internet. I want to get into this whole agent commerce space, but we pride ourselves on kind of making sense of the conversations for the broader public. So for our audience, for me, how do you describe Cloudflare? Let's just start there.
Yeah. So Cloudflare is rebuilding the internet to make sure that it has everything that it should have had if we knew what we were getting into in the very beginning. It's interesting if you go back and read the original specifications for the internet that we all use today. This was never supposed to be the thing that carried live video like we're doing now. It was never supposed to be the thing that handled commerce. It was supposed to be sort of a silly almost toy research project on the side. There was an official kind of secure internet which was being developed at the same time that never really took off. And so instead this thing that had no real security, had no real reliability built into it, but was incredibly cheap and efficient to run. That's what took off. And so Cloudflare started out by saying how can we basically make it so that anyone who was putting content online could have security. How could we put a firewall effectively in the cloud? Over time we've developed an entire developer platform where a huge percentage of the new sites and applications that are being built online are developed through and because of that, you know, a huge percentage, more than 20% of the web now sits behind Cloudflare and that gives us sort of a catbird seat both to understand what's happening and what trends are going on but then also to help define what the future's going to look like for the internet as a whole.
And I think you've actually described, I love this, and maybe it was you, but it's almost like you guys are the bouncer. You decide who gets in the door or not. You're kind of like that blocker. Is that a simple way to think about it?
Yeah, I think that's probably the right description for where we started. I think that it's probably not complete today, but you know, whether it's we're sort of the force shield or the bouncer, we help make sure that those sort of rules are followed. But then on top of that increasingly you can develop whole applications on us and increasingly we're also thinking about things like, you know, if there's going to be a new business model of the internet what is that going to look like?
Well, that's actually I'm probably leading the witness because I was thinking about your mission which was I believe when you started, build a better internet. That mission still is your mission, right? But it's different.
With an asterisk I think, which is you left one really critical word out which is our mission is to help build a better internet. We don't think we can do it alone. And so we try to partner with, you know, both civil society organizations, other commercial organizations, people who are standard setters, regulators and governments around the world because I think this is something that, you know, the internet when it's at its best is governed sort of from the bottom up. It's not controlled by any one organization and so our mission is to help build a better internet. Our mission is definitely not to build a better internet because I think that's a real threat and a real disservice to all the amazing people that have worked to build the internet into what it is today and I hope we help continue to push that forward but we can't do it alone.
100%. You know, you have this incredible view from which you sit within Cloudflare and the work that you guys have been doing the last, I guess, 16 years now. When I think about the volume of data, exponential curve of data, there's a lot of us that aren't seeing what you're seeing. Can you just without getting too detailed, you just tell us broadly what you're seeing? Not the last 16 years, but really, Matthew, like the last three years, what are you seeing in terms of like the biggest change in data that the rest of us aren't even aware of? Maybe start there.
Yeah. So the first thing is when we started Cloudflare and for most of the internet's history there's always been automated traffic. So Google has crawled the internet in order to build their search index. There's been, you know, automated malicious traffic that's been out there but it sort of hovered around 20% of the total traffic that has been online. I was asked last year actually at WebSummit, so in late 2025, November of 2025, you know, when did I think that automated traffic, agentic traffic would cross human traffic online. And we pulled all the data in order to sort of give a good estimate for that. And at the time it looked like it was going to be in the second half of 2027. At South by Southwest, just a few months later in March of 2026, I was asked the same question again. And we went back and revisited the date. I expected that the answer would be the same, but it actually moved up quite a bit, which was now agentic traffic, automated traffic was going to pass human traffic in the first half of 2027. And so I was really surprised when in May of this year, the team came to me and said, 'You won't believe it, but now that automated traffic is 57% of traffic online, it's already past human traffic, which is now only 43%.' And if those trends continue, in five years we anticipate that automated traffic, that agentic traffic will be a thousand times more traffic than humans online. Not because humans are going to stop using the internet. We think human traffic will hold pretty steady over that period of time but you're just going to see just this explosion in traffic which is generated by these agentic systems.
And, you know, we can all see that if you've ever used ChatGPT, if you've ever used Grok, if you ever used Claude and you ask it a question, you know, if I ask like I'm shopping for a digital camera, you know, if I'm doing that personally as a human, I have a finite amount of time. I have a finite amount of patience. I might visit five websites and, you know, visit, I don't know, 10 pages per website. Your agent's going to visit literally a thousand times that much and see just more because it has almost infinite patience and almost infinite time to be able to do that. And so that alone is going to drive just an enormous amount. That's having consequences as well because these agentic systems are getting significantly smarter. If you think about like search in the past, when you search for something, Google would return, you know, 10 blue links, the search wasn't over when they got the links. That was just the treasure map. And then it was your job as the human to click on the different links and then go find what it was you were actually looking for. Search engines are changing from being search engines to now being answer engines. So, if you've run a Google search recently, the results that you get back at the top often answer your specific question without you having to click on anything. And that's even more dramatic with a service like ChatGPT or Anthropic Claude or Grok or any of the other systems that are out there.
What that's doing though is it's actually putting real pressure on the business model of the internet. Because the business model of the internet for the last 27 years was really defined by Google and was defined by ads and then to a lesser extent subscriptions. The ads and subscriptions, like, agents don't click on ads. And so as a result, if people are getting their answers without having to go to the original sources, then they're not going to be actually serving those ads. And the people who are creating that content, it's getting harder and harder for them to actually make money. To give you some sense, in Google of old, you know, for every two pages they would scrape, they would send you about one visitor. Today it's up to about 40 pages Google scrapes for every one visitor that they send you. And that's the good news. In the case of ChatGPT, it's about 3,000 pages scraped for every one visitor they send. For Anthropic, it's over 100,000 pages scraped for every one visitor they send. And that keeps getting harder and harder every single year, which means that ads-based business model is really under threat.
So you have sort of two things happening simultaneously. The amount of traffic which is being generated online is going through the roof. And that's, you know, someone has to pay for the bandwidth, someone has to pay for the server, someone has to pay for the compute that's behind that. And then the business model which has been based on ads is under pressure and threat which means that there's less money to pay for those things. And so that I think is forcing a real rethink of what the business model of the internet is going to be. And I think the most interesting question certainly in tech, maybe the most interesting question in the world is what is the future business model of the internet going to look like. How are we going to, in the world which is increasingly going to be driven by agents, make sure that the content creators and the people who are putting those applications online are able to get paid themselves and we're able to support the infrastructure to have what is, you know, maybe a thousand times more traffic online than we're seeing today. I think those are fascinating questions and that's what we're spending a lot of time thinking about at Cloudflare.
Okay. So there's a dozen things I want to kind of unpack there. So I'm following very much of what you're talking about. It's a lot of the work that I do in the kind of space that I live in in Agentic and the marketing automation capabilities that I work in for the last number of years. I want to start with the search moving to AI. So can you talk about when you talk about scraping or crawling and kind of the foundation of how the web has been built over the last 28, 30 years primarily by Google. Can you just help our audience understand this what I call often the give-get, the value exchange? Can you break that down? You just touched on it. I just want you to share that so people fully understand when I search for a new bicycle, new camera, and I get these blue links and I see banner ads. Can you break that down for us a little bit first and then we'll go from there.
So the way that Google historically worked, so the Google of 10 years ago, worked was that they would run an automated crawler, an agent effectively that would run across the internet and it would make a copy of the internet and it would put it in an index and then they would use that copy when someone searched for something in order to send them, you know, to what they thought was the right place online.
Like a camera for instance, right? Buying a digital camera, you want to buy, you know, a Nikon Coolpix camera.
And so they have essentially made a copy of the entire internet. They know everybody that sells digital cameras. They know all the review sites. They've got copies of that. They've put it in a giant database that they run. And generally, historically, for every two pages that Google would make a copy of, in exchange, that was the give from the site owners, they let Google make a copy of that, the get was that Google would then send one visitor back to them. And that's the average across the entire internet. In the case of a regular review site, it was actually probably much more favorable than that. You might be that for every one page that Google copied, they might send you 10 visitors or something like that. But on average, for every two pages they copied, they would send you one visitor. And then what you could do with that visitor was a number of different things. You could either put an ad in front of the visitor, in which case you got paid by the advertiser to show that ad to them. You could charge the visitor for some sort of a subscription like the New York Times, the Wall Street Journal does. Or you could actually sell them something. You're selling digital cameras, they've got traffic that's back to you to buy a digital camera, and you could be an e-commerce platform to be able to do it. Those are the financial things.
On the other side, there are also some non-financial things. You know, you think of something like Wikipedia. Wikipedia, all of the content that's created there is just done by volunteers. Nobody is making money from, you know, updating a Wikipedia article. And yet, people do it. But one of the reasons people do it is for the ego of knowing that actually people are reading their stuff and doing things. And a lot of the internet was created not just for financial reasons, but for ego reasons. And so all of those things over the last 27 years that Google has existed, Google has really helped facilitate. They did things like building out AdSense in order to make it easier for anyone to put ads on their pages. They helped build subscription platforms for people to help do that. They gave people much better attribution to be able to do e-commerce more efficiently. And then even just for the pure ego, they built things like Google Analytics that let you see when people are actually visiting your site and reading your content. And so whether it was financial or just ego-driven, a lot of what the internet had been before was really facilitated by Google and the ecosystem that they created.
Okay. So using your camera example, right, I put in Nikon. I want to buy a $100 camera for my 12-year-old son versus I want to buy an $8,000 professional grade camera. People all across the globe have been doing that. They get these hyperlinks and then they search, they go to the about page, they go to the product page, and they go through the process. But now I'm seeing my own behavior and you can speak to this in detail. I might just go to Claude or Chat or Gemini is a little bit different because they are part of Google. But that behavior shifts now. I'm not maybe going to the about page and the product page. I'm just asking questions. I'm not seeing the hyperlinks. Can you walk us through what is starting to emerge?
Yeah. So I think, you know, if you look back to when ChatGPT first emerged people were using it kind of as a research tool to augment the search that they were doing. So they would say I want to buy a digital camera for my 10 or 12 year old son. I want to spend $100. What would you recommend? And it would make some recommendations but people tended to then go off and find things because it wasn't great at it. What's happened is the interface has stayed the same. The ChatGPT interface is exactly the same as it was but the models behind it have gotten significantly better and people are learning that gosh I can really trust these things. And so if you ask now a ChatGPT or a Claude or a Grok, any of these systems like what is the best digital camera for my kid who has these requirements and in my budget which is this, oftentimes it will just say this is the best one and by the way here's the best place to buy it. And as a result there's a lot less of you exploring around and hunting for it and in the process that means that there's not an ad that anyone has sold to you. There's not a subscription anyone has sold you. Yeah, there might be one e-commerce company that still gets that and that's why actually when you look at who is winning and who's losing in sort of this world of agent commerce for now the e-commerce companies are the ones that are actually still seeing this drive real business to them.
But the other part is that, you know, let's imagine that someone just as a hobbyist had created a site that was out there that just reviewed every digital camera that had ever been created. And they wrote detailed, detailed, detailed reviews about it. They didn't make any money off it. They didn't have any ads. They didn't have any subscriptions for it. They just did it as a passion project because they really cared about this. What's hard is that even for that ego piece, those people aren't getting any sort of satisfaction because the crawlers are coming in from, you know, OpenAI or Anthropic or even Google now, they're taking all of the work that that person had done and they're importing it and now that's being their work of having said, you know, for a 10-year-old for a $100 budget, here's the best camera. That's now getting sucked up into this super intelligence that we have. And as a result there are fewer and fewer visitors to go back to even the original source. So even if you're just doing this for ego the amount that you're getting back from it is much less. And we see this in some really specific ways, the number of contributions to Wikipedia is dropping because fewer people are actually reading Wikipedia. And if people aren't reading your stuff then people don't actually want to, you know, even if it's a volunteer job, they don't want to do it. There's not as much meaning to the work that they're doing. And so from all of these different places, I think it's putting pressure on what has been the magic of the internet, the magic of the web over the last 30 plus years. And it's going to change and require us to rethink things for what's going to happen for the next 30 years.
So this value exchange and this give in simple terms. I want to ask you how you define or describe two words, a publisher. How do you define a publisher in this context? And how do you define a content creator? Can you talk about that because I think that helps us understand what's being scraped and that sort of thing.
Yeah. And I'm probably a little sloppy between those two terms because they're probably kind of interchangeable to me. Oftentimes if I were really thinking about a publisher, I'd be thinking about somebody who's aggregated some sort of major site together. So Condé Nast obviously is a publisher or People Inc., those are publishers. They have multiple different brands that are behind them. And then under each of those brands, they're going to be people who are creating content, the authors, the journalists, the folks that are doing that. I think content creators could be literally anyone. The fact that you put something online, you're obviously a content creator. I'm at some level a content creator. And we've got to just think about what are the incentives for them. Obviously, the publishers are often, you know, driven primarily, I would say, by monetary interest, the ads, subscriptions or e-commerce like that piece. I think content creators have a much broader set of concerns. It's actually, and for me it's actually easier for me to figure out how to solve the economic interest. I think one of the things that I'm still struggling with trying to figure out is how in this world where more and more of the content which is being created is actually consumed through an AI system, how do you solve the ego piece? How do you make sure the person who's doing that work to come up with that Wikipedia entry or that camera review site that they're just doing as a passion project, how do you make sure that they still feel like they're relevant in a world where people aren't consuming the content directly, but they're consuming effectively a derivative of it from the AI agents. And I think that's going to be, you know, I don't have a perfect solution for that yet.
So, I want to get into the actual transaction that's happened in the past 28 plus years and where you think it's going and what you've built for it. Before I do that, can you give us a little sense of the scale that you guys work at, the number of transactions or requests every second I think. Can you just talk about the scale? Because I think that'll help all of us understand the insights that you have gleaned over the years.
Yeah. So, again, more than 20% of the internet uses us. We see every day basically anyone who's online will pass through our network. In any given second we're doing over half a billion transactions, internet transactions per second. The scale is just mind-boggling in terms of what it is. If you look at the top 10,000 websites online it's 40% plus are using Cloudflare. Almost everyone who is interacting online is passing through our network, you know, usually hundreds or if not thousands of times a day. And again, when we're doing our job right, we're just in the background making sure that you have a fast, secure, reliable experience as you're using any of our customers.
So, taking that volume, that scale, and those insights you can glean from it. You touched on, you know, Google scraping and now that's changing because of AI like ChatGPT and Anthropic, OpenAI and others. Can you double back on to some of that and help us understand, hey, OpenAI is now doing this but the give-get kind of conversation, that value exchange, wait a second they're scraping content what are they giving back? And even the numbers of Anthropic because even in the last six, eight months since you were at Davos those numbers have changed dramatically. Matthew, can you share with us what you're seeing there?
Yeah, I think the way to think about this is that what we're trying to put a metric around is how much harder has it become to be a publisher, a content creator, and still either make money from your job and what you're doing. And I think the answer is in the case of Google, it's about 20 times harder today than it was, you know, just a few years ago to be able to actually make money. So, if you were having to publish, let's say that you had to publish a thousand articles in a year in order to cover your costs, now you're going to have to publish 20,000 articles in order to cover your costs in order to be able to get that same thing. And again, that's a radical simplification, but that's I think a way of thinking. And that's with Google, with OpenAI, with Anthropic, you would literally have to publish millions or in the case of Anthropic, hundreds of millions of articles in order to be able to get the traffic back to substitute for what you were doing.
In all of these cases, the story is the same, which is that people aren't actually reading the original sources. They are more and more trusting the AI summary of those sources. And as they do that, less and less traffic is going back to the original source, which means that if you rely on either advertising or subscriptions, then it's much much harder for you to actually have a viable business going forward. And we all see this, publishers are going out of business left and right. There's consolidation in the industry as people are trying to say, how can I get more efficiency by having more content that's there in order to be able to cover what my fixed costs are. And that's just putting a lot of pressure on everything that's there.
What I worry about is if we don't figure out what a new business model is for content creation, and this is before we even start to talk about the challenges for small businesses and other things. If we don't figure out something for that, you know, I think we're going to see things like journalism really suffer. We're going to see things like creative storytelling get much, much, much harder. What actually gives me a lot of hope here is that the AI companies themselves actually need original content. They need humans to be out there going and doing these things. And so that's part of why I think that there's a real hope that we may be able to figure out what a new business model for the internet looks like. And that, you know, even though right now it feels somewhat dire, I actually think that we might be on the cusp of what could be a truly golden age of content creation because you have a very willing buyer for new original interesting content in the AI companies.
You have a whole bunch of journalists who have been for the last little bit playing a game where what they were doing was trying to generate content as inexpensively as possible and then put articles and headlines in front of it that generated as much sort of provocation and rage as possible to get people to click on them. And no one's enjoyed that. It's not been any better. I remember sitting with a new media CEO who was bragging about how they could generate a piece of content for very little money and then they would literally A/B test headlines to see which ones produce the largest cortisol response because again, if you're selling ads, that's what you want to do.
And so I think we also need to be just really honest and realistic that while, you know, the business model of the last 27 years of the internet has been amazing in that it's created the internet, it's created everything that we've seen, it's also done some really bad things. It's caused us to really divide the world more. It's caused us to literally rage bait people into clicking on things. And all of those things are really challenging to making sure that you actually find, you know, what is the truth. Traffic has always been a really poor proxy for quality. Like there's a lot of things that humans stare at that aren't really good for us. You know, traffic accidents on the freeway, pornography, salacious violence, like none of that is actually kind of appealing to our greater angels. And I think a lot of that is embodied by what is just how do we have more traffic, more attention and that's the opposite of how do we create truth? How do we create beauty? And those things. So like the business model of the last 27 years of the internet is dying. Full stop. But I'm kind of optimistic that this gives us an opportunity to think about what could that future look like and how do we design a business model that actually rewards not just traffic creation but actually knowledge creation.
Listen, you're not going to get an argument from me. I think if we had Tristan Harris or Jonathan Haidt or others on the call with us, I think they would certainly weigh in on the good and bad of the last 20 plus years. I want to go back to, you know, Google going from two to 20 and Anthropic and OpenAI, the volume of scraping to returning. Is that the same for anybody? So, for instance, for Walmart, New York Times, for this podcast, Disruptor Confessions, for Nikon, is it across all of that kind of content that's out there? Is it neutral?
It's the average across everyone. I think that you're seeing people who are just purely sources of information are getting hurt more and things that are easier to summarize are getting hurt more. So long form storytelling isn't getting hurt as much as digital camera reviews because again one is sort of facts and the other is story and it's hard to summarize, you know, reading the cliff notes of a book isn't the same as reading a book. And so when people are just looking for facts, I think AI is consuming that more. Same thing is true. You see more for media companies, they're getting hit harder than e-commerce companies, at least so far. We'll see if that changes in the future, but I think right now that's the case.
But the real trend, the reason the numbers keep getting worse is not because the AI companies are scraping more. The scraping has held fairly consistent. There are more AI companies and so the overall scraping has gone up. But what's really changed is actually the consumer behavior. And the reason the consumer behavior has changed is because the models have gotten better. People are trusting them more. If you just think about your own behavior, how many times do you run a search and you don't click on anything, right? You just read the summary that is in the Google overview or how many times are you going to ChatGPT or going to Claude without actually running a search at all. I think all of those things are consumers behaving and the general trend has been that we've seen since the launch of ChatGPT essentially every year the more that consumers trust these systems has been doubling year-over-year and that means that as you trust it more then you don't have to go back to the original source or read that and so again that's just continuously putting pressure on what has been the historic business model of the internet.
Matthew, so a couple thoughts. I want to get to Cloudflare's Content and Independence Day. I want to talk about that. I want to actually have you define what is Agentic Commerce today, where is it headed. So, I want to talk about both of those, but I can't help but think our listeners and people watching this going, 'Wait a second. If I go to ChatGPT and ask about a camera, that seems great. I don't have to scan 20 different blue hyperlinks. I don't have to look at all the advertising. It's gonna summarize based on my budget, my needs, my whatever the consumer, the customer is saying, value in that. Can you touch on that first?'
Answer engines are better than search engines. Full stop for consumers. This is what consumers want and it's why it's going to win. So, this isn't like, I think anyone who sort of says, 'Oh, let's try and preserve the internet of yesterday,' is sort of missing the point. Consumers don't want to go on treasure hunts most of the time. What they want to do is get the answer as quickly as possible and these AI systems are better at doing that than search engines are and so over time they're going to win, full stop. And so I think that's like, it's not about, and I think anyone who's like, 'Oh we need to preserve the publishers of yesterday,' like no we don't. What we need to do is preserve journalism and we need to preserve knowledge creation and we need to preserve storytelling. But whether the New York Times or the Wall Street Journal or the Financial Times survives, those are all institutions I like and admire and in most cases subscribe to, I'm not sure that the goal is just to say how do we make sure yesterday survives. The world is changing and I think what we need to say is the business model is going to change. That means there are going to be different winners and losers, but hopefully we can create a system where we're constantly seeking truth and then have incentives to publish that truth in as open a way as possible.
I think those are the things that we should be, as we think about what the change looks like, we should be trying to figure out how to define the new structures that actually facilitate that. We should learn from some of the mistakes of the last, you know, at least 30, but really 3,000 years of being what has been largely rewarding traffic and eyeballs as opposed to rewarding real intellectual knowledge creation and truth and truth-telling.
Just a quick break in the conversation to ask you to please subscribe to Disruptor Confessions. It's free. It just takes five seconds and it keeps these conversations coming. Thank you. Now, back to the show. I think one of the things that I'd love to have you just define for us, what is agentic commerce today? Where do you see this going?
