Why Everyone Is Wrong About AI (Including You) | Benedict Evans

It seems to me right now you could do like a double blind test of the same prompt given to Grok, Claude, Gemini, Mistral, DeepSeek. I bet most people wouldn't be able to tell which is which.

The very high level threat to Google is that you have this moment of discontinuity in which everybody resets their priorities, reconsiders their defaults. And so it's no longer just the default that you go and use Google. There's this sort of question for Apple around does this actually change the experience of what the smartphone is, what the ecosystem is. Does it end up kind of getting Microsofted in the sense that...

It's funny, my, I suppose my take on AI, controversial take on AI, rather like my controversial take on crypto, is being a centrist in that it seems to me very clear this is like the biggest thing since the iPhone, but I also think it's only the biggest thing since the iPhone. And there's a bunch of people who think no, it's much more than that, it's more like computing. And then you've got people going around saying no, this is more like electricity or the industrial revolution or, you know, transhumanists or something. My sort of base case is to say this is kind of another platform shift and all the new stuff will be built around this for the next 10 or 15 years and then there'll be something else. And so the impact on employment will be kind of like the impact on employment from the other platform shifts and the impact on the economy and productivity and intellectual property, and there'll be a whole bunch of different weird new questions just like there were a bunch of different weird new questions before, and then in 10 years' time it'll just be software.

Well, that's the, there's a famous book about financial bubbles called 'This Time Is Different' because people always say this time is different and it always is. Like the dotcom bubble was different to like the late '80s and the Japanese financial bubble was different to, you know, pick any other bubble you want. They're always different. But that doesn't mean they're not a bubble. And the same thing here. I have a diagram I use a lot from 1995. This research firm made a diagram of something they called cyberspace, because it wasn't clear it was just going to be the internet. It was clear that everyone was going to have some kind of computer thing connected to some kind of network. But remember the phrase information superhighway, which sort of conveys that it would be centralized and controlled by cable companies and phone companies and media companies, which is sort of how everything had always previously worked. It wasn't clear, no, it was going to be the internet. It wasn't clear the internet was going to be kind of radically decentralized and permissionless and anyone could do what they wanted. It wasn't clear the internet was going to be the web and only the web because there were all these other things going on. If you look at Mary Meeker's first big public internet report from 1995, she has a separate forecast for web users and email users. And she thought email users would be way bigger. It wasn't clear like that was all one thing. And then it wasn't clear that it was about the browser. It wasn't clear that the browser wasn't where the value capture was because Microsoft crowbarred its way into dominance in browsers, but that turned out not to matter. And then all the value is in search advertising and social, which were five years later and 10 years later. And so like you can be very, very clear that this is the thing and then still be completely unclear how it's going to work. The same thing with mobile internet. Just funny. Mobile internet now, it's kind of like saying black and white television or color television. Desktop internet, mobile internet, black and white TV, color TV. No one really says mobile internet anymore. It's like talking about e-commerce. You're starting to have people talk about physical retail and retail. But it wasn't clear, you know, I was a telecoms analyst in 2000 and it was very clear mobile internet was going to be a thing. It was not clear that there would be basically small PCs. Like that was the fundamental shift of the iPhone is it's a small Mac. It's not a phone with better UI. It's a small Mac. And it wasn't clear that the telcos would get no value. It wasn't clear Microsoft and Nokia would get no value. It wasn't clear it would take 10 years before it took off. And it wasn't clear it would replace the PC as the center of the tech industry. I mean, everyone was talking about, well, what's a mobile use case? What would you do? You'll do some things on your mobile phone, but what? But obviously your PC will be how you use the internet. And of course that's not how it worked. And so like we kind of forget because now we don't see it because now it just kind of became part of the air we breathe how weird and strange and different all these things are. There's something I love talking about which is the rise of automatic elevators. So until the '50s, elevators were manually operated. They were basically vertical streetcars. They were trams. They were pub trains. And you have a driver who has a lever with an accelerator and a brake. If you've been into a New York co-op, you may have seen one of these. They call it an attended elevator. There's a lever. You push it that way to go down, middle to stop, that way to go up. And then in the '50s, Otis creates the Autotronic, I think it's called the Autotronic elevator, which had electronic politeness, which basically meant the infrared thing that stops the door closing. But if you get into an elevator now, you don't say, 'Oh, I'm going to use an automatic elevator with electronic politeness.' It's just a lift. We kind of forget how weird and different all the other things were. And yes, this is new and weird and different in a bunch of kind of strange, confusing, confounding ways we can probably talk about, but we sort of forget that other things were weird and strange and different too.

I'm pretty sure people thought Microsoft had an advantage on the internet and Google and Meta had an advantage on mobile and everyone thought IBM was going to win PCs. Like once IBM made a PC, that was it. It's all over now. And we kind of forget that like there were PCs before and then IBM made one and that kind of became the standard, but then IBM lost it.

The master of my college at Cambridge said that history teaches us nothing except that something will happen and, you know, there's always the example and the counterexample. So with any new kind of, any new platform shift, and the term platform shift itself is, you know, it's a useful term, but you have to be careful not to be trapped by your terminology and get into this sort of arguments about, well, is it a platform shift or is it not a platform shift and how do you define a platform, like shut up. Like, you know, the thing is when any, with any of these sort of fundamental technology changes, the incumbents always try and make it a feature and they try to absorb it. And the same thing outside of technology, existing companies try and absorb it and they use it to automate the stuff they're already doing. And then over time you get new stuff, you unbundle both the incumbent's intake and you unbundle existing companies because of something that's possible because of this new technology. So you can always kind of jump into the new thing and sometimes the new thing kind of really is just a feature and sometimes it's no, it's a fundamental change in how everything works. And sometimes that sort of contingent, you know, there's this whole sort of parlor game, like drinking game that historians play about kind of historical inevitability, you know, well, what would have happened if that battle had been lost or if that politician had been assassinated or not assassinated. And it depends. Sometimes the answer is, well, no, nothing, and everything would have been completely different. And sometimes the answer is, well, no, then, you know, what if Napoleon had won at Waterloo? Well, then he'd have lost another battle six months later. Like nothing would have changed. The whole environment had changed. What if, you know, the revolution hadn't happened in spring of 1917? Then it would have happened in the summer. Sometimes it's like really clear. I mean, I always think the Kodak example here is kind of interesting.

Because, you know, like it's like the cliche that people say, oh, Kodak had digital cameras and they didn't get it or they ignored it or they didn't want to do it because it would destroy their business. But then you go and look at it and it's like, well, that was 1975 and the thing they had was the size of a refrigerator. Like, no, that was not a consumer product. And it took until the late '90s before the technology was actually viable as a consumer product. So, of course, like they didn't do it in the '70s because you couldn't. What actually happens is once it starts happening, Kodak go all in on digital cameras. At one point, they were the best-selling digital camera vendor in the USA. And if you look at their annual reports of the time, they think this is going to be great because they're going to sell way more photo printers. So they're selling these inkjet photo printers. People are going to produce way, way more photos. So they're going to take way more. They're print. They're going to print them all. Two things that screw Kodak. One of them is smartphones. And you could argue that what actually screws Kodak is not the camera. It's the social media and it's not printing anymore. Because that's what killed, that's one side. The other side of it is that film was this high-margin product where they had a bunch of unique intellectual property and digital cameras are a low-margin commodity where they were competing with the entire consumer electronics industry with no differentiation. And so even if you go all in into that market, it's still a crappy market where you've got no differentiation. So you can kind of put all of these things on the table and shuffle them around and say, well, in hindsight, obviously, Kodak was screwed and in hindsight, obviously Google was going to be able to make the jump and in hindsight, you know, in hindsight, and yeah, maybe.

So I think it's, I'd be nervous about knowing what the margins are in AI because we've had, you know, depending on who you ask, like the price to get a given result has probably come down by two orders of magnitude, but then that was the state of the art two years ago and now there's a new thing which is more expensive. And so there's an awful lot of kind of shifting planes and shift, you know, there's a lot of algebra and all the variables and the algebra all changing at the moment. So it's kind of hard to quite know what that is. I think the obvious Google threat right now is that Google shows you a bunch of links and results and ideas and those could now be solved in a different way. Well, let me kind of go back a second. The very high level threat to Google is that you have this moment of discontinuity in which everybody resets their priors and reconsiders their defaults. And so it's no longer just the default that you go and use Google for this search or that search, like maybe Bing is 10% better on that search. In fact, as we saw from the Google TAC trial last year, actually Google is still the best search engine by quite a long margin relative to the other traditional search engines. But what we have now is like a reset of the playing field. And Google has a whole bunch of advantages as to why they might win in that playing field. But there's a reset both of what the product is and how you sell it and your org structure around selling it and do you have the right politics and the right org structure to build that and the right incentives and internal conflicts and then the consumer behavior kind of get reset as well.

So I'd actually, I think it's actually the opposite, which is that everyone's kind of using the same data, which is you need such an enormous amount of generalized text that the amount that Google has or that Meta has is not actually enough to move to be a kind of fundamental difference in what you can train with.

So it depends. Push back. Yeah. I want to say so it depends. So the models that we're training now, we're training on text. So that's not really being trained on YouTube. We saw this lawsuit around book copyright with Meta that they downloaded a torrent of pirated books because guess what? They don't have enough text and it's not the right kind of text and they don't have lots of prose, they've got lots of short snippets of text. So I think the generality of LLMs is you just need such an enormous amount of data that everyone kind of needs all the text that there is and all the text that there is is kind of equally available to anyone. So we've kind of like the data is a level playing field effectively.

So obviously Google has, you know, an enormous repository of scraped data because they read the web all the time, but anyone else with a billion dollars can go out and do that, right? Or you can go and download the Common Crawl from AWS.

There is this sort of like people parody of like the, like suddenly magically this thing grows and becomes amazing and learns everything. I don't think we're at that stage now. I don't think anyone really knows when it would happen. So it's very sort of impressionistic. I think another answer might be you kind of have to be very careful looking at headlines and thinking like what exactly is that telling me? So Anthropic has done a bunch of things where they say like the AI was threatening to blackmail me.

And you read the story and you think, okay, you asked what's basically a story generating machine, please tell me a story of what you would do if in this situation where most people would probably say X and the machine says probably X. And you say, my God, it said it would do X. It's like, well, yeah, it would blackmail based on human behavior. It would blackmail you.

It's kind of like the reductive ad absurdum of this is, this is a point somebody else made, that like you write 'murder is good' on a piece of paper and you put it in a photocopier and you press go and you say, my God, the machine says murder's good. Well, no, you told the machine to say that. And that's what these Anthropic studies are. They're basically you tell the machine to say a thing and then it says it. Like, well, you haven't proved anything.

So, I think regulation of AI is sort of the wrong level of abstraction. Talking about regulating AI as AI is the wrong level of abstraction. And it's like saying we're going to regulate databases or regulate spreadsheets or regulate cars. Well, we do, but not like that. When you regulate stuff, there are trade-offs. Like, you learn about this in your first year in economics class. Like, regulation has costs and consequences, and it's not necessarily, you know, there's always a trade-off. And often you're making product decisions or engineering decisions that do actually have trade-offs. There's like a three-way trade-off of like what's good for the product, what's good for the consumer, what's good for competition, what's good for the company, what's good for the consumer.

Well, what's your objective? Is your objective to have a nice press release?

A long time ago, I used to get these questions about like how can we replicate Silicon Valley? And I always feel like the answers to those questions as much as possible is like, you can't. I mean, occasionally there are things you can do, like you can create funding structures, you can make it easy, you can, you know, you can try and jumpstart startup ecosystems, you can try and jumpstart funding availability, but most of the answer is things like getting out of the way. I think that the idea of trying to create national champions is very hard. Now that almost kind of becomes an economist's question rather than a technology question. How do you create national champions? Where has that worked? Where has that not worked? I'm sure there's a bunch of books and papers about, you know, where does industrial policy work? Where does it not work? From a technology analyst perspective, I think of this in terms of A, what are you doing that would make this harder, and B, think of this as just more startups. What are the things that we're doing that make it harder to develop that ecosystem without picking a winner? It's not about picking a company and backing them. It's like if you do like this ridiculous law that California had a year or two ago, if you treat this as like nuclear weapons and you say this is incredibly dangerous and we need to have it under extremely tight control so that nobody does anything bad with it, which is basically the EU approach. Go back to your economics class. Policies have trade-offs. To govern is to choose. You're making a choice when you do that. And you're choosing that has costs. Personally, like most people in tech, I think the idea that this is all going to kind of produce bioweapons and take over the world and kill us all is just idiotic. Like I think it's just a bunch of kind of childish logical fallacies within that. But you have to be conscious. You say if we're going, you know, the kind of the Biden approach to generative AI very explicitly was to say this is sort of social media 2.0. Like social media 1.0 was terrible and destructive and bad. And I think there's a, I don't agree with that. I think there's a huge dose of moral panic within that. But be that as it may, if you make a decision that says we are deliberately and explicitly going to make it really hard to build models and really hard to start a company that builds models and really hard to do anything with any of this stuff, then guess what? It's kind of like, you know, the mayoral election in New York today. Like if you make it really hard and expensive to build houses, houses will be more expensive. You've made that choice. If you do that, you cannot then complain that houses are more expensive. You can choose that, but you can't complain.

Part of this is that like in most non-emotive fields, we kind of do, like you understand, people understand that, you know, if you make, you know, more employment regulation tends to produce slower growth but more protection for employment and you're choosing a trade-off. People kind of, I think everybody on both sides of that equation understands that that's a trade-off and you're choosing one versus the other. The point is you can have a fully functioning free market and you can regulate some of the negative externalities of free markets that anybody in any part of the economic spectrum understands there are negative externalities to free markets. You can also have like a government provided alternative. You can have the government do the fire department where you have some of the kind of most obvious gaps between the US and Europe it seems to me sometimes are in places where you kind of have neither. So the US neither has a government-controlled health care system nor a free market health care system. Do you see what I mean? You have neither a government-controlled housing which you have in like weird places like Singapore nor a free market in housing. So you kind of break the free market. So you stop the price signaling. This is like the great insight of Hayek is that pricing is a signal. Pricing is an information system. It's telling people what's wanted. It's not just a signal of worth. It's a signal of demand. There's a fascinating book I read a while ago called Red Plenty which is about Soviet central planning in the '60s, '70s, '80s and it's about sort of what happens when you have a central planning that just cannot cope with the level of complexity of a sophisticated economy in the '60s and '70s as opposed to let's make grain and tractors and locomotives in the '20s and '30s and steel which already kind of works. But once you actually have a sophisticated industrial economy, central planning can't handle the complexity. And so you try and create incentives and structures around that while not having pricing and that just doesn't work. I suppose there's a sort of a generalized point which is like a market economy is a system and if you pull a lever here something will move there and you can't just pull a lever here and say, well, I don't want that to move because it's a democracy. It will move anyway and so you have to understand how the system works and understand what consequences you want from that and what your parameters are within this.

It's funny. I have a draft thinking about like what LLMs do to web search and publishing and discovery and e-commerce and like big foot, hand-wavy, fuzzy, all of that stuff. And I was sort of thinking about this and there's a book written by a French academic sort of 20 years ago or something called How to Talk About Books You Haven't Read which sounds very kind of snide. But kind of his point is that like there's the book you read when you were 17 and you really didn't get it. And if you read it now you'd get it. And there's the book that like he's got this kind of list of like there's the books that everybody else has read so it's to say you've read them. There's the books that like you've read three other books by that writer so you don't really need to read this one too. You get it, like do you need to read another Malcolm Gladwell book? Like if you've kind of got the Malcolm Gladwell experience. So there's this sort of generalized sense of pattern and accumulation of what you've seen, what you've half seen, what you half remember. There's also I think, you know, what your viewers, listeners might notice is I kind of have two modes, two or three modes. I have a mode that's sort of discursive and slightly rambling and free associating and I'll kind of spiral off in different directions and hopefully come back to the point. And then there's a mode where I want to try and pin the thing down and break it apart and say what are the two, three, four things that are happening here, which is what you see in the slides is no, like what is it? It's this and then this and then that and does that capture that? That's a way of trying to understand what this is. I try and work out what I think about this, how I understand it, how you can break it apart by kind of pinning it down. The thing about data is, and the thing about the slides and the analysis is like I'm always asking, who cares? And I'm always asking, yes, but what actually matters here? Why are you showing me this slide? Why am I showing you this chart? And so you kind of have to ask like, well, what are the actual questions? What are the questions we're not asking on AI that we should be asking? I mean, we're asking, okay, well, there are some people who are saying all the value capture is going to be in the models. There's this kind of funny split between people who are just talking about the models getting better and everybody else is saying, well, all the value is going to be in the application layer and, you know, what are the companies and let's fund Cursor and let's fund all of this stuff and why isn't there a consumer breakout yet? And other people are saying, what do you mean there isn't a consumer breakout? Everyone's using ChatGPT, to which the answer is, well, not exactly, which is my data point. Some people are using ChatGPT, most people look at it and don't get it still, which is fascinating. There's this kind of core where's the value capture question. Then there's like a bunch of questions we could have asked two years ago where we don't have an answer. Will the error rate ever be controllable or manageable? Will you ever get to a model that knows when it's wrong, which to me seems like, given a statistical system, seems like a contradiction in terms, but maybe there was a bunch of kind of, you could make a list of like a dozen questions we could have asked in early '23. We don't really have answers to any of those. I mean, there were some people asking, are these things commodities? Will China catch up? A true answer even then was obviously yes, of course, which is what happened, which DeepSeek kind of demonstrated. But we don't have that many new questions since then. The thing that I puzzle about right now is first of all there's this whole nexus, as I said, of like what is SEO for LLMs? You know, we have infinite product, infinite retail, infinite media. How will you choose what to buy? What happens if I go to an LLM and say, what mattress should I buy? What life insurance should I get? That poses dozens of questions we don't know yet. Then there's a question around like the differentiation in the LLMs as product. Like it seems to me right now you could do like a double blind test of the same prompt given to Grok, Claude, Gemini, Mistral, DeepSeek. Do a double blind test. I bet most people wouldn't be able to tell which is which. That question of like is there product differentiation? Can there be product differentiation around the LLM as consumer product? Because right now the models are commodities but ChatGPT is way, way, way, way more usage. So ChatGPT is at the top of the App Store rank. Gemini bubbles between like 50 and 100. None of the others were in the top 100. Same in Google Trends, same in the usage numbers, same in the revenue. There's revenue for a corporate API. Corporate is a whole other story. But as a consumer thing, it's like ChatGPT is now like the brand. It's the default. It's the Google. You use it because you've heard of it and none of the others have broken it. Is that where we are now? Is that...

Yeah. And there like there's like the browser project, there's people trying to do it now but it hasn't worked, hasn't got traction.

Yeah. And all the other people that tried to do photo sharing.