AI: Intelligent until Proven? Benedict Evans on Hallucinating Mirrors

Well, there's a lot of caveats. There's a lot of sort of definitional questions embedded within that. So, there's an old observation from, I think, the '70s that AI is whatever doesn't work yet. Because once it works, people say, 'Well, that's not AI, that's just software.' And it kind of sounds like a joke, but it's not. If you go back to the '60s, databases were looked at as a sort of artificial intelligence, and we were going to use databases to do translation, and databases were going to solve everything. And now we just think, you know, when you go to a bank machine or swipe your card to use the subway, you don't think, 'Ah, I'm using AI now,' or 'I'm using a database now.' It's just a piece of software. In fact, you don't even really think of it as software anymore. And so AI in generality is sort of all the stuff that we haven't made to work yet.

You can see this just sort of in the last 10 years. Sort of 10 years ago, machine learning started working, and that meant that speech recognition, natural language processing, image recognition now just kind of works. And so you can go into your phone and search for pictures of your dog or your child, and it just finds them. And 10, 15 years ago, that really didn't work, or it worked sort of half the time, and now it just works. And so you don't look at that and say, 'Ah, thanks to AI, I can find pictures of my dog.' It's just software. And so there's a sort of challenge in generality in talking about AI because it's sort of, you know, there's almost kind of a contradiction in terms in that if it's here and it's working, it's not AI anymore. AI is another way of thinking about the frontier of the stuff that doesn't work. So that's kind of one way of thinking about this.

So a more kind of tangible way to talk about this is that we have this technology called large language models, or generative AI more generically, that's kind of started working in the last two years. And we're trying to work out what it is, how to understand what it is, how to understand what it would be able to do, and what the right level of abstraction to understand it would look like. And, you know, if in a sense just to kind of link those two points, you may remember when people were talking about the information superhighway. Yeah, yeah, yeah, yeah. Oh, long, long ago. Very hard to say what that actually means. If you talk about the internet, that's a lot easier, it's a lot easier to say what the internet is. And then it's one level further in, you can talk about the web, and then we have like a tangible, specific thing. And even once we're talking about the web, it was difficult to get what it was going to be and how it was going to work. But at least today you've got like a tangible, specific technology. And nobody today says that, you know, the web means there will be no more, you know, national governments or something. Whereas people did kind of make those statements about the information superhighway. And the same thing sort of applies when we talk about AI. You know, AI, when you say AI, then suddenly we're like the monkeys in the beginning of 2001 looking at the black monolith. So we have this sort of specific technology called large language models. And there's a few other things around that, but large language models are the main thing that people are interested in. And we're sort of trying to work out, well, first of all, what is it that these things are going to be able to do? How much better are they going to get? And secondly, what's the right way of understanding what they are and how to think about what they are right now, never mind what they might be able to do in sort of two or three years.

And so then you have sort of a range of excitement. From, you know, there were some people who think, 'Look, these things are going to scale perhaps indefinitely, certainly several orders of magnitude from what they are now, because we basically made them work by scaling them, and the scaling will kind of continue. And so that will get us something that looks sort of like human.' And there are some people within that who think, 'That will get us to something that is sort of equivalent to human intelligence.' Then there are people who will say, 'No, this is a sort of fundamental paradigm shift in what you can do with software. And so we should compare it to the iPhone or the web or the PC.' Like very big deal, very important, but fundamentally life kind of continues more or less in a more or less recognizable way. And then there's a kind of a small number of people who say, 'No, this is all, none of this is going to work. It's all nonsense.' And I don't think that's a particularly kind of useful, sensible way of looking at this.

But I suppose what I'm trying to kind of capture is that there's a lot of uncertainty as to just kind of tangibly how this technology will change. And then there's a lot of uncertainty in how we should think about what it is. And you can make these sort of very reductive statements. Like, you know, go back in history, you can sort of look at the web and you can say, 'Oh, well, that's just, you know, teenagers talking to each other.' And you can look at the web and say, 'Oh, it's very slow. You'll never be able to do video on this. No one will use that.' And you have to think, 'Well, yes, what's the right level of understanding this? How is it going to improve? What are the paths for it to improve?' But of course, you also have to remember that just because the web works and smartphones work doesn't mean that automatically anything else will work as well. So you've got this kind of great cloud of uncertainty as to how big will these things get, how good will they get, what do you do with them?

I mean, I think so. Sort of if we go into this in a little bit more detail, is there a kind of a good way to understand these technologies? So if you think about the history of automation, and automation is also an interesting word. You could kind of propose things go from science fiction to science to technology to automation to automatic, and then you just don't think about it anymore. You don't say, 'Oh, my car has an automatic gearbox now.' In fact, you don't talk of an elevator. Is that an automatic elevator? Meaning it doesn't have an attendant anymore. In the '50s it was an automatic elevator. Yeah. So if you think about the history of automation, we never made a mechanical horse. We made a steam engine. We never made a steel man walking around a factory. You make conveyor belts and machine tools and pick-and-place machines and so on. And what all of that automation does, from, you know, steam engines to sewing machines to machine tools, machine is an interesting word here as well. Yeah, yeah.

It is. So what they're all doing is you're saying, 'I'm going to take one task and I'm going to abstract it and I'm going to explain it to a computer.' And generally what you're doing is you're taking a task that's very hard for people to do, but easy for people to explain. So it's very hard for you to draw the same hole exactly the same way a thousand times. It's very easy for you to explain that you want the hole. It's very hard to calculate a million people's taxes or a million mortgages in your head, let alone do it in a day, but it's very easy to explain how you calculate a mortgage. You can write down a series of logical steps. And so what all of that automation does, from like the industrial revolution right the way through to computerization in the '80s and '90s, is that you are writing down logical steps and then giving them to a machine, creating a machine that follows those logical steps. Whether that's a steam engine or an ATM in a bank. Right? Go and check the bank balance, go and check the account data, go and produce this many bank bills, etc.

Now, however, there is a problem here because there's also a class of a problem that is very easy for people to do, but very hard for people to explain. And there's someone whose name I forget who kind of posed this as a paradox, that the things that are easy for people to do are the hardest for us to tell computers how to do. So it's very hard to tell a computer how to walk. It's also very hard to tell a computer how to recognize images. And so in the 1960s, people wanted to do translation because practically they had a huge number of intercepted Russian signals and not enough people spoke Russian. And so they thought, 'Well, this is easy. This is a database problem. We'll get a German dictionary and an English, Russian dictionary and an English dictionary, and we'll do a database match, and we'll just do all the translation on a computer.' And of course, language doesn't work like that. And so it turned out that it didn't work.

And the same thing with image recognition. So you would think, 'Well, how do I recognize a tank in a satellite photograph?' If this is the thing you're trying to do, or, you know, in practical terms, 'How do I recognize a cat?' And you think, 'Well, it's just that you've got a grid of pixels. So what do you do?' Well, you look for edges. And then you make a texture detector, and you try and detect fur. And then you make something that can look for pointed ears. And then you make something that can spot eyes. And then you try and look for two eyes close together. And you build up all these logical steps. And I think, is it Aristotle and Plato, or Socrates and Plato, that said, you know, 'Define a man.' And it's a featherless biped, and then somebody holds up his plucked chicken and says, 'Here is your man.' Yeah, yeah. And so the problem is you think it's easy to write down these logical steps, and, you know, 10, 15 years later you've written 745 logical steps, and it still doesn't work. Yeah, yeah.

And so then what machine learning does is say, 'Instead of making this a logic problem, let's make it a statistics problem.' Yeah, sure. And so we'll get the machine... Yes. And so the machines are good at statistics. So we'll give the machine a million pictures labeled cat and a million pictures labeled not cat, and let it work it out. And it'll make a model, and then we'll give it a picture and say, 'Which sample set does it match? Set A or set B?' This is machine learning. This was not working 10 years ago. It turns out to generalize fantastically well. And so to begin with, you'd go to a big company, and you could say, 'Holy, look at this. So can I go on your podcast?' I don't know. You'll say, 'Amazing.' Mm-hmm. It can recognize cats. And they would say, 'We're a bank. We don't have pictures of cats.' But it turns out, I like it, it's great, but we don't have pictures of cats.

And it turns out that actually the right way to understand this is pattern recognition. Once you understand this is pattern recognition, then obviously a bank can think of things you can do with pattern recognition, like fraud detection, for example, sentiment analysis, translation. There's all sorts of things you can do with pattern recognition. And we've sort of spent the last 10 years thinking, 'Well, what could we turn into pattern recognition?' And all sorts of problems that maybe we didn't realize were pattern recognition, or that you could turn into pattern recognition, and then you can automate them with machine learning. So that's what the last... That was the last wave of AI. Now, all of this is sort of a long way of saying, so what people actually want to do is not make something that could recognize cats. What they actually wanted to do is to make something that is intelligent in the sense that we say people are intelligent, or indeed that a dog or horse or a monkey is intelligent. And in a way that a calculator or washing machine is not intelligent. So a calculator can do superhuman maths, but we don't think of it as intelligent. A washing machine can wash clothes better than we can, but we don't think of this as intelligence. It's just a machine. Again, this word machine. What we want is... So then you have this sort of philosophical placeholder that we call general intelligence, which is what people or dogs or chickens or all sorts of creatures have on some kind of vague sort of multi-dimensional scale. And this is really kind of a philosophical placeholder rather than an actual technical concept. It's just, well, clearly there's something that animals have that an ant doesn't have, and that a washing machine doesn't have. How do we make that? And again, back in the '80s, people tried to make this by making lots of logical steps. They think, 'Well, we're just going to teach it common sense.' Yeah, we're going to write all these things down. We'll just write down how we understand the world. And of course, you can't do that. And so the concept of an LLM is to say, 'Well, if you can recognize cats by giving it a million pictures of cats, can you teach it how to reason by giving it trillions of examples of reasoning?' Which worked, as Geoffrey Hinton displayed.

Well, so maybe. And so that gets you to this question of what is it that these things are doing? They do something that looks like reasoning. It is not. It is not. Well, it's a mirror. Well, so this is, if we knew that, then we would be in a different place. The problem is we don't know that. So then you get to some sort of different opinions here. So kind of a good example is, and in fact, you can ask an LLM a question. And if you ask it a question about something that's very specific, requires a very specific, where there is a very specific, very detailed answer. And so I remember at the beginning of last year I asked ChatGPT, 'Write a biography of Benedict Evans.' Because, like, there's a lot of stuff about me on the internet, and I know all the answers because I'm an expert in myself. And so it says, 'Benedict Evans is, you know, a technology analyst, and he's widely, you know, mentioned, and, you know, he's very clever,' all of which is true. And it says, you know, and he has a degree in English from Oxford. Reload. He has a degree in history from Cambridge. Reload. He has a degree in economics from the LSE. Reload. He has a degree in philosophy from Oxford. Reload. And the same when you go through my career, it'll say first job was at Goldman Sachs. First job was at McKinsey. First job was in investment banking. First job was with Atlas Ventures, and so on.

And the thing is, those are all the right kind of degree. They're all the right kind of job. And what it's really doing is saying, what would a biography of someone like Benedict Evans probably look like? Hallucinating, yeah, yeah. Well, hallucinations of the machine. If somebody had asked me, 'Can you give me a thousand-word biography of yourself?' this would be really useful because I don't have one. And I could take that and fix it. Mm-hmm. If you're working at an advertising agency, and you're saying, 'Give me 50 ideas for a paragraph about a new brand of shaving soap,' there isn't really a wrong answer to that. So the question of whether the answer is wrong kind of depends on the question. There are questions that have very specific, precise, correct answers. And there are questions that only have better and worse answers. They might have terrible answers, but there isn't one correct answer.

So I was going to just sort of make a kind of final point. So I'll wrap up. But you can look at this thing and say, 'Right, clearly what this is doing is kind of making a pattern.' And the pattern may or may not be doing a database lookup. It's making a pattern that may or may not be what you want, depending on why you asked the question. Mm-hmm. However, so then there is a debate. Maybe as these models get bigger, they will get to 99.999% accuracy, and it won't matter that they don't know the difference between true and false. They'll just always be right anyway. It may be that as the models get bigger, some kind of what people call structural understanding will emerge, and they will actually know the difference between correct and not correct. It may be that they never get there, and that the models just stall out. It may also be, and this is kind of a question for, you know, philosophy students rather than engineers, it may be that people don't have that understanding either. We just think we do. And that our models, so to speak, are really good, and that explains cognitive bias. It explains the fact that we're bad at statistics. It explains optical illusions. And what all of those kinds of mental flaws in our own brains are telling us is we're descended from monkeys. We're optimized for picking up fruit and climbing through trees. And we have a model that's designed to do that. And every now and then you can see that our model is not actually as perfect as we think it is. You can find holes in it. And maybe these systems are just kind of doing the same thing, but coming at it from a different direction in a different way. Meanwhile, does it matter? Like, these things are still useful. And so then there's this whole other conversation, 'Well, fine. That's a great conversation for two drunk philosophy students at 2:00 in the morning in a bar. But meanwhile, what do we do with these things?'

Well, it's interesting. I mean, you could make these kinds of comments about clothing. You know, you cannot survive north of the Arctic Circle without technology enhancement in the form of certain clothes, or in the form of fire. So, you know, we have always been reshaping the world around us and creating tools around ourselves. And, you know, going back to my point about elevators, there's a fascinating book, I think, by a German academic called 'A Cultural History of Elevators,' in which, among other things, it points out that there was a sort of debate as to whether an elevator is a room or a hallway. So when you go in, do you greet people? So I think in Germany, when you get into an elevator, you greet people. And in... Yeah. You would be pretty rude if you didn't. Yes. And in Britain, that would then be impolite, it would be rude, because you're... Is it... Are you in a hallway or a room? And an elevator is like... Is it... Maybe this is a vertical streetcar. It's like a private tram that goes up and down. So all these... And now, of course, we don't think about any of this. It's just an elevator.

And so things sort of disappear into the background, and we don't see them or think about them. There's a similar sort of narrative about the cost of light that I'm sure you're familiar with. And if you think about reading fairy stories, they have these phrases, certainly in the English versions as well, they are like, you'll find a path, magical palace in the forest, and it will say it was ablaze with light. And it doesn't really mean anything to us in an age of cheap electricity, but in an age where a candle is like a week, wait, a week of wages, a building ablaze with light is something amazing. So, I mean, I think the same thing applies to computers now. Like, you know, my first experience with a computer was what was then called a microcomputer, made by a company called Sinclair in the early '80s, that my father brought home one Saturday. And I had a PC when I was a student, and there was a sort of an interesting shift, and maybe kind of a generational shift, in that I arrived at university in 1995. And when I arrived at university, I think I was the only person in my year in my college who had a PC and knew how to use it and wasn't studying computer science. Oh. And then in the third year, everyone in the first year had a PC. And the impetus for that was the web. But before the web, there was actually not much reason for most people to have a PC unless you either wanted a PC or you needed to run like Excel. Or maybe you were playing games, which most people weren't doing the same kind of games. But for most people, you kind of look at it and you think, what do I do with this? This is very clever, but this isn't useful for me. And the web was a sort of a huge shift in the addressable market for PCs. Smartphones kind of had a sort of similar effect 15 years later, 10 years later. And then they disappear. I mean, my father gave my son an old radio. And my son said it's an automatic speaker. Because you don't have to tell it what to play. You just press the button and it plays. Okay. Yeah, he mistakes them. Yeah.

So there's several different questions embedded within that because if I'm using some kind of automated assistant, AI assistant on my device, that could take a couple of different forms. So it might be that the AI itself is abstracted away entirely and I don't see it. And what happens is a new email arrives and the notification on the home screen has a summary of what the AI thinks is in the email rather than the first two lines of the email. And very quickly, of course, that's not AI, that's just software. And so that's, you know, but that, you know, so then I can say, right, what was the restaurant that my mother mentioned last week? And it will go and look in my email and in my text messages and try and find that. And before AI, the most that it could have done, before LLMs, the most it could have done is do a text search. It realizes this is probably a text search and it would do a text search for restaurant. Now, it can have a much more generalized understanding of what this is, a much looser, more fuzzy understanding, and it's much more likely to be able to find the answer if you don't actually say restaurant. So that's kind of one answer to this.

And so another way to think about this is, and it kind of goes back to my point about, you know, the answer might or might not be wrong depending on why you asked it. So, you know, if somebody else wanted a biography of me and they did what I described earlier, it would be full of mistakes. It would be useless. If I did it, it's very useful because I can fix those and so forth. Yeah. And so one could imagine exactly the same question and producing exactly the same answer being useful or useless depending on who's asking and why. Exactly how it's framed. Yeah. And it... well, not even that. It's the same question. It's what do you need it for and why? Now, a much more tangible kind of practical product question is you need to frame this in a way that the user understands. Either you need to look for use cases where either the user understands the limitations of the product or you can wrap it in enough tooling, enough pre-processing, post-processing that you can make sure nothing terrible happens. Or you're using it in a situation where it doesn't really matter if it's wrong. So either the errors need to be obvious, or they don't matter, or the user understands exactly what this is or how it works. Where it becomes a challenge is if, for example, you try and just use this to replace Google, because then you will say, like, what are the symptoms of appendicitis? Or, you know, is it safe to give chocolate to my dog? And it may or may not give you the right answer. And it will look like it's giving you the right answer.

I mean, this is something you mentioned with ELIZA. Part of the challenge of an LLM is that the creation of language is perfect. It creates perfect prose. It's all grammatically correct and lined up. And that sort of hides the fact that the model underneath it may not be perfect. So it will give... Now, this is why people talk about, like, the overconfident undergraduate, you know, all the... Like, because it... And that's sort of, again, that's anthropomorphization. You know, what's actually happening here is it's just generating patterns. It has no idea of concept of truth. It's very good at generating, but the language, the bit that makes that the language, that bit's perfect. The bit that says which university did Benedict go to, well, what it's actually doing is saying what university is plausible. Yes, it's plausible, and that's the point. Plausible might be what you want. Yeah, yeah. I mean, imagine if you were writing a novel and you were saying, give me a biography of a character like this. Well, then you wouldn't care. Yeah, which is plausible. What would this kind of person probably be?

So, I mean, when I was living in Silicon Valley, I went to see the Computer History Museum. And the interesting thing about this is seeing the sort of continuum from cash machines and adding machines in the late 19th century and typewriters, which are basically the same thing, through punch cards, and then punch cards become magnetic, they have been magnetic cards, and then they become tape, and then the tape becomes a hard disk. And then the software and then the processing goes from being metal rods to being impulses stored in transistors or stored in capacitors and valves. And so the point is there isn't like some binary moment when computing happens. It's just a kind of a continuum all the...

Way back to like, it's it well it it is, but it goes kind of goes back to like, you know, it's like it goes back right. It's like, you know, reading. It's like if you read a religionist and he says he doesn't say they have a different god. He says they worship this same, they call something and believe he's also the god of the harvest. And so there are all these, you know, that everything is kind of fuzzy. There isn't some moment when computing just kind of happened in 1968.

I would say so.

[laughter] Well, but you are but another level of abstraction up. You know, you're giving the computer instructions to do things. You've made a computer program. Like for practical purposes, that's what you've done. You've made a spreadsheet and then you change this number and other numbers change somewhere else. This is software. Um, it's a very abstract kind of software. If I make a document in Word, is this software development? Well, it's not there's not like a kind of a very very clear distinction that says well that's software development and that isn't software development. It's like when you're make creating something that does things, you're kind of making software. You're kind of making a program.

So, um, and and one of the ways that I kind of I I kind of keep thinking about how to understand what you can do with an LLM is to imagine seeing the first computer spreadsheets back in the early late '70s early '80s. So, people had the guy who built VisiCalc, guy called Dan Bricklin, has these stories of showing this to accountants. And they would be flabbergasted because they would think that's like a two weeks of my work and I could do this in five minutes. Cuz I can just change these numbers and it will just generate the next 10 years of numbers and I don't need to calculate them one by one. It will just do all of them.

There's actually a video we mentioned elevators. There's an ad from the early '90s for Microsoft showing somebody in an elevator going to a meeting. Opening a laptop, opening an early laptop and changing a spreadsheet. And you look at it and you're like, wait, when does he get to the bit that's interesting? And no, like I can change this number. And that number changes, too. And everyone in the elevator was like, "Wow." And this is the... That's the ad.

So, let me let's give it a go on a much more practical way way of thinking about this. And kind of getting to what we're going to be talking about when you compare this to a GUI. So, before GUI, if you wanted to do something with a computer, you had to learn the keyboard commands. You may remember we had these kind of cardboard overlays. You had these cardboard overlays that you slotted on top of your keyboard so that you knew what to press to save it or to print it or to make something bold. And with a GUI, you don't need to learn all of that. You can just click on something. Now, that's that's new stuff that you have to learn. But, this is a huge change in who can use computers, how many things can be done in software, how much of software can be done.

So, how much of software can do. So, you have this huge change in how many tasks can be automated with software. However, someone needs to have made the thing with the GUI. So, if you want to do your taxes in Poland, you can do it with a GUI, but someone needs to have made software that knows the Polish tax code. Um, now, in principle, I could go to an LLM and say, "Hey, I need to do my taxes in Poland. How would I do that?" And it could say, "Hmm, you'll go and read the Polish tax office's website and work out what documents you need." And say, "Right, take a photograph of your mortgage statement, take a photograph of your bank statement. Here are the expenses you need to document."

Chunk chunk chunk, 30 seconds later, okay, here is a PDF. Here is what you need to fill into the... Here is the PDF you need. Here is what you need to fill in. And that's not science fiction anymore. It's engineering. Now, it might not actually work quite that well. And but it's not no longer unfeasible that it could do that.

You know, a much more kind of a kind of an intermediate case might be, you know, imagine you're work running working for a company and you... Imagine you I I you built some software and you want your users to give them all your bills. So, give me your rent, give me your utility bills and then I'll do something, I'll do a price comparison for you. Five years ago, I what I needed to do is I need to get a copy of every single bill in Germany. I need to know what all the different gas companies, all the different phone companies, all the different credit cards. And I need to build a software that does Deutsche Bank's five different credit cards. Yeah. And then the bank's credit card and the four different ways they do the credit card and the different ways that Sparkasse does it. And the way that your gas bill might look. And I build all of those one at a time in software. And then you take a picture of your bill and I'll work out which one it is. Now, that's an API call.

Now, I can just say, "Hey, ChatGPT, what bill is this?" [laughter] And then you might also need to check it and say, "Is this what we think it is?" And the user might never know that this was AI. They certainly wouldn't know it was an an auto an AI assistant.

And so, what I'm kind of getting at with all of this is like the levels of abstraction of automation just keep shifting. And the stuff that the computer can do keeps changing. And the stuff you can automate keeps changing. And if you grew up with it, you think, "Well, of course that's what computers do. Of course they They always did that." And if you're, you know, if you're in your 50s, you think this is... There's an old um... There's a joke by Douglas Adams who wrote, you know, The Hitchhiker's Guide to the Galaxy. He said something like, "Anything that was created before you were 10 is just the natural order of the world. Anything created between the age of 10 and 30 is interesting and exciting and you can have a job in it. Anything created after you're 40 is a threat to civilization."

Um, there's a scandal in the UK around the post office.