There's a lot I would like to ask, but as an economist, I would just assume away the problems and try to figure a solution. But I would like to invite somebody who I think is much better placed to be able to do that. Our chair of our computer science department, David Garth, Professor Garth, he's working on the human-computer interaction lab. His research sits right at the intersection of people and machines and he's been thinking quite deeply about how people interact with computer systems and how they constrain our behavior. So we thought it would be great if he could come and lead the next part of the conversation. So please welcome Professor David Garth.
It's wonderful to sit down with you. Before we begin, I think would you be able to confirm that you're a human by identifying the stop lights?
Very well done. So we'll have a quick conversation here and I want to let you know that we will have an opportunity for audience questions a little bit later on. We'll do that through Mentimeter. So there'll be a link and instructions on how to prepare your questions at that point. But I do want to start our conversation by thinking about these transformations in society. You mentioned at the beginning of the talk you talked about how powerful these things are and how good they are for science and for medicine. But when I talk to people sometimes in their day-to-day experience they find these tools to be underwhelming and they make mistakes or they're not as impressive as they think. How do we account for this disparity in people's experiences?
Okay, so the first thing to say is that the AI that you're interacting with now is the worst AI you will ever see. It's getting better rapidly all the time. So if you interacted with ChatGPT 3.5, it couldn't do simple arithmetic or it sometimes stumbled on simple arithmetic. It made quite a few confabulations. 5.4 is now much better. It can do simple arithmetic fairly reliably. It can tell you how many letter Rs there are in the word strawberry. And it can do some quite sophisticated reasoning. So early versions, for example, couldn't answer the question, which I'm not going to ask you to answer because I once did this on stage and the host couldn't answer the question, which was very embarrassing. So I'm not going to ask, I'm sure you will know the answer to this, but don't say anything. Early ones got the following wrong. Sally has three brothers. Each of her brothers has two sisters. How many sisters does Sally have? And the guy on stage in Las Vegas asked me for an example of what ChatGPT couldn't do. And I gave this and then put him in the headlights and he said six. The right answer is one. Because all three brothers have the same two sisters and one of them is Sally. So early chatbots couldn't do that. Now they can do that. Now they won't get that wrong. So they get better all the time. And if you're underwhelmed by the current chatbots, just wait till next year.
A good example that I heard recently is my car wash is only about half a mile from me. Should I walk there or drive? And some chatbots they get that right and some get that wrong. So also those chatbots when they do that the next question was okay so I walk there but I need the car and so I think the chatbot said you should bring it with you. Bring the car with you when I walk there. So do these kind of confabulations make it difficult to talk about these bigger existential risks?
Yeah, they make you think it's not a problem. But the other point about this intelligence is it's jagged in the sense that it's very good at some things and not so good at other things. And its abilities at different things isn't the same as a human's. It can be much better than you at some things and a bit worse than you at other things. But it's coming up all the time. So when you find it doing something dumb, that doesn't mean it can't think at all or that it's a really dumb thing. It just means it's dumb at that. But it can still be very smart at lots of other things.
The training in these language models is language training, right? This is fundamental to the way that we've defined these things. Trained on particularly text language and we know that that's only sort of a subset of our own thinking. We think and then we speak and you mentioned how language...
Noticed you're Donald Trump.
It's very easy to get Trump laughs in Canada.
It's an effective strategy. I agree.
You mentioned how language is an imperfect conduit for information. These things that are trained on language, does that limit their ability to do thinking? Are they restricted to thinking in language the way we might not be?
Okay. So, there's two issues here. There's a philosophical problem and a practical problem. The philosophical problem is like this. If you locked a child in a room with a radio and that was his only experience of the world, just the sound waves on the radio, could it understand the world? And most philosophers would have told you no, but that's what these large language models are doing essentially. And the answer is yes, but it's not a very efficient way to understand the world. In particular, it's not a very efficient way to understand space. So large language models can actually, once trained only on language, can actually deal with ambiguities like the trophy would not fit in the suitcase because it was too big where they understand that it is the trophy and the trophy would not fit in the suitcase because it was too small where they understand that it is the suitcase. So they're understanding stuff about space that big things won't fit inside small things purely from language. But that's difficult. It's much easier to understand that if you can do vision too and it's even easier if you can manipulate things. So multimodal chatbots are going to find it much easier to understand stuff. But the philosophical problem is is it possible to understand things like space just from language. I think the answer is yes. But it's very tedious and needs huge amounts of data. You're much better off if you can see things and interact with things.
And tedious and huge amounts of data is exactly what they're good at. They have huge amounts of data and they can spend all the time they want to think about it.
The language that we see, that is their sort of chain of thought, their reasoning, right? When we watch it make a decision. I should do this but I should not do that. That's also language. Is there thought underneath that or is that sort of a representation or a simulation of thinking? What does that reasoning mean that we're watching it do?
Okay, I think that is thinking. I think that little voice you have in your head when you're thinking, you know I forgot to do that and I've got to do this and oh god that voice, that is thinking. We think with words. And the point about words is they're a universal modeling kit. They're like Lego, but you can model anything that way. You can model the layout of this room. You can model what you suspect somebody else feels about you. You can model what you think about Republican congressmen. That uses rude words, but you can model anything with them, but that doesn't mean they're your only way of modeling things. So you also have images and something somewhat more abstract than images, thoughts about space that aren't in words. You actually have other ways of representing things. Like when I'm in my workshop and I'm thinking about one thing and I'm looking for a hammer, I sort of walk around the workshop going like this. And this is my representation that I'm looking for a hammer while I'm thinking about something else. So we have many other ways of representing things other than words. Words are kind of universal, but if you have those other things as well, you're going to be smarter.
And so the large language models that we're looking at today are trained exclusively on language.
Not all of them. No, things like Gemini are trained with images too.
Images and also movies and what about like you mentioned just briefly before being able to interact with the real world, reaching out, touching things?
Right, that would make them even better at understanding space and things like weight and force. The problem with that is the real world has a natural time scale and you can't give training data for doing this that goes thousands of times as fast if it's really doing it. You can play back training data, but if you have it interacting with the world, you get a natural time scale for the world, which means you can only get a certain amount of experience. So, you only live for two million seconds. Sorry, two billion.
Two billion seconds. Well, you're going to live for two billion and a little bit.
I'm already in the little bit.
That means you can't get more than two billion one-second experiences, even if you didn't sleep. Which is very little experience compared with a large chatbot. And if the chatbot can get these real world experiences, even time limited, they'll still be able to share them with each other in the same way.
That's the point of the sharing.
Yeah, you can now have 10,000 of them. Each of them is going slowly, but they can share what they learned. So, they can get through a lot more data. And for AI agents, that's going to be very important because an AI agent that's interacting with other AI agents on the web, there's an actual time scale to that. You can't just speed that up and run it through very fast because you need it to be actually living in that world and interacting. But if you can have lots of them that learn from each other, that share what they learn, they're still going to be able to learn a huge amount.
You mentioned about AI agents talking to each other, perhaps inventing new language. Have we seen an AI making up a new English word to represent an idea that it doesn't have a word for, or is it restricted to the words that it's been trained on?
So yeah, here's one of the fibs. They're not actually trained on words. What they do, the first thing they do is break words into little pieces. Which try to be roughly like morphs. So un at the beginning of a word has some meaning to it. That's a very good piece to have. N at the end of a word is a very common piece. They have lots of pieces like that. But we have seen AI agents creating new things. I mean, not new individual pieces because they're defined ahead of time. But we've seen them making up new combinations of pieces. Apparently to talk, we don't know what they're using it for, it's just beginning to happen.
Kind of frightening as well.
You talked about the idea of a maternal instinct, having an AI that's trained to be a mother, in order to sort of be well aligned with what our intuitions might be. And you mentioned that the difficulty of predicting what might happen if you had a well-trained AI and maybe a rogue AI that was not behaving like a mother, more like a disconnected uncle or something like that. What about a conflict between maternal AIs that represent different facets of society or different countries? Like is our mother AI going to go and fight against some mother AI from some other country?
That's a huge problem, right? You just have to go to a girls' soccer match to see.
And that assumes that all of the people who are developing AI are going to try to build those instincts in.
Right. So when people talk about alignment, people talk blandly about alignment with human values.
Well, our values don't align with each other. Some people think it's justified to drop 2,000-pound bombs on schools to kill a terrorist and other people don't. It's like asking draw a line that's parallel to these two lines. Well, you can't. So that is a huge worry. In particular, if we have AI mothers, what's to stop them getting into big fights? Now maybe the AI mothers will be sensible enough to realize on the whole you do better by cooperating than by fighting.
There's some other people we'd like to realize that. This is of course the challenge is the people who are developing these AIs today aren't building them to be mothers. And so how do we have that conversation of encouraging the people who are developing these to take these ideas into account?
Well, it's difficult. The best I can think of is if you look what happened with climate change, the big energy companies didn't want any regulations. Eventually the general public got to understand there was climate change despite the best efforts of Murdoch newspapers and they provided pressure on the politicians from the other side to do something about it. Climate change is simple by comparison because we know what the answer is. The answer is stop burning carbon. Here we don't even know what the answer is. But I think our best hope is for the general public to understand this is very dangerous stuff and needs to be regulated for all of those short-term threats that I listed at the beginning and for this long-term existential threat. We need to be putting a lot of work into how we're going to survive this and the general public needs to understand more about it so they understand what the danger is. And that's why I'm here.
And so what role then does the university, the academy have in helping to inform regulation, inform governments that might be interested in doing regulation? What can our role be?
Well, in the old days people would ask the experts and we were the experts. Now people say why do we need experts? So we need to keep informing people. In the states, for example, there's about half of the attorneys general of the different states are suing the National Academy of Sciences trying to get them to withdraw a report on climate change because they argue that it doesn't represent the other side. I mean, we're in a terrible situation. So, we've got this huge event that's about to happen, the super intelligence, and we're at a time when, well, maybe it's suddenly going to get better because Viktor Orban didn't get re-elected, but the political systems just aren't right for dealing with it. At the end of the Second World War, we were much better off. People really didn't want any more wars. They set up the UN and that was a very good time for nuclear weapons to arrive because at least the governments were fairly reasonable about trying to prevent the harms.
But in this case we have people who are seeing great benefit from these AI tools and also not a lot of pressure to regulate. So what does good AI regulation look like? What can that be?
Okay, so there's an analogy that the big companies use. They try and get you to see it this way. You have a car, right? And developing AI is like the accelerator and regulation is like the brakes. And they're trying to get you to see it that way because most human reasoning doesn't work logically. It works by analogies. Trump tries to get you to think immigrants are vermin. And if you think immigrants are vermin, then rational argument won't do any good. They're trying to get you to think that regulation is like brakes. It's going to spoil the fun. What you should think of is regulation is the steering wheel. The whole point about having rules is it's fine for people to make lots of money by inventing wonderful new things as long as those things are good for society as a whole. So, the point of the rules is you can make money but not by doing bad things. You have to do good things. You can make money by interpreting medical images much better. Great. But you can't make money by getting teenage kids hooked so they never leave their bedroom. You shouldn't be allowed to make money like that. That's the point of rules.
