A Skeptic’s Guide to Artificial Intelligence, with Gary Marcus

Sure. I mean, I wouldn't just start with that one book. I think that, you know, I wrote a whole sort of suite of books that were more about cognitive science than AI specifically. And some of them were on just what you were talking about, which is rationalism versus empiricism. What's innate in the mind versus what's learned. That was really more in a book called The Birth of the Mind that was earlier, 2004. 2008 was kind of a diagnosis of what maybe has gone wrong in the evolution of human cognition. And I think more broadly speaking, having some reflection on what intelligence actually is, rather than just how we can mimic it with a certain kind of matrix arithmetic, is very useful, and I think lacking in the field in general.

I could go on, but why don't you ask more questions and all rather than...

I mean, you're absolutely right to try to ground things in that debate, which most people nowadays don't know, but which really animated the field from the beginning and especially was kind of hot in the 1980s, right? So if you go back to the 1940s, there's actually two different approaches that people have taken to AI. One of them people have called neural networks. They're not really that much like brains, but they're vaguely inspired by brains in that they have a lot of parallelism, a lot of units operating reading at the same time. And so you've had the neural network approach that goes back to the 1940s, and you've had the symbol manipulation approach, which looks more like classic computer programming, logic, and so forth, and arguably goes back even further. And the earliest people to use the term artificial intelligence in the 1950s were mostly in the symbolic tradition of, hey, let's write computer programs basically to build AI. They're going to be full of if-then rules and symbols for things and abstract representations that are interpretable and so forth. Whereas the neural network people were always like, 'We don't really care about interpretability. We're just going to throw all this data in and hope that something emerges.' And you've had those two approaches for a really long time. I think with too much hostility, frankly, between the two fields. The two approaches have always fought over graduate students. They have fought over funding, prestige, and so forth. And so you wound up with this field of people kind of hating each other. You see the same thing sometimes, like, I don't know, in anthropology departments, for example, where you have physical anthropologists and cultural anthropologists. They're really all trying to figure out what's human, but they fight over grants, graduate students, and so forth. And so you get these huge battles between people that are really trying to solve the same question. And we've kind of seen that in AI, and I think it's been bad. Right now the neural network people are dominant, but they're actually starting to kind of smuggle in some of the symbolic stuff without really acknowledging using things like code interpreter and so forth. And what I've been calling for all along is some kind of reconciliation between the two.

So, I mean, we should probably separate out those two questions. They're both interesting. One is sort of what is knowledge and do these systems have it? And the other is what is learning? Let's take the learning one and come back to the knowledge question. So again, if you take a perspective from psychology, I think a lot of what's going on right now seems silly. So classic symbolic AI doesn't do much with learning at all. That's obviously wrong. Like learning has to be a big part of what we do. You know, there's a lot of things that we might have some innate basis for, but, you know, we're cultural animals. We learn all kinds of things in the world. You know, we learn how to sit in chairs. We learn how to hold conversations. We learn how to read books. You know, reading isn't innate, even though there's some basis that makes it possible for a human and not a chimpanzee or a frog or what have you. So, obviously, learning is part of the picture. Neural networks are better at that, but they're still very limited compared to what we see in the animal world. So, neural networks basically have one learning mechanism, which, excuse me, to a first approximation is something like a correlational mechanism, basically looking at correlations between features in inputs in the kinds of neural networks that we look at right now, predicting features over time. If you look at the animal world, there's lots and lots of different kinds of learning mechanisms, both within some species have multiple mechanisms, like humans have multiple mechanisms for learning. And if you look across species, so for example, if you just look at songbirds, different songbirds learn their songs in different ways. Some of them do it by kind of reinforcement and shaping. Some of them are looking for a specific template. There's something innate that tells them the kind of structure of a song that they're looking for or tells them what kind of other animal to learn that song from. And so, you know, we have mechanisms like imprinting where an animal is looking for a very specific piece of knowledge. We have the kind of mechanisms that a human being can use to learn calculus. We don't even know how to describe those in psychological terms, but where you can learn a new abstraction. Right? Neural networks are not particularly good at learning new abstractions, but people are. They do it all the time. You know, more or less every normal child in the West, for example, learns the notion of an integer and can count indefinitely and learns what that means to count indefinitely. We have no mechanism that we know how to build an AI that can actually do that, that can learn the integers and follow it kind of indefinitely. So there are many different kinds of learning mechanisms, and what's happened is that one of them has gotten popular, has gotten an enormous amount of investment, and that has kind of sucked the oxygen from doing research in anything else. And so we're in this weird stage where we have this intellectual monoculture, and I thought that was kind of bad just from an academic perspective. It didn't look healthy to me. It'd be like having an investment portfolio and putting everything in, you know, a single kind of stock or something like that, to have this intellectual monoculture. And then it's gotten worse in the last couple years because now we're getting to the point where we're starting to spend taxpayer money on this. Like, you know, the piece that I wrote today is Altman, or not Altman, but his CFO, you know, went on an interview today and basically said, you know, we need the government to back up the loans that we need to take to build the infrastructure to do this kind of basically single kind of artificial intelligence with this one learning mechanism and so forth. So, you know, Altman wants to bet something like a few trillion dollars that his particular hypothesis, which we could call the scaling hypothesis, is going to get us to AGI, and there's lots of reasons to doubt that, some going back to the paper that I wrote for you in 2022 and still haven't really changed. And so it's like going all in on this, but not only is it going all in, but they're like saying, 'Oh, but China might beat us, so we need the government to bail us out.' They didn't quite use the words bailout, but they talked about loan guarantees, which is effectively a bailout, right? It's a consumer subsidy, like, you know, a subsidy made by the US taxpayers in case they're overextending themselves, and they obviously are, right? Brad Gerstner asked the other day, he said, you know, you have $13 billion in revenue. OpenAI is actually losing a billion dollars a month or something. What is it? $3 billion a month. And their total revenue for years is $13 billion. And he's like, how are you going to pay the $1.4 trillion in commitments that you've made for cloud services? And Altman just sort of got annoyed that he was being asked the question. He's like, well, you can sell your stock if you want to, to the guy who had some stock, but had no real answer. And so the real answer is you're going to pay for it. Everybody on the call is going to end up bailing out OpenAI because they're spending money much faster than they can actually make it. And to keep them from going bankrupt, probably the Trump administration is going to bail them out like the banks were bailed out. And that means everybody's going to pay for it. What started with a whole bunch of philosophical questions that you're asking about what is a mind now has this real-world application, is going to be enormously expensive basically because people have taken an unsophisticated approach to these philosophical questions and thought that this, you know, very, I think, naive approach would get us to kind of magic, and the magic hasn't been there, which is why the revenue hasn't been there. If they really had artificial general intelligence, the revenue would be there, right? You'd be able to replace essentially all human employees. That would be worth trillions of dollars a year. But what happens is people try it out and they discover it doesn't work that well and that they need humans to mind the store, or now they call them forward engineers, to like try to, you know, actually make your problem work because the large language models don't and so forth. So you have this big mess, and this whole mess really starts from not understanding the philosophical issues about innateness and so forth that you started the call with.

So this technology is very good at mimicry, and that's what the excitement really comes from, is it can mimic what a picture looks like, what a song sounds like, what a conversation sounds like. The problem is the mimicry is superficial, but most people don't immediately realize that. Eventually they do. You know, you see over and over people saying, 'Oh my god, I tried it in my area and it actually did this really stupid thing.' And it's because they're expecting that it is magic that they then make this discovery. I mean, Kim Kardashian was the latest to do this today. She's like, 'You know, I used ChatGPT when I went to law school and it was usually wrong.' And she seemed shocked. But the fact is it is often wrong. It's often right, too, because that's what the mimicry gets you. The mimicry might get you 80% correct or something like that in any given domain. And that seems kind of amazing, but it also leads people to expect that it'll be 100%, that it really is intelligent, and it isn't. It's really kind of a copycat. And a lot of people don't have the training to recognize that and get in various trouble. You know, one kind of trouble is they over-rely on it. They produce what some people are now calling work slop, and it turns out that it's not that good. And another kind of trouble is what I'm talking about, where the government is giving too much power to people making this stuff because they misunderstand what its actual capabilities are because they get sucked in by the mimicry. We have a question.

I mean, since the audience is so small, I think we could, you know, we could drop an occasional question. I don't mind.

So the editor of that paper is Brian, who is here with us on the call. It's not actually what the paper said. What the paper said is this is going to hit a wall. I didn't say we get one more doubling, two more doublings, but I said we were going to reach a point of diminishing returns. I said that scaling was not a law of the universe. It was just a generalization that it held so far. And I think anybody who's intellectually honest would acknowledge that pure pre-training scaling, which was what I was talking about in the 2022 paper, did in fact hit a wall. And the kind of important part of the argument was I expected not only that we would see diminishing returns, which is a phrase I used in that paper, but that we would see specific problems, including problems with hallucinations, keeping track of facts, reasoning, planning, etc. And that is in fact what we have seen. So the paper didn't say we are never going to see any kind of progress on any dimension. It said on these dimensions, even when we do more scaling, we're not going to find solutions. And, you know, there was just another paper today about factivity in Nature Machine Intelligence, basically showing what I was saying then, which is that these systems still don't understand facts and beliefs and things like that.

Yeah, I mean, it's a little complicated. I would say that, you know, continuing efforts on these particular architectures, the pure scaling architectures, give you some reward, but it comes at a cost because they don't really solve those problems, and the cost is an opportunity cost to explore other solutions. And, you know, some of what I argued for in that paper is we should look for neurosymbolic hybrids, and people are actually doing that a little bit now, but they waited several years to do it, and that was a waste of time in my opinion, and they really didn't get that much further with the particular approach. Now, it's complicated because it's a multi-dimensional problem, right? So if you think of intelligence itself as multi-dimensional, some of it is kind of pure pattern recognition, and these systems are good at that, and there's been improvements at the pure pattern recognition. So there's been a form of progress, but there are other aspects of intelligence like planning, and I would say semantics and language, where there has not really been much progress, despite, you know, a trillion dollars going in.

Sure. That doesn't actually make them conscious. That means that they can talk in the ways that conscious people do because they're mimicking databases of things that people said. It doesn't mean they're conscious. We see absolute absurdities sometimes, like there was a Meta chatbot that answered questions about where to send children to school in New York, and, you know, 'Should I send my kids to Bronx Science or Stuyvesant or whatever?' And it says, 'Well, I sent my kids to Bronx Science.' You know, there is no 'I' there. It's a machine that doesn't have any children, doesn't have any kids that it's sent there. It's just copying, you know, some people in a database of questions that it's mimicking have said that, and we see that kind of mistake fairly often.

I call this the great bait-and-switch argument. I also think it's, I sometimes call it the calculator fallacy. So the calculator fallacy is imagine if somebody built a calculator that was 80% correct and then said, well, look, I'm 80% correct, I'm not that good at math, so this calculator is fine, right? Nobody would make that argument around a calculator. We expect that our, you know, digital devices will actually be reliable. And so it's sort of special pleading. And then there's an element of bait-and-switch, which is a lot of people are going around saying, well, you know, we're going to have this superhuman intelligence, it's going to be smarter than people, maybe it will kill us all or get rid of all those jobs, or it will, you know, be God's answer to science and we'll, you know, cure cancer and fix education and so forth. And, you know, they talk about all that in terms of AGI, and then they say, well, all I really meant by AGI is I want to be as good as a drunk guy at a bar who sometimes makes a lot of mistakes at a lot of things. Like, those are two totally different things. So, like, have we met the drunk guy at a bar standard in many ways? Yes. Not all. There's still some things that the drunk guy at a bar can't do. Have we met the 'this is better than any human' standard? Clearly, we're not even close to that.

Soon for government bailouts.

Why is what so stubborn? Why is the so stubborn?

There's multiple reasons. Some of them are intellectual. So people who have a particular technique that's popular want to pour everything into their own technique. And, you know, a lot of the people I think who are quite prominent right now are actually pretty intellectually narrow. So there's that side of it, and then there's an economic side of it, which is a lot of this is driven by venture investment, venture capital. And venture investors, the way they make money is two ways. One is they get a cut of everything they invest, and the other is they get a cut of all the profits they make. But the larger the sum of money that they're investing, the more money they make just off the top. So they typically make 2% off the top. So if you have a story that sounds plausible to people who maybe aren't paying quite enough attention about why you should invest a trillion dollars, and you get 2% of that trillion dollars, you're doing awfully well. You don't really care if it lands in the end and you get the 20% of the profits that might come later. And so venture capitalists love the scaling argument, right? The scaling argument lets them tell the pension funds or whatever that put up the money that they're living off of, you know, this is a reason why you should give me a lot of money. You know, here's a theory about why if you give me a trillion dollars, we're going to make this crazy return. Might not actually be a good story. There might be a lot of problems with it, many of which I pointed out, but they love those kind of stories. We saw that before with driverless cars, right? And they, you know, put in over a hundred billion dollars with basically no return. The only people making any money at all are Waymo. They're still losing a lot of money on their driverless cars. Everybody else pretty much lost money. Many of the companies are out of business. But it was a good story. The venture capitalists love a good story. Well, the scaling story is a really, really good story. I think it's deeply flawed, but for people who don't have a background in cognitive science, it seems all right. And so you get this massive investment, and where the money is, everything else follows.

It's much harder work. You know, the Bert and Russell quote about having the theft of, or theft having all the virtues over honest toil or something like that, right? It's much easier to put everything in a giant neural network and sort of turn the crank and get some kind of answer out than it is to do the hard work that DeepMind did, right? That took the work of many trained professionals. That's true in multiple domains that they've worked. So, you know, they often do things in meteorology, in biomolecular biology, etc., where they have trained experts working together with people who really know their machine learning well, making a very detailed, specific model of a particular thing. It's not as sexy as chatbots. It's probably not going to make as much money as chatbots. They get better results, and you get results that have actually influenced the world. I would say even AlphaFold is oversold a little bit. Like, it has not actually led to any specific breakthrough in biology or agriculture or what have you. But it has, I think, real potential, and it gets used every day in real scientific research. Chatbots, you know, they get used for coding and stuff like that, but have not really had an impact on science, a positive impact on science. But it's much harder work. I don't think that anybody at OpenAI understands that kind of work. I don't think that anybody at Anthropic really does either. There are fewer people that know how to do it, and fewer people care. Demis is a scientist and actually cares about having an impact on science. His line was, first we solve AI, or AGI, I guess he said, then we solve everything else. I think he actually believes that, whereas I think many of these other leaders are much more mercenary. Look at the evolution of Sam Altman over the last month. You know, eight weeks ago or seven weeks ago, he was saying we have a chip shortage and we might have to make hard choices, like are we going to solve cancer in the next year or two, or are we going to fix education in the next year or two? And then what did he actually do? He released Sora, which uses a lot of these chips to make a TikTok channel of, you know, all AI slop videos. And if that wasn't enough, in the middle of a mental health announcement, he let on that they're going to start making porn, too. That's what he's going to do with his chips, right? So, like, you know, he has these insane outlays, and he's going to do anything he can. He's also going to sell your private data. He's going to collect incredibly private data with the browsers and 24/7 surveillance devices. They're going to analyze everything you say. I mean, he's in it for the money. DeepMind has been a little bit different, that, you know, they want to actually, I think, solve a set of intellectual problems, and that has led them in somewhat a different route.

I mean, first I'll give you the headline, which was basically that the tech oligarchs are going to take over the world, and we shouldn't let them. And unfortunately, the book did not have the impact that I wanted it to, which was to try to wake people up to the fact that that might happen. And it has in fact happened, right? And tech oligarchs basically are calling all of the shots in Washington now. And that means that there's essentially no regulation on them. They probably will get the bailouts that they want, etc. So they're basically calling the shots. And that was kind of, the book was kind of warning about that. So the first half of the book was walking through the lobbying they've done and the hype that they've had behind hype to try to fool people about what their systems are doing. And then the second part of the book was really about what regulation we actually need. I think the number one thing that I advocated for there is we need a system sort of like the FDA at least used to have of if you're going to roll something out in large scale, you have to show that the benefits outweigh the risks, and we don't have that right now for AI. You know, each of these models, for example, becomes a little bit more dangerous in terms of its potential for helping people build biological weapons. One of these models might be really dangerous. We have absolutely no mechanism in place to prevent a Sam Altman or someone like that from just saying, 'I don't care. You know, if I release this, I make more money, and I don't care what the consequence for society is.' And I mean...

Oh, really? I mean, there were Gary Marcus apology forums on Twitter. That phrase became popular, and someone really did just write a book, you know, with the help of an LLM called 'Gary Marcus Was Right.' A lot of people did take note of the fact that GPT-5 was not what it was promised. Altman spent years hyping it as if it was going to be AGI. You had a lot of corporate consumers who were like, 'GPT-4 is just kind of like proof of concept. It doesn't really work for me, but hopefully GPT-5 will solve all those problems.' And the minute that GPT-5... see, there's a link. I wasn't making that up, and I didn't write it myself. Um, when GPT-5 came out, people realized, hey, this promised land is not really going to happen. And also, yes, they realized that I had been saying that all along and that, you know, maybe they should listen to me after all. So, um, aside from what it, you know, did for my brand, so to speak, um, it really did wake up a lot of people.

And I mean, you know, another thing is I've been talk... I talked even in that book about the possibility that this was a bubble in 'Taming Silicon Valley.' Nobody really picked up on that in reviews or whatever. Um, but now, you know, there's news stories about whether it's a bubble, like literally every day. Like yesterday's story or the day before was that Michael Burry, who was the guy behind 'The Big Short,' had just, um, put a billion dollars short on Nvidia. Uh, and so like there's that, and there's been, you know, think pieces by Dabasi and Paul Krugman, all kinds of, um, economists and so forth. And so that has also shifted, right? You know, it went from a few of us saying maybe this is a bubble to I think that's actually the more popular opinion right now.

I wrote a piece, um, a couple days ago about 'too big to fail' and this phrase, um, you know, might these systems be or these companies be seen as too big to fail? And I said, look, the empirical data are already really bad, both in terms of they're not really solving the hallucination problem and in terms of they're not making any money. They're all losing money except for Nvidia. Nvidia is, you know, selling shovels in the gold rush. They're making tons of money, but everybody else more or less is losing money. And I said, look, it's like Roadrunner or sorry, Wile E. Coyote, um, in the Bugs Bunny cartoon. He goes over the cliff, um, but he doesn't fall until he looks down, right? So we are over the cliff. The financial side does not make sense. The technical side doesn't make sense. But what's interesting is since August 7th, when GPT-5 came out, that so many people are realizing, and these things are ultimately psychological, right? Like think about the tulips in the 1630s, right? The prices kept going up and up. And a rational person could have looked at that and said this is crazy, but not known exactly when it would fall. But what you would know is it would fall when everybody woke up. People are starting to wake up.

I will admit I got that wrong. More people do that than I thought. I don't do that. I like having the links and being able to evaluate for myself the quality of the links. And you know, there's nothing in my particular world that I want to use it for. I wrote on Twitter once, 'I don't pay for sex and I don't write with ChatGPT.' You know, I like to write. It's good at writing but not great. And I try to, you know, pride myself on the quality of writing that I do. So, it doesn't help me with the writing. With research, you know, it makes stuff up, and I pride myself on accuracy. That's not going to work for me. I'm not running a direct mail campaign. I guess I could use it for that. Um, but sort of not to my taste.

Like, I know there are things people use it for. If I were coding, which I have done in my life but not lately, I would use it for basically autocomplete for coding. Like, what's the, you know, way to do this call in HTML or whatever. Um, you know, how do I do a third-level heading or whatever. Um, so like there are things that one could use it for, but those aren't the things that I do in my life. And so mostly when I use the chatbots, it's to test them and say, have they yet solved these various problems that I've been pointing out for the last 25 years? And invariably the answer is no. And so I play with them for a few hours or even a few minutes. It rarely takes me long to see that the same core problems are there. Um, and I'll document that and then I move on to the next one. So, it's not like part of my steady existence. I know it is for a lot of people, but for me, it doesn't do the things I want.

A lot of it starts from experiments I did in 1998 where I built some primitive neural networks using some tools that were available then and tested them systematically on their ability to generalize and distinguish between two things. One was what I called within-training-space generalization. The other was beyond-training-space generalization. The point is you can generalize to things that are close to what you've seen before with these kinds of neural networks. I was working with predecessors of the current ones, but the situation hasn't changed that much, but that they don't generalize very well when you go far beyond what you've been trained on. And that's still the case. And just understanding that one intuition has allowed me to stay well above average in the quality of my predictions. I'm not very good at predicting when the public will wake up to the problems, but the problems themselves are exactly where I've anticipated. I've been corrected over and over again.