Where is the cutting edge of AI research leading today, and how are some companies already putting it into action? Let's talk about it with Cohere Chief AI Officer Joelle Pineau, right after this.
Welcome to Big Technology Podcast, a show for coolheaded and nuanced conversation of the tech world and beyond. Today, we're going to look deep into the state of AI research, where the cutting edge is leading, whether there are limitations with the current methodologies, and how some companies are already putting this technology into action in a practical way. We're joined by the perfect guest. Joelle Pineau is here. She's the chief AI scientist at Cohere. Joelle, welcome to the show.
Thank you. Glad to be here.
So, for those that don't know Joelle, she is a researcher who's been at this for a long time. You and I met actually maybe a month after ChatGPT was released, and everybody was asking whether AI was sentient. You at that time were the head of the fundamental AI research division at Meta. You're also a professor at McGill, and currently you're the chief AI officer at Cohere. We've had Aidan Gomez on the show. He founded the company in 2019. He's also one of the authors of the "Attention Is All You Need" paper, which basically kicked off the generative AI moment. So Cohere is seven years old at this point, six, seven years old, for the kids out there. It's raised 1.6 billion, it's worth 7 billion, and it sells AI to enterprise. So that sets the stage.
Let's talk a little bit about AI research. There's so much discussion. People have been talking about whether AI research is going to hit a wall, and whether these new methodologies — things like putting reinforcement learning on top of large language models, going through reasoning, teaching the models to use different tools — there's so many different opinions of where to focus right now. So what, in your opinion, is the cutting edge of AI research, and where do you think it's going to lead?
Well, I'm certainly not worried about research hitting a wall. There's so many questions that we need to work on right now, and I'd separate it into two interesting angles. One is like, what are the right problems to be solving right now? What are the things that the models, the current generation of models we have, can't do? And then there's a question of like, how do we go about it? Like, what's the hypothesis that may give us the clue to how to solve some of these problems. So in terms of what problems to solve, I think an important one is what do we do about memory? Machines have the ability to remember tremendous amounts of information. You're just like stocking it in there. The hard part is knowing like when to pull on what piece of information to make a prediction, to generate information, to reason. And so having this ability to be a lot more selective about all the information you've seen in context is super important. And already transformers were an important piece of that. You know, "Attention Is All You Need." Well, it turns out it's not all you need. You need a little bit more than that. You need the ability to reason about information at different time scales, at different granularity, and so on and so forth. So there's definitely a good piece of work to be done there, which really involves — and now we talk about the how — you know, the choice of architecture, the choice of learning mechanisms, the type of datasets, the type of use cases that we need to look into.
Another big research theme is on building world models. We hear a lot about world models, which are essentially the ability to take in all this information and predict the effect of actions. You know, so when we talk about causality, how are actions transforming the world? This is what a world model should be able to do. World models are absolutely essential when you want to build agents, because these agents are going to take actions which are going to change the world. You want to be able to predict these effects. So whether you're building robots, and then we talk about physical world models, but also the agents getting deployed on the web, you need to build digital world models so that these agents, you know, whether they're making financial decisions, communicating on your behalf, organizing meetings, that they have the ability to predict the consequence of their actions. So that's a big theme, and there's a lot of different hypotheses about how to go about building these world models. And the third theme I'll highlight — and there's many more, but let me at least pick out the top three — is about how do we build in reasoning efficiently. And right now a lot of the reasoning methods are still quite thorough, based on sort of forward search methods and learning the right reward function. But I do think there's, you know, like the transformer moment for reasoning and choosing action and being able to plan at different levels of granularity. We're still far away from doing that. And so there's all sorts of ways that it's being baked in, you know, LLM as a judge and things like that, where AI systems give feedback to AI systems in order to train them, is still very early days.
Okay. I want to dig into a lot of what you just said. Let's start at the beginning. Let's start with memory. Is memory and continual learning two sides of the same coin? I mean, there's this idea that the models can search the web and they can find something in a session, but as soon as you close that session, they forget it. And I guess the reason why I'm going there is because a way that some people have suggested solving both of these is just making the context window massive and then just becoming efficient in the way that you navigate that.
What do you think about that hypothesis?
The two concepts are related, but they're not exactly the same. And so memory is really about how do you address sort of what information to pull in in the context of the task you're trying to solve. Continual learning makes the assumption that the context keeps on changing. Therefore, what you've learned keeps on changing. So there's a notion of non-stationarity that is really key to continual learning. I confess I have a little bit of trouble with continual learning as a concept because I feel the community has never been able to nail like how do we articulate the problem in a way that we all agree on it, and so everyone who does work on continual learning takes a different flavor of it, which makes it, at least in my eyes — and I haven't worked a lot in this area — but makes it a little bit hard to know whether we're making progress or not. On memory, it's a little bit more standardized. The tension really is about, it's a question of efficiency and relevance. So the way to measure whether you're doing that is a little bit better standardized, and you don't want to be just sort of remembering everything. And so it's a little bit better standardized how we articulate the tasks.
Okay, let's go. We're going to touch on both of those now, and then we'll keep going down the list. With continual learning, maybe I'm, you know, so far removed from it, I'm not struggling with it. So I'll give you my caveman thought about what this is, and you can help us break it down a little bit. I mean, the problem has been articulated that the models, they don't change as they go about all these — I mean, think about how powerful it would be if, let's say, the GPT model, which is speaking with 800 million or maybe more, by the time this comes out, million people a week, could — I mean, might be scary actually — but could internalize those conversations and learn from the discussions that it's having. That would almost, you know, I agree with you that the wall, we're not at the wall, but the question is, is there going to be enough data to keep making these machines smarter? And you know, as they have these conversations, that opens up that ability to continue to grow and learn. But the model stays static despite all these conversations that it's having with people. Isn't that the problem?
And I mean, don't get me wrong, right? Like, I absolutely believe we need to address the fact that these models need to keep on evolving. I have no doubt about that. I just mean right now the progress in the research community that's working on continual learning isn't necessarily connecting to the work that's going on on scaling. Now, the models that are released, right, you know, they keep on evolving. I would say, you know, the generative models we have today, whether it's ChatGPT, whether it's Gemini, whether it's our, the Command models that the Cohere team is building, these models keep on improving. It's just we don't necessarily let them improve online, but we, you know, we ship at definite times, like a release of a model which has a particular characteristic. The advantage of doing that, frankly, is you can really test the model before you put it out there. You can put it through its paces in terms of performance, in terms of safety, and so on. And I would be a little bit reluctant to just let the model keep running on its own because, you know, the learning can go very, very fast, and you can switch out of a mode that seems completely reasonable very quickly, which, you know, we have seen a few times in the past.
Yeah, I think we might be thinking about one of the same instances when Microsoft had this bot called Tay. Yes. I'll tell you a story. I actually broke the news of Tay, that Microsoft had this great bot, spoke with the people, wrote the first story about it when I was at BuzzFeed. I pinned it to my Twitter profile. I went to sleep on the West Coast, and I woke up with all these messages being like, "Hey, that chatbot that you wrote about, the fun teen chatbot, is actually espousing Nazi ideology. You might want to unpin that tweet." And it was because it kept learning. So okay, maybe continual learning, you know, if it's done, because it has to also be done with some sort of fine-tuning where you want to make sure that behavior — maybe it's preemptive fine-tuning even.
Well, let's not release continual learning till we've achieved continual testing.
That sounds like a very reasonable plan. All right, memory. What makes it so difficult? I'll tell you one story. My Friday co-host, Ranjan Roy, and I, we both went into Gemini on Google and on Gmail, and we asked, "Can you find the first email that I ever sent with my wife?" Couldn't do it. Is that because there's so many emails in there that actually, like, applying AI to try to figure out like what conversations have been had, is that difficult, or is it kind of a product problem from Google? Like, why is memory so difficult, and how are we going to end up — like, how is the research community going to tackle this?
I mean, like, it's a little bit difficult to diagnose just from your description. I feel like I'm a little bit like, you know, a surgeon who's, you know, on the phone hearing the description of the patient. So, I won't necessarily venture, you know, a precise diagnosis for your case. But nonetheless, I don't think it's that difficult to figure out. I mean, I'd have to know what information is the bot pulling from, right? Like, just in terms of like visibility and privacy. Did you give it access to all of the information it needed to answer that? And that's the first one. And you know, we do a lot — you know, go back to what we're building at Cohere, actually — like, we do a lot of deployments on-site. So sometimes it's just a question like we didn't activate the access to the right information to do it. So you need to figure out whether that access to the right information is there. And there's all sorts of reasons that you may not want to give the bots access to all of your information all the time. So that's one practical consideration. The other one is like retrieving the right information. And so, you know, did the query match how the information was encoded? Because in most of these, you may not want to just leave the information in raw form. It gets very expensive. I mean, you're one person, but at the scale that some of these companies are operating, you have to compress it, which we often call embeddings. So you create like embeddings of this representation. And so it may not have embedded the information properly. And then there's like retrieving that information, and maybe it retrieved like 10,000 different items and didn't rank this one close to the top, and so it didn't generate the right response. But it could be that it knows of it, it just didn't show up at the top. So there's like a few different reasons which makes it hard. One of them is like the access to the information when it's encoding that information. Then it's like retrieve the information at the right moment.
But when this stuff works, it's pretty magical. I was just in Claude, actually, and I noticed that Claude's memory capabilities have really improved. I was speaking — so I love to upload the transcripts of my interviews and like just, you know, get a grading out, like, give me a rating on a variety of metrics. You decide to tell the bot, you decide.
Do you agree with the ratings that the bot is giving you?
Usually, well, you've trained it well.
So some are good, some are bad. I actually had Gemini do a bunch of ratings, and it was like five of five on all categories, and I was like, that is wrong. And then I went to ChatGPT and Claude, and they were actually much more reasonable about it. But one of the interesting things that Claude did when I asked it this week, it started comparing it to the other interviews I had done. And it said, you know, you actually hit better points on this one, and this is why this one didn't resonate in my opinion.
Did you benchmark it with a sample of your audience?
That's probably the next — and it'll probably, when I, because I'll take data out of the podcast analytics and drop it in these bots, it's going to be able to cross-reference. So when it works, it's magical. And you know, you've identified this as one of the areas where AI research really needs to, you know, concentrate, and this is the cutting edge. How good can this get, and what do you think? Do you think that it's at a moment of real progress, or is it sort of party tricks to be able to get Claude to be able to do the things that I talked about?
The question on rating specifically, like analyzing the information and sort of distilling some feedback —
More about the memory, the fact that it can call back memory in particular.
No, I mean, we're making good progress on that. You know, extending the context length is kind of the easiest way to go about it, but there's quite a bit of progress that is being made on this.
Okay, let's talk about reasoning. You mentioned reasoning as a cutting edge moment. The problem is efficiency. Is that really the issue here? I mean, so reasoning is the model basically goes step by step. It tries to answer, checks the answer, tries a different answer, then eventually decides, okay, this is probably what they want, and then it spits something out.
Yes. I mean, that roughly happens this way. I think the challenge is really being able to plan at different levels of sort of temporal granularity, right? So in terms of how you execute actions, let's say, you know, you're planning a trip, right? You're not going to start by thinking of like, what are the shoes that I put on to go on my trip, right? You're going to start by thinking like roughly what season, roughly what, you know, part of the world do I want to go visit. You start from the top level, and then you take it down a notch, which is like, okay, you've identified like a rough time, a rough place, like let's get more precise on the time and the place, and maybe the activity, and like maybe who you want to go with. And then you take it down another notch, right, and that's when you start booking your reservations and so on. But sometimes, you know, you'll hit a blocker on the reservation, and you can't get the flights or the hotel you want, and then you'll pop back up and say like, do I change my dates? Do I change my place? Do I change who I go with? I'm not going to bring the kids because then we can, you know, have more options. So we can pop back up in terms of level of resolution. That's the part that the reasoning models don't do. They do really well at like one level of granularity. So you've got a robot, you give it all these like motions for the hands, what the body motions, it can plan essentially to control the motors at that level of granularity. But the going back and forth between different levels of sort of resolution of action, it's really hard. So on the technical terms, we call it hierarchical planning. That's really hard to do that decomposition and keeping the information relevant as you go back and forth.
Is that just a limitation of the large language model? Because the fact that an LLM can even do this in the first place — like again, like it started with predict the next — do it at the word level.
And out of the word level, you do get the higher level. It is really impressive. I think that's the part that probably shocked a lot of people. They expected at, you know, back in 2023 or so, they expected that as you're generating tokens, you're not going to be able to generate sort of big ideas or a bigger plan. And yet, it's pretty remarkable that it does it. Which is why you get sort of, you know, different opinions in terms of some people thinking like, hey, like it's already impressive, like let's just keep on pushing that way of doing things, and we will unblock this, and other people being a lot more skeptical that you'll achieve it. Explain that a little more. So as it's typing, as it's — I mean, I think Andrej Karpathy basically explained that the transformer is a computer, and every time you generate a new token, you're going through a piece of computing. So the more you type, the bigger the computer is that you use.
Yes. The more, I mean, the more information goes in, and the bigger your representation is.
And so, but are you saying that as this happens, the computer is effectively already thinking ahead? I'll give one example. Claude, just to go back to some Anthropic research, they published this amazing research where they asked Claude to write a poem. And as it's writing the first line, it's already activating features in the model that's thinking what rhymes with that. Which is amazing because again, it's technology that predicts the next word. But as it's predicting the next token, it's already thinking the next sentence, which to me is just mind-boggling.
Yeah. And so I mean, this is why to some degree the emphasis on code and the ability to build representations of code and generate code is so interesting, because when you look at code, and you know, for people who've programmed before, the code has that structure, that hierarchical structure, it's encoded in. Anyone who looks at a bunch of code, even if it's not necessarily a language you understand, you understand the notion of functions and variables and libraries and so on. And so those different levels of granularity of the project, it's encoded in there. And so there, you know, there's some hope that by training enough on code, the machine essentially infers these kinds of structural cues.
Fascinating. So that, I mean, like you talked about the fact that this technology — and this is sort of the thing that sort of makes my head explode a little bit — the fact that this technology is able to do these things that you wouldn't think given the architecture it is supposed to do. Same with if you think about video models and image models. And by the way, one of your former colleagues, Yann LeCun, would always talk about how to generate — and I know he has some criticisms of video models — but to be able to generate AI video, you really have to be able to predict and plan what's going to happen in the physical world. Absolutely. And there's some embedded intelligence that even leading researchers, I don't think, fully get. That when you, for instance, ask a model — just to use Yann's favorite example — to drop a pencil, there's so many permutations of where that can go. And now the models, without like, I mean, without having lessons of physics, understand that it drops and maybe hits the table and might bounce, but bounce up.
Yep, because it's seen enough data from objects that are dropped that have these kinds of behavior.
But try to predict what's the behavior of a similar object dropped on, you know, on a different planet, and probably the prediction is wrong because all of the data was taken with our gravity constant.
Yes. I mean, I will say as I'm talking about this, I did just see a video generated where a man's fingers came out of a styrofoam cup as he was holding it. So there's room to — a lot of room for improvement.
Now, there is some talk — Demis Hassabis was on recently talking about how Google's video models in some way have like these world model capabilities. They do understand the physics, and you brought up world models as another area where this technology really has the potential to grow. It's the cutting edge, still kind of undefined. Yeah. I will say, and you know, going back to the caveman here, I'm a little bit confused about why, for instance, like one of the examples that you brought up earlier was that if you want a model to be able to like go out and like complete financial transactions and understand the implications of financial transactions, it has to know how the world works. But can't you just teach that in text? Can't you teach it like, if you use my credit card and, you know, buy anything online, I will go bankrupt, like in text or even number logic, and therefore don't do it? Like, why does — and I think world models is like, these models need to understand gravity. Why does a model need to understand gravity to learn these basic rules of sort of the way that the world works?
Well, and this is why earlier I sort of distinguished between like physical world models and digital world models, right? It's possible that you can actually build really effective agents, web-based agents that don't understand the concept of gravity. And it's possible you can build physical world models for robots that don't need to understand, you know, the functioning banking system. And so you can define the word world as being like a contained environment. And so, but if you want to deploy the agent on that environment, then it does need to understand the rules of that environment quite well. The challenge is getting enough coverage of data for all the possible futures, right? And all the different ways that the world could evolve subject to various events, various events happening. So a lot of the cases today where it's actually most beneficial is where there's like a place for the human on the table. And I'll give you an example, right? People talk a lot about using chatbots for customer service, right? Like chatbots should be — you should just like plug them in, and they will answer all your questions. They'll be available 24/7 and so on. In reality, and there will be of course many chatbots deployed for these kinds of cases. But you know, like one of the use cases we've seen that works really well is actually to have the bot like pull together all the relevant information. You do customer service, you pull together all the relevant information from many different sources as opposed to like following a script being just a chatbot. Pull together all that information about, you know, the documents, the documentation that accompanies the system, the case on the client, the different problem description that you have. You pull all together that, and then you pose a diagnostic, and then you pose a few suggested actions, and then you keep a human in the loop to validate the plan and to carry out the action. And so that means that the human, you know, can — and these are more complicated cases than just like your cell phone plan or something like that. But nonetheless, in those cases, like what would have taken a long time, you know, maybe, you know, half an hour to pull together all that information, distill it for a human, now you can reduce that down to like a 20-second, you know, analyze, verify, and carry out the action. So if you have that ability to combine the human and the AI agent, actually you get often some much more powerful results, and it means if your world model isn't complete, humans in the loop, they figure out the pieces that's missing. They give that extra information, and then you bring that information back to train your agent. Then you get continual learning.
