The scaling laws have been remarkably robust. There's a lot we don't know yet in terms of the vulnerability of these systems.
Today we have one of the leading minds in AI, Joelle Pineau. Joelle is the chief scientist at Cohere. Maybe you don't need to buy the Galacticos. Why do you have like an Andrew Tulloch, a Daniel Gross, an Alex Wang, and the Galacticos assembling?
I used to be quite skeptical that neural networks were necessarily the ultimate solution to machine learning. I seem to be quite wrong on this one. Knowing what you know, what do you not let your children do?
Eat too much sugar. I don't have a lot of patience as a scientist for people who are predicting the extremist scenarios. The catastrophic risks of AI, you know, AI becomes our overlord kind of scenario.
If I gave you $10 billion, what would you spend it on first?
Ready to go?
Joelle, it is so great to have you in the studio. I've heard many great things from Nick, Aidan, Shrep. So, thank you so much for joining me.
Thank you. Happy to be here.
Now, you spent over six years at Meta and I want to start there because it's a very transformative time and place. What are the biggest takeaways for you from that time and how did that shape your mindset to how you think today?
Well, I was there from 2017 to 2025 and you have to see just how much AI changed over that period of time. And what we were really focused on is fundamental AI research. And what you know, one thing that I've learned is just sometimes how long it takes to prove out a hypothesis. We feel like AI is moving at the speed of lightning. But in fact, there's some things that it just takes a few years to mature, to get the right optimizer, the right compute, the right data for that to really make a difference.
I look at where we are today and everyone kind of goes, it's here, it's here, it's here. And then you actually look at what a lot of the leaders have been saying recently where it's like actually, you know, Andre was saying it's not the year of the agents, it's the decade of agents. Sam's kind of pulling back too. Yeah. Have we got over our skis and we're actually kind of all pulling back realizing that time is the factor we need to rely on?
Well, I'll give you an example. You know, I've been in research for a couple decades now. I've been working on reinforcement learning for over 20 years and suddenly everyone's talking about reinforcement learning, you know, since the advent of reasoning models, agents and so on. So, you know, sometimes you have to be a little patient with these ideas and the right algorithmic tweak, the right context, the right problem domains just opens up the magic.
I was listening to Andre yesterday and he said in this show that reinforcement learning is terrible.
Less terrible than 20 years ago.
Have we overinvested in RL-based methods at the expense of maybe like more scalable alternatives?
I'm still super bullish on RL in that like the concept itself is so fundamental, you know, this idea of training through a system of rewards of indicating what's valuable and what's not valuable through numerical values like that is so fundamental, it's not going away. Now you know where we're maybe getting a little bit ahead is thinking that just RL out of the box is going to give us AGI, that part a lot less so. You know, if you look at the curve of progress, RL is terribly inefficient and so the amount of signal you need to get in order to really shape the behavior of a model is far from where we are today. And so we'll need to figure out how to really deal with this learning efficiency problem.
You're probably thinking, what did I get myself in for? And I don't blame you. I ask questions that I think everyone else thinks, but I'm not afraid to say I don't know. Why is RL so inefficient?
There's a few reasons. You're going to get me on like a deep topic. There's a few reasons.
One is the fact that RL is about sequential decision-making. So, you know, think about you're starting at a point, you need to figure out what you're going to do next and you might pick the right side of the branch or the wrong side of the branch and then like the road keeps on splitting. So, every time you make a mistake, it sort of compounds through the length of the series of actions you're making. So that means like the amount of error you can make can be very, very large and to get it right is quite difficult. Sometimes people compare it to like a needle in a haystack is like finding the right solution in RL. So there's that part. The other part that's hard is the fact that to train the system, to train the models, you have to essentially take actions to learn. You can't learn from static data. You can learn some things from static data but actually, you know, to get the right policy you need to test it out and so that means you need a simulator, you need to get the synthetic data, all of that can be really expensive also. And so we have difficulty getting like just a variety of environments and simulation to test RL.
When we look at the cost curve for RL, you said you've been working on it for 20 years, have we seen that dramatically come down, will we see it continue to dramatically come down, or is it a case of it is just a fundamentally expensive method of training?
It's come down especially in domains where we have good reward functions. So the place where most people started hearing about RL is around the AlphaGo time. You know, the game of Go which was sort of one of the goals for AI, many people thought we were at that time we were still a decade away from being able to have machines play Go at the level of humans. And out comes a team from DeepMind, you know, goes off, plays against the world champion and shows that RL can basically do it. And so I would say, you know, in cases where we clearly know what's the goal, we can write down precisely the reward function, we're good, we can make a ton of progress. So that's why you're seeing progress in mathematics, very well-defined reasoning tasks, games, these kinds of things. RL to shape the behavior of models to get them to be social creatures, that we have no idea how to do. I mean, I don't know if you have children but like shaping their behaviors, you know, the number of times you can repeat the same thing and still they do something else, and so there's something there. You don't know how to write that out mathematically. And that's where I think we're still in for some hard work.
Okay. So, we still have some hard work. When we look at the training versus the inference market today, we've had so much weight on training so far and it's been incredibly costly and expensive. And then I hear everyone say, well, actually, inference is 95% of the market and that's where it's all going and that's where Nvidia will make most of their money. How do we think about the cost curve applied to training versus inference and where it sits today?
I think there's a lot of different variants and you know, if you'll allow, maybe I'll pivot to like, you know, anyway, where I'm going with Cohere now. Just you know, I joined Cohere less than a month ago, super exciting company.
I think one of the things that Cohere is doing is actually to develop AI models that run on premise, so that means enterprise bring it in, they run it locally, so the company has to worry about the training of the models. Obviously, we want world-class models for the needs of enterprise, doesn't have to worry about the inference, doesn't have to worry about the inference cost. The, you know, the client's customers have to figure out what's the right way for them to digest the AI. That means like there's a lot of motivation to have very efficient models so that they can run really efficiently on premise. So, you know, we get caught up into one paradigm, but there are other paradigms as well.
If they're the ones paying for the inference, is there not less incentive to make them efficient? Because sort of, you're not the one paying for it. If you're the one paying for the inference costs, God, I want it to be as efficient as possible because it's my dollar going to that. But if it's IBM's dollar, I mean, yeah, I would love it to be efficient, but we're not paying for it.
We're still in the early days of AI adoption in enterprise, so what's good for the client is good for us.
Totally. Totally get you. What's the biggest challenge about capital efficient AI today? I know that sounds strange when you look at the economics so to speak. What's the biggest challenge?
There's a lot of challenge today. I think in terms of the economics of AI, I think one of the biggest challenges is the fact that it's very hard to have predictability, right? Everyone wants to know when are we going to hit the breakthrough. Everyone wants to know how many GPUs do I actually need? Everyone wants to know like what's the return I can expect. There's just a lot of uncertainty built into the system. A lot of that is because there's a lot we don't know about this technology. And so that means we have to take in quite a bit of risk when you're building out, whether you're building out your data center, whether you're building out your workforce, whether you're trying to figure out, you know, how much data to curate. And so that makes it difficult for a lot of people. People want answers and this is a world where we don't have that level of predictability compared to other industries.
Does progression happen in kind of a linear fashion or does it happen in step functions like AlphaGo, like a DeepSeek, which depending on kind of what you believe suggests a lot of efficiency in terms of model improvement. Is it step function or is it linear?
I tend to decompose different ingredients that lead to progress. You know, people often talk about like the algorithms, the data, the compute. I think in general, compute and data have a more linear effect on progress. You build more compute, you run bigger models, you can typically get better performance. You feed in more data. It's not just quantity. You need to worry about quality and diversity as well. But roughly, it's more linear-ish with respect to the data. The algorithms are the ones that have the nonlinear effect. And so you can explore lots of ideas and then something like the transformer comes along and just changes the paradigm. And it's not just your transformer, you know, on the optimization side suddenly we hit upon Adam which is a technique to do the optimization of your model, changes in the paradigm. Reasoning, suddenly we start thinking about how to put that in the loop, reasoning, and it changes the paradigm. So those ideas tend to have a nonlinear effect.
The challenge with these algorithmic ideas though is that actually it may take a long time to prove themselves out. So like the paper can be sitting out there. There's thousands of papers coming out. The idea is sitting out there and we may not think to try it with the right data at the right scale with the right combination of hyperparameters and so you don't notice that effect for a while. So it's hard to predict and it's nonlinear more on the algorithmic side than I think on whether it's data, compute, even talent or other things.
With respect to Google, I mean, transformers obviously birthed in Google and sat as papers for many, a couple of years. When you mentioned that you compute, algorithms, data, if we just kind of go through them to understand, everyone suggests that it's weird, there's two different worlds, like scaling laws exist, just throw more compute at it when you look at data center investment, when you look at all the kind of desirability of compute, and then again you have GPT-5 seemingly focusing on efficiency and other signals. Do scaling laws play out from here and if so for how long?
The scaling laws have been remarkably robust. They don't play exactly as we expect but still they've been remarkably robust. Lots of people have bet against scaling laws in the past and I would say overall, you know, we've seen a pretty robust effect. They don't work alone. We also need these algorithmic innovations. But most of the time, you know, I wouldn't bet against it.
On the algorithm side, is that the hardest to innovate on? You could think about, but you can buy more compute, might be hard, but you can buy more compute and data, there are different ways whether it's synthetic or human, is algorithms the hardest to innovate on?
It's certainly the most creative work to be done and you know the space of ideas is so wide that I would say it's hardest in the sense that like you can move in, you know, I'm a researcher at heart, you can move in so many different directions and picking the right one, you don't know until you get there whether it was the right one or not. It's a little bit like reinforcement learning. So in that sense I think it's the most interesting one. It's the most frustrating one and it's the most difficult one certainly from an investor's point of view to know where to put your chips.
Speaking of kind of knowing where to put your chips and moving from purely a research lens with Meta to now also building product. Is there ever this like inherent conflict between intellectually interesting research with the need to productize and monetize and how do you think about that?
I mean, one of the reasons I'm really excited to be joining Cohere actually is because like we're at a stage where AI is really starting to be useful. Maybe not as useful as people think it is, but we are there. And by working on AI that's going into enterprise, I feel we're going to get such an interesting signal of what works and what doesn't work. You know, we keep on talking about, you know, AGI and AI for the masses and so on, but actually like when you need to sell AI to a business, you get a real signal of what works, what doesn't work. And that's what I'm most curious to see. And you know, we've been using these academic benchmarks for many years. You get some signal, but it's not the same as getting this to do productive work. So I'm curious to learn out of that. You know, we're going to get new types of data. We're going to get I think a lot of insights that are then going to drive the research ideas. I think that's the other thing to think through when you have a large space of ideas to explore. Getting that feedback signal from the real world is super useful to guide you through that search of ideas.
I just had a great chat to David Cahn at Sequoia who said that he thinks a good barometer for utility value within enterprises is like does it have the ability to replace the work of your bottom 5% in any category. He says like we overestimate a lot. Can it replace the bottom 5% in any function? And if it can, that's a very meaningful improvement. Do you think that's a good barometer and how would you advise an enterprise on whether something's useful or not as a yardstick?
I, you know, I prefer in terms of a barometer of productivity something a little bit different, which is to say can most of your employees do 10x the amount of work with AI versus on their own. That to me is actually a better barometer. I think human and AI have very complementary abilities. So to just like flat out replace a portion of your workforce is actually pretty unrealistic. Some may try and some may be slowing down their hiring. But I actually think...
Respectfully, I think 10x-ing your work feels more unrealistic. Is that not a bigger ask? I'm almost more intimidated by 10x-ing my work.
Oh, I don't think that's unrealistic at all.
Yeah, in a timeline that is the next couple of years.
Yes.
I'm sorry, how does that actually shape out then?
I think you have to identify very concretely the types of work that you are delivering. But I think we're starting to see like, you know, Hollywood quality productions being made in a matter of hours. We're seeing, you know, to take a super concrete case like machine translation. And if humans are doing the machine translation compared to machines doing it, you go from hours to seconds on long form text, multi-page documents. And so for a lot of work, it's not like AI can do all of the work. Humans still need to ask the right question. They need to verify the information. They need to shape the tasks. But once the task is well defined, the product are clear, like all the design considerations are fed into the prompt, you press the button and you've got an answer in seconds for something that used to take sometimes weeks and months.
So completely hear you and understand that. I'm just kind of re-trying to reevaluate a belief that I had for the last few months, which is like for, I'm a venture ambassador for a fund, for all of us to make money, we need to see the transition from kind of human labor budget to AI spend. And it's with that transition where we obviously see the TAM massively increase and we make a lot of money. But when I hear you say that, I suddenly question that assumption. It's like the barometer for whether we make money, because you're suggesting that actually we don't replace the human labor budget. It just makes us 10x more efficient. Is that correct?
Yes. And I think, you know, people, I've, you know, there's a lot of nuance to all of that and some work will be harder to get that same level of efficiency gain whereas other work you'll see 100x in terms of efficiency gain, but I do think that for a lot of the work that's happening right now that's absolutely feasible.
Where do you think, I'm sorry, where is the efficiency gains most tangible? It goes back a little bit to this notion of what are the tasks that we can specify.
Well, so in any case where we can be very precise about what a great result looks like, we'll be able to make that task automatic much more easily than tasks that are much more nuanced and have a lot of complexity.
Ambiguity in the specification of the task is what's hard for our machines.
How have you seen enterprise reaction to this? There's fear from workers sometimes, there's excitement from leaders, there's apathy sometimes. How have you seen and measured enterprise response?
A lot of the workforce can be reasonably fearful about job displacement. There's a lot of also individuals who have, you know, a bit of an instinctive reaction to change and change can be hard for a lot of people and we're seeing a lot of change in a very short time span. And so I think there's also a generational effect. I think for some generations that change is more jarring. I think for the younger generations, I have teenagers, young adults at home. For them, yeah, it's just natives. They just, you know, kind of, you know, are going to grow up with that technology in a different way than some of the older generations.
It's interesting. Sam, speaking of kind of children there at home and how they engage with it. Sam Altman said that kind of young people engage with it as like an OS to the world and AI is that companion for them and then older people kind of use it as a next-gen Google. Do you agree with that? And do you see that in your work?
I see a lot of people using it as a tool more than as a companion. You know, people have this, you know, Swiss knife in their work life all of a sudden that can be super helpful, but that's really most of what I see.
Totally get you. What are enterprises' biggest challenges with AI adoption at scale?
For many enterprise, you know, one of the challenges is to make sure that the AI comes in and can be integrated in their workflows, their processes, their information. And so the challenge is to deploy in a way that allows them to exploit all of the information systems that they already have. And some of them have accumulated these over decades. And so that's, you know, some of the work that remains to be done.
Integration with existing systems and data flows.
And that's, you know, of course that's something we see a lot of at Cohere because we do on-premise deployment. You know, one of the things we focus on the most is data confidentiality and security so that enterprise can exploit all of that information. So that's top of mind for us. But it's also a huge opportunity. I would say there's a big interest in that. But making sure to get that compatibility I think is a challenge. In many cases for people, you just, you know, change is hardest for people and so you have to get them curious about using the technology. Many people feel they have to get it right the first time. And I really think like a spirit of exploration and curiosity is much better suited to the phase of maturity of the technology that we have today. We don't have all the answers of how it should be used. That's going to come from people on the field.
Security is a topic that we quite often glaze over especially when investing in kind of application layer AI tools. What does no one know about AI security that people should know?
With respect to AI security, I think there's a new front that's opening up with the development of agents and frankly there's a lot we don't know yet in terms of the vulnerability of these systems. With LLMs, we're starting to get a better understanding. We've had, you know, quite a bit of red teaming exercise and jailbreaking and so on. And so people have identified different risk vectors, prompt injections, things like that, which are vectors for malicious actors to interfere with a system. With AI agents, we haven't seen that. And one of the features of computer security in general is often, you know, it's a bit of a cat-and-mouse game quite frankly, like there's a lot of ingenuity in terms of breaking into systems and then you need a lot of ingenuity in terms of building defenses and so we just have to stay very active in that sense.
What are the potential vulnerabilities though in an agent world?
In terms of agents, you know, we worry a lot about hallucinations in LLMs. The parallel in agents is impersonation. So agents that come along and are essentially impersonating entities which they don't legitimately represent and in doing so taking actions on the behalf of these entities where they don't legitimately represent, whether it's infiltrating, you know, banking systems and so on. And so I do think we have to be quite lucid about this, develop standards towards it, develop ways to test for that in a very rigorous way. Okay, there's ways to reduce that risk drastically. You run your agent, you know, completely cut off from the web. You're reducing your risk exposure significantly. But then you lose access to some information. So depending on your use case, depending on what you actually need, there's different solutions that may be appropriate.
Totally get you. That's a really hard one because like then verification becomes the most important thing. But then it's like who's the arbiter of verification? Is it governments? Is it companies? How does one think about that? Who says you're a valid agent versus an invalid agent?
Governments can be good for defining standards on which we all agree. Companies are much better at building the solutions at scale and deploying them.
Do you think governments are good at setting the standards when you look at AI and where we're at? And then when you look at the sophistication levels of government programs or decision makers with respect, they're just a little bit behind. Do you think they are actually equipped?
I don't think you should look at where governments are in terms of necessarily AI regulation. AI as a field is so incredibly young and fast-moving by nature and there's some good in this. Governments are moving a little bit more cautiously and usually need to benefit from our knowledge to make good policies. And so I do think you can look at other fields in terms of regulation. You know, you look at aviation, the security record for aviation today compared to where we were 50 years ago is just incredible. And governments have played a role in defining that in terms of standards and in terms of what are the norms and so on. I'm quite hopeful, I'm an optimist about this, maybe it's my Canadian side, that governments can play a useful role. In many cases, you know, clear standards actually means reducing uncertainty for a lot of companies in this space, but we shouldn't expect that to be ahead of the technology. I think that would be the wrong order of things in some sense. We need to develop that technology with enough of a creative space and we need to learn fast and then develop the right guardrails for that technology from the real learnings we have.
We mentioned there kind of governments and their role. When I had Nick on the show, he was saying actually the benefits of not being an American company given some geopolitical challenges sometimes. I'm just intrigued. Do you think we will have these kind of sovereign models for each geo? You know, we have Mistral in France. You, Cohere, obviously in Canada or founded in Canada, but I know you've got global kind of HQs. Do you think we will have these sovereign models and regionalized winners?
I do think it's healthy that there are models that are getting built in different places around the world, not just in the US and China right now. I think this is healthy in terms of diversity of thoughts. I think it's healthy in terms of having a greater amount of people with access to technology. I do think for Cohere, you know, the vision isn't to be a Canadian company. Like the vision is to be a global AI company. And I think yes, you know, we have a headquarters in Toronto. We have teams that are distributed around the world. We have a great team here in London as well as in the US and in France and other places. And so, you know, having that ability to deploy models that operate across the world, I think it's going to be absolutely an important part of the strategy for Cohere. I think there's a great opportunity. What it gives us to be headquartered in Canada is like a sensitivity to the fact that it's not always a one-size-fits-all solution. You know, I go back to the research we've done. We've done leading work in terms of multilingual models. And it turns out it matters. You go to Japan, you go to Korea and they do want models that work well in their language. People in the workforce are still operating in the language of the country. So having a company that is attuned to that, that values that internationalization of models is actually important on the global market.
Totally get that. On the team building side, obviously Canada has great talent. You mentioned obviously some in London as well. What have been your biggest lessons, observations on team building in this like talent frenzy that we're in? Also, how are you analyzing that?
One of the things that's important when you're building a team for AI, I do think you need people who have vision, who have like a sense of like what can we create just because we're in a space where there's so much innovation that is still needed. So you need an ingredient of vision. That can be one, two, three people who bring that ingredient of vision. You need people who have amazing execution muscle. Like they don't care that it's their idea. They care that if the team agrees on an idea, they are just going to push this and get it done. They're going to build the system. They're going to run the experiments. They just have that technical rigor to execute. And then you need people who kind of...
Like keep the team together, who have like this sense of like who needs what to operate well and who are that social glue. You know, humans are still social beings and that social glue in a team matters a lot. Where I've seen it fail is to have just sort of one type of person inside the team. Um, I don't think it becomes that productive to put a bunch of AI superstars all together in a room without the execution machine, without the social glue. I don't think you get necessarily the same result. So, I'm a big believer in building teams with diverse, complementary skills.
So, you can't just buy the Galacticos.
I don't think you need to. I think you really have to be thoughtful about putting people in a group. The other thing that helps a lot is for the team to have focus. You know, if it goes in all sorts of different directions, you'll lose that power that you get from people working together. So, having a lot of clarity — what's the north star? What's the goal? Where are we going? Even if over time that needs to change, but that level of clarity is required for everyone to be working in the same direction.
Can I be so blunt? If you don't need to buy the Galacticos, why do you have like an Andrew Tul, a Daniel Gross, an Alex Wang and the Galactic assembling? Like, is that wrong?
You do need a few of these like uber talents in the team. There's a relatively, you know, short number of people who just understand this technology very deeply. You do need some of this talent and if you can afford it, you should get some of that talent. But you don't need all of your team. You need like a team with complementary skill as well.
Does that create a good team? Like if I gave you, you know, $10 billion to go build a team and you could buy a couple of these luxury star players — I feel like it's Top Trumps cards for like sports teams, but you can buy a couple. Does that create a good team when one is a three billion dollar person and then the rest of just average $50 million people?
Yeah, I wouldn't say no if someone offers me the opportunity to hire. There's definitely some really talented people in the field and they deserve to be fairly compensated. This technology is going to make a lot of people very rich and have major effects in terms of society and so, you know, we should be rewarding the talent, but I'd be very thoughtful about what are the teams that I put together and how do they work together rather than just like, you know, hire a roster of superstars without being thoughtful how they're going to work together.
So it's so funny, so because of the impact that you can have in these teams, actually the multi-billion price tags that you see can even be justified.
Time will tell. I don't think it's necessarily needed to go at that scale, but time will tell.
If I gave you $10 billion, what would you spend it on first?
