Hi everyone, thanks for tuning in. Really awesome episode coming up today. I'm Wilson, your host of the Make It Click podcast by Borderless AI. We have Canadian researcher, programmer, entrepreneur Nick Frosst on the show today. Nick is the co-founder of the enterprise-focused LLM Cohere. Really looking forward to the conversation with Nick. I learned a lot. I'm sure you will too.
Hey Nick, good to meet you. We've connected a couple times briefly here and there. Would love just to kind of... okay, first off, a lot of people know this about you, but over 3.5 million people know this about you: you have a band.
Oh, yeah. When I first reached out to kind of kick this one off and get this going, have you on the show, somebody on your team was like, "Hey, trying to pin him down, but he's traveling, touring with his band."
I didn't bring my Sharpie for the autograph, but I will get that. Tell me a bit about that because you have 3.5 million active listeners on your band, Good Kid.
Yeah, Good Kid. Yeah. We're like an indie pop rock band, somewhere in between The Strokes and Panic! at the Disco.
Okay, nice. Big Strokes fans. They're gonna be in Toronto this year, I think.
Nice. And what's older, the band or the company?
The band, actually. The band is older.
Started in university. It was like five of us. We were all programmers. So we started playing music together back then and we just never stopped.
What's it like juggling on the company side? How does that work?
It's fine. I'm very compartmentalized about it. When we tour, I work out of the back of the bus. We have a few buses and there's a room in the back of one of them and I just set up shop. I work there during the day and then I play some music and then I go to sleep and then I wake up in some different city because the bus will have driven during the night. And then I'll do it again. There's something almost monastic about it that I like. There's a routine.
So the name of this show is called Make It Click. People watching want to just understand stuff, and it's typically entrepreneurship, creative, Canadian. You probably didn't think you'd get to 3.5 million monthly listeners on Spotify on the band, and maybe you thought you'd get to this number, but latest public valuation on Cohere: 7 billion. That's nuts. That's crazy even to you, is it?
Let me give you two answers. The one answer to both of those questions is like no, both of those numbers are too big for a person to engage with. Like a person, unless you are very strange or broken, cannot really engage with a valuation above—I don't know—assume you crossed some threshold and we crossed that a while ago—where you can't really engage with that as a person. In terms of music, I can engage with this because I play shows. The largest show we played over the last run was I think 2,000, sold out, something was like 2,500. So that was— I know what that feels like. I know what playing to an audience like that feels like. But in terms of having 3 million people listen to us monthly, no, a person, you can't really engage with that. You can't really imagine what that would look like.
On the other hand, as a person, I don't really engage with it. I can't really imagine I would get there. Can't really do it. On the other hand, I believe in our business. When I look at our valuation, I'm like, man, we got a lot more value we can add. And there's a lot more things we can do. So I think, in some sense, I think we're kind of undervalued. I think we can add more. I think we can build more. I think we can make more for the customers that use us. And on the band, I'm like, man, we play good music. More people might like it if they listen to it. So it's kind of both of those things.
Well, you got a new subscriber in me a couple weeks ago when I heard we locked this show and I was like, okay, now I gotta get some merch. Cohere has some pretty dope merch. Startup merch is tough to get right, but you guys crush it.
We have a background in it. We have some new stuff coming for Toronto Tech Week actually.
Oh nice. Toronto Tech Week for those that are going to be watching this show—it's going on right now, this week in Toronto. You're obviously doing some speaking stuff?
Yeah, we have some stuff coming up.
Cool. On the funding you guys have raised—so 7 billion, and in lifetime you've raised like 1.5 billion. Where does all that go? So if I'm like an educated guy watching this show and I work in banking or something, maybe at Scotiabank, I know you guys have a massive customer in RBC, and I'm just trying to make it work in my head—where does all that money go?
So the process of creating large language models, which is what we do—we make large language models and we sell them to businesses, to enterprises—is really three components that go into it. There's compute: so we spend a lot of money to get access to computers that are needed for training the models. There's data: so we have to get data to train the models on, we have to create that data, we work with a bunch of people who create the data. And then there's engineering talent and scientist talent. So it's really compute, data, and talent. That makes up the vast majority of our spend.
Now, we're kind of interested in this industry which is a wild industry. But you know, on the one hand we're a large company—the numbers you're throwing around are big, you can't really engage with this person. On the other hand, there's maybe 10 or 11, 11 now, companies in the world that can make foundational models. Definitely under 20. Yeah, definitely under—I think maybe 11 or 12. And we spend orders of magnitude less on compute than our competitors. Like truly we have spent 1/100th, maybe even beyond that, less on compute than Anthropic, than OpenAI, than Grok, like xAI or SpaceX, whatever is going on there. So on the one hand, yeah, we've spent an enormous amount of money on this. On the other hand, it pales in comparison really to what some of the other players in the space are doing.
And why is that? Just because they've been able to—I guess with OpenAI, do they get a boost because of the consumer slant, and you guys are B2B and it's trickier to raise money?
Well, we're only B2B. So that means that we're not interested in making the biggest model possible. We're interested in making a model that is efficient to deploy and that when you deploy it privately into a customer's environment, it adds enough value to what they're doing that it warrants paying for it. So if you're training a big model—like the models that are powering the consumer offerings from ChatGPT, from Claude—those are many trillions of parameters, and even the inference of those is often losing money. So we make models that are much smaller, but are competitive on the things that our customers care about.
There's a bunch of stuff we don't train the model to do. Like our model doesn't generate images, because that's not actually—super useful. It's not—none of our customers are asking us to generate images. They're instead asking us to make sure, like, hey, when it answers a question for an employee, make sure it cites it correctly. Hey, when it extracts quarterly earnings from a PDF, make sure it puts them in a good graph, things like that. So we train the model to be really good at those things. So then we can deploy it. Because we're training smaller models, we can spend less on compute. And then also, because we don't have a consumer offering, we don't need to be spending on that compute for inference to get a bunch of consumer customers in. So those things together have meant we've been able to be significantly more capital efficient.
Yeah. Lots to unpack there. So that's the capital efficiency piece. That's very helpful. When you guys started the company, I think your Series A was like 40, 40 million bucks?
You know, it's been many years. I forget the exact number.
You go to McDonald's every time you raise money?
Yeah. When we close the deal.
When you close the deal. That's like the tradition. When you guys started the company, founded 2019?
Yeah. Was that the idea—hey, B2B, enterprise?
Yeah. We always thought that this technology was going to be most useful for work. Yeah. We always thought that when we started in 2019, 2020, things are very different, right, like the world was different. And so we were to begin with, we were interested in mostly building an API, and we have an API, but that's not where most of our customers are. Most of our customers are on-prem deployments. And that's because they want to use LLMs on real data, on data that is private, on data that has regulations around where it can go. And the only way to give access to models to that data is to have the models deployed where the data is. So that's why we do on-prem. We started doing that a few years ago.
So maybe folks that wouldn't necessarily understand on-prem watching at home—help me understand that. How the engineers work within that, what customer support is like?
So imagine if you're like a healthcare company, for example. You have a bunch of healthcare data. We work with several healthcare customers. You have a bunch of private data, like people's personal private healthcare data, and there's rules, there's laws about where that data can go. So hospitals will have internal infrastructure, they'll have computers that are in the hospital, and they might have rules that say, hey, we're not going to send this data anywhere but here.
Now healthcare is something that is really burdened these days. It needs to be more efficient. Everybody's trying to figure out how to process information better, get better outcomes, get more efficiency. And LLMs are useful there. You can have an LLM help process meeting transcriptions, summarize for a doctor, for example. Or you can have an LLM read through an insurance claim and figure out what information is in there that needs to be processed correctly and stuff like that. But you can only do that if the LLM can have access to that data. And if you're using a consumer application like ChatGPT or something, you're sitting at home and you're using it. You're sending all the text that you have over to some server. It's being processed and then sent back to you. And that's not going to work if you have healthcare data. You have to have that model be where the data is. And so we'll deploy privately in their environment. Nobody sees that data but the customer. Like we don't see it, we don't have access to it. It stays private. And yet they can still have an LLM help them with administration and things like that.
Got it. And what's your thought on this whole—like you mentioned insurance, you mentioned healthcare—how do you pick the verticals to go after? Do you kind of go out there and whoever answers the phone, you go build for them? Because you have healthcare, you have insurance, and RBC is a massive customer. Fin, we're all over the place.
We're all over the place. And one of the things that joins them all together is that we're in places where a lot of our customers have data that needs to stay private. They have places where there's regulations, like regulated industries is a place where we can add a lot of value.
Got it. And what about the startups that are raising big rounds, maybe not as big as your rounds, but big rounds that are doing this just for that vertical? Are they using you guys as a model and they're verticalizing? I think of us—we're doing it in HR. There's companies doing this in insurance. You probably know many more. What's your take on them? Are they wasting their time?
No, I don't think they're wasting their time. I think there's useful things that can be built around there. The thing that we have as a model maker—because we make the model and again there's like 11 companies that can do this—a, we get to choose how to deploy it. So we can deploy it privately if we want. B, we know entirely what goes into making it. So the customers, the customers that have regulated industries, they also care about the provenance, they care about security. They want to know what's going on in that model. And C, we can customize that model to the customer if we need to.
If they say, hey, like Fujitsu is a customer of ours, they needed a model of a particular size that was better at Japanese, and we can do that. We can take our model, change the pre-training data—not just fine-tune it, but actually change the way we pre-train it to get it to be better at Japanese and on the footprint that they need—and then we can ship it into their environment to use it. So for the companies that are doing something like taking an LLM as a backbone, using that API and building an offering for a particular vertical, they can surely make something useful, but they can still only have access to the data that their customers are okay sharing with wherever that LLM is hosted. And there's some places where that's fine—like HR, I think that's not an issue. But other places where that's not totally fine.
You mentioned—you don't need to list them all—but under 20 companies making models. All of them massive funding rounds. You guys just—maybe backing up—you guys just made two acquisitions in Germany?
Yeah. Germany and Canada.
Germany and Canada. Walk me through those for those that maybe don't know that yet. What was the rationale around that? Why'd you do it?
Yeah. One is Reliant AI. We're super excited about them. And that team's out of Canada and Germany. They're healthcare, life sciences. And as I mean, as I've been using that as example, so you can probably see why having a team that has that expertise and those brilliant researchers and those connections working with life sciences, pharmaceutical, healthcare stuff already, is a natural fit for us. And you'd be working with them to train models in that category. Yeah. And then tailor our models specifically to administrative processes within healthcare.
Now, there's some companies that they kind of do that—they would look at you as a customer and they go out and find companies like a Scale AI. What—do you work with any companies like that? What's your thought on that?
You mean for data creation? Yeah, we've worked with a handful of data creation companies over the years. I think all of the LLM makers do. It's changed a lot every few years. And one of the things that has been beneficial for us is having our own in-house annotation team.
So we have a bunch of in-house annotators. And I think that's one of the reasons why often if you use a lot of the Chinese LLMs, they often feel very similar. They often—you can kind of tell the provenance, like where they came from. And that's because they're training on synthetic data from each other.
And we use a lot of synthetic data, but we also still have a big in-house annotation team that creates data. And so we're able to use that to make sure that the model is particularly good at the things we care about. Inline citation is a big thing. We've always trained the model so that if you give it a document and you ask it a question, it will answer the question and tell you specifically where it got that information. And that's because you really need that if you're going to use it in a business setting, in an enterprise setting—versus like if you're just doing creative writing or you're just having an idle chat, it doesn't matter. But if you're trying to use it in a business setting, in an enterprise setting, you really need that.
So we have our annotators spend a whole bunch of time making sure that it's good at that. So yeah, we do a lot with in-house annotation, but we do use data from people who make data as well.
Yeah. Okay. And the company in Germany?
The company in Germany is Alphα. Great company. They've been doing a lot of research for a long time. We're really excited about Europe and about Germany and like being there.
Yeah, it's a really interesting moment right now.
I think the world is really looking around at AI sovereignty and thinking, hey, this is a really foundational technology. How can we make sure that this technology is being deployed and built in a way that empowers the people that are using it, the companies that are using it, in the countries where it's deployed. And especially these days, people are interested in having access to tech that doesn't come from America. People are interested in having access to tech that doesn't come from China. Because they see the ways in which using tech from there is building in—there's an off switch, right? Like if America wants to shut off your access to tech, they can do it.
So us joining up with Alphα is really exciting because it allows us to double down on what they've built in Germany and allows us to deepen our connections there, deepen our roots there, keep building out of Canada and Europe, to bring the tech to the rest of the world in a way that is sovereignty-enabling.
Yeah. And very much enterprise-focused. You guys—like you don't need to go very far when you look you guys up. Very enterprise-focused. There's no slant strategy to go consumer?
And what's your take on the overall kind of competitive landscape in that—again, about let's call it 15 to 20 folks doing this, teams, big teams, heavily funded teams. Is there like consolidation? Will that happen? Will all these folks be around? I know you've been kind of not controversial, but you're like, hey, I get the bubble thing, but this isn't like ultra bubble.
Yeah, it's definitely a weird time. Is there going to be consolidation? Yeah, I think there's probably going to be one company that's going to own the consumer. Maybe one or two companies that's going to own the consumer.
So that's like winner-take-all, the consumer?
Maybe a few, maybe a few take all. Yeah. And the reality of that is just that it's so expensive to create these models, and even now all the consumer companies—the economics are very strange.
Right. The economics are pretty weird. The margins are off. The margins are odd. You gotta spend—especially if you're using a really big model. Kind of what's happening is all the consumer companies, they're all racing to have the best model. The way to have the best model is to make the biggest one. That's pretty well established in the literature—to make a really big one that'll be generically best and you'll get like a point better on some leaderboard or something. This costs a ton.
But then those models are really expensive to run. And so if you have somebody paying $20 a month or something, they're probably using way more than $20 a month of your compute cost. And that means what's happened so far is the consumer companies have mostly been burning through a whole bunch of money to keep people using their tech as opposed to swapping over to somebody else's.
I think there's a real good place for LLMs within consumer applications. Like as an individual, as a consumer, I use it. There are times when I'm like, oh, this is a perfect task for a consumer application. It doesn't happen as often as my work. And that's just because there's more velocity in my work life—I'm trying to automate as much as possible in my life as you can. Whereas in my personal life and like music as an example, I'm not trying to write lyrics faster.
Dinner with your family, out with your partner.
I'm trying to hang out. I'm trying to be slow. So I still like it—like Perplexity is nice. It's nice to have that as an interface for asking idle questions. But when I'm using it at work, I don't need a question machine for the internet. I need a question machine for all of our internal data. So I need something that has access to our stuff too.
But yeah, so I think there are applications for this, but I don't think the cost has been sorted out yet. And I think right now is an unstable moment in which everybody's kind of vying and a handful of those are going to work. You know, you've also seen a lot of companies spend a huge amount of compute and then not end up with a good model, right? Like SpaceX—xAI is a pretty interesting example right now. They just said they're renting all their compute to Anthropic.
And I don't know if that isn't a white flag of, hey, we're not in the model business anymore. I don't know what is.
So they're pretty much going—I was reading about this—pretty much say, hey, we've gotten too much of this, we need to sublease it.
SpaceX is a strange one. That filing is not particularly confidence-boosting. So I don't really know what they're doing, but they're a company that ostensibly is trying to make models. If you're trying to make models, you need access to compute. Like we're always trying to get more access to compute. And they've decided instead of using that for their own models, they're going to rent them to somebody else. I think that's about as clear as you can go to saying, hey, we're actually not going to make the models. This is not our deal because it's really challenging to do and we tried to do it and it didn't work.
And I think you might see similar stuff like that happen with the other consumer players. I don't know what that's going to look like over the next few years. I think Google's in a really good position. They already have the internet infrastructure. They already have compute. They already actually make a whole bunch of money, right? They have pretty good balance sheets. So I wouldn't be surprised if Google ends up being the one that kind of wins that space.
And meanwhile, there's still nowhere near enough LLM in work. Like in many places, there's way too much LLM in personal lives. People are already quite reasonably frustrated with it. But in their work life, if you take a look, it's like you're still doing pretty mundane things that LLMs are really good at doing and you should not be doing. And instead, you should be strategizing, you should be thinking. So we just stay focused on that.
Yeah. Dope. And then what's your take—there's a company and we kind of talked about this briefly, but I just had a really cool dude on the show. Jack Newton—I might get this wrong—no, I'll get this right. 500 million ARR. He's been at it 18 years. Clio, he's CEO co-founder of Clio.
So this AI wave's come along and legal—one area that AI has been massively applicable towards. You have companies like Spellbook, Harvey, Clio. So they're kind of going into law firms, large law, small and medium. They'll find their way, but they're going into law firms and kind of automating a lot of lawyers' work. And these are—so how do you guys go into a company and automate work? Like, what do you do for RBC where it's like this is true value now—I can be a banker and now I can give financial advice. I don't need to do Excel all day.
Totally. So we go in with models, and we go in with an agentic framework for using them, and we go in with a search stack. So the models are the things—like the generative models, the things that write text—that's the thing you're talking to. The agentic framework is the thing that, you know, the chat interface layer, the app layer. And then the search stack is the thing that allows the model to say, hey, I need to find a relevant document to this. Where is that? So that's like embedding models, rerank models. And we deploy all of that privately.
So the end result is that you take somebody in capital markets at RBC, for example. Their job used to be—someone used to, they're going to track X number of companies and every time they release quarterly earnings they're going to read through them, come up with a strategy, write a memo, share that with somebody. That's like a big process. Now they have access to this. It can help them do that. It can write drafts for them that they can validate and now they can track more companies and they can suggest with greater accuracy the moves they think their client should make.
So it's just like helping take off the burden of that cognitive load of boring tasks and allowing them to instead think more critically about where they think the market's going to be.
I mean, you may be biased because you're building it and leading one of the largest companies doing it, but I think that's the place to be. Like you're B2B—just from being a business owner, building a business, selling into companies—it's—
Well, it's a lot—I think it's the place to be, too. It makes the margins look a lot better because you deploy privately, so we're not paying for that inference compute. Way stickier. Healthier revenue. It means we're focused on adding value now. Like we're not just saying, hey, pay us $20 a month so that when AGI comes you'll have access—we're instead like, hey, is that useful for you? Is it worth it for you right now? Is it worth it for you today?
Yeah, it's gotta be. Sales cycles are long.
Sales cycles are long. Yeah. It's like another year. But it's just been a lot. I feel like the incentives are aligned a little better and it's nice.
Because we're also not doing like—we're not doing ads. We don't have anything—when I see people talking about putting ads into LLMs—
Yeah. Well, that's what Perplexity is doing.
Yeah. That's tough. And consumers are so fickle.
That's true, they are. I still use Perplexity though. So it hasn't bothered me enough to make me switch yet, but—
Yeah, it's like really fickle. My first company—
This is kind of funny. Embarrassing, I don't know, like looking at what you've done. My first company was a dog walking app.
Okay. So like we competed with like rover.com, wagwalking.com. We were called GoFetch.ca. Uber for dog walking, Airbnb for dogsitting. Doggy daycare was like a big subscription business—people would subscribe weekly for a dog. But it's consumer and it's tricky, man. It's like so brutal. You're running the ads, you gotta pay 300 bucks for a consumer. They only get a dog walk twice and then they take it off your app and text the walker instead.
It's the same thing here.
The business I started now, you know, we're global payroll, right? It's a B2B business and we're selling like AI into HR companies and it's a way better business. So I look at what you guys are doing versus like very consumer-slanted AI. It's like this is where you want to be.
Yeah. I like the incentives a little more. Like I don't—I like when you're working with a business and you're B2B, they need something from you. And you have it and you can talk about how much it's worth for them to pay you and how much it's worth for you to give it to them. And then once you do that, you keep checking in, make sure it's still going well. But like everybody's kind of aligned. Whereas when you're consumer, you're undercutting the price of things to get people in. You're figuring out how to monetize their attention.
You end up especially with LLMs, you end up with these weird things where you're trying to optimize for people to stay engaged so that you can feed them more ads or something. So then you end up training the model to ask like, you know, inane follow-up questions because somebody wanted a metric that says, well, how many conversations do they have? And it's just—our model doesn't do that. You ask our model to do something, it does it. People never say, would you like me to follow up? Here's an answer. It does a thing. That's it.
So I think in like—I feel that in lots of tech, but in LLMs in particular, I find the incentives have a weird way of working their way into the training of the model. And that's upsetting to me.
Yeah. Yeah. Yeah. What's your— I guess backing up—
How do you manage your day? Like, are you filling it up with meetings? I mean, you were early on time to this. You seem like a pretty balanced, level-headed guy. What's the day like? Do you have a daily dashboard? Do you do one-on-ones? Are you looking at burn every day? Because there will be a lot of founders that crush it financially that will never get to these sorts of numbers. They'll raise a series A, series B, and they might sell a company for 200, 300 million. But they won't raise 1.5 billion, 7 billion valuation. So how do you actually manage the day?
Super accomplished. I'm 33.
I don't really know. My wife tells me. Yeah. But like, how do you manage your day?
Yeah, I like to speak to people. So, I don't write a lot. I spend more time talking to people. I don't send big emails. I write stuff on Slack. And then occasionally when there is a document that needs writing, I will write a long document, but that's only when there's some very specific thing I want to communicate to many people. I try to get it into a page or two. But I spend a lot of time talking to people. I do a lot of one-on-ones. I do a lot of group meetings with my reports to talk about things. I do a lot of work with customers, so I talk to them and then I follow whatever schedule they need me to be on. I try to book a lot of time to do nothing. I try to book time for coding or ideation or thinking. So I have blocks in my calendar that just says 'coding' and I used to code during that time. Now I sometimes code things or sometimes just think about things or try to have blocks of time that are idle.
And what does that look like? Are you in an office? You're not being bugged?
Yeah. At Cohere, we don't have individual offices. So I've got a desk there. I don't have an office.
I've been to the company's office.
There you go. Yeah. And it's a really nice space by the way. Yeah. Really what you guys have done is old Slack.
Yeah, one of the floors is old Slack.
So I'll be there, or I work from home like one day of the week, or I'll be traveling for work or on tour where I'm working in the back of the bus or in some hotel or somewhere. I think the thing that I find useful, I don't really mind context switching at all. I know other people need a really long time to get into something and I don't really mind going from one thing immediately to the next thing. It's never bothered me. I've always found it actually if I want to—it's often useful. I find if I context switch to something and then think about something completely unrelated for a bit, when I come back to the problem, whatever I was thinking about, progress will have been made. I'll come back to it with new eyes. So I think that's one of the reasons why I like the life I lead right now. It's nice to have a meeting, spend the day doing work, leave briefly to think about something else, come back, work, leave, sing some songs, play a show, come back, think about something. I actually like that switching.
And when you're doing thinking time, is it pen and pad or are you just puttering, wandering?
No. I'm often writing little bits of code or trying things out or reading things.
So like in the middle of the day, you'll read stuff?
