We have Aidan Gomez, the co-founder and CEO of Cohere.
The ability to adapt to those environments and change yourself and the company to fit, that's been the only way that we've survived. So getting into Google the first time was like total fluke, and I actually shouldn't have been an intern at Google, and I found that out on my last day. I'd be sitting at the gym and in between sets I'd have my stack of papers with my pencil and just like writing notes. So every moment of free time was dedicated to studying.
Solutions integrators like Accenture and Deloitte, they are going to make a killing because it's like a total refactor of the tech stack. You may be seeing the return is going to be AI, but your investors may have more short-term goals.
Well, I would not say that. I think we're definitely in the point where investors have just said no, like, sorry, we're not going to give you that extra X billion dollars. If I could capture every thought in your head for your entire, you know, hopefully 100-year lifespan, that would be my... It's cool again to talk about enterprise products. I'm excited for that.
Ladies and gentlemen, we have an exciting show for you here today. This is somebody I have known for a while but are happy to have in the house with us here in person. We have Aidan Gomez, the co-founder and CEO of Cohere, one of the leading companies in all things AI. And Aidan, I want to have this conversation with you for a long time, so thank you for coming over.
Yeah, thank you for having me. I'm sorry for being so late, braving London traffic, the holidays.
We want to make this awesome. No, thank you. We're going to get a lot into Cohere, what you guys do, and the state of AI, the state of enterprises adopting AI. But when we first met, the thing which I think we really bonded on and what really struck me was your childhood and upbringing in Canada, because a bit unconventional but also something a lot of people watching this can probably connect to. So tell us a little bit about where you grew up and how you got into computers.
Yeah, so I grew up in Canada in rural Ontario on a pretty big plot of land that was mostly covered in forests. So it was a very Canadian upbringing. Both my parents are immigrants to Canada, but still every spring we would tap the maple trees, hang up the buckets, collect the sap, boil it down. So yeah, it was a super in-touch-with-nature upbringing. It came with challenges. One of those was that it was basically impossible to get high-speed internet. It still is, actually. We have Starlink now, which is way better than anything that came before, because initially it was just dial-up, right? We had a telephone wire that we had to run to the house, and we were doing dial-up. Then we went to whatever the previous gen of satellite internet was, and if it was cloudy or like there was a storm, fully gone. So yeah, it was a little bit tricky, but I think that scarcity or lack of access really made it fascinating for me. It made it really gripping. There was this thing that was just out of reach. I could experience it when I went over to a friend's house, but at home you didn't have access to it.
So how did you get into computers, internet, everything else?
Yeah, so we had computers and like video games at home. I couldn't connect them to the internet effectively. I had dial-up, and so I started getting really interested in how to make it faster. And so how to improve my computer, that led me to teaching myself how to code. Then I started making websites for myself, and yeah, it was just sort of like a deep curiosity and fascination with this technology that was just not working for me but was working for everyone else.
Nice, wow, okay. And I think it's very similar to our childhoods in a way that we're a little bit older than you, where the lack of actual working internet for long periods of time, in our case, running up the phone bill. So you had a ticking meter going up every single moment I was online, and that gave you constraints that made you have to be more efficient, you had to do things faster. Kids these days have no idea. They don't know what dial-up modem was like. And the sound of dial-up, it still keeps me up at night sometimes. This is super recent, this is like I probably shouldn't be telling the story, but a few years ago my younger brother, we still don't have proper internet, right? So he's using his mobile network and just racking up massive bills. My dad giving him shit. Yeah, so it's still alive and well, don't worry. Think before you download, kids.
Okay, so fast forwarding, and you're in the middle of Canada by the way, Maple, Maple, it's so Canadian, it's as if you're an ad for the country. All of a sudden you wind up with arguably or one of the Godfathers of AI, recent Nobel Prize winner Jeff Hinton. So talk to us about your journey from being there in the Canadian wilderness to now being in the forefront of AI.
Yeah, I would say that was a lot of luck. I just happened to be born, my closest university was U of T, which is where Jeff taught. And so of course when I became a computer science student at U of T, I was raised in an ecosystem that just lived and breathed AI. So I sort of say that I was raised into AI from scratch. But that's not enough, right? Like if you have a celebrity professor at your university, there's many universities that have that. What we had at U of T was Jeff was so open and welcoming to the student body. He would always have undergrad interns. He taught for ages. He wasn't teaching when I got there, but he really fostered a doors-open policy with the school. And so when I was starting to learn neural networks, I just emailed him. I emailed him and I said, hey, you know, your ReLU activation function, I think for some, I don't know why I said this, but I think it should be sinusoidal instead of monotonic. And he wrote back to me this detailed description of why it should be monotonic in his mind. And yeah, it was just really welcoming and open.
Give us a fun Jeff Hinton anecdote.
There's a lot. So when I was interning for him in Toronto, I started at Google in the Bay Area and then I went back to Toronto to finish my undergrad. Either Jeff reached out to me or I reached out to Jeff, and he said, hey, come intern with me at Google Toronto. And so I was in Toronto, I was interning for him, and he really wanted me to do this one project, which was called, I've even forgotten the name now, but something to do with distillation, this idea of class-wise distillation. And we were going to use some recent results in information theory to inform a new version of distillation. But I was obsessed with the lottery ticket hypothesis, which was this idea that inside a neural network, in all of these billions and billions of parameters, there's actually a tiny number that actually count. You can throw away 99.9% of the weights in a model, and there's really like this little lottery ticket in there that defines the entire behavior of the model. So I got super interested in that and I got distracted and I was working on this other thing. And so he was constantly just giving me shit for getting distracted, working on the wrong thing, and trying to steer me back to the thing that he wanted to do. And so in the end, at the end of that internship, I felt bad, I was hassling him so much, I was going off doing my own thing. And so I carved him out of a piece of wood that I chopped down on the property at my home, a little diagram in the wood of the research that he wanted me to do, the structure of it. And then later I found out that before he went into computer science, he was training to be a carpenter. And so it was so perfect. That's awesome. But yeah, I don't know if he still has my little gift, but that was my 'I'm so sorry, instead of doing this you could have just done the research'.
I was right, you know, you should have been a great researcher instead of building these silly companies. And you know, I always, when I talk to one of my lecturers, they always give me that. Okay, so if I'm thinking about the montage sequence movie of your childhood, I have you chopping down wood in Canada, very politely, in the wilderness.
Conserving bandwidth, learning to code with a lot of gaming. Then you're winding up with essentially this Gandalf-like figure, but the right person, on AI before AI was really hard, right?
But I think the next important milestone would be your participation in, I'm not even going to use the word arguably, I would say maybe the most important academic paper of the last decade, who knows, maybe of our lifetime, which is the 'Attention Is All You Need' paper. So talk to us about that. How did you wind up there? What was your memory of the work leading up to it and when it came out?
Yeah, so getting into Google the first time was like total fluke, and I actually shouldn't have been an intern at Google. And I found that out on my last day when Lukasz, my manager, he was starting a goodbye party for me and he's saying, okay everyone, Aidan's headed back to school. Aidan, what year are you in in your PhD? How much more have you got to go? And I was like, I'm going back to third year undergrad. And he was like, what? We don't hire undergrads. How did it happen? Did they just forget or vet you? And I was like, I'm a big emailer, cold emailer. I was emailing a lot of my favorite paper authors. And so I was sending emails to loads of folks at DeepMind and Google Brain and all the places where the papers that I was reading, because that's how I learned. I just printed out stacks and stacks of papers. And initially to read a single paper it would take me like two weeks, because I would just hit a word that I don't know, and then I'd have to look up that paper and read that paper first. And so it was such a slow process. And I'd be sitting at the gym and in between sets I'd have my stack of papers with my pencil and just like writing notes. And so every moment of free time was dedicated to studying, learning stuff. But as a kid, I viewed AI researchers as like gods. And so their emails were on the paper. You could email God. It was right there. And so I would just say like, hey, I just read your paper on this, I have this follow-up idea, what if we did do do do do. And eventually a few of them responded, and some folks at Google Brain. And so they invited me to interview. My resume got passed around, and then I found Lukasz. And Lukasz had such a cool project idea. It wasn't the Transformer. Instead, Lukasz, he's always running like a decade ahead of where he should be. The research that he does today, it'll actually be ready, the world will be ready for it in a decade. But what he was doing was we called it 'One Model to Learn Them All', and his idea was we're going to model every single dataset that humans have created. And so fully multimodal, like text to image, image to text, audio to text, everything, all the different combinations. And we're just going to throw all the data into a model to process it. And now the world's kind of getting to that, right? Like now we have Omni, we have these models. Exactly. So you know, it's caught up. But yeah, that was the project that I joined to do.
I want to make sure everyone gets the most important piece of what you just said, which is no, no, no, not even that. There is this amazing imagery of Aidan workouts, academic papers, nothing else, right? Sets and papers, nothing else. That's all you need. All you... I mean other people at the gym are going, how much is he working out? You know, usually you take notes. Nerd. I don't know what the right... it's like, it's cracked, it's very kino. Actually, it's a good question. So I'm sure a lot of people watching this are thinking of how do I reach you, and they're probably reading papers that you have written or people you work with have written. What is a good way to attract your attention?
I get a lot of cold emails, and I wish I was that guy who was passing it forward and responding to everyone. Unfortunately, that is not... I do respond. The times I respond, usually it's someone who has an idea that they're testing with me. They're not asking for my time, they don't want something from me. They just come to me with an idea, they say, what do you think about X? And those I usually engage with. Or they say, Aidan, I'm in a situation that I need help with. I've heard that you've been in this, maybe it's starting a company, maybe it's, I don't know, I'm from Canada and I'm trying to blah blah blah. If it's something that I feel I have a unique ability to help answer or help them get through, then I'll be much more willing to put in the energy to help. If it's something that anyone could answer for them, yeah.
It's kind of like, what's the secret to a good cold email, either ones you received or ones you've written?
You know, I don't know. Maybe for the ones I've written, I never ever was cold emailing to ask for something. It was always just to talk to the person, to share an idea, test their intuition against one of my ideas, basically testing my ideas. That's usually what I reached out for. It was never to get a job, it was never to become an intern.
Got it. Yeah, I find it very hard when somebody says I want 15, 20 minutes of your time, because I wish I could do it, but there's so many of those, it's very, very hard. And sometimes somebody really stands out and you do it, but usually you're right. I think when somebody sends you an app or something written, then you can very, you can respond and you can engage and it's more fun. Well, there you go, kids. Between your sets at the gym, shoot me an email and get back to working out. Okay, so let's get to the meat of it all. So actually, wait, before that, I want to talk about 'Attention Is All You Need', right? So you talk about getting to Google, what was your memory of the work leading up to the paper and of the paper itself?
So from my perspective, Lukasz and I were working on this framework for training models called Tensor2Tensor, and the whole point was every modality, super generic, right? So you could plug it into literally any sort of data, any sort of model, convolutional, autoregressive, whatever. So we were working on this framework, and at the same time I was sitting next to Noam Shazeer at the office, and so we were kind of aware of what he was doing. We invited him to start building, it didn't have a name at that point, but start building his stuff in our system. And so that led us towards attention-based autoregressive models. And then at the same time there was a third paper. So there was 'One Model to Learn Them All', there was 'Attention Is All You Need', and then there was 'Depthwise Separable Convolutional' blah blah blah, which was much, much worse branding. That one never took off for some reason. I think I know why. But it was with François Chollet. And so that again was like, if you actually look at that paper now, if you go back and read it, very similar to the Transformer, with convolutions instead of with attention. And the results were really, really close to each other. Oh wow. If you look, because we benchmarked them on the same problem, the same translation problem, and they both performed kind of close.
Got it. François is a legend. I think he just left Google.
And it struck me as how important he had an impact on Google, because Google actually wrote a whole blog post thanking him for his contributions, which so rarely happens. Okay, so I think let's get to the bit where you're working now. So you are now co-founder and CEO of Cohere. For those ordinaries who are not familiar, give us the short summary of what Cohere is and what you folks are trying to accomplish.
Yeah, we build large language models and we focus on enterprise. And so making them easy to adopt, making sure the enterprises don't get locked in, and making them comfortable. So we're one of the few that hasn't taken one of these cloud recycling deals, cloud puts in 5 billion, you pay it back over two years type deal. We're one of the few that hasn't done that. And so we're available on all clouds, everywhere, as well as on-prem. So for anyone who's super paranoid about their data, we're the best choice, right? And the second piece is customizability. And so because we build the models from scratch, we have a lot of places where we can intervene. We could do a custom pre-training, we could do post-training, SFT, RLHF, and so we can offer a lot of customization, much more than just take the final checkpoint and try to fine-tune it. We can do something much more substantial and significant. So that's sort of what Cohere does.
Now, I think unpacking that, there's a few, if I think about say the flowchart of the top AI companies out there, and you folks are one of them, there are a few choices, and you've gone on a certain choice. For example, there's consumer and enterprise, and obviously some folks do both, but you've gone down the enterprise part of the tree. And as you said, OpenAI has a partnership with Microsoft, Anthropic with Amazon, you are cloud agnostic. So why these particular choices?
Yeah, so I think for market, for our customers, not getting locked into one ecosystem is really, really important, right? If one of your largest expenses as a company is cloud, and you're spending a billion dollars a year, you really want to make sure that you don't get squeezed on that line, because it's a big one. So you need optionality. You should very much be reticent to buy something that's only available on one cloud. And this has been a really important buying decision for data, for databases. Getting locked in to one has been hugely, hugely dangerous. And I think similarly in AI, this is going to be as key an infrastructural piece of your tech stack that you don't want to get trapped into one of these ecosystems.
And why enterprise versus consumer? You know, when the whole ChatGPT AI wave started, it was all consumer. Consumer was very sexy, very hot. And now I feel like the tides are turning towards enterprise. Enterprise is sort of where people are seeing value. But why make the conscious choice for Cohere to look at enterprise as a market?
That was always the way that we wanted to have value in the world. So the way that we wanted to create value was by enabling everyone else to adopt the technology. Like we saw that this is going to become a foundational piece of the infrastructure of the tech stack. There's going to be a resurfacing of all products because we have an interface now that we just didn't have before. We have language as an interface. And so because of that, that was the direction that we wanted to go, to basically empower everyone else to adopt it and facilitate that change. We never even dipped our toes on the consumer.
