Chef Aidan Gomez. Aidan, welcome to Hon. It's a pleasure to have you. Most people here may have first heard of you just a few weeks ago. Cohere made headlines by buying the German AI startup Aleph Alpha, and our federal minister for digital affairs who joins us tomorrow personally championed that deal. This alone will make a lot of people here very curious about who you actually are. So let's start with you. I think the most well-known anecdote about you is probably that you grew up without internet in a cabin in the woods an hour and a half outside of Toronto. I heard you hated it at the time. Do you miss it today?
I do miss it. Yeah. I think I was super lucky to grow up disconnected from technology and I think it's actually what inspired my drive to contribute to it because when something feels just out of reach, like everyone else has it but you don't, there's nothing more motivating than that.
It was also just like a really, yeah, like you said, I didn't appreciate it when I was in it. You know, having no internet and being in the middle of the woods. My driveway to my house was like a kilometer and a half long. I couldn't walk to see my friends. But now I crave that. Like I live in London. I live in Soho, which is like the middle of London. And every year I have to go back to the woods and I'm from Canada and you know the maple syrup season starts around March, February. So we go and tap the trees, make maple syrup. It's a really special place for me.
In Germany, we constantly debate whether we fall behind internationally when our children don't have tablets in kindergarten and learn to use ChatGPT in primary school at the latest. Would you say that you're the living proof that that's nonsense?
Well, kindergarten might be early for iPads. I would also want to let the child's brain develop a bit on its own before introducing that. I don't know if I'm evidence that it's a good idea, but I'm evidence that not having that much access to tech at least doesn't hurt you that much.
Mhm. After school, you ended up at the University of Toronto. That's where Geoffrey Hinton was a professor, a famous AI researcher, and later a Nobel Prize laureate. You stumbled upon his work almost by accident, as far as I know. Is it true that you pitched him research ideas without even knowing that he was a legend?
Yeah. Yeah, he was just, I was reading papers on AI and he was one of the authors of one of these papers and his email was there and so I just, you know, now he's a Nobel Prize winner and super famous but I just shot him an email and he actually replied to me.
So I think that really speaks to his character that some random undergrad with a bad research idea can shoot him an email and he actually took the time to respond.
Yeah, maybe you were not so random at all. By the age of 20, you got an internship at Google and became a co-author of one of the most famous AI papers of all time. Please tell us in simple terms, why was 'Attention Is All You Need' such a shift and when did you recognize it?
Well, I was co-author with this guy right in the front, Jakob, who's an incredible German computer scientist and was a fantastic mentor.
And he will be on stage tomorrow.
Sorry. He will be on stage tomorrow. He'll be on stage tomorrow. You're invited to see his talk, listen to him.
Um yeah, what was the question?
It was, so why was 'Attention Is All You Need' such a shift for the AI landscape?
So I think the special thing about 'Attention Is All You Need' was its ability to scale. It was built so that, you know, back in the day we used to train everything on a single GPU and the GPUs we had back then were tiny. We could like maybe fit millions of parameters into these cards. And when we were doing the work at Google, we wanted to scale that up to like training on 16 GPUs or like, you know, dozens of GPUs. And that was a huge shift.
So we designed the architecture in a way that was very compatible with that scaling, with no real sense, or at least certainly I didn't have any real sense of what was going to come. And then it turned out that because we had designed it for scaling, training on multiple chips instead of just one, as the world started to realize that scale and the immensity, the size of these models mattered so much, it became the go-to architecture because it was the one designed for scale. And I think maybe Jakob will agree. What's perhaps most shocking coming up on next year, our 10-year anniversary of the model, is that we're still using the transformer. We're stuck on it, you know. And I think as scientists, we crave, like we don't hold it too dear, you know. We don't want our creation to live and succeed forever. We want humanity to beat it. We want to succeed. We want others to succeed on top of our work. And so I'm very excited for what's next. We unlocked the scaling era. I think maybe the next era is efficiency. Humans are so remarkable in our ability to learn in such a short amount of time with such limited data. These models do not have limited, they need tons and tons of data. And so maybe the next generation of architecture or training methods will come about as a product of finding efficiency, finding data efficiency in these models instead of just scale, make them bigger.
I mean you started here to drive the research forward. What you do today is considered an alternative to OpenAI and Anthropic. You built AI and software that businesses can use to deploy chatbots and agents and automate processes. So what sets you apart and what's the next thing you will do?
So the big thing that sets Cohere apart is that we're focused on critical industries. So think the bedrock of the economy, telecom, energy, the grid, healthcare, financial services, defense, government. And the reason why we're so focused there is that our product is very tailored to privacy and security. So instead of asking our customers to send their data to us and we host it on some big cloud, some hyperscaler, instead what we do is we send our infrastructure, our models and our product to them to sit on their infrastructure. So I like to say that while we can serve on the cloud, etc., if you want to use an American hyperscaler, please do, it works, but we can also serve a kilometer under the surface of the ocean in a submarine. And so these varying degrees of privacy can be completely achieved with us.
