Hey, Bilawal here. Before we start the show, I have a quick favor to ask. If you're enjoying the TED AI Show, please take a moment to rate and leave a comment in your podcast app. Which episodes have you loved and what topics do you want to hear more of? Your feedback helps us shape the show to satisfy your curiosity, bring in amazing guests, and give you the best experience possible.
So I'm a millennial, and when I was growing up, there was a toy that had us in a chokehold. Not Tickle Me Elmo, I'm talking about Tamagotchi. You know, these were those little digital pets that lived on egg-shaped key rings and never let you sleep. They were super needy, and if you didn't feed them long enough, you'd wake up to find a gravestone where your pet used to be, with a little ghost floating next to it. Personally, I was very happy to consign the Tamagotchi to the annals of history. But believe it or not, they've got a weird connection to some pretty cool AI stuff happening today.
I'm Bilawal Sidhu, and this is the TED AI Show, where we figure out how to live and thrive in a world where AI is changing everything.
Hi, I'm Bilawal Sidhu, host of TED's newest podcast, the TED AI Show, where I speak with the world's leading experts, artists, and journalists to help you live and thrive in a world where AI is changing everything. I'm stoked to be working with IBM, our official sponsor for this episode. Now, the path from GenAI pilots to real-world deployments is often filled with roadblocks, such as barriers to free data flow. But what if I told you there's a way to deploy AI wherever your data lives? With Watson X, you can deploy AI models across any environment: above the clouds, helping pilots navigate flights; on lots of clouds, helping employees automate tasks; on-prem, so designers can access proprietary data; and on the edge, so remote bank tellers can assist customers. Watson X helps you deploy AI wherever you need it, so you can take your business wherever it needs to go. Learn more at ibm.com slash X and start infusing intelligence where you need it the most.
Something about the way we're working just isn't working. When you're caught up in complex pay requirements or distracted by scheduling staff in multiple time zones, you're thinking about the complexity of working in Monterrey while based in Montreal. You're not doing the work you're meant to do. But with Dayforce, you get HR, pay, time, talent, and analytics all in one global people platform, so you can do the work you're meant to do. Visit dayforce.com slash dothework to learn more.
If you make decks at work, you should make the switch to Canva Presentations. Canva presentations might be the most visually impressive presentations you'll ever use. Start with a designer-made template and add images, videos, and graphics from Canva's massive media library. You can even add smooth slide transitions and data animations to really set your presentation apart. You'll love the presentations you can quickly and easily design with Canva, and your audience will too. Love your work with Canva presentations at canva.com.
So there's a company called Hugging Face, like the emoji smiling with outstretched hands. Back in 2016, the co-founders Julien Chaumond, Thomas Wolf, and Clement Delangue had this idea: they could use AI to create an online Tamagotchi for teenagers, a chatbot, a friend that was always around for them, a constant companion that responded to their input. While they were tinkering with their AI chatbot, they started thinking about the actual infrastructure underpinning their product. Like, what goes into an AI chatbot? You need natural language processing, right? Which means you're going to need data, massive amounts of data. You're going to need a library of pre-trained models to mediate the interactions between computers and human language. Basically, you need a lot of complex parts just to run a simple chatbot. And right around this time, in 2017, Google dropped their seminal 'Attention Is All You Need' paper, which introduced the world to Transformers. So while Hugging Face was tinkering with their fledgling product, they had a lightbulb moment. This software stack they were creating to build and improve their chatbot was perfect for the shift to the Transformer architecture, and they wanted to share it with others. So they shared their code, their best practices, and their findings with the public. And each time they shared something, they got such a positive response from their followers and users that they didn't want to stop. They decided to build a public library of AI software so anyone, anywhere, building anything with AI could access the information they needed. Next thing you know, Hugging Face made a full pivot away from the chatbots and put all their resources into this mission. And today, their site is the go-to resource for all sorts of folks interested in AI, from software engineers to weekend coders to computer science students. I use it myself every single week. And Hugging Face includes so much more than code. It's a common space for anyone using, building with, or even just interested in AI to learn more and do more with this amazing tech. Want to create your own comics? Someone made a comic book generator and shared it on Hugging Face. Looking to include images from other countries in your datasets? Yep, they've got that too. Want to share the latest release from IBM and NASA on AI-enhanced climate predictions? There's a message board for that as well. It's pretty wild, right? Of course, not everyone wants to go open source, and yeah, there are definitely advantages to keeping your cards close to the vest. But I spoke with Thomas Wolf about why open source works for Hugging Face and why it matters, not just for the major players in the industry, but for the rest of us too.
Well, Thomas, welcome to the show.
It's a pleasure to be here. Thanks for having me.
Cool. All right, so let's talk about your origin story. I obviously want to chat about Hugging Face, but I'd actually love for you to start with your personal background, because you are what I would call a triple threat. You're an engineer, you're a scientist, and you're a lawyer. Could you chat a bit about who you are and your journey into artificial intelligence?
So I grew up in France, and that's also where I met one of my co-founders, Julien Chaumond. We were playing together in a band, a rock band in Paris during our engineering school. And after engineering school, I went to do research in quantum physics, statistical physics, basically working on superconducting materials. I really liked that, it was really nice, but it was moving a bit too slowly for me. And so at the end of a postdoc after my PhD, I was like, I need to do something else. And I really liked writing at this time, and so one of my friends was a lawyer and told me, if you do law, basically you write a lot. So I was like, okay, why not? And so I moved to patent law, which is half technical and half legal, I would say. And so I did that for six years, started to get progressively my own clients, bigger and bigger, and a lot of them were startups and a lot of founders. And at the end, in the years like 2014, 2015, some of them were starting to do deep learning. So some of the intellectual property strategies I designed were around how can you protect stuff that were around at that time: image recognition, object segmentation, that were really the first things that work with modern AI techniques, right? And I was very surprised that I didn't know anything about AI, because this was basically the same equation as physics equations, but just written by ML people.
Anchor us in time. What year, around what year was this when you're looking at these kind of perception AI patents?
So that was 10 years ago, 2014, 2015.
Yeah, so like right in the early days of ImageNet and all that stuff starting to pop off.
Exactly, exactly that time. And so that's where I started to do some evening classes on my own, reading papers, reading books. And so that's when I contacted back to Julien, where we actually just ended up posting roughly the same thing on AI on Facebook, and I was like, oh yeah, I'm also reading this. And he said, yeah, maybe we're going to start a startup in New York, so why don't you join and do some science back, and then you'll find maybe a more serious job in a couple of months. And basically what happened is I never moved out to find a more serious job, because the game company that was Hugging Face at the beginning, so the early idea, and that's why we have this non-serious name, was to make a kind of modern version of a Tamagotchi, a modern version of this kind of funny little being to interact with. And when I say modern, I mean powered with all this early Gen things. So basically at that time, we thought the only thing we could do was understand images better, and we were like, that's still really cool. You can recognize selfies, you can maybe react to how someone is looking at you on emotions. But quickly came some of the early NLP breakthroughs in generation, so with LSTMs at the time, and we were like, oh, maybe we can even generate text. And so basically we were exploring all of this, building this game. And that's a bit when happened, I'd say, the change or the morphing of Hugging Face into what it is today, which happened basically by trying on our side as well to give back. So if we do some research, we find some stuff, we should also write our first paper, we should also publish our code. And after a couple of months, we were like, yeah, when we look at all the curves, in a startup you're always looking for this exponential, you're always looking for where is something growing crazy. And so the game was working nicely, we had a very nice linear increase, we're reaching 300 million messages. But the open source indicator, the stars and the issues, they were really growing exponentially. And we also realized we were very much excited about this idea of open source. Julien is also a very, very strong proponent, even more than me, I would say, of radical open sourcing, radical open sharing. And this I think made a lot of sense, and that's where basically the mission of Hugging Face kind of started to come. So not really from the beginning, but it became kind of obvious that that was both something that was growing very strongly and at the same time something we believed a lot in. So basically that's when we pivoted for our Series A in 2019.
Talk to me about that pivot. You know, clearly y'all were one of the earliest companies that were looking at these sort of machine learning primitives at your disposal, let's say, for visual understanding and trying to build a consumer experience around it. I'm sure that involved building your own stack to kind of do that. This is probably the early days where there were no companies such as yourselves making that easier. And so kind of packaging that up and tackling open source, that must have been interesting, because usually you hear about the reverse, where you've got a developer-centric company that goes consumer and boom, they get suddenly a lot of interest and adoption. Y'all feel like the complete reverse case. Can you talk a little bit more about that pivot and shift as it happened?
Yeah, so in this time we were doing a lot of trials in many, many directions, both on the interaction we wanted from the user. Okay, is it better to be on mobile? Is it better to send like NLP? Is it better to use images? What is the most interesting thing? But also more generally, how do we want to be as a company? How can we get a little bit of excitement? You also have to picture that back in the day, we were really underdogs. We were basically French founders, so still in the US, but for the rest, we had no PhD in ML, we were not in the field at all versus Google people. And so we had a lot of imposter syndrome and wanted to build some credibility. And it was also the time of the chatbot company, the early versions, and we kind of wanted to steer away from this and not be put in the same bag. We were like, okay, we're trying to build something serious, we believe in AI, we're not just repurposing a couple of templates and making that into a chatbot. So that was part of this thing with the idea of, okay, if we publish, if we communicate about what we do, we're going to be taken more seriously, and it's going to be easier to raise, it's going to be easier to hire people. So it was really decided, which actually ended up being very true. And I think being open source and having an open source approach is still a really great idea to hire great people, to be visible when you're starting a little bit outside of the traditional path. And yeah, it doesn't mean you have to do everything open source, but it can be a very big part of the mix.
You know, a big part of why y'all have so much adoption, especially in the early days, was some of your open source libraries, right? Like the famous Transformers library. Now, for the audience that might be uninitiated, how would you explain what a Transformer is and the importance of these open source libraries and even datasets?
So a Transformer is basically the software of AI. It's the workhorse, the backend that can make all the AI models we see today. And they define how you combine the numbers. An AI model is basically just a big set of matrices, so a big set of numbers, and you need to combine them in a certain way to get the output that you want. It can be an image created from a text prompt, it can be text generated from text. So the first step is you usually convert the input into numbers. You convert your text into numbers by associating a number to each word. And then the way these numbers are combined is defined by the architecture. So Transformer is one way to combine them. And basically, Transformer is also the name for the big libraries that we have. We have many, many open source libraries now because we expanded much wider, not just text, but that's kind of where we started. And this library, I would say what was really special about it was two things. The first thing was it was very, very easy to use. So basically, I remember when I met the researcher who created this BERT model, which was the first widely used model, so before GPT-1, right before, it was a bit around the same time. It was the first model that people who had been working in the field for a long time would basically drop everything they were doing to try this new model and run to it and try to test it. And so when you reduce the entry barrier, it helped them a lot. So we had a lot of people thanking us, okay, I could try this model and I could modify it and try to tweak it, understand how it was working, and basically switch my whole research around this new model in minutes, thanks to the very easy access that your library gave to us. So the first power users of our libraries were really ML researchers, AI researchers, and that's really how it started. And then the second gradient was that we started to include a couple of models in it. So you have one model released by a company, and we see that a lot today, and basically a couple of weeks later, a couple of months later, their competitor released another model which is slightly better. And so you have this kind of, I think, quite healthy and generally nice competition between researchers or teams building models to get the best model. So it's very nice, but when you are participating in this competition, you also want to quickly be able to try the new model and to compare them together and to understand, okay, what are the differences? Why is the new model from Google better? And the nice thing about the Transformers library when I created it was it was really flexible, and so I could quickly add new models in it. And basically, I remember when OpenAI released GPT-2, it was one day, in basically a weekend. So I worked on it on Saturday and Sunday, I had added it in the library. It was released on a Friday, and on Monday everyone could use it in the library. So basically, people just keep the same library and they just switch, you know, like one argument, and they could use GPT-2. So this was the beginning of the idea of the Hub. And then we added a lot of social features and more things on top of it, but that was the beginning of an idea that there will be a diversity of models, there will be more and more AI models coming, and it's very nice if you can easily move between them and try them and compare them in this single environment.
It's super exciting because, like you mentioned, the ML community, at least historically, has just been so transparent and forthcoming. And I think that creates those opportunities for virality, right? If people come across a cool new thing, oh, BERT's super cool, GPT-2 is super cool, they're going to jump into it, they're going to post about it, other ML engineers and researchers are going to look at it, and then on and on, and then sort of the flywheel continues. And that's quite a beautiful thing, because the way my first exposure to Hugging Face was even the next layer, making this more accessible, was Hugging Face Spaces. Every time a new computer vision paper would drop, or a new diffusion paper, diffusion Transformer thing would drop, I'd go to the Hugging Face Space and try it out. Oh, Segment Anything Model by Meta is out? Let me just upload an image and see what it does. And that was exciting, right? Because it suddenly contextualizes some cool new research, and it's not the same five videos associated with a paper that get reposted again and again and again. You can see it with your own data. So if you have to describe what Hugging Face does now and the role that it plays in the open source community, how would you describe that?
It's really this place where people who make models and the people who use them, and actually a lot of them are now the same, and I think it's very nice that that was also our idea. If we democratize ML and AI enough, everyone could be a model builder. And actually, just before talking with you, I was playing with my kids and we were trying the new Flux model, so the image generation from Black Forest Labs, which is totally amazing, so good. And so we went just exploring Spaces, trying it, and then find tuning on a couple of images. And you think, okay, an 11-year-old can now fine-tune an ML model. And so the frontier between ML builders and ML users is really, really thin, and that's very good, I think, because our idea was always, AI, we think the very long-term vision is we think AI should be a common good. For me, as I said in my physics view, everyone can learn about general relativity, quantum physics. Anyone can learn about this. These are like common good knowledge from all humanity. And I think it's great if, you know, only Microsoft would know everything about quantum physics, that would be very sad, right? And I think AI is a bit the same. It's a very fundamental revolution. It's a technique that's going to change very obviously a lot of things we do, and it should be something that anyone could understand. And if anyone wants to understand now how this thing works, they should be able to do that, because it's going to control so many things in our life. And the best way to understand something is, first, to share it in an open source way, but also to be able to use it very easily. You go to a Space and you're like, okay, how does this model work? You try a little bit, you try a couple of example prompts, you try, if it's a chat model, you try to ask a couple of tricky questions. And Spaces has helped us a lot to democratize this aspect and make Hugging Face this kind of place where you can find models and you can also understand them. And the third thing we added is datasets, because in the end, what we are discovering is that the data that you put in these models is basically the core thing that makes everything work, the quality of the data and the quantity. And so a couple of years ago, maybe three years ago now, we started this second approach around datasets. So we also started a second library called Datasets, and we started to host datasets on the Hub. And the idea was, let's open this new black box. Okay, so we have the models, we've opened roughly the black box of the model, we've opened weights with open source around the model code, but the dataset should also be accessible. If you want to understand AI, you have to understand how to make a good dataset, so we have to push people to share them in an open way. And this has been growing also very strongly on the Hub. We have, I think, roughly maybe 300,000 datasets, but I would have to check because this is exponentially increasing. And this is very exciting, I think, because when you have all of these things open, basically you can really say, okay, now anyone can train a model and basically join the field.
I think that's really it. It's so interesting to hear you talk about openness both on the model side and, I think, the dataset side gets less attention. To your point, yeah, these models are like, if they're open weights, it's like this inscrutable set of matrices. What do you do? All you can do is use it, maybe you can fine-tune it or whatever. But when it comes to the real, I guess, like, I don't know if oil or gold is the right analogy, that has distilled wisdom into these models, it comes from the data. And making that openly available is so huge too, especially if you've got an eager community that's kind of, rather than just a select group of people at various labs working on this stuff, you can have the whole community, including indies and hackers, working on it. Which sounds exciting, but it almost makes me want to go back to the bigger question here, which is, in the past, in previous interviews, you've said that we need to fight for open source. Why is open source important and why should we fight for it?
Yeah, it's a big question. It's important for many reasons. Maybe the most practical one is because of balance of power. I think open source is a great way to make sure that you can have many people participating in the AI revolution, and not only a small set of well-funded labs, but basically anyone, any smart person from anywhere in the world who has a nice idea can join. And because they can start from this openly accessible weights and data, they're almost at the level of the top lab from the get-go. So that's really nice. I think it brings, in my opinion, a lot more people around the table. So it's really great to catalyze new research ideas, to basically accelerate knowledge in the field, but also to include other voices. We also tend to think sometimes that AI is just built by a small set of ML people, you know, math-focused, white males, for instance. But I think it's really nice if everyone can join. So basically, if everything is open, a lot more people can join and participate.
