AI of the Hurricane | Thomas Wolf (Hugging Face) & Robin Rombach (Black Forest Labs)

Awesome. This is a little bit of a funny format because we are supposed to have a discussion. So I have to get into the role of a moderator. What is Hugging Face?

So I would say across all your various experience across Stability, Canvas, you've brought eight out of the top 10 models in terms of downloads on the platform, and three out of the top five models in terms of likes. I think we were discussing just before on stage that the total number of downloads for all the models you've shared open source is about half a billion downloads. That means half a billion times someone thought, 'Okay, I want to run this model locally, fine-tune it, build something on top of it.' Can you tell us a little bit, Robin, on how you managed to reach this top spot?

Yeah, that was actually the model that launched all our operations in video. Before that, we were everything on text; that's when we moved into image and video.

I think this latent diffusion work is maybe not extremely known unless you're very technical, but it's a very good example of something that happened again recently. When DeepSeek came and became extremely famous in January, one of the very surprising things from them was also that they were focusing a lot on efficiency. They had this very strong constraint on compute because of export restrictions; they didn't have enough GPUs. Not only did they use that to release a top-performing model, but like you did with latent diffusion, they used this constraint as a catalyst. Just like when you did latent diffusion, you were trying to solve the problem that images have way too many pixels and are too big to be efficiently trained. But when you moved to this efficient thing, the constraint added to your work and let you reach the top spot. Later, Stable Diffusion and its extensions were not just efficient but also top-performing. I'm wondering if you still think it's a viable path, or if you think this is actually a strength that you've had.

Interesting. Do you feel like for Flux, to tell a little bit the story: when you were at Stable Diffusion, you built this Stability brand, and then you moved to starting Black Forest Labs. It's probably good to remind that it's extremely recent. You started Black Forest Labs basically in the summer of 2024, so about one and a half years ago.

One and a half. And so you started with your flagship model Flux, and you also quite quickly went into this idea of making various derivative types of models. There was the Schnell, the Dev, the Pro, and maybe soon a smaller version as well. Can you tell us a little bit how you see this versus what I would say like a team like Gemini or something are trying to sell, or maybe like OpenAI's Sora, which is basically a single large model?

Let's live a little bit in monetization, because people tend to oppose open source and say if you go open source, you can't sell your stuff. I think it's very interesting that Black Forest Labs has taken this very strong open source approach but also managed to generate enough revenue apparently. So tell us your feedback from having this half billion downloads. Do you think this visibility, this top spot you have at Hugging Face, has been in your experience after one year an interesting way to sell premium models?

I'm actually curious to get your thoughts on this because there's a very heated debate online about full open source versus open weight. Open weight often means a non-commercial license. I think it's a good point, and people often ask us: what is a good open source model? But also some people who open source and don't get the benefits come back and complain, saying 'I open sourced it, I didn't solve my visibility or mind share problem.' The first thing I try to tell them is that you cannot just open source once and hope that you will have a very strong brand built out of it. There's a reason why models like Qwen are constantly top downloaded models. One reason is the right size and price point, but another reason is that it's a team that has been constantly open sourcing good models for the past two years. People have started to associate them with good quality open source models. I think Flux has the same thing: you open source regularly a couple of things, so the community builds around it and starts to know you as a household name. Another counterexample is a company I was working with recently, GLM. They open sourced once, then didn't do something for two years, went closed source, and what they discovered is that people kind of forget about you. The community doesn't stay active; it disappeared. When they open sourced again this summer, they saw a revival of activity and usage of the first model. So the first thing is that when you start to go on an open weights approach, you have to think a little bit like coding: it's a community you're going to have to keep maintaining. You cannot just say 'we're giving you once and that's it.' What usually good open source looks like is that regularly you try to update your models, release a smaller version that's faster, update the ways, release a blog post on how you train it, release a fine-tuning dataset. It's a whole approach you have to do if you want to really get all the fruits of an open source approach. That's usually the main difference between organizations that grow like Qwen or DeepSeek, which have 100,000 followers on Hugging Face, and some one-shot attempts that fall short.

A lot of them. I think the text field is quite well covered. There is still maybe room for regional models because we tend to forget that these models encode some values and are trained by teams from some areas. If you take a model trained by a US team, it will be English first, most of the data is going to be English. If you want to use it to process Polish, you might want something else. Chinese models will be different. There's a lot of debate about what is encoded. So I think there's still definitely room in text for more European models trained in those languages. But maybe even more fun is the extension to new modalities. There is a lot of activity, and the top trending models right now are more speech models, text-to-image is very strong, text-to-video is very strong. Personally, in the past few months, I've been really mind-blown by all the world models, the interactive world models where you have an image and you can actually start moving in the image. There is almost nothing in open source right now. So there is definitely a lot of room for attention and usage to be tackled. There are some teams, but it's mostly the same combo that we see. So there's a lot of room there.

That too. Yeah, I think world modeling is something that is extremely exciting, and there's still so much research to be done. How do I actually achieve real-time rendering performance? How do I couple this with a long context window to process and remember minutes of video? That's fascinating. I think we'll probably see way more of that in the coming months.

I think we are already seeing a trend towards more unified large general models, but they are large and slow. It will be interesting to see how much adoption larger models will see in the future, even when they are open source and available on Hugging Face. But that's definitely something that is working. You're paying that trade-off. I do also see the need for very specialized models that do a certain application very well. They might be fine-tuned on some data or confined to the task of text-to-image generation or image editing. I think we'll probably see both. I think of it as the more specialized models as kind of visual rendering or intelligent rendering, and then the more unified models that are what everybody talks about when thinking about general artificial intelligence.

Like a text-to-image diffusion model that you have to prompt. For example, you come up with a meme design. You have to provide a very detailed prompt, but then it understands that prompt in a way that it knows where to put text, where to put a line break, how to combine it with the rest of the image. That to me is more of the visual intelligence, visual rendering. But importantly, it only involves generation. When you think about more general models, that also includes perception like image understanding or video understanding. That's where I draw the line.

Super interesting. Think about a classic text-to-image model. You usually optimize the kind of images it puts out. But now you can augment that and optimize on the text that actually comes into the image, and then augment that to audio or other modalities in the future. I think that's super promising.

Two things quickly come to mind. One is how do you make the models two orders of magnitude faster? The second is how do you design the memory module I was talking about in the context of world models? You can either go the brute force approach and have everything in the token sequence like in typical transformer models, and that will work to some degree. But when you think about very long video context that is very high resolution, you will hit limits even with a lot of GPUs. That requires a lot of future research, smart projects.

100%. I think making money with the stuff is important. We want to have sustainable funding for our own research to continue doing this. I think it's important that when we talk about building it from Europe, we are building it for the world, for everyone. There's a global audience that uses these models. That sometimes gets a little bit lost when you talk about building AI in Europe. Yesterday I was in Berlin at an AI summit with lots of politics, and it sometimes feels a little bit defensive and too overfocused on Europe alone. The ambition has to be global. For these research problems, the one thing that matters is that you have a really good team with a lot of experience in the space and access to compute. If you don't want to sit next to your compute cluster, it doesn't really matter where you build your team. Europe, Germany, southern Germany in particular, has such a huge talent density. I think it's possible. We are hopefully one of the examples of how you can do that. What you get here is the ability to focus and go out into the world with this technology.

Not isolated, but we have the ability to focus when we want to. We also have an office in San Francisco. I'll fly there in a week and a half. It's important, the collaboration between us here in Europe and the team in the US. That's reflective of how we approach this problem. I'm actually curious: your co-founder Clem at some point said that it wouldn't have been possible to build Hugging Face anywhere else than in the US. Would you still think that's true? That's a key question for any European company: how can we build a global company out of Europe? That's a key danger and the key question you need to ask very early: how do I think global? For us, the way to solve that was to build out of the US first. That's why we started in New York. You have to picture that back in the day, eight years ago, raising money for AI in Europe was really tough. AI was not even a field in 2016; it was a chatbot field, and chatbots had a pretty bad reputation for good reason. So raising for this idea that AI could be big, you could only do that in the US. That being said, if we had to start again today, I think we would ask ourselves much more the question: could we not just start it here and do the same global thing? But even if we had started here, I think we would probably have tried to do something like you did: open an office in the US very quickly, or at least take a look at the US market. People complain about the fragmented market in Europe, but there's an amazing unified market you can tackle: the US market. So you should just go tackle that market very quickly. From that point of view, we would very likely also have had an office in New York very quickly, an office in the Bay Area very quickly. So would Hugging Face, the remote company, look so different? I'm not even sure.

Roughly just expanding the scope. Right now, we're building a lot in robotics and AI for science. We're very excited about this area. Generally, there's a lot of exciting things to be done now in AI outside of just the text field. Vision is one part, but robots, physical AI, is a new one. Science with proteins is a new one. We're very excited about this kind of organic expansion on the ground we've already built.

Thank you.