The Hollywood industry as we know it today will either cease to exist or it will have to change radically. Will Smith Eating Spaghetti was from 2023, about 3 years ago I think. We've been making images since 2016, and for me, Will Smith Eating Spaghetti is already a masterpiece of quality. Do the exercise of looking up the type of images that could be generated in 2015, and you will see that on that scale, Will Smith Eating Spaghetti, which seems, I don't know, very glitchy, is actually not glitchy at all compared to where we were 10 years ago. And in fact, we have a study, we did an analysis with hundreds of people called The Turing Reel, which is a kind of test where I show you real or generated images and try to make you distinguish which is real and which is generated. And statistically, we realized that no one could tell the difference. It is too difficult. It's a coin toss.
Hello everyone, welcome to Inferencia, the podcast where we talk about technology and innovation. Today as always, José Murillo from Primero II greets you with my friend José Luis Abau from Perpetuo. And José Luis, today we have the pleasure of interviewing Cristóbal Valenzuela, CEO of Runway ML. Tell us a little about him, José Luis.
The truth, José Antonio, is that there are few people in Spanish as qualified to talk about artificial intelligence as Cris. Cris is the CEO of Runway, as you mentioned, which is a company with quite a big promise. They give you everything you need to do everything you want. Which is a promise that is only possible in the age of AI. What that means pragmatically is that Runway is a company that uses artificial intelligence to generate images and video. And not just that, they have recently transitioned to also creating what we call world models, which is a different way of thinking about AI. If we usually think of these AIs as a probabilistic calculation that takes previous information to predict the next token, world models go a step further and try to take all the information we have about the world to simulate the world so that we can have infinite worlds, perhaps an infinite Minecraft, an infinite GTA. And for good reason, Cristóbal has been named one of the 100 most influential people in AI by Time Magazine.
José Luis, hey, I think two things worth adding and highlighting about Cristóbal is one, Runway is an extraordinary company, and importantly, Cristóbal is one of the Latinos who has built one of the most successful companies on the technology frontier in the United States and in the world. Runway has today become one of the most successful companies in selling tools to creative teams for image creation, content editing, and video production. Today Runway has raised more than 300 million dollars from some of the best investors in the world, like Sequoia. And something that I find especially cool about them is that they haven't just built a very good business creating creative tools, but they also have a very strong research component. So, as you say, they have this very successful creative tools business today, and at the same time, they are one of the companies innovating at the frontier of creating World Models, and what they've achieved is really cool. But, for example, a couple of years ago, this movie Everything Everywhere All at Once, which won the Oscar, I loved it, it blew my mind, and there’s this part where there are some rocks that are moving, and it turns out that the model used to make that scene and other animated scenes in it were Runway's. And today, today, today Runway is important for creatives, it's doing research at the frontier, and it’s used by some of the most important directors and production houses in Hollywood. The most important ones. So yes, I think it’s a luxury to be able to talk to Cristóbal today.
It seems like they really give you everything you need to do anything you want. I can just see Cristóbal on the call now, José Antonio, so let’s jump in with him and talk for a while.
Cris, to start, I want to focus a bit on using AI to generate images and video. Usually, when people interact with AI, it’s with a model like ChatGPT or Claude from Anthropic, which is a predominantly text-based model where you input text and it generates text. I’d be interested in understanding how artificial intelligence changes behind the curtain when you are generating images or video versus when you are doing text.
The truth is that the process of generating language or pixels is actually very similar in a certain way. You can group images and videos together because, basically, the idea of generating pixels is quite similar. The only difference is that if you think about what a model like ChatGPT does, the model is trying to predict one word after the next word, right? They are mathematical probabilistic models that take a sequence of words, and what the model tries to do is predict the next word in that sequence. That, very simply put, is what ChatGPT does. We do something very similar, but instead of predicting the next word, we try to predict the next pixel. What is the next pixel in a sequence of pixels. Now, another way to think about that is if you have an image or a frame or a simple image, my task as a research company is to create an algorithm, a system that can be capable of predicting the next frame in that image and the next frame after that. And maybe if you remember from college, or school, you might remember that one of the things is that video is an optical illusion. Video is an illusion of movement and the illusion of movement is generated because we are able to repeat, I think it's 12 frames per second. So, that is basically the main idea of what we do when we work with images and videos, that we try to predict in a sequential way the next frame or the next pixel in an image.
Cris, maybe as something interesting is whether LLMs have evolved a lot in the last 3 years, let's say, since the launch of ChatGPT. That evolution in LLMs, in your words, operates somewhat similarly to the creation of the next pixel, has the fact that they have evolved so fast led to the evolution of image and video creation advancing faster as well?
Yes, of course. Many of the techniques that have been used in language models can be used in the creation of pixel prediction. Well, they are relatively similar problems, they don't translate into a one-to-one relationship. There are many things that are much more specific regarding the pixels themselves that don't necessarily apply to the creation of tokens or words. But I think there are many techniques that are usually relevant and one more important consequence is that nowadays, for example, our research uses many models, many LLM models and language models to conduct research. So, you are working to improve our problem. We use language models. Our language models basically today are researchers that help us and work with us on improving those algorithms.
What are, when you said there are some things that are different from pixel generation versus tokens for language, what are some of those things that are different?
Well, first of all, the space and the problem is different because, in a way, what you are doing with LLMs or language models is predicting words, and the probability and distribution space of a pixel and pixel generation requires a different kind of approach. One of the most important things, for example, very relevant, is that you've probably heard that an image is worth a thousand words, that sentence, right? It is important to remember that because if I show you a book generated by AI or I show you a text or a document generated by AI, your human ability to read the document and understand, for example, where the AI went wrong or what part needs an iteration, it is not as easy or fast as if I show you an image and you tell me very quickly what is wrong with the image. Our eyes have evolved for hundreds of millions of years to be able to detect the slightest imperfection in an image. If there is something in the world that is imperfect, we will see it immediately. And that imperfection means that the quality and the way we generate images must have a different quality standard. It must have another standard for ensuring it works, which is much easier for anyone to detect than it is for a language model. And that means that the way we measure, the way we benchmark, the way we ensure that the model is working the way it should be working, has to be very, very good, and there are different techniques to be able to do that in video and images than there are in text.
I would love to go a bit deeper into this because it does seem, I think you're hitting the nail on the head, right? The way we measure the output of an artificial intelligence that gives you text is different from an image. But it would also seem, at least as a user, that at least in early image generation models, it was indeed something much harder to achieve. I think of the perennial question, why is it so much simpler for an AI to write a good poem than to make a hand with five fingers?
I don't know if it's that much simpler. I think it's more... But first of all, I think that today, it's not that it's solved, but we are quite close to solving the problem of generating images of any type, any quality, any resolution, any kind of change. I mean, the big challenge for image generation two or three years ago was, as you were saying, a hand with six fingers, and the model it was very hard to be able to make an image without six fingers. Nowadays that is totally resolved and you can make a hand with however many fingers you want. But I think it has more to do with perception, with that in the end the evaluation of whether a model works or not will always be a human or in the majority of cases it will be a human. And that perception of how good a model is is a perception. The model can generate code and the code can look impressive, but there is a way to know if it works, which is use the application the model created or try to use the webpage the model has created and you will realize what works. No, that takes time. What takes less time is being able to see the flaws in a video. If you are shown a 10-second video and there is a small glitch in a small part of the video, your eyes are immediately going to catch it, the illusion is already broken. It doesn't work for me, the video doesn't work. And this happens to us a lot, for example, when I imagine you have seen movies with very bad visual effects. When you are watching a movie with very bad visual effects, you lose focus, I mean, the story takes a backseat because it is so distracting the type of effect that looks so fake. That is exactly a bit of what we have tried to solve for the last 8 years that we have been working on this.
Hey, Cris, I mean, and part of what you are saying is the importance of an image being perfect and how easy it is for a human to distinguish when an image is not. What have you done at Runway and also like in the industry in general so that in the last few years we go from having, you know, like in this image of Will Smith eating spaghetti terribly to being able to make videos as realistic as those we have today at Runway?
A couple of things have happened. In fact, Will Smith Eating Spaghetti was from 2023, 3 years ago I think. We have been making images since 2016 and the progression of image quality, for me, Will Smith Eating Spaghetti is like a masterpiece already of quality. Because if you do the exercise of looking for the type of images that could be generated in 2015, you'll see that at that scale, Will Smith Spaghetti, which seems, I don't know, very glitchy, is actually not glitchy at all compared to where we were 10 years ago. And if you map that trajectory, what we could do in 2015, what we could do in 2023, what we can do today. I think one of the main reasons is basically scaling laws applied to video. Scaling laws basically say—it's a law about how these models are trained—that if you scale the size of the model you're training with more data and more compute, you will improve the quality of the model and the output. And today this law, I think, applies quite, quite well to pixel-generation video models, where if you scale the size of these neural networks, you will get better, and better, and better results. And you can kind of see that with this scale of results that I'm telling you about, from 2015 to today, and also.
Something you say about that, Cris, is, I mean, I understand you, the more data, the more compute, the model improves and gets better, but you're saying something different. You're saying that you need that model to be perfect so that the image doesn't have errors that are visible to the human eye. When did we cross that benchmark?
I mean, I think that benchmark is crossed today. We actually have a study called—we did an analysis with hundreds of people called The Turing Reel, which is a kind of test where I show real or generated images and try to make you distinguish which is real or which is generated, and statistically we realized that no one could tell, it's too difficult, it's a coin toss. And, I mean, in terms of quality, that's why I think that quality is somewhat solved. What still needs a lot of improvement is, well, the cost, to make it much more effective, more efficient. And the second thing is the ability to control and iterate. Once I generate an image, how can I modify it in such a way that it is exactly what I'm thinking and what I'm trying to do? And there is quite a lot of progress and quite a lot of advancement in that sense, but I think there is still a gap in trying to make these models as controllable as possible.
I couldn't agree more, perpetually. For example, the designs we use for our Friday campaign are made with AI, and at least once a week someone asks us for our designer's contact, so I think we've crossed that threshold.
And yes, it happens to me too, quite early on people were asking me what advertising agency we use to make our videos. And I send them, I think it still exists. If you go to Pasarela, Pasarela. Agency on the internet, Pasarela is the word, it's the Spanish version of 'runway,' and our careers page. So if you want to see how it's done, you have to...
It's incredible. And maybe moving on a bit, but it intrigues me a lot. If you have any idea when we'll reach a world where most of the images we see on the street are already made with it. It surprises me now to go to towns on the outskirts of Mexico City and see taco stands where the advertising image is clearly made with a ChatGPT prompt. Given that we're moving so fast, do you have any idea when it will become ubiquitous and we'll have so many images with it?
That's happening today. It's already, probably, in the double digits or, like, 10, more than 20%, I would imagine, of the images being created today in the world are created with AI, and we are going to get very close to a world where the majority of the pixels you see on a screen or printed in the real world will come from a model; and it sounds, it sounds crazy, but it's really not that crazy. If you think about 150 years ago, most or all of the images in the world were hand-painted, they were hand-painted by painters, and the only way to write or capture a moment was with paint. Every image 150 years ago that was in the world was a hand-painted image. And then came photography and computers, and that changed. Today, I think the vast majority of the images and content you see in the real or digital world were clearly not hand-painted, but were created by a camera, by a video camera, by a computer. And I think this is, basically, to me an evolution of that where, very soon, a large part if not almost all the content you see, digital or in physical form, is going to be generated by a system, but the truth is that, today, it might not matter, but I think in a couple of years it won't matter. In the same way that when you see an advertisement or you see a movie, you never ask yourself, 'Oh, but was that made with a computer or with a camera?' Basically that question moved to the background.
Cris, hey, and on that subject I was watching an interview you did 3 years ago with Jason Calacanis and you were talking about the future of entertainment, you said that Darren Aronofsky is like your favorite director. I also really like cinema since I was a kid, I watch a lot of movies, I'm a fan of the Oscars. In fact, José Luis and I often watch them together. I think frequently about a world where there is an abundance of content creation and where, if today every year there are 10 movies nominated for the Oscar, we could live in a world where every year there are 100,000 or a million movies of that caliber. How do you think the Hollywood industry evolves in a world where there is so much abundance of creativity and content?
Ah, good question. The truth is it's hard to know. I think what I believe is most likely to happen is that the Hollywood industry as we know it today, either won't exist or it will have to change radically. And I think that is more than anything the consequence of this technological change that is redesigning and reinventing the way we understand movies, what a movie is. What does making a movie mean? First of all, that is the first question for me, and I think perhaps a couple of years before this revolution, making a movie meant going through a studio, going through an extremely expensive process, difficult, quite hard for many people to access. And I say, because I always wanted to make a movie when I was younger, I always wanted to be a cin—as a teenager, something like that. Of course, of course. And in Chile, I mean in Chile, there was no film industry and it was very difficult to make a movie, it was very hard to study, I never had access to a camera, so my capabilities, my possibility of making cinema was quite limited. And I believe that this limitation is not a creative limitation, nor one of ideas, nor of capabilities, nor of intention. It is an economic and technical capacity. It’s that the technology was not at the necessary level where I could do that without having to go through a series of processes. And that is the fundamental role of technology. The fundamental role of technology is to take something very difficult or very complicated or that has been very closed off to a very specific group of people and make it extremely easy and accessible for everyone. If you think about this, in communication, talking to a person in another part of the world 50, 60 years ago was extremely difficult, extremely expensive, extremely complicated. Only a very small group of people in the world could do it. Nowadays it is totally free and open and easy and you can talk to anyone at any time in real time. For me, cinema, or if you think not only about cinema, if you think about storytelling, about the idea of telling stories visually, today we are reaching the point where you don't need a budget, you don't need a production, you don't need a studio, you don't need a producer, you don't need anything to bring a good idea to life. What you need is the idea and it is basically limited by how good your idea is, nothing more. And that for me is quite important because if you think about Hollywood, the Hollywood industry is a quite small industry. In the Hollywood industry, I think there are between 2.5 to 3 million people and it is quite small because it is quite expensive. Now, what happens if we have billions of people making films at that level? Well, the need to have Hollywood in the first place I think disappears and that is my main point.
How cool. How do you say it? Just an idea on that is, I find the narrative around artificial intelligence to be very unfortunate and negative, regarding everything that could be lost and the jobs that could be lost to AI. And how few people talk about the abundance of AI and, for me it’s like: Yes, I agree, that is the perfect example. But I imagine a lot that if, for example, in today's world of journalism, obviously Bloomberg and the Wall Street Journal remain very important for headlines; many of the news stories I consume are from Twitter and from columnists who write independently, and perhaps the world of entertainment and content might be like that. Maybe you have opinions too, José Luis, but I get very excited about that world of content abundance.
It sounds truly incredible. Also, one thing that has fascinated me a lot is, in a way, the decentralization of stories. There's a very crude example, but you've probably seen the 'frutinovelas.' I'm addicted now, I mean, they pop up on my Twitter every day, but it's very curious to think that these are frutinovelas that aren't centralized. Anyone can add chapters, can add stories, and I think there's an element where that can happen. Perhaps the next Star Wars is something Cris creates now that he has infinite resources to make a Chilean Star Wars, and then we can adapt it and generate a new version of it, and IP fundamentally changes its role in this post-AI world. That is what intrigues me a lot.
Yes, maybe something that brings it all to the center is what does it mean to create, what does it mean to define what a movie is, what is a drama, what is a film, what is art. I think what this technology is doing is making us question many of those things we considered untouchable, right? Many people thought that, well, like what's happening a bit with Hollywood—I've spent a lot of time in recent years traveling to Hollywood and being with great producers and great artists and great directors—and, in general, I think Hollywood, a large part of Hollywood, forgot that Hollywood is the consequence of great technology. The reason why that industry exists, why there is a profession called film director, was as a consequence of a technology that is the camera 140, 150 years ago, which at that time was totally considered a toy and something totally not serious. I mean, the history of cinema, a large part of the first experiments in moving images, comes from the Lumière brothers, at the beginning of the 19th century in France, and thanks to great works that today are like historical milestones of cinema, right? I don't know, The Arrival of a Train at a Station is probably one of the first films that everyone is shown in film school as the master milestone. Well, the Lumière brothers thought that what they were doing was total nonsense, a complete waste of time, that cinema... why would anyone want to spend their time doing this? And it was such a fascination, yet at the same time a lack of vision, that they got bored and said, 'This is going nowhere, let's do something else,' and they stopped making movies or stopped experimenting; and of course, while they were quite visionary at first, they were also quite wrong and mistaken. And to me, what’s happening today is very similar, where 'frutinelas' or any type of media experiment or use of technology today sounds like a toy; but, well, in a couple of years it might be a fully validated art form where we have people 100% dedicated to this. 'Frutinelas' at the Oscars, that would be great.
Something that José Luis and I have talked about a lot is the potential for artificial intelligence to one day have judgment or be creative. Like something you often insist on: 'Hey, a younger Cris in Chile today will soon be able to use artificial intelligence to create his movie with a low budget and without having to own a camera.' And you sort of start from the idea that this young Cris could be very creative and have many ideas and use artificial intelligence to make them real with many more tools at hand. Well, what about now that the AI itself can be creative and can have those ideas from scratch just like a human? Does that excite you, or how do you think about it?
I mean, of course, I am excited by the capacity to have a collaborator or an agent or a system that is constantly stimulating you intellectually or creatively. It sounds incredible. I mean, the capacity to have a tutor or to have a creative assistant that is provoking you to think in a different way about the world, that helps you solve problems. No, no, I don't feel that. I find it hard to think why anyone would be opposed to that. It doesn't mean you have to consume it every single day. That you have to live with it, but the option to have that possibility, I think, is fundamental.
And you think it’s possible, too—I mean, do you imagine an AI being able to have the same creativity that a young Cris could have?
Of course. I think one of the things we haven't realized in this technological revolution of the last 10, 15 years is that we are going through a process very similar to what we went through when we realized that Earth is not the center of the universe. For a long time, we thought that this planet, this Earth, this world we know was the center of the universe. And when we realized that it wasn't like that, that there are actually quite a few other things that are the center of the universe, I don't even know if there is a center, but clearly that the center of our solar system is the sun and not the Earth, we changed our perception a bit of who we are and how we are. And I think something similar is happening with our ability to see ourselves as intellectual or creative agents. Or with intelligence, we are creating an intelligence in an alien way, different from us, which is capable of doing things that we could never do. And I don't feel that is going to happen, it is already happening; that is, it is already possible to start this way and I think it will be quite positive in every sense for the future of humanity.
Since we are talking about frontiers, I would like to return a bit to the subject of models and especially talk about world models. We have already talked about image and video generation, and it seems very logical to me. If we are already capable of using AI to generate videos of seconds or minutes, but as a transition, what follows is generating eternal videos, eternal worlds that you can continue, which is basically a world model, or at least my rough understanding of them; but I want to ask you, how would you define this? What is the difference between thinking about a world model and generating videos using AI?
Hey, quickly, how do you say 'World Model' in Spanish? It's 'modelo mundial.' 'Modelo del mundo.' Yes, I suppose. 'Modelo del mundo,' I think. World Models.
And what is it, Cris, then? A world model.
And José Antonio, we have this constant debate about how we translate things. That is, like, that is a big topic.
How difficult, right? In this day and age.
