Matt, do you want to kick us off? What are the big things that happened in the last calendar year in your mind?
Well, yeah, it's been kind of an amazing year. I do think for people who just started paying attention to the sort of new AI and deep learning revolution in November of '22, when ChatGPT came out, things probably seemed very fast because we had Stable Diffusion come out over the summer previously in '22, we had ChatGPT in November, and then just a few months later GPT-4 came out, sort of demonstrated a new level of capabilities that were shockingly improved over what was there previously. And so I do think there's a set of people who sort of extrapolated from those three data points in terms of the progress that was going to happen from there, and it's incredible actually how quickly people can adapt to these new things. And there was a point at which kind of ChatGPT blew everyone's minds, and then a few months later it was sort of just something we accepted as part of the world, as part of reality, and we can very quickly enter this kind of slump: 'Where's the new model? GPT-4 is boring now. When do we get GPT-5? Come on, Sam, put it out already. What are you waiting for?' You know, this kind of thing. So I think we had a little bit of that this year, and then over the course of the year though, what we started to see — and I'd been waiting for this for a while — but we started to see real adoption taking off. And so, you know, back in 2020, GPT-3 had come out. In '21, when I was running GitHub, we put out GitHub Copilot, and it was I guess one of the first LLM commercial products. And I thought that immediately after that there would be this rapid wave of commercialization of large language models because developers would have seen how capable they were, what they could do, and start building products with them. But it really took this kind of ChatGPT moment before entrepreneurs, developers, and product people started to do this. And so what I think happened in the last year really is the starting gun was fired on the actual exploitation of these raw capabilities, the building of products, the working of them into companies and organizations. And now what we see, the adoption's been incredible. ChatGPT has rumored to have gone from basically zero to two billion in revenue in just about a year. That's amazing. There's, you know, organizations like Stability that would put out models that have, you know, hundreds of millions of users. You have Midjourney, you have now Google in the game finally with Gemini. I thought it was exciting to see them not only release Gemini but very quickly follow up with Gemini 1.5. And so there's a way in which Google has clearly internalized the pace here and the need to iterate and ship. And so I don't think a lot of people realize how far ahead Google was for a decade.
Yeah, right. So Google had really developed a lot of the early tech here and said it's not ready for release, right? They were being responsible to a great degree. And then how do you not release after ChatGPT? They're shipping pilled now. They're going to ship. They're going to ship. Yes, Sora was, I mean, pretty like a holy moment, right? And someone is doing it right at OpenAI with ChatGPT and then Sora. But I remember last year you told us it used to take like 30 seconds to generate on Stability a single image, and then it was down to a second, and you've advanced technology orders of magnitude since then.
Yeah, so I think if we can put the thing up, but yeah, put up the slide, Stable Image. And then if we actually go to the next slide, talk about speed here. Should I click this? Let's see. Oh, we can click this. Give an idea of where it's gone. So image was kind of one of the things that kicked this off in 2022. All of these images can be generated on a MacBook, just from description. And now we've perfected text and other things. But the next step after you can generate all these beautiful images is that you want to move to control. So the models are just the first step. You have to have ChatGPT and other things to make them useful. But then you want to be able to take that guy and say upscale, and that's all we said, and upscale him. You want to say replace a lion with a cat, and we can now do that real time. You know, or tiger with a unicorn, replace background with a forest. So all of this you can do pretty much real time because if you look at the next slide from this — okay, I said it was 20 seconds to one second. This is live, real time. You can just type and it automatically adjusts the cat, it gives him a hat. But then if you look at the optimization of that with the next slide — oh, we missed it. Oh, okay, well, we missed it. Go back one slide, sorry. With this process here, we're just releasing a new distillation model today. We've got it to 200 cats with hats per second. 200 cats with hats per second.
So that's your speed of image generation. That's an image generation — 50 milliseconds per image?
Yeah, exactly. It's because there's not enough cats on the internet, right? So we're going to add more cats to the internet. Oh, 5 milliseconds. 5 milliseconds. Yeah, but I think with the new chips that are coming, Nvidia is going to announce another one, we'll get up towards 1,000 images per second. So that's real life. That's live video times 30. Yeah. And so OpenAI's innovation, there were two major things. There was the Transformer architecture, the language models, and diffusion that drive the media models. They combined the two together. So our new image model, Stable Diffusion 3, which is the best performing image model, combines those together as well. So if you go forward a few slides from here — next one, skip — another one, another one, that's the upscaling. So you basically said upscale this image on the left and you get the image on the right. We'll have that real time in a couple of years. So your boxy video games will look a lot more realistic, just literally as you can take an old game and play it in surreal life. Yes. But if you go to the next slide, these videos with our video model, all generated on a consumer graphics card with 5 GB of VRAM. So I mean, the point is, this is in everybody's hands. It's in everyone's hands. And again, we haven't even optimized the data. So in an hour — next slide — we're releasing the world's most advanced 3D model. So all of these are generated just from descriptions. And so the fastest version of this does it in one second. So you can generate that dragon. But then the next step is you'll be able to control every element, make its horns bigger, make these adjustments.
So how long would that take for a normal creative person in kind of industry?
A huge amount of time. But now it works on the edge and it works even faster in the cloud. So a future here where you can be describing the video game you want created and it's generating all the characters and generating the play. Yeah. And so if you go back a few slides to that thing with all the nodes, I think this is an important thing. Sorry, I didn't do the slides properly. I think one of the things that we're having right now is this is a system we built called ComfyUI that's used by just about everyone now. So you take the face, the pose, the dress, and then you have that output. But if I share that image with you, it reconstructs the entire flow. Because last year was the year of creation, models that create. Then ChatGPT, ComfyUI allowed you to control and compose them. And the next bit is bringing that all together. Because ChatGPT, all the knowledge that you build from writing your speech, you don't have files anymore. You have flows with these models and assets there. I think that's the next step because, you know, when you go beyond just spitting out ideas to be able to control them like that, that's a huge deal.
Amazing. And you're announcing this in an hour. The 3D model is releasing in an hour, open source to everyone. Give it up for Emad here. You really have — and I don't know if you're able to say this — I mean, I am curious about the idea of how far are we from having an AI that I can have a conversation with and say, 'I'd really like to create a new business that does this, this, and this,' and just have that like a brainstorm partner AI, and it will do the incorporation, it will write the code, it will generate the marketing materials, and it will be able to — you know, we're all entrepreneurs here. That's all we, you know, find a problem, build a business, find a problem, build a business. How far are we from that reality of an AI-created business?
Well, I'll come back to the second part in a minute. But an AI that can be your thought partner and really staff up a company, I think it's already happening gradually, and then maybe suddenly. We have — I think companies will — these models are neural, they're neural networks. And so I think the right way to think about it is that companies in the future will just be increasingly neural, and there will be more and more of the company and what used to be the departments of the company that are single models or swarms of models that are doing work. I think it'll, you know, at some point, probably very soon, we'll look back on 2024 and say, 'God, do you remember when companies had like hundreds of people in the finance department doing accounts payable and just like transforming information from one form of text to another, essentially, and doing this coordination?' And so I think you will have some form of both existing companies that just adopt more and more AI because it gives them advantages, it makes them more efficient, maybe it gives them better customer service. People enjoy the responsiveness, intelligence, politeness, clarity, etc., of the AI counterparty. And you'll also have new companies that'll be AI-native, started from scratch, neural from the beginning. So some people may remember when Instagram was acquired by Facebook for a billion dollars. Everyone was just marveling at the fact that a small company, 13 employees, 13 people, could be worth a billion dollars. How is that possible? And the joke was, you know, 'Gosh, could you ever get down to a one-person company that's worth a billion dollars?' And well, like, this is clearly going to happen. Maybe eventually a zero-person company.
We talked about this getting ready, and I said, when are we going to have a scenario where a government — and we have some governance in the room here — says we're going to create a new regulation that allows an AI to incorporate on its own in our jurisdiction? I think it's a winning scenario because all of a sudden the AI incorporates, you get the tax base, and every AI-incorporated company will move there. I think Wyoming's done that.
What's that? Wyoming has done that. They have a new structure called a DAO, a decentralized autonomous organization.
Who has Wyoming? Wyoming of all places. Yeah, we have some Wyoming fans here in the audience. Yeah, so technically that could happen today. Amazing. I mean, I do think that that is a — so the speed of wealth creation. So let's talk about what are the wow moments that might be possible in the next year. What are some crazy wild moments that you might see?
I think one of the most important things is the accuracy and then these long context windows.
So explain what the long context window is again.
So you're absorbing information right now and it's quite high definition because you can see everything here and other stuff, but you're still writing it down. And the reason our organizations get big is because text is a lossy transmission format. We lose so much context. The final PDF loses all of that stuff. Now with the new Google models — Nat has some amazing companies in this space as well — you can upload hundreds of thousands of words, thousands of documents, and the AI can interpret them all at once. There doesn't need to be trained on it. So you can upload all of your ideas and say, 'Build a business based on this,' and it will do that. Or you can upload like a whole bunch of movies and then tell it to write a script that incorporates all of that, and it will do that inference time. That again is something superhuman. But we all have these massive repositories of all these ideas we've had. Being able to dump that now and then the AI, without having to be trained, spit back answers, ideas, and things like that, I think is a really huge step, along with that composition step that I kind of discussed before. I think.
Yeah, I totally agree with that. I think there will be a couple things coming probably pretty soon. It's hard to predict exact timelines on these things. Sometimes things happen faster than you expect and sometimes a little slower. But one of the clearly amazing, clearly possible now products to build is a voice-to-voice model that's indistinguishable from talking to a human, maybe for a conversation of up to a couple minutes. Where what happens after a couple of minutes? Well, maybe you can kind of just tell somehow that it's not quite human after a couple of minutes. I'm setting a milestone that I think is achievable this year. Maybe if you can do two minutes, you can do 12 minutes. I don't know. But yeah, you know, it's actually about all the technology there. It all just has to be integrated. And so you need the sort of the ability to recognize speech is there, the ability to interpret it with a language model and generate responses is there, and then the ability to turn that text into incredibly realistic voices there. And kind of putting that all together into a package that has very low latency, that's talking the way we talk, where you can kind of interrupt me, and maybe there's an avatar that's giving you this human-like — I mean, Aristotle was very impressive, but I knew that that was not a real person. And so I think we could — yeah, he was a real person. Yeah, that's right. And then I think the other thing that's a very big deal is this idea of autonomy and agents. There's been a lot of talk of it with AI over the last year. Today these things are not agents. They're tools. They're call and response. You go to ChatGPT, you type something, you hit enter, you watch the response kind of stream back. And I think what people don't necessarily understand is that when these language models are responding to you, it's almost like a rap battle. They have a fixed amount of time to generate each word, and so they can't sort of sit there and ponder for a minute, you know, what they're going to say. They have to talk to a metronome. And so that's why when you see the words that they're writing out, that's not like they've thought about it a lot and then you see the words. It's actually the thinking is happening during the output.
And would you say that's how a human does it too? I often — I mean, a lot of times, sometimes I'm like really pissed off at what came out of my mouth because I didn't think about it in advance.
Yeah, yeah, I think so. I mean, sometimes you have a conversation with a smart friend and you're just forming the words to respond, you have a better idea. And so I think we definitely do that too. But the agent, the autonomy is about kind of increasing the unit of work you can trust the AI to do without making a mistake. So right now you can ask it one question, get a response, you interpret it, you figure out the next thing. But what if it could go do 10 or 100 steps? You're talking about an AI business. You know, do you trust it to come up with the title of the blog post that you're going to post? Or do you trust it to actually come up with the whole idea of the marketing campaign, right? Come up with a strategy for how to execute it, all of the content it's going to generate, the partners it reaches out to and negotiate advertising or whatever it's going to do, have those conversations, that'll be multi-steps successfully all the way to like measuring its results without supervision. And I think that agency thing, we're starting to see it in programming. There was a very impressive demo a week or two ago of a company called Cognition that I think it was maybe the — it was arguably the first really impressive demo of working agents in AI. And they did it with GPT-4. So not with a brand new model. They did it by being very clever about the way they squeeze and distill the intelligence out of GPT-4 by repeatedly calling it and analyzing and evaluating its results and choosing the best ones. And so sort of going from rap battle to like draft and redraft and sort of think and ponder about it a little bit harder mode. And it can do hundreds of steps in programming successfully. So it's amazing.
Okay, I was going to ask you one quick question, then please jump into that. How far out can you imagine right now?
About three years. Three years is a max.
Yeah. How about you, Nate?
You can dream. Yeah. So I actually — the thing I find easier is to think about the long term because sort of the asymptote of where we're — where we're — yeah, and that's the key question of our time is the asymptote. It's like we know this is going to get better. When does it slow down and what's the shape of the curve to get there? You mind I cut you off? Apologies.
