The following is a conversation with Wojciech Zaremba, co-founder of OpenAI, which is one of the top organizations in the world doing artificial intelligence research and development. Wojciech is the head of language and code generation teams building and doing research on GitHub Copilot, OpenAI Codex, and GPT-3, and who knows, 4, 5, 6, N and N plus 1. And he also previously led OpenAI's robotic efforts. These are incredibly exciting projects to me that deeply challenge and expand our understanding of the structure and nature of intelligence. The 21st century, I think, may very well be remembered for a handful of revolutionary AI systems and their implementations. GPT, Codex, and applications of language models and transformers in general to the language and visual domains may very well be at the core of these AI systems. To support this podcast, please check out our sponsors. They're listed in the description.
As usual, I'll do a few minutes of ads now, no ads in the middle. I try to make these interesting, so hopefully you don't skip, but if you do, please still check out the sponsor links in the description. It's the best way to support this podcast. I use their stuff. I enjoy it. Maybe you will, too.
This show is brought to you by Paperspace Gradient, which is a platform that lets you build, train, deploy machine learning models of any size and complexity. I love how powerful and intuitive Gradient is. If you're already in the machine learning world or are interested in joining it, I highly recommend Fast AI, which is a set of courses by Jeremy Howard and others that introduce you to machine learning. And the reason I bring that up is they use and recommend Paperspace Gradient. One of the nice things, you can host notebooks on there. You can swap out the compute instance at any time. So you can start out with a small scale CPU GPU instance and then swap out something super powerful once you're ready, once you need it. I'm also really excited about what they're calling workflows, which provides a way to automate machine learning pipelines on top of Gradient compute infrastructure. It makes it really easy to build a production app because all the orchestration is reduced to a simple YAML file. To give Gradient a try, visit gradient.run/lex and use the sign up link there. You'll get 15 bucks in free credits which you can use to power your next machine learning application. That's gradient.run/lex. If you're interested in future machine learning projects I'm doing in terms of tutorials and educational stuff, maybe Gradient is worth a try because I'm likely to use Gradient as part of that. Anyway, like I said, gradient.run/lex.
This episode is also brought to you by Indeed, a hiring website. I've used them as part of many hiring efforts I've done for the teams I've led in the past. They have tools like Indeed Instant Match, giving you quality candidates whose resumes on Indeed fit your job description immediately. I think hiring is the most important component of a successful company, no matter what stage your company is in. I've been lucky to get a chance to interact with some very successful people in the entrepreneurial and the business world and I think that universally the thing that is common between them is the ability to build a great team. So you should definitely use the best tools for the job and Indeed should be in your toolbox right now. Get a free $75 sponsored job credit to upgrade your job post at indeed.com/lex. Get it at indeed.com/lex. Offer valid through September 30th. Terms and conditions apply. Join 3 million businesses that use Indeed by going to indeed.com/lex.
This episode is also brought to you by Blinkist, my favorite app for learning new things. Blinkist takes the key ideas from thousands of non-fiction books and condenses them down into just 15 minutes that you can read or listen to. I can recommend a bunch of books including Sapiens by Yuval Harari. I recently announced that I'm doing a book a day for a few days and actually had some chaos come up in my life which threw a wrench into that. But I will return to this project and I'm excited by the idea of sort of spending an entirety of a day reading and really sort of living through the story. But I use Blinkist to select possible non-fiction books, and I also use Blinkist to reflect on the books I've read in the past, whether I want to revisit them. Go to blinkist.com/lex to start your free 7-day trial and get 25% off a Blinkist premium membership. That's blinkist.com/lex, spelled B-L-I-N-K-I-S-T. blinkist.com/lex.
This show is also brought to you by Grammarly, a writing assistant tool that checks spelling, grammar, sentence structure, and readability. Grammarly Premium, the version you pay for, offers a bunch of extra features. My favorite is the clarity check, which helps detect rambling, over complicated chaos that many of us can descend into. I'm a big fan of quality of writing emerging through the editing process. So, the stream of consciousness where you just throw a bunch of stuff on the page and figure out which of it sticks. For me, the end result of a great piece of writing is as simple as possible and like Einstein said, not too simple. But I find it's almost difficult to arrive at something that's too simple. Grammarly is available on basically any platform and major sites and apps like Gmail and Twitter and so on. Do more than just spell check. Get your point across more effectively with Grammarly Premium. Get 20% off by signing up at grammarly.com/lex. That's grammarly.com/lex.
This episode is also brought to you by Eight Sleep and its Pod Pro mattress. It controls temperature with an app. It's like magic. It's backed with sensors. It can cool down to as low as 55 degrees on each side of the bed separately. I've been hanging out with Andrew Huberman this week and he's actually just a mastermind of temperature control and all kinds of devices. He had this nice explanation of what parts of the body to apply cooling to for it to be actually effective. It's fascinating to see what kind of effect temperature control has on you. I think during sleep for me it does magic. Even with air conditioning, just having a warm blanket and a cold bed, it makes me really look forward to the naps and the sleep. They have a Pod Pro cover, so you can just add that to your mattress without having to buy theirs. But their mattress is nice, just so you know. Again, it can track a bunch of metrics like heart rate variability, but cooling alone is honestly worth the money. Go to eightsleep.com/lex to get special savings. That's eightsleep.com/lex.
This is the Lex Fridman podcast and here's my conversation with Wojciech Zaremba.
You mentioned that Sam Altman asked about the Fermi paradox and the people at OpenAI had really sophisticated interesting answers. So that's when you knew this is the right team to be working with. So let me ask you about the Fermi paradox about aliens. Why have we not found overwhelming evidence for aliens visiting Earth?
I don't have a conviction in the answer but rather kind of probabilistic perspective on what might be, let's say, possible answers. It's also interesting that the question itself even can touch on your typical question of what's the meaning of life because if you assume that we don't see aliens because they destroy themselves, that kind of upweights the focus on making sure that we won't destroy ourselves. But at the moment the place where I am actually with my belief, and these things also change over time, is I think that we might be alone in the universe, which actually makes life more, or let's say consciousness life, more kind of valuable, and that means that we should more appreciate it.
Have we always been alone? So what's your intuition about our galaxy, our universe? Is it just sprinkled with graveyards of intelligent civilizations or are we truly, is intelligent life truly unique?
At the moment my belief that it is unique, but I would say I could also, you know, there was like some footage released with UFO objects which makes me actually doubt my own belief.
Uh, yeah I can tell you one crazy answer that I have heard.
Um so apparently when you look actually at the limits of computation, you can compute more if the temperature of the universe would drop down. So one of the things that aliens might want to do if they are truly optimizing to maximize amount of compute, which maybe can lead to or let's say simulations or so, is instead of wasting current entropy of the universe, because by living we are actually somewhat wasting entropy, then you can wait for the universe to cool down such that you have more computation. That's kind of a funny answer, I'm not sure if I believe in it, but that would be one of the reasons why you don't see aliens. It's also possible. See, some people say that maybe there is not that much point in actually going to other galaxies if you can go inwards. So there is no limits of what could be an experience if we could connect machines to our brains. While there are still some limits if you want to explore universe.
Yeah, there could be a lot of ways to go inwards too once you figure out some aspect of physics we haven't figured out yet. Maybe you can travel to different dimensions. I mean, travel in three-dimensional space may not be the most fun kind of travel. There may be like just a huge amount of different ways to travel. It doesn't require a spaceship going slowly in 3D space to space time. It also feels, you know, one of the problems is that speed of light is low and the universe is vast.
Yeah. And it seems that actually most likely if we want to travel very far then instead of actually sending spaceships with humans that weight a lot, we would send something similar to what Yuri Milner is working on. These are like a huge sail which is at first powered. There is a shot of laser from earth and it can propel it to quarter of speed of light. And the sail itself contains a few grams of equipment and that might be the way to actually transport matter through universe. But then when you think what would it mean for humans, it means that we would need to actually put a 3D printer and 3D print a human on other planet. I don't know, play them YouTube or let's say or like pre-3D print like huge human right away or maybe a womb or so.
With our current techniques of archaeology, if a civilization was born and died long enough ago on Earth, we wouldn't be able to tell. And so that makes me really sad. And so I think about Earth in that same way. How can we leave some remnants if we do destroy ourselves? How can we leave remnants for aliens in the future to discover? Like here's some nice stuff we've done like Wikipedia and YouTube. Do we have it like in a satellite orbiting Earth with a hard drive? Like how do we back up human civilization for the good parts, or all of it is good parts, so that it can be preserved longer than our bodies can.
That's a kind of a, it's a difficult question. It also requires the difficult acceptance of the fact that we may die and if we die we may die suddenly as a civilization. So let's see. I think it kind of depends on the cataclysm. We have observed in other parts of universe that births of gamma rays, these are high energy rays of light that actually can apparently kill entire galaxy. So there might be actually nothing even to protect us from it. I'm also when I'm looking actually at the past civilizations, so it's like Mayans or so, they disappear from the surface of the earth and one can ask why is it the case. And the way I'm thinking about it is, you know, that definitely they had some problem that they couldn't solve and maybe there was a flood and all of a sudden they couldn't drink, there was no potable water and they all died. And I think that so far the best solution to such problems is, I guess, technology. So I mean if they would know that you can just boil water and then drink it after, then that would save their civilization. And even now when we look actually at the current pandemic, it seems that once again actually science comes to rescue and somehow science increases size of the action space and I think that's a good thing.
Yeah but nature has a vastly larger action space.
Yeah, but still it might be a good thing for us to keep on increasing action space.
Well, okay. Looking at past civilizations, yes, but looking at the destruction of human civilization, perhaps expanding the action space will add actions that are easily acted upon, easily executed, and as a result destroy us.
So, let's see. I was pondering why actually even we have negative impact on the globe because if you ask every single individual they would like to have clean air they would like healthy planet but somehow it actually is not the case that as a collective we are not going this direction. I think that there exist very powerful system to describe what we value, that's capitalism. It assigns actually monetary values to various activities. At the moment the problem in the current system is that there are some things which we value, there is no cost assigned to it. So even though we value clean air or maybe we also value lack of distraction on let's say internet or so, at the moment these quantities, companies, corporations can pollute them for free. So in some sense I wished or like that's I guess purpose of politics to align the incentive systems and we are kind of maybe even moving in this direction. The first issue is even to be able to measure the things that we value, then we can actually assign the monetary value to them.
Yeah. And that's so it's getting the data and also probably through technology enabling people to vote and to move money around in a way that is aligned with their values. And that's very much a technology question. So like having one president and Congress and voting that happens every four years or something like that, that's a very outdated idea. There could be some technological improvements to that kind of idea.
So I'm thinking from time to time about these topics but it also feels to me that it's a little bit like, it's hard for me to actually make correct predictions what is the appropriate thing to do. I extremely trust Sam Altman our CEO on these topics. I'm more on the side of being I guess naive hippie.
That's your life philosophy. Well, I think self-doubt and I think hippie implies optimism, those two things are a pretty good way to operate.
I mean still it is hard for me to actually understand how the politics works or like exactly how the things would play out and Sam is really excellent with it.
What do you think is rarest in the universe? You said we might be alone. What's hardest to build is another engineering way to ask that. Life, intelligence, or consciousness. So like you said that we might be alone. Which is the thing that's hardest to get to? Is it just the origin of life? Is it the origin of intelligence? Is it the origin of consciousness?
So let me at first explain you my kind of mental model, what I think is needed for life to appear.
So I imagine that at some point there was this primordial soup of amino acids and maybe some proteins in the ocean and some proteins were turning into some other proteins through reaction and you can almost think about this cycle of what turns into what as there is a graph essentially describing which substance turns into some other substance and essentially life means that all of the sudden in the graph has been created a cycle such that the same thing keeps on happening over and over again. That's what is needed for life to happen. And in some sense you can think almost that you have this gigantic graph and it needs like a sufficient number of edges for the cycle to appear.
Then from perspective of intelligence and consciousness, my current intuition is that they might be quite intertwined. First of all, it might not be that it's like a binary thing that you have intelligence or consciousness. It seems to be more continuous component. Let's see if we look for instance on the even networks recognizing images and people are able to show that the activations of these networks correlate very strongly with activations in visual cortex of some monkeys. The same seems to be true about language models. Also if you for instance look if you train agent in a 3D world, at first it barely recognizes what is going on. Over the time it kind of recognizes foreground from background, over the time it kind of knows where there is a foot and it just follows it. Over the time it actually starts having a 3D perception.
So it is possible for instance to look inside of the head of an agent and ask what would it see if it looks to the right and the crazy thing is initially when the agents are barely trained these predictions are pretty bad, over the time they become better and better. You can still see that if you ask what happens when the head is turned by 360 degrees for some time they think that the different thing appears and then at some stage they understand actually that the same thing supposed to appear. So they get like an understanding of 3D structure. It's also very likely that they have inside some level of symbolic reasoning, there particularly symbols for other agents. So when you look at Dota agents they collaborate together and they have some anticipation of if they would win battle. They have some expectations with respect to other agents. I might be too much anthropomorphizing how the things look for me but the fact that they have a symbol for other agents makes me believe that at some stage as they are optimizing for skills they would have also symbol to describe themselves.
This is like a very useful symbol to have and this particularly I would call it self-consciousness or self-awareness. And still it might be different from the consciousness. So I guess the way how I'm understanding the word consciousness, let's say the experience of drinking a coffee or let's say experience of being a bat.
That's the meaning of the word consciousness. It doesn't mean to be awake. Yeah it feels it might be also somewhat related to memory and recurrent connections. So it's kind of like if you look at anesthetic drugs they might be pre like they essentially they disturb brain waves such that the memory is not formed.
Mhm. And so there's a lessening of consciousness when you do that.
And so that's one way to intuit what is consciousness. There's also kind of another element here. It could be that it's this kind of self-awareness module that you described plus the actual subjective experience is a storytelling module that tells us a story about what we're experiencing.
The crazy thing, so let's say I mean in meditation they teach people not to speak story inside of the head and there is also some fraction of population who doesn't have actually narrator. I know people who don't have a narrator and they have to use external people in order to kind of solve tasks that require internal narrator. So it seems that it's possible to have the experience without the talk.
What are we talking about when we talk about the internal narrator? Is that the voice when you're like reading?
Yeah, I thought that that's what you are referring to.
Well, I was referring more on the like not an actual voice. I meant like there's some kind of subjective experience that feels like it's fundamentally about storytelling to ourselves. It feels like the feeling is a story that is much simpler abstraction than the raw sensory information. So there feels like it's a very high level abstraction that is useful for me to feel like entity in this world. The most useful aspect of it is that because I'm conscious, I think there's an intricate connection to me not wanting to die. So like it's a useful hack to really prioritize not dying. Like those seem to be somehow connected. So I'm telling this story of like it richly feels like something to be me and the fact that me exists in this world. I want to preserve me and so that makes it a useful agent hack.
So I will just refer maybe to the first part as you said about the kind of story of describing who you are. I was thinking about it even so, you know, obviously I like thinking about consciousness. I like thinking about AI as well and I'm trying to see analogies of these things in AI. What would it correspond to? So OpenAI trained a model called GPT which can generate pretty amusing text on arbitrary topic.
And one way to control GPT is by putting into prefix at the beginning of the text some information what would be the story about.
You can have even chat with GPT by saying that the chat is with Lex or Elon Musk or so and GPT would just pretend to be you or Elon Musk or so and it almost feels that this story that we give ourselves to describe our life it's almost like things that you put into context of GPT.
Yeah, but the context we provide to GPT is multimodal.
So GPT itself is multimodal. GPT itself hasn't learned actually from experience of single human but from the experience of humanity. It's a chameleon, you can turn it into anything and in some sense by providing context it behaves as the thing that you wanted it to be. It's interesting that people have stories of who they are and as you said these stories they help them to operate in the world. But it's also interesting, I guess various people find it out through meditation or so, that there might be some patterns that you have learned when you were a kid that actually are not serving you anymore and you also might be thinking that that's who you are and that's actually just a story.
Yes. That's a useful hack, but sometimes it gets us into trouble. It's a local optima.
It's a local optima. You wrote that Stephen Hawking, he tweeted Stephen Hawking asked what breathes fire into equations, which meant what makes given mathematical equations realize the physics of a universe. Similarly, I wonder what breathes fire into computation. What makes given computation conscious? So how do we engineer consciousness? How do you breathe fire and magic into the machine?
So it seems clear to me that not every computation is conscious. I mean you can let's say just keep on multiplying one matrix over and over again and maybe gigantic matrix you can put a lot of computation. I don't think it would be conscious. So in some sense the question is what are the computations which could be conscious. I mean so one assumption is that it has to do purely with computation that you can abstract away matter. Other possibilities that it's very important what's the realization of computation, that it has to do with some force fields or so and they bring consciousness. At the moment my intuition is that it can be fully abstracted away. So in case of computation you can ask yourself what are the mathematical objects or so that could bring such properties. So for instance, if we think about the models, AI models, what they truly try to do, or like models like GPT, is they try to predict next word or so. And this turns out to be equivalent to compressing text. And because in some sense compression means that you learn the model of reality and you have just to remember where are your mistakes, the better you are in predicting. And in some sense when we look at our experience also when you look for instance at a car driving, you know in which direction it will go. We are good in prediction and it might be the case that consciousness is intertwined with compression.
It might be also the case that self-consciousness has to do with compressor trying to compress itself. So okay I was just wondering what are the objects in mathematics or computer science which are mysterious that could have to do with consciousness. And then I thought you know you see in mathematics there is something called Gödel's theorem which means okay if you have sufficiently complicated mathematical system it is possible to point the mathematical system back on itself. In computer science there is something called the halting problem, it's somewhat similar construction. So I thought that if we believe that under assumption that consciousness has to do with compression, then you could imagine that as you keep on compressing things then at some point it actually makes sense for the compressor to compress itself.
Consciousness is meta compression.
Thank you. So but do you think if we think of a Turing machine, a universal Turing machine, can that achieve consciousness? So is there some beyond our traditional definition of computation that's required?
So it's a specific computation and I said this computation has to do with compression.
Mhm. And the compression itself, maybe another way of putting it is you are internally creating the model of reality in order to simplify reality in order to predict what's going to happen.
And that also feels somewhat similar to how I think actually about my own conscious experience. So clearly I don't have access to reality. The only access to reality is through a cable going to my brain and my brain is creating a simulation of reality and I have access to the simulation of reality. Are you by any chance aware of the Hutter prize? Marcus Hutter, he made this prize for compression of Wikipedia pages and there's a few qualities to it. One, I think, has to be perfect compression which makes, I think that little quirk makes it much less applicable to the general task of intelligence because it feels like intelligence is always going to be messy, like perfect compression feels like is not the right goal but it's nevertheless a very interesting goal. So for him, intelligence equals compression. And so the smaller you make the file given a large Wikipedia page, the more intelligent the system has to be.
Yeah, that makes sense. So you can make perfect compression if you store errors. And I think that actually what he meant is you have algorithm plus errors. By the way, Hutter was PhD advisor of Shane Legg who is DeepMind co-founder.
Yeah. So there's an interesting, and now he's at DeepMind. There's an interesting network of people. He's one of the people that I think seriously took on the task of what would an AGI system look like. I think for the longest time the question of AGI was not taken seriously or rather rigorously and he did just that. Like mathematically speaking what would the model look like? If you remove the constraints of it having to have reasonable amount of memory, reasonable amount of running time complexity, computation time, what would it look like? And essentially it's a half math, half philosophical discussion of how would a reinforcement learning type of framework look like for an AGI.
Yeah. So he developed a framework even to describe what's optimal with respect to reinforcement learning. Like there is a theoretical framework which is as you said under assumption there is infinite amount of memory and compute. There was actually one person before, his name is Solomonoff. Hutter extended Solomonoff's work to reinforcement learning but there exists a theoretical algorithm which is optimal algorithm to build intelligence and I can actually explain you the algorithm.
Yes. Let's go. Let's go. Let's go.
So the task itself you can...
Can I just pause how absurd it is for a brain in a skull trying to explain the algorithm for intelligence. Just go ahead.
It is pretty crazy. It is pretty crazy that the brain itself is actually so small and it can ponder.
How to design algorithms that optimally solve the problem of intelligence. Okay. All right. So what's the algorithm?
So let's see. So first of all the task itself is described as you have infinite sequence of zeros and ones. Okay you read n bits and you are about to predict n plus one bit. So that's the task and you could imagine that every task could be casted as such a task. So if for instance you have images and labels you can just turn every image into sequence of zeros and ones then label, you concatenate labels and that's actually the, and you could start by having training data first and then afterwards you have test data. So theoretically any problem could be casted as a problem of predicting zeros and ones on this infinite tape. So let's say you read already n bits and you want to predict n plus one bit and I will ask you to write every possible program that generates these n bits. Okay. So and you choose programming language, it can be in Python or C++ and the difference between programming languages might be there is a difference by constant, asymptotically your predictions will be equivalent. So you read n bits, you enumerate all the programs that produce this n bits in their output. And then in order to predict n plus one bit you actually weight the programs according to their length. There is like some specific formula how you weight them and then the n plus one bit prediction is the prediction from each of these programs according to the weight.
Like statistically you pick, so the smaller the program the more likely you are to pick its output.
So that algorithm is grounded in the hope or the intuition that the simple answer is the right one.
It's a formalization of it.
It also means like if you would ask the question after how many years would the sun explode, you can say hm it's more likely the answer is two to some power because it's a shorter program.
Yeah. Okay. And I don't have a good intuition about how different the space of short programs are from the space of large programs. Like what is the universe where short programs run things.
So I said the things have to agree with n bits. So even if you have a very short program and there's still some, if it's not perfect with prediction of n bits you have to store errors, what are the errors and that gives you the full program that agrees on n bits.
Oh so you don't agree perfectly with the n bits and you store the errors.
That's like a longer program, slightly longer program because it contains these extra bits of errors.
That's fascinating. What's your intuition about the programs that are able to do cool stuff like intelligence and consciousness? Are they perfectly, like is there if-then statements in them? So like is there a lot of exceptions that they're storing?
So you could imagine if there would be tremendous amount of if statements then they wouldn't be that short. In case of neural networks, you could imagine that what happens is when you start with an uninitialized neural network it stores internally many possibilities how the problem can be solved and SGD is kind of magnifying some paths which are slightly similar to the correct answer. So it's kind of magnifying correct programs and in some sense SGD is a search algorithm in the program space and the program space is represented by the wiring inside of the neural network and there's like an insane number of ways how the features can be computed.
Let me ask you the high-level basic question that's not so basic. What is deep learning? Is there a way you'd like to think of it that is different than like a generic textbook definition?
The thing that I hinted just a second ago is maybe the closest to how I'm thinking these days about deep learning. So now the statement is neural networks can represent some programs. It seems that various modules that we are actually adding up to, we want networks to be deep because we want multiple steps of the computation. And deep learning provides the way to represent space of programs which is searchable and it's searchable with stochastic gradient descent. So we have an algorithm to search over a humongous number of programs and gradient descent kind of bubbles up the things that tend to give correct answers.
So a neural network with fixed weights that's optimized, do you think of that as a single program?
So there is a work by Christopher Olah where he works on interpretability of neural networks and he was able to identify inside of the neural network for instance a detector of a wheel for a car or the detector of a mask for a car and then he was able to separate them out and assemble them together using a simple program for the detector for a car.
That's like a, if you think of traditionally defined programs, that's like a function within a program that this particular neural network was able to find and you can tear that out just like you can copy and paste from Stack Overflow.
So any program is a composition of smaller programs.
Yeah. I mean the nice thing about the neural networks is that it allows the things to be more fuzzy than in case of programs. In case of programs you have this like branching this way or that way and neural networks they have an easier way to be somewhere in between or to share things. What do you think is the most beautiful or surprising idea in deep learning in the utilization of these neural networks, which by the way for people who are not familiar, neural networks is a bunch of, what would you say, it's inspired by the human brain, there's neurons, there's connections between those neurons, there's inputs and there's outputs and there's millions or billions of those neurons and the learning happens by adjusting the weights on the edges that connect these neurons.
Thank you for giving the definition that I was supposed to do it. But I guess you have enough empathy to listeners to actually know that that might be useful.
No, that's like so I'm asking Plato what is the meaning of life? He's not going to answer. You're being philosophical and deep and quite profound talking about the space of programs which is very interesting. But people are just not familiar what the hell we're talking about when we talk about deep learning. Anyway, sorry. What is the most beautiful or surprising idea to you in all the time you've worked on deep learning and you worked on a lot of fascinating projects, applications of neural networks, it doesn't have to be big and profound. It could be a cool trick.
Yeah. I mean, I'm thinking about the trick, but it's still amusing to me that it works at all.
Yeah. That let's say that the extremely simple algorithm, stochastic gradient descent, which is something that I would be able to derive on a piece of paper to a high school student, when put at the scale of thousands of machines actually can create the behaviors which we called kind of human-like behaviors. So in general any application of stochastic gradient descent to neural networks is amazing to you. So is there a particular application in natural language, reinforcement learning? And also what do you attribute that success to? Is it just scale? What profound insight can we take from the fact that the thing works for gigantic sets of variables?
I mean the interesting thing is these algorithms they were invented decades ago and people actually gave up on the idea. And you know back then they thought that we need profoundly different algorithms and they spent a lot of cycles on very different algorithms and I believe that we have seen that various innovations, let's say like transformer or dropout or so, they can help. But it's also remarkable to me that this algorithm from the 60s or so, or I mean you can even say that the gradient descent was invented by Leibniz in I guess 18th century or so, that actually is the core of learning.
In the past people are, it's almost like out of maybe an ego, people are saying that it cannot be the case that such a simple algorithm could solve complicated problems. So they were in search for other algorithms. And as I'm saying, I believe that actually we are in the game where there are actually frankly three levers. There is compute, there are algorithms and there is data. And if we want to build intelligent systems, we have to pull all three levers and they are actually multiplicative. It's also interesting. So you ask is it only compute? People internally they did the studies to determine how much gains they were coming from different levers and so far we have seen that more gains came from compute than algorithms but also we are in the world that in case of compute there is a kind of exponential increase in funding and at some point it's impossible to invest more, it's impossible to invest 10 trillion because we are speaking about the let's say all taxes in US.
You're talking about money, there could be innovation in the compute.
That's true as well. So I mean there are like a few pieces. So one piece is human brain is an incredible supercomputer.
And they're like a, it has 100 trillion parameters.
Or like if you try to count various quantities in the brain, there are neurons, synapses. There are small number of neurons. There is a lot of synapses. It's unclear how to map synapses to parameters of neural networks, but it's clear that there are many more. So it might be the case that our networks are still somewhat small. It also might be the case that they are more efficient than brain or less efficient by some huge factor. I also believe that there will be, you know, at the moment we are at the stage that these neural networks, they require thousand x or a huge factor of more data than humans do. And it will be a matter of, there will be algorithms that vastly decrease sample complexity, I believe so. But the place where we are heading today is there are domains which contain million x more data, and even though computers might be a thousand times slower than humans in learning, that's not the problem. Like for instance, I believe that it should be possible to create superhuman therapies by, and even simple steps of doing it. And you know, the core reason is there is just, machine will be able to read way more transcripts of therapies, and then it should be able to speak simultaneously with many more people, and it should be possible to optimize it all in parallel.
Well, now you're touching on something I deeply care about and think is way harder than we imagine. What's the goal of a therapist?
What's the goal of therapist? So okay, so one goal, now this is terrifying to me, but there's a lot of people that contemplate suicide, suffer from depression, and they could significantly be helped with therapy. And the idea that an AI algorithm might be in charge of that, it's like a life and death task, the stakes are high. So one goal for a therapist, whether human or AI, is to prevent suicide ideation, to prevent suicide. How do you achieve that?
So let's see. So to be clear, I don't think that the current models are good enough for such a task because it requires insane amount of understanding, empathy, and the models are far from this place. But it's...
But do you think that understanding, empathy, that signal is in the data?
Um, I think there is some signal in the data. Yes. I mean there are plenty of transcripts of conversations and it is possible to, it is possible from it to understand personalities. It is possible from it to understand if a conversation is friendly, amicable, antagonistic. I believe that, you know, given the fact that the models that we train now, they can, they are chameleons, that they can have any personality, they might turn out to be better in understanding personality of other people than anyone else. And they...
Yeah. Interesting. But I wonder if there's some level of multiple modalities is required to be able to be empathetic of the human experience. Whether language is not enough to understand death, to understand fear, to understand childhood trauma, to understand wit and humor required when you're dancing with a person who might be depressed or suffering, both humor and hope and love and all those kinds of things. So there's another underlying question which is self-supervised versus supervised. So can you get that from the data by just reading a huge number of transcripts?
I actually, so I think that reading huge number of transcripts is a step one. It's like the same way as you cannot learn to dance just from YouTube by watching it. You have to actually try it out yourself. So I think that here that's a similar situation. I also wouldn't deploy the system in the high stakes situations right away, but kind of see gradually where it goes. And obviously initially it would have to go hand in hand with humans. But at the moment we are in the situation that actually there is many more people who actually would like to have a therapy or speak with someone than there are therapists out there. Like, you know, I was so, so fundamental, I was thinking what are the things that can vastly increase people's wellbeing. Therapy is one of them. I think meditation is another one. I guess maybe human connection is a third one. And I guess pharmacologically it's also possible. Maybe direct brain stimulation or something like that. But these are pretty much options out there. Then let's say the way I'm thinking about the AGI endeavor is by default that's an endeavor to increase amount of wealth, and I believe that we can vastly increase amount of wealth for everyone and simultaneously. So I mean these are like two endeavors that make sense to me. One is like essentially increase amount of wealth, and second one is increase overall human wellbeing.
And those are coupled together.
And they can, like I would say these are different topics, one can help another.
And you know, therapist is a funny word because I see friendship and love as therapy. I mean so therapist broadly defined as just friendship is a friend. So like therapist has a very kind of clinical sense to it. But what is human connection? You're like, not to get all Camus and Dostoevsky on you, but you know, life is suffering and we seek connection with other humans as we desperately try to make sense of this world in the deep overwhelming loneliness that we feel inside.
So I think connection has to do with understanding. And I think that almost like a lack of understanding causes suffering. If you speak with someone and you feel ignored, that actually causes pain. If you are feeling deeply understood, that actually, they might not even tell you what to do in life, but like a pure understanding...
Or just being heard. Understanding is kind of, that's a lot, you know, just being heard. Feel like you're being heard. Like somehow that's an alleviation temporarily of the loneliness, that if somebody knows you're here with their body language, with the way they are, with the way they look at you, with the way they talk, you feel less alone for a brief moment.
Yeah, very much agree. So I thought in the past about the somewhat similar question to yours, which is what is love, rather what is connection. And obviously I think about these things from AI perspective. What would it mean? So I said that, you know, intelligence has to do with some compression, which is more or less like you can say almost understanding of what is going around. It seems to me that other aspect is there seem to be reward functions, and you can have a, you know, reward for food, for maybe human connection, for let's say warmth, sex, and so on. And it turns out that various people might be optimizing slightly different reward functions. They essentially might care about different things. And in case of love, at least the love between two people, you can say that the boundary between people dissolves to such extent that they end up optimizing each other's reward functions.
Yeah. Oh that's interesting. Celebrate the success of each other.
Yeah. Or in some sense I would say love means helping others to optimize their reward functions. Not your reward functions, not the things that you think are important, but the things that the person cares about, you try to help them to optimize it.
So love is, if you think of two reward functions, you just, it's a condition you combine them together.
Maybe like with the weight, and it depends like the dynamic of the relationship.
Yeah. I mean you could imagine that if you're fully optimizing someone's reward function without yours, then maybe you're creating codependency or something like that.
I'm not sure what's the appropriate weight. But the interesting thing is I even think that the individual person, we ourselves, we are actually less of a unified entity. So for instance if you look at a donut, on one level you might think, oh this looks tasty, I would like to eat it. On another level you might tell yourself, I shouldn't be doing it because I want to gain muscles. And you know, you might do it regardless, kind of against yourself. So it seems that even within ourselves they're almost like kind of intertwined personas. And I believe that self love means that the love between all these personas, which also means being able to love yourself when we are angry or stressed or so.
Combining all those reward functions of the different selves you have.
Yeah. And accepting that they are there. Like, you know, often people they have a negative self-talk or they say I don't like when I'm angry. And like I try to imagine, try to imagine if there would be like a small baby Lex, like a five years old, angry, and then you're like, you shouldn't be angry, like stop being angry. But instead actually you want the Lex to come over, give him a hug, and he's like, I say it's fine, okay, you can be angry as you want, and then he would stop.
Or maybe not. But you cannot expect it even.
Yeah. But still that doesn't explain the why of love. Like why is love part of the human condition? Why is it useful to combine the reward functions? It seems like that doesn't, I mean I don't think reinforcement learning frameworks can give us answers to why. Even the Hutter framework has an objective function that's static.
So we came to existence as a consequence of evolutionary process, and in some sense the purpose of evolution is survival. And then this complicated optimization objective baked into us, let's say compression, which might help us operate in the real world, and it baked into us various reward functions. Then to be clear, at the moment we are operating in the regime which is somewhat out of distribution where even evolution optimized us. It's almost like love is a consequence of cooperation that we've discovered is useful.
Correct. In some way, it's even the case if you... I just love the idea that love is like the out of distribution, or it's not out of distribution. It's like as you said, it evolved for cooperation.
Yes. And I believe that cooperation, like in some sense, cooperation ends up helping each of us individually. So it makes sense evolutionary. And in some sense, you know, love means there is this dissolution of boundaries, that you have a shared reward function. And we evolve to actually identify ourselves with larger groups. So we can identify ourselves, you know, with a family, we can identify ourselves with a country, to such extent that people are willing to give away their life for country. So we are wired actually even for love. And at the moment I guess maybe it would be somewhat more beneficial if we would identify ourselves with all of humanity as a whole. So you can clearly see when people travel around the world, when they run into a person from the same country, they say oh which city you are from, and all this, like all of a sudden they find all these similarities, they befriend those folks earlier than others. So there is some sense of belonging. And I would say I think it would be an overall good thing to the world for people to move towards, I think it's even called open individualism, move toward the mindset of a larger and larger group.
So the challenge there, and that's a beautiful vision and I share it, to expand that circle of empathy, that circle of love towards the entirety of humanity. But then you start to ask, well where do you draw the line? Because why not expand it to other conscious beings? And then finally for our discussion, something I think about is why not expand it to AI systems? Like we start respecting each other when the person, the entity on the other side has the capacity to suffer, because then we develop a capacity to sort of empathize. And so I could see AI systems that are interacting with humans more and more having conscious-like displays. So like they display consciousness through language and through other means. And so then the question is like, well is that consciousness because they're acting conscious? And so, you know, the reason we don't like torturing animals is because they look like they're suffering when they're tortured. And if AI looks like it's suffering when it's tortured, how is that not requiring of the same kind of empathy from us and respect and rights that animals do and other humans do?
I think it requires empathy as well. I mean I would guess us or humanity also make progress in understanding what consciousness is, because I don't want just to be speaking about the philosophy but rather actually make a scientific, to have a, like you know, there was a time that people thought that there is a force of life and the things that have this force, they are alive. And I think that there is actually a path to understand exactly what consciousness is. And in some sense it might require essentially putting probes inside of a human brain.
So the goal there, I mean there's several things with consciousness that make it a real discipline, which is one is rigorous measurement of consciousness, and then the other is the engineering of consciousness, which may or may not be related. I mean, you could also run into trouble, like for example in the United States, the DOT, Department of Transportation, and a lot of different places put a value on human life. I think DOT values nine million dollars per person. Sort of in that same way you can get into trouble if you put a number on how conscious a being is, because then you can start making policy. If a cow is 0.1 or like ten percent as conscious as a human, then you can start making calculations and might get you into trouble. But then again that might be a very good way to do it. I would like to move to the place that actually we have scientific understanding what consciousness is. And then we'll be able to actually assign value. And I believe that there is even the path for the experimentation in it. So you know, we said that you could put the probes inside of the brain. There is actually few other things that you could do with devices like Neuralink. So you could imagine that the way even to measure if an AI system is conscious is by literally just plugging into the brain.
I mean that assumes that's kind of easy, but the plugging into the brain and asking a person if they feel that their consciousness expanded. This direction of course has some issues. You can say, you know, if someone takes a psychedelic drug they might feel that their consciousness expanded even though the drug itself is not conscious.
Right. So like you can't fully trust the self-report of a person saying their consciousness is expanded or not. Let me ask you a little bit about psychedelics, because there have been a lot of excellent research on different psychedelics, psilocybin, MDMA, even DMT, drugs in general, marijuana too. What do you think psychedelics do to the human mind? It seems they take the human mind to some interesting places. Is that just a little hack, a visual hack, or is there some profound expansion of the mind?
So let's see. I don't believe in magic. I believe in science, in causality. Still, let's say, and then as I said, I think that the brain, that our subjective experience of reality is, we live in a simulation run by our brain, and the simulation that our brain runs, they can be very pleasant or very hellish. Drugs, they are changing some hyperparameters of the simulation. It is possible thanks to change of these hyperparameters to actually look back on your experience and even see that the given things that we took for granted, they are changeable. So they allow to have an amazing perspective. There is also, for instance, the fact that after DMT people can see the full movie inside of their head, gives me further belief that the brain can generate the full movie, that the brain is actually learning the model of reality to such extent that it tries to predict what's going to happen next.
Yeah. Very high resolution. So it can replay reality.
Extremely high resolution. Yeah. It's also kind of interesting to me that somehow there seems to be some similarity between these drugs and meditation itself. And I actually started even these days to think about meditation as a psychedelic.
Um, do you practice meditation?
I practice meditation. I mean I went a few times on the retreats and it feels, after like after second or third day of meditation, there is almost like a sense of, you know, tripping.
What does a meditation retreat entail?
So I mean you wake up early in the morning and you meditate for extended period of time, and...
Yeah. So it's optimized, even though there are other people, it's optimized for isolation. So you don't speak with anyone, you don't actually look into other people's eyes.
And you know, you sit on the chair. Vipassana meditation tells you to focus on the breath. So you try to put all attention into breathing, and breathing in and breathing out. And the crazy thing is that as you focus attention like that, after some time there starts coming back some memories that you completely forgotten. It almost feels like you have a mailbox and then you are just archiving email one by one. And at some point there is this amazing feeling of getting to mailbox zero, zero emails, and it's very pleasant. It's kind of, it's crazy to me that once you resolve these inner stories or inner traumas, then once there is nothing left, the default state of human mind is extremely peaceful and happy. Extremely, like some sense it feels, at least to me, the way how when I was a child, that I can look at any object and it's very beautiful. I have a lot of curiosity about the simple things. And that's where usually meditation takes me.
Are you, what are you experiencing? Are you just taking in simple sensory information and are just enjoying the rawness of that sensory information? So there's no memories or all that kind of stuff. You're just enjoying being.
Yeah, pretty much. I mean, still, thoughts are slowing down. Sometimes they pop up, but it's also somehow the extended meditation takes you to the space that they are way more friendly, way more positive. There is also this thing that we've, it almost feels that we are constantly getting a little bit of a reward function and we are just spreading this reward function on various activities. But if you stay still for extended period of time, it kind of accumulates, accumulates, accumulates. And there is a sense that at some point it passes some threshold and it feels as a drop is falling into kind of ocean of love and bliss, and that's very pleasant. And as I'm saying, that corresponds to the subjective experience. Some people, I guess in the spiritual community, they describe it that that's the reality. And I would say I believe that there are all sorts of subjective experience that one can have. And I believe that for instance meditation might take you to the subjective experiences which are very pleasant, collaborative, and I would like the world to move toward a more collaborative place. Yeah, I would say that's very pleasant and I enjoy doing stuff like that.
I wonder how that maps to your mathematical model of love with the reward function combining a bunch of things. It seems like our life then is we have this reward function and we're accumulating a bunch of stuff in it with weights. It's like multi-objective. And what meditation is, is you just remove them, remove them until the weight on one or just a few is very high, and that's where the pleasure comes from.
Yeah. So something similar how I'm thinking about this. So I told you that there is this story of who you are. And I think almost about it as, you know, text prepended to GPT.
And some people refer to it as ego. Okay. It's like a story of who you are. Okay.
So ego is the prompt for GPT-3 or GPT.
Yes. Yes. And that's a description of you. And then with meditation you can get to the point that actually you experience things without the prompt.
And you experience things as they are. You are not biased toward the description of how they're supposed to be. That's very pleasant. And then with respect to the reward function, it's possible to get to the point that there is dissolution of self, and therefore you can say that your brain attempts to simulate the reward function of everyone else, or everything. That there is this love which feels like oneness with everything, and that's also very beautiful, very pleasant. At some point you might have a lot of altruistic thoughts during that moment, and then the self always comes back.
How would you recommend if somebody is interested in meditation, like a big thing to take on as a project, would you recommend a meditation retreat? How many days, what kind of thing would you recommend?
I think that actually retreat is the way to go. It almost feels that, as I said, meditation is a psychedelic, but when you take it in a small dose you might barely feel it. Once you get the high dose, actually you're going to feel it.
So even cold turkey, if you haven't really seriously meditated for a prolonged period of time, just go to a retreat. How many days? How many days?
Start with a weekend one.
Weekend. So like two, three days.
Mhm. And it's interesting that first or second day it's hard, and at some point it becomes easy.
There's a lot of seconds in a day. How hard is the meditation retreat, just sitting there in a chair?
So the thing is actually, it literally just depends on your own framing. Like if you are in the mindset that you are waiting for it to be over, or you are waiting for nirvana to happen, it will be very unpleasant. And in some sense even the difficulty, it's not even in the lack of being able to speak with others. Like you are sitting there, your legs will hurt from sitting.
In terms of like practical things, do you experience kind of discomfort, like physical discomfort of just sitting, like your butt being numb, your legs being sore, all that kind of stuff?
Yes, you experience it, and then they teach you to observe it rather. And the crazy thing is you at first might have a feeling toward trying to escape it.
Yeah. And that becomes very apparent that that's extremely unpleasant.
And then you just observe it, and at some point it just becomes, it just is. It's like I remember Ilya told me some time ago that he takes a cold shower, and his mindset of taking a cold shower was to embrace suffering.
Yeah. Excellent. I do the same.
Yeah, it's my style. I like this. So my style is actually I also sometimes take cold showers.
It is purely observing how the water goes through my body, like purely being present, not trying to escape from there.
And I would say then it actually becomes pleasant.
It's not like, ah well, that's interesting. I'm also, I mean that's the way to deal with anything really difficult, especially in the physical space, is to observe it, to say it's pleasant.
I would use a different word. You're accepting of the full beauty of reality, I would say. Because say pleasant, but yeah, in some sense it is pleasant.
That's the only way to deal with a cold shower is to become an observer and to find joy in it. Same with like really difficult physical exercise or like running for a really long time, endurance events, just anytime you're exhausted, any kind of pain. I think the only way to survive it is not to resist it, is to observe it. You mentioned Ilya, Ilya Sutskever. He's very, he's our chief scientist, but also he's a very close friend of mine. He co-founded OpenAI with you. I've spoken with him a few times. He's brilliant. I really enjoy talking to him.
His mind just like yours works in fascinating ways. Now, both of you are not able to define deep learning simply. What's it like having him as somebody you have technical discussions with in the space of machine learning, deep learning, AI, but also life? What's it like when these two agents get into a self-play situation in a room? What's it like collaborating with him?
So, I believe that we have extreme respect to each other. So I love Ilya's insight, both I guess about consciousness, life, AI. But in terms of the, it's interesting to me because you're a brilliant thinker in the space of machine learning, like intuition, like digging deep in what works, what doesn't, why it works, why it doesn't. And so is Ilya.
Yeah, I'm wondering if there's interesting deep discussions you've had with him in the past or disagreements that were very productive.
So I can say I also understood over the time where are my strengths. So obviously we have plenty of AI discussions. And you know, I myself have plenty of ideas, but I consider Ilya one of the most prolific AI scientists in the entire world. And I think that I realized that maybe my super skill is being able to bring people to collaborate together. That I have some level of empathy that is unique in AI world, and that might come from either meditation, psychedelics, or let's say I read just hundreds of books on this topic. And I also went through a journey of, you know, I developed all sorts of algorithms. So I think that maybe that's my superhuman skill. Ilya is one of the best AI scientists, but then I'm pretty good in assembling teams. And I'm also not holding to people, like I'm growing people, and then people become managers at OpenAI, like I grew many of them, like research managers.
So you find places where you're excellent, and he finds his deep scientific insights is where he is, and you find ways the puzzle pieces fit together.
Correct. Like, you know, ultimately, for instance, let's say Ilya, he doesn't manage people. That's not what he likes. Also, I like hanging out with people by default. I'm an extrovert and I care about people.
Oh, interesting. Okay. Cool. So that fits perfectly together. But I mean, I also just like your intuition about various problems in machine learning. He's definitely one I really enjoy. I remember talking to him about something I was struggling with, which is coming up with a good model for pedestrians, for human beings that cross the street in the context of autonomous vehicles. And he immediately started to formulate a framework within which you can evolve a model for pedestrians through self-play, all that kind of mechanisms. The depth of thought on a particular problem, especially problems he doesn't know anything about, is fascinating to watch. It makes you realize like, yeah, the limits of the human intellect might be limitless. Or it's just impressive to see a descendant of ape come up with clever ideas.
Yeah. I mean so even in the space of deep learning when you look at various people, there are people who invented some breakthroughs once, but there are very few people who did it multiple times. And you can think if someone invented it once, that might be just a fluke. And if someone invented it multiple times, you know, if the probability of inventing it once is one over a million, then probability of inventing it twice or three times would be one over a million squared or to the power of, which would be just impossible. So it literally means that it's given, that it's not luck. And Ilya is one of these few people who have a lot of these inventions in his arsenal. It also feels that, you know, for instance if you think about folks like Gauss or Euler, you know, at first they read a lot of books, and then they did thinking, and then they figured out math.
And that's how it feels with Ilya. You know, at first he read stuff, and then he spent his thinking cycles.
And that's a really good way to put it. When I talk to him, I see thinking. He's actually thinking. Like he makes me realize that there's deep thinking that the human mind can do. Like most of us are not thinking deeply. Like you really have to put in a lot of effort to think deeply. Like I have to really put myself in a place where I think deeply about a problem. It takes a lot of effort. It's like an airplane taking off or something. You have to achieve deep focus. He's just, what is it? His brain is like a vertical takeoff in terms of airplane analogy. So it's interesting, but it is, I mean Cal Newport talks about this as ideas of deep work. It's, you know, most of us don't work much at all in terms of deeply thinking about particular problems, whether it's in math, engineering, all that kind of stuff. You want to go to that place often, and that's real hard work, and some of us are better than others at that.
So I think that the big piece has to do with actually even engineering our environment such that it's conducive to that. So see, both Ilya and I, on a frequent basis, we kind of disconnect ourselves from the world in order to be able to do extensive amount of thinking.
Yes. So Ilya usually he just leaves iPad at hand. He loves his iPad.
And for me, I'm even sometimes, you know, just going for a few days to a different location, to an Airbnb. I'm turning off my phone and there is no access to me.
And that's extremely important for me to be able to actually just formulate new thoughts, to do deep work rather than to be reactive. And the older I am, the more of these random tasks are at hand.
Before I go on to that thread, let me return to our friend GPT and let me ask you another ridiculously big question. Can you give an overview of what GPT-3 is, or like you say in your Twitter bio, GPT N+1, how it works and why it works?
So GPT-3 is a humongous neural network. Let's assume that we know what a neural network is. You gave the definition. And it is trained on the entire internet and just to predict next word. So let's say it sees part of the article, and the only task that it has at hand is to say what would be the next word, what would be the next word. And it becomes really exceptional at the task of figuring out what's the next word. So you might ask why would this be an important task? Why would it be important to predict what's the next word? And it turns out that a lot of problems can be formulated as a text completion problem. So GPT is purely learning to complete the text. But you could imagine, for instance, if you are asking a question, who is the president of the United States, then GPT can give you an answer to it. It turns out that many more things can be formulated this way. You can format text in the way that you have a sentence in English, you make it even look like some content of a website elsewhere which would be teaching people how to translate things between languages. So it would be EN colon text in English, FR colon, and then you ask the model to continue, and it turns out that such a model is predicting translation from English to French. The crazy thing is that this model can be used for way more sophisticated tasks. So you can format text such that it looks like a conversation between two people, and that might be a conversation between you and Elon Musk. And because the model read all the text about Elon Musk, it will be able to predict Elon Musk's words as if it would be Elon Musk. It will speak about colonization of Mars, about sustainable future, and so on. And it's also possible to even give arbitrary personality to the model. You can say, here is a conversation with a friendly AI bot.
And the model will complete the text as a friendly AI bot.
So I mean, how do I express how amazing this is? So just to clarify, a conversation, generating a conversation between me and Elon Musk, it wouldn't just generate good examples of what Elon would say. It would get the syntax all correct. So like interview style, it would say like Elon colon and Lex colon. Like it's not just like inklings of semantic correctness. It's like the whole thing, grammatical, syntactic, semantic. It's just really, really impressive generalization.
Yeah. I mean I also want to, you know, provide some caveats. So it can generate a few paragraphs of coherent text, but as you go to longer pieces it actually goes off the rails. If you try to write a book it won't work out this way.
What way does it go off the rails by the way? Is there interesting ways in which it goes off the rails, like what falls apart first?
So the model is trained on all the existing data that is out there, which means that it is not trained on its own mistakes. So for instance if it would make a mistake then, so to give you an example. So let's say I have a conversation with a model pretending that it is Elon Musk, and then I start putting some, I start actually making up things which are not factual.
Like Twitter. But I got you. Sorry. Yeah.
Like I don't know. I would say that Elon is my wife, and the model will just keep on carrying it on.
Yes. And in some sense if you would have a normal conversation with Elon, he would be what the...
Yeah. There would be some feedback between. So the model is trained on things that humans have written, but through the generation process there's no human in the loop feedback.
So it's magnified, like the errors get magnified and magnified. And it's also interesting. I mean first of all humans have the same problem. It's just that we make fewer errors and magnify the errors slower. I think that actually what happens with humans is if you have a wrong belief about the world as a kid, then very quickly you will learn that it's not correct because you are grounded in reality and you are learning from your new experience.
But do you think the model can correct itself too? Won't it through the power of the representation, and so the absence of Elon Musk being your wife information on the internet, won't it correct itself? There won't be examples like that, so the errors will be subtle at first.
Subtle at first. And in some sense you can also say that the data that is not out there is data which would represent how the human learns. And maybe if a model would be trained on such data, then it would be better off.
How intelligent is GPT-3 do you think? Like when you think about the nature of intelligence, it seems exceptionally impressive. But then if you think about the big AGI problem, is this footsteps along the way to AGI?
So let's see. It seems that intelligence itself, there are multiple axes of it. And I would expect that the systems that we are building, they might end up being superhuman on some axis and subhuman on some other axis. It would be surprising to me if on all axes simultaneously they would become superhuman. Of course people ask this question. Is GPT a spaceship that would take us to the moon, or are we building a ladder to heaven, that we are just building bigger and bigger ladder, and we don't know in some sense which one of these two...
I'm trying to, I like Stairway to Heaven. It's a good song. So I'm not exactly sure which one is better. But you're saying like the spaceship to the moon is actually effective.
Correct. So people who criticize GPT...
They say you guys are just building a taller ladder, and it will never reach the moon. And at the moment, the way I'm thinking is this is a scientific question. And I'm also, in heart, I'm a builder, creator, and I'm thinking let's try it out, let's see how far it goes. And so far we see constantly that there is progress.
Yeah. So, do you think GPT-4, GPT-5, GPT N+1, will there be a phase shift, like a transition to a place where we'll be truly surprised? Then again, like GPT-3 is already truly surprising. The people that criticize GPT-3 as a ladder to heaven, I think too quickly get accustomed to how impressive it is that the prediction of the next word can...
Achieve such depth of semantics, accuracy of syntax, grammar and semantics. Do you think GPT-4 and 5 and 6 will continue to surprise us?
I mean definitely there will be more impressive models. There is a question of course if there will be a phase shift. And even the way I'm thinking about these models is that when we build these models, you know, we see some level of the capabilities but we don't even fully understand everything that the model can do. And actually one of the best things to do is to allow other people to probe the model to even see what is possible.
Hence the using GPT as an API and opening it up to the world.
Yeah. I mean, so when I'm thinking from perspective of there like obviously various people that have concerns about AGI including myself. And then when I'm thinking from perspective what's the strategy even to deploy these things to the world, the one strategy that I have seen many times working is the iterative deployment that you deploy slightly better versions and you allow other people to criticize you so you actually or try it out, you see where are their fundamental issues. And it's almost you don't want to be in that situation that you are holding into powerful system and there's like a huge overhang then you deploy it and it might have a random chaotic impact on the world. So you actually want to be in the situation that you are gradually deploying systems.
I asked this question of Ilya, let me ask you this question. I've been reading a lot about Stalin and power. If you're in possession of a system that's like AGI, that's exceptionally powerful. Do you think your character and integrity might become corrupted like famously power corrupts and absolute power corrupts?
Absolutely. So I believe that you want at some point to work toward distributing the power. I think that you want to be in that situation that actually AGI is not controlled by a small number of people but essentially by a larger collective.
So the thing is that requires a George Washington style move in the ascent to power. There's always a moment when somebody gets a lot of power and they have to have the integrity and the moral compass to give away that power. That humans have been good and bad throughout history at this particular step. And I wonder, I wonder we like blind ourselves in, for example between nations, a race towards, yeah, AI race between nations. We might blind ourselves and justify to ourselves the development of AI without distributing the power because we want to defend ourselves against China, against Russia. That kind of logic. And I wonder how we design governance mechanisms that prevent us from becoming power hungry and in the process destroying ourselves.
So let's see. I have been thinking about this topic quite a bit but I also want to admit that once again I actually want to rely way more on Sam Altman on it. He wrote an excellent blog on how even to distribute wealth. And he proposed in his blog to tax equity of the companies rather than profit and to distribute it. And this is an example of a Washington move.
I guess I personally have insane trust in Sam. He already spent plenty of money running a universal basic income project that like gives me I guess maybe some level of trust to him but I also love him as a friend.
Yeah. I wonder because we're sort of summoning a new set of technologies. I wonder if we'll be cognizant like you're describing the process at OpenAI but it could also be at other places like in the US government, right? Both China and the US are now full steam ahead on autonomous weapon systems development and that's really worrying to me because in the framework of something being a national security danger or a military danger you can do a lot of pretty dark things that blind our moral compass and I think AI will be one of those things. In some sense the mission and the work you're doing at OpenAI is like the counterbalance to that. So you want to have more OpenAI and less autonomous weapon systems.
I like these statements, like to be clear, like this is interesting and I'm thinking about it myself but this is a place that I put my trust actually in Sam's hands because it's extremely hard for me to reason about it.
Yeah. I mean one important statement to make is it's good to think about this.
Yeah. No question about it. No question about even like low-level quote unquote engineer. Like there's such a... I remember I programmed a car, RC car, like went really fast like 30, 40 miles an hour and I remember I was like sleep deprived so I programmed it pretty crappily and it like, the code froze so it's doing some basic computer vision and it's going around on track but it's going full speed and there was a bug in the code that the car just went, it didn't turn. It went straight full speed and smashed into the wall. And I remember thinking the seriousness with which you need to approach the design of artificial intelligence systems and the programming of artificial intelligence systems is high because the consequences are high. Like that little car smashing into the wall. For some reason, I immediately thought of like an algorithm that controls nuclear weapons having the same kind of bug. And so like the lowest level engineer and the CEO of a company all need to have the seriousness in approaching this problem and thinking about the worst case consequences.
So I think that is true. I mean what I also recognize in myself and others even asking this question is that it evokes a lot of fear and the fear itself ends up being actually quite debilitating. The place where I arrived at the moment, might sound cheesy or so but it's almost to build things out of love rather than fear.
Yeah. I can focus on how I can, you know, maximize the value, how the systems that I'm building might be useful. I'm not saying that the fear doesn't exist out there and like it totally makes sense to minimize it but I don't want to be working because I'm scared. I want to be working out of passion, out of curiosity, out of the, you know, looking forward for the positive future.
With the definition of love arising from rigorous practice of empathy. So not just like your own conception of what is good for the world but always listening to others.
Correct. Like at the loft where I'm considering reward functions of others to infinity is like a sum from one to n where n is 7 billion or whatever it is. Not projecting my reward functions on others.
Yeah, exactly. Okay, can we just take a step back to something else super cool which is OpenAI Codex? Can you give an overview of what OpenAI Codex and GitHub Copilot is, how it works and why the hell it works so well?
So with GPT-3, we noticed that the system, you know, the system trained on all the language out there started having some rudimentary coding capabilities. So we're able to ask it, you know, to implement addition function between two numbers and indeed it can write Python or JavaScript code for that. And then we thought we might as well just go full steam ahead and try to create a system that is actually good at what we are doing every day ourselves which is programming. We optimized models for proficiency in coding. We actually even created models that both have a comprehension of language and code. And Codex is API for these models. So it's first pre-trained on language and then I don't know if you can say fine-tuned because there's a lot of code but it's language and code. It's also optimized for various things like let's say low latency and so on. Codex is the API that's similar to GPT-3. We expect that there will be proliferation of the potential products that can use coding capabilities and I can speak about it in a second. Copilot is a first product developed by GitHub. So as we're building models we wanted to make sure that these models are useful and we work together with GitHub on building the first product. Copilot is actually as you code it suggests you code completions and we have seen in the past there are like various tools that can suggest how to like a few characters of the code or the line of code. The thing about Copilot is it can generate 10 lines of code. It's often the way how it works is you often write in the comment what you want to happen because people in comments they describe what happens next. So these days when I code instead of going to Google to search for the appropriate code to solve my problem I say oh for this array could you smooth it and then you know it imports some appropriate libraries and say it uses numpy convolution or so that I was not even aware that exists and it does the appropriate thing.
So you write a comment maybe the header of a function and it completes the function. Of course, you don't know what is the space of all the possible small programs it can generate. What are the failure cases, how many edge cases, how many subtle errors there are, how many big errors there are. It's hard to know, but the fact that it works at all in a large number of cases is incredible. It's like a search engine into code that's been written on the internet.
Correct. So for instance when you search things online then usually you get to some particular case like if you go to Stack Overflow people describe one particular situation and then they seek for a solution but in case of Copilot it's aware of your entire context and the context is oh these are the libraries that they are using, that's the set of the variables that is initialized and on the spot it can actually tell you what to do. So the interesting thing is and we think that Copilot is one possible product using Codex but there is a place for many more. So internally we tried out, you know, to create other fun products. So it turns out that a lot of tools out there, let's say Google Calendar or Microsoft Word or so, they all have an internal API to build plugins around it. So there is a way in a sophisticated way to control Calendar or Microsoft Word. Today if you want more complicated behaviors from these programs you have to add a new button for every behavior. But it is possible to use Codex and tell for instance to Calendar, could you schedule an appointment with Lex next week after 2 p.m. and it writes the corresponding piece of code and that's the thing that actually you want.
So interesting. So what you figure out is there's a lot of programs with which you can interact through code and so there you can generate that code from natural language. That's fascinating and that's somewhat like also closest to what was the promise of Siri or Alexa.
So previously all these behaviors they were hand hardcoded and it seems that Codex on the fly can pick up the API of let's say given software. And then it can turn language into use of this API.
So without hard coding it can find, it can translate to machine language.
Correct. So for example this would be really exciting for me like for Adobe products like Photoshop which is I think ActionScript, I think there's a scripting language that communicates with them, same with Premiere and you could imagine that that allows even to do coding by voice on your phone. So for instance in the past like as of today I'm not editing Word documents on my phone because it's just the keyboard is too small but if I would be able to tell to my phone, you know, make the header large then move the paragraphs around and that's actually what I want. So I can tell you one more cool thing or even how I'm thinking about Codex. So if you look actually at the evolution of computers, we started with very primitive interfaces which is a punch card and punch card essentially you make holes in the plastic card to indicate zeros and ones. And during that time there was a small number of specialists who were able to use computers and by the way people even suspected that there is no need for many more people to use computers. But then we moved from punch cards to at first assembly then C and these programming languages they were slightly higher level they allowed many more people to code and they also led to more of a proliferation of technology. And you know further on there was a jump to say from C++ to Java and Python and every time it has happened more people are able to code and we build more technology. And it's even, you know, hard to imagine now if someone would tell you that you should write code in assembly instead of let's say Python or Java or JavaScript. And Codex is yet another step toward kind of bringing computers closer to humans such that you communicate with a computer with your own language rather than with a specialized language. And I think that it will lead to an increase of number of people who can code.
And the kind of technologies that those people will create is innumerable. It could, you know, it could be a huge number of technologies we're not predicting at all because that's less and less requirement of having a technical mind, a programming mind. You're opening it to the world of other kinds of minds, creative minds, artistic minds, all that kind of stuff.
I would like for instance biologists who work on DNA to be able to program and not to need to spend a lot of time learning it and I believe that's a good thing to the world. And I would actually add, so at the moment I'm managing Codex team and also language team and I believe that there is like plenty of brilliant people out there and they should apply.
Oh okay. Yeah. Awesome. So what's the language and the Codex? So those are kind of overlapping teams. So it's like GPT the raw language and then Codex is like applied to programming.
Correct and they are quite intertwined. There are many more teams involved making these models extremely efficient and deployable. For instance, there are people who are working to, you know, make our data centers amazing or there are people who work on putting these models into production or even pushing it at the very limit of the scale.
So all aspects from the infrastructure to the actual machine learning.
So I'm just saying there are multiple teams while the team working on Codex and language I guess I'm directly managing them. I would love to hire if you're interested in machine learning. This is probably one of the most exciting problems and systems to be working on because it's actually pretty cool. Like program synthesis, generating of programs is a very interesting problem that has echoes of reasoning and intelligence in it and I think there's a lot of fundamental questions that you might be able to sneak up to by generating programs.
Yeah. The one more exciting thing about the programs is that so I said that in case of language one of the troubles is even evaluating language. So when the things are made up you need somehow either a human to say that this doesn't make sense or so. In case of program there is one extra lever that we can actually execute programs and see what they evaluate to. So the process might be somewhat more automated in order to improve the qualities of generations.
Oh, that's fascinating. So like the... Wow, that's really interesting.
So for the language the simulation is actually to execute it as a human mind. For programs there is a computer on which you can evaluate it.
Wow. That's a brilliant little insight that the thing compiles and runs. That's first and second you can evaluate on a, like do automated unit testing. And in some sense it seems to me that we will be able to make tremendous progress. You know we are in the paradigm that there is way more data. There is like a transcription of millions of software engineers.
