This episode of Grey Matter is in partnership with Startup Grind, a global community of entrepreneurs. At the 2019 Global Startup Grind conference, OpenAI co-founder and CTO Greg Brockman and Greylock partner and OpenAI investor and board member Reid Hoffman discuss if the current artificial intelligence boom can scale to AGI, also known as artificial general intelligence.
That is great energy for the end of a day, so let's kick it off because I think the subject's super important. OpenAI, why did you help found it? What's the mission? What's the way that people should understand OpenAI?
All right, well, I'll just tell you a little about my personal story. So I first got excited about AI when I read Alan Turing's 1950 paper on the Turing test. And I was used to programming as something where you think really hard about a domain, you understand it, you write it down in this obscure way that we call a program, and then suddenly anyone can get the benefit of it. And I always thought that was amazing. And then you read this paper and instead Alan Turing says we're not going to program an answer to this, we're going to learn an answer. And the idea that the machine itself could figure out how to solve a problem that I could not even describe how to begin to approach, that for me was just the most mind-blowing thing ever. So I got very excited, I went to my professors, I said, hey, can I do some NLP research? They said sure and they pointed me out some like parse tree, like probabilistic grammars. And I was looking at that, I was like, this is just not going to scale, this is not how we're going to solve this. And I was very sad. And so I decided to work on practical things like building startups. And I spent the next decade trying to do that. I was at Stripe for five years. And when I was leaving, looking around, you know, for me, I think that's about when we first met, that's right, and top of the list was AI. It's a question for me, you know, it's always been clear that the most important problem that I could hope to contribute to is artificial intelligence and really building machines that are able to do materially useful things in the world and do things that humans cannot and help us, you know, achieve new heights. It's just a question of when. And you know, this was about 2015 and three years into the deep learning revolution, things were starting to work. The default for me would have been starting an AI company, try to get some cash flow and then maybe five years later contribute back to general intelligence development. And I was talking to Sam Altman and he said, why not just go right for the general intelligence development? Like it feels like we're on this exponential curve and it's hard to know where that's going to top out. And so that's where we started. I think that we brought together a group of people who thought very similarly to me in that story that I just told, where I think that for a lot of the people who were at OpenAI and were there at the founding, that general intelligence has just been the most important problem that they could imagine contributing towards. And it was the right time to come together and to actually have a serious effort to build it and make sure that it went well.
And so what is the founding mission? Like if people were trying to understand what is the kind of the moonshot of OpenAI and what is the things that they should watch for from it, what are the things that they should anticipate the work will be heading in the direction of, what would you tell them?
So the mission of OpenAI is to ensure that artificial general intelligence benefits all of humanity. And when we say artificial general intelligence, we're talking about machines that can outperform humans at most economically valuable work, which is a very, very high bar. I think that again, it comes down to if you can actually build a system like that, it's clearly going to be transformational and that there's a question of when and exactly how fast that happens. But I think that making sure that goes well is clearly crucial. And you think about technologies that have come to market over the past couple decades, it's pretty clear that getting those right is hard even when you have a long time to figure it out. I feel like today we're just starting to see some of the implications of choices that were made very early on in the internet. With general intelligence, I think things can go even faster and so we really have to make sure we get these things right ahead of time. And so the main thing that we work on is actually development. We actually are trying to build these systems, we're trying to push forward the state-of-the-art and we've made a number of breakthroughs in different areas. But we also work on safety, as that's making sure that you build powerful AI systems that they actually do what the designers intend. But there's also this other hard problem which is, well, what should be the values of these systems and how should they affect society? And we have a policy team whose job it is to work with governments and other stakeholders and try to shape answers to those questions.
And how should people understand the 'open' part of OpenAI? What does 'open' mean in this context?
Yeah, so this is I think a misconception, right? I think that you see OpenAI and you think, great, it's an open source project. And sadly, I don't think that being an open source project to AGI is the best way to benefit the world. We've actually published a document called the OpenAI Charter where we document exactly how we think about what we want to release and how we want to benefit. And one point there is that due to safety and security concerns that we're going to have to decrease traditional publication in the future. And the thing that really matters to us that is core and invariant is that mission that we want to make sure that general intelligence benefits everyone and that is baked very deeply into our structure and our ethos.
And so sketch a little bit the research program. I mean, probably people are fairly familiar with you get a lot of compute, you get a simulated world, you get self-playing agents, you have an early state on set of games. But how does that research program look like as kind of like this is the vector in which AGI might get constructed? And then what's the theory of intelligence that correlates with that research program?
The way that I always like to think about this is starting with the AI that we all know and love. So for the past, you know, seven years now, deep learning has really come on the scene. Deep learning is not a new thing, right? It's been around for 60 years and it's just now that it's really starting to work. And at the core, what's going on is that we're starting to have massive computational power, that we have an algorithm of a deep neural network which is massively scalable. Right, that one thing that we found is just a pattern that just holds across domain after domain is if you scale up a neural network with appropriate data, you tune the architecture in the right way, it will work better. And then there's a third thing which is the availability of data. Now the thing is that these building blocks of AI are starting to change in a very interesting way. One is data. And so to your point, so if you're doing reinforcement learning, you have a simulator, you have a game like for example Dota where we've had some breakthroughs, you're suddenly spending computational power to generate your data, right? The agent is actually playing against itself, is generating data that it can then learn from. That you're learning from your own experience in the same way that a human bumbles around the world and sees things and learns from that experience. Now that's actually starting to work very well. I think there are a number of groups that have really amazing reinforcement learning breakthroughs and it's actually been very interesting to see. 2013 is the first time that you started to see exciting things with deep Q-learning and 2016 or so with AlphaGo and 2018 we have and now StarCraft. And I think that we're really starting to reach the point where if you have a simulator, you can solve that problem in a very meaningful way. But there's something else that humans do that none of that addresses, which is we somehow just look at the world, you look at, you know, a baby who just really just experiences and somehow is able to extract meaning from that. And you're somehow able to take all this past experience and apply it to a new problem. That's not something that describes the AI systems that we're used to, but we're starting to see signs of life with AI technology doing this. So we'll actually be releasing some results soon on doing this in language. This is something that actually we and a number of other groups have had neat results on and we're starting to see that same sort of exponential progress where you just train a model to predict the next word in text and it's able to start learning to generate text that is materially surprising. And actually you're able to use that past experience to solve normal tasks like classification or question answering that normally you would think you just need a big data set of questions and answers for. And so I think that we're starting to see a shift in terms of the story, right? That people used to say deep learning, it's just pattern recognition, you just have question answer, I learned to answer new questions like that. But we're starting to see that's not quite true. It's also taking action in the world and it's also looking at the world and extracting structure and information.
Characterize a little bit about how both the Dota result and the StarCraft results were progress on this kind of like what does it look like? Because you know, obviously a relatively untutored person says, oh, computer, you know, AI stuff with learning algorithms playing computer games, okay. Right, there's been some intelligence programmed in these games before. From someone who's not up close and personal, what's the thing goes, ah, that's a really interesting set of results?
Yep. So I think it's all about distance from the real world. If you look at something like chess, it used to be that people said, hey, if we solve chess, we've solved intelligence. What else is there? Turns out quite a lot. Turns out that you could solve chess just by looking through more positions than a human possibly could. And you look at something like Go, which people said, okay, this is the new milestone, we can't just look through all the positions. And suddenly you mixed in a neural network which is basically an artificial intuition with looking through more positions than a human could. Those two things together could solve the problem. But you still have an issue with transferring this to the real world. In the real world, I can't look through positions, what does it even mean, right? I can't run this talk that we're doing 100,000 times to pick the best one of those. And so you need to somehow have an artificial intuition that can perform at the same level as the best humans in some domain. And what's going on in these competitive esports games is it's exactly that. You have these very long games that you have to do planning, that you have to be able to manage all these different units, you have different time scales you have to plan at. And you have humans that dedicate their lives to these games. You know, I think that in some ways esports gets a bad rap, but if you actually look at these players, they're actually professionals, right? That they live in houses together year-round training to become as good as possible. And so if you exceed their capabilities there, you've learned something meaningful.
We'll get to the exponential curve in a second, both on the hardware and the other things, because this is obviously one of the things that's super interesting with the AGI thing. But what does this architecture of iteratively better games, what does that mean for how you think about what human competencies and human cognitive capabilities are and why that's an interesting research program for that?
So I think that one thing that's great about games is that human designers have put in a lot of effort to make sure that they're actually challenging, right? That they do exhibit different skills, they're very interpretable, right? That you can look at behavior in a game and you can see, okay, this is what a human would do here and you can see it's learned this, it hasn't learned that. One thing that's also nice is that it's a lot of system one thinking, right? That the system two thinking, I think, is another piece that we're missing right now. And that it might be that you should probably give the Kahneman on system one, system two. You might remember from your college psychology textbook, system one thinking is kind of reflexes, right? It's the instincts and kind of the emotional lizard brain. And system two is the deep thought, right? Is, you know, kind of what you do when you're thinking very hard about a problem. And I think that what we have right now by analogy looks a lot like almost an insect, right? That you have, if you think about our Dota agent, it's been trained in the simulation for many, many thousands of years, right? It's hyper-optimized for this one environment and it can actually respond to things that are totally alien to it in that world, right? You have a human come in and play, it's the first time it's ever seen a human, right? It's spent millennia playing against other bots, who knows what those bots do, they definitely don't do what humans do, and somehow it's able to respond and able to operate well in that environment. But it's never going to teach you calculus, right? And I think that that's one gap that we need to overcome.
And as we look at the game playing, what should we look at as the future of kind of mixed human and agent in terms of how the humans and agents can play together?
I mean, that was one of the things that we were, you know, working on on the Dota environment. And that's right, yeah. I think that we'll actually have some interesting things to show there. And I think that, you know, at the end of the day, AI is not about beating humans, right? It's really about enhancing what humans can do. And so I think that one thing that's very exciting is starting to see not just how can we outplay, but how can we actually make something that's materially different where you have a collaboration between humans and AI. And I think this is actually a very untapped area, but we're starting to reach the point now where the systems are so competent that they actually have meaningful value to add to the human experience.
Now let's go back to the AGI kind of exponential curve. What's driving the expense? Yes, leads to the okay, so we're at insects maybe now, but we are actually in fact trying to get to human level. What are the drivers of that exponentiation?
Yep. So one core driver that I think was very surprising to us to discover just how fast it's growing is computational power. So we published a study sometime last year where we just looked at the most computationally intensive results from the past six years. And what we found is that there was more than 10x growth in that computation each year, right? And it wasn't, this was a planned thing, it wasn't that people even knew about it. We published this and everyone was surprised, including the people who had contributed to those results. But it's just the emergent phenomenon of all the money that's pouring in, all these people wiring together systems, people building faster chips and wiring them together in bigger systems. You look at devices like Google's TPUs, we track about 47 different hardware startups that all have a unique approach. And what's going on at a hardware level, at a silicon level, is that we've more or less run out of juice on Moore's law. It's kind of hard to fit more transistors on there when you're talking on the nanoscale. But what we haven't tapped out is parallelism. And turns out the neural networks, just like the thing up here, massively parallelizable. And so yeah, you can just wire together all these chips and you're going to get benefit.
So is the 10x increasing compute scale that you're talking about is actually not transistors on the chips but actually in fact a better ability to get a bunch of the chips working together in a parallel scale?
That's right. So it's kind of the product of a few different phenomena. And one of these is wiring together massive systems. It used to be that in 2012, you know, if you look at the history, that there was a group at Google who started out trying to train neural networks on 16,000 CPU cores and were blown out of the water by two grad students on two GPUs, which is a wonderful David and Goliath story, but I don't think it'll ever happen again. Because you look in 2014, people figured out how to use eight GPUs in a single server. And then starting in 2017, I believe, is when we had a linear scaling up to 1,024 GPUs. That's about a million dollars worth of hardware right there. And people are continuing to scale further and further. So one exponentiation is compute, which is through the parallelization, which works very naturally with parallel distributed processing neural networks.
What are the other areas that make the exponential curve?
So one that I think is often very undersung in these conversations is actually algorithmic progress. And it's a very subtle thing because on the one hand, deep learning, you can almost think of as one algorithm, right? It was the late 1950s that Rosenblatt released his perceptron and we're basically just playing with advanced perceptrons right now. But at the same time, there are fundamental advances that people make and often these fundamental advances, they're one-line changes to the existing code, right? Figure out which line it is, that's the hard part.
