No One Is Ready For This — Ilya Sutskever on Superintelligence

I do maintain here's something which I predict will happen. That's a prediction. I maintain that as AI becomes more powerful, then people will change their behaviors and we will see all kinds of unprecedented things which are not happening right now. So Ilya Sutskever, if you don't already know him, is basically one of the architects of deep learning. He's known primarily for his early work on AlexNet, the breakthrough neural network that pretty much kicked off the entire modern AI revolution, and of course for his time at OpenAI, which he co-founded, where he helped build GPT-3 and GPT-4 as their chief AI scientist. But he left and now he runs a secretive new startup called Safe Super Intelligence. Their mission is exactly what the name says: build a superintelligence and somehow keep it safe. And in this brand new interview with Dwarkesh Patel, Ilya actually gives us rare insight into what they're working on. He also talks about the end of scaling, the missing piece of real generalization, how AGI could change human behavior, and when superintelligence might actually arrive. So yeah, this one's going to be a doozy. Let's get into it.

All right. So, one of the first things Ilya declares in this interview is that the age of scaling has basically sucked all the air out of the room. Everyone is just scaling bigger and bigger models because that's the only thing they know for sure works. But no one really has any new ideas. According to him, what we actually need now is more research, not more compute. Check this out.

Yeah.

Actually on that, you know, there is this Silicon Valley saying that says that ideas are cheap, execution is everything, and people say that a lot.

And there is truth to that. But then I saw someone say on Twitter something like, if ideas are so cheap, how come no one's having any ideas? And I think it's true too. I think if you think about research progress in terms of bottlenecks, there are several bottlenecks. If you go back to the if you and one of them is ideas and one of them is your ability to bring them to life.

Which might be compute but also engineering.

He even brings up AlexNet right after this and talks about how it was created using only two GPUs and how even the first Transformers were only built on dozens of GPUs. Today we have millions of GPUs that are radically more powerful and efficient. Yet everyone's building the same LLMs and the same world models. This is part of the reason why Ilya went off to do his own thing. And to understand what he's really chasing at Safe Super Intelligence, you first have to understand what actual AGI is, at least how Ilya defines it, because that not only gives us clues into what SSI is working on, but it also explains why scaling the current systems alone won't get us there.

Yeah.

Yeah.