Ilya Sutskever on AI pre-training
From Ilya Sutskever "Superintelligence is Self Aware, Unpredictable and Highly Agentic" | NeurIPS 2024 · · Wes Roth
“Pre-training as we know it will unquestionably end. Pre-training will end. Why will it end? Because while computers growing through better hardware, better algorithms and larger clusters, right, all those things keep increasing your compute, all these things keep increasing your compute, the data is not growing because we have but one internet. We have but one internet. You could even say, you can even go as far as to say that data is the fossil fuel of AI. It was like created somehow and now we use it and we've hit peak data and there'll be no more.”
On , Ilya Sutskever, Cofounder at OpenAI, spoke about AI pre-training during Ilya Sutskever "Superintelligence is Self Aware, Unpredictable and Highly Agentic" | NeurIPS 2024 on Wes Roth.
About Ilya Sutskever
Ilya Sutskever, cofounder of Safe Super Intelligence (SSI), received the NAS Award for the Industrial Application of Science from the National Academy of Sciences on May 5, 2026. The award recognized his pioneering contributions to the deep learning revolution, including work on AlexNet, word2vec, sequence-to-sequence learning, and the GPT, DALL-E, and CLIP models. Sutskever was not present in person but provided a recorded acceptance message. In his recorded remarks, Sutskever expressed gratitude for the award and acknowledged his collaborators. He stated that AI will "unquestionably and unequivocally continue to grow in impact, influence, disruption and potential." He described both the positive potential and the risks and downsides of AI as "staggering," and said that the progress of AI will compel people working on it to act in unprecedented ways to realize its positive potential.