An automated system that can be operated by a small number of people and is capable of fundamentally developing new technologies, I think, just has very profound consequences for governance and just balance of power. There are some useful domains like individual math problems or individual programming problems where AI outperforms the vast majority of humans.
Okay. So as you are saying that in some incentives are aligned that like making pro capability, you know, and make this capability useful, we require essentially being able to make it safe. You have this potentially extremely powerful technology which clearly deserves some level of control and thought put into how it's deployed. Which as it becomes more and more diffused it becomes harder and harder to control. When we interact with a social network, the content that we see is suggested to us by an AI and even though it's written by people, there are so many voices, right, that it's really the AI that crafts the narrative that we see.
Yeah, of course. Again, I think you make an important point that at the time when this was set up it was a very different world and maybe kind of everyone was just trying to figure out their best guess.
And welcome to Before AGI. When we talk about AI, our conversation often drifts towards the big picture, utopian dreams and dystopian anxieties. But beneath these sweeping visions lies a more tangible story, one that's less discussed, but equally important. How exactly did we get here? What does it truly mean to work at AI's frontiers today? And perhaps most interestingly, what's next? To unpack these questions, I'm joined today by two important figures who witnessed and co-created the full arc of OpenAI's and AI's evolution. Jakub Pachocki, OpenAI's chief scientist, and Szymon Sidor, one of just a handful of technical fellows at the lab. They have been around since the early days, long before OpenAI became synonymous with ChatGPT, back when the idea of an independent AI lab was still audacious, experimental, and uncertain. We'll take a step back to those early moments. What drew them into this venture? The doubts and excitement of pioneering days and how the culture and vision at OpenAI have transformed along the way. We'll also get practical. What's it genuinely like to be a researcher navigating the edge of possible in AI? And amidst the optimism, what keeps them awake at night about the future they are helping build. Let's get into it. Szymon, Jakub, welcome to the podcast. I'm so happy to be able to host you here. So, we will be talking mostly about AI here. But before we talk about AI, I want to talk about you two because you know, it's the first time when I have two guests on the podcast and there is a reason for that because you both are unique individually but also as a power duo kind of. I think to me you are like similar to like there is Jeff Dean and Sanjay Ghemawat kind of power duo who made Google great. You are the duo that made OpenAI great. So essentially, how did it happen? Like how did you first meet?
Um, we actually went to high school together.
We both ended up at the third high school in Gdansk, which was in Poland. And we were drawn there in a big part by our computer science teacher, Professor Chubovski. And we both very much engaged with his method of teaching, which was giving the students a lot of freedom, really not trying to impart knowledge directly but trying to focus on showing the students something interesting and giving them a lot of room to pursue it. And not just within his class but also, you know, making space in general and teaching to kind of pursue your own objectives in education, in life.
So Szymon, so when did you actually meet? So you kind of went to the same high school together but like, do you even remember the first time you saw each other?
The first time you saw each other, I think probably one of the computer science camps. So essentially these are the camps that are organized for like the talented people in Poland just for them to get together and start studying more advanced topics in computer science.
Yeah. Yeah. Yeah. It was pretty awesome because we would spend, I think, I mean actually maybe you spent more, but my recollection is we would spend about two months every year on those computer science camps. There would be like four week-long camps and one month-long camp in the summer. Yeah. Yeah. Most of the computer science inclined students in our class would go to all of those camps.
Yeah. So and you know after that I guess you parted ways. Szymon went to Cambridge and then to MIT but kind of to start the PhD program but I guess in the end you realized this is not for you. For Jakub, you did your undergrad in Poland but then you moved to CMU for a PhD, then you know did the postdoc at Harvard and then you reunited at OpenAI. So before we get there actually like when you met each other like the first time, like did you know that you will meet again or was it like at that time like did you keep in touch or like kind of it only happened later?
Yeah. So I think actually the time that I remember as the time that our friendship grew was when we were going to study in the US and it was like kind of experience. It was kind of like a little bit of a leap of faith and I feel like we reassured each other on this journey even though we went to completely different universities.
I see. Okay. So let's move to this exactly reunited under the OpenAI banner like this was like 2017 something like that and again this was a strange time because you know at that time AI had or like the deep learning revolution already started but I would say it's still well compared to today it's pretty low key but you know it was definitely not as hot of a thing as it is right now. So when did you conclude that AI is the thing for you by the way? Because I think you did not do much AI before that. So like what was the AI enlightenment moment for you?
In high school and before high school I became very engrossed in programming competitions. I became very passionate about the idea that like there is some problem and it's not clear at first at all that like it is solvable but you can write a program that makes the computer do that in like a reasonable time, you know, and this kind of becomes like proving like what are different things that computers can do. And this is an idea that I became extremely excited about and I saw a natural continuation of that is working on theoretical computer science and kind of building fundamentals for understanding like what are computers capable of, what are they going to be able to do in the long term. And really at that time I thought of AI as something that like a true reasoning AI as something that will take a long time to develop, will take much larger computers and will take like very solid mathematical fundamentals compared to what we had then. And so that's kind of what I wanted to work on and what forced me to re-evaluate my timelines.
So just to clarify, so at that time you were still working on like PhD in theoretical computer science, maybe you were curious about AI but not really thinking this is the thing to do and then what changed?
Yes. The thing that forced me to re-evaluate was AlphaGo. I kind of thought of, I thought chess was like the kind of big milestone in AI up to that point where there was like this difficult feat against humans and you kind of understand how chess works, right? There is this brute force tree search with some heuristics that allows something like Deep Blue to play at a competitive level and subsequent chess systems. But in Go, the search space is so much larger that it's very hard to construct such a program. And you know, this fact that like oh there's this game and the search space is so large and like our algorithms just like cannot really tackle it to be competitive with the best humans, that was like clear evidence that like there's something deep that we're missing. And you know, maybe if there was like some other result that like had solved Go by like, you know, developing better heuristics maybe that wouldn't be as compelling to me but the fact that you could take the same system that was being used for vision and just apply it there and use it to steer the search, that forced me to really think hard about like, you know, how much of this theoretical foundation do we need or is it maybe something that we should view as a physical phenomenon that we need to understand.
And then you know you knew that okay this is time to act.
Yeah. Yeah. Actually one of my first practical experiences with deep learning was before I joined OpenAI during a catch up with Szymon.
Yes. Actually let's talk about Szymon because I think Szymon got on AI train in a bit more steady way because I think your PhD was kind of about AI. It probably was the old type of AI. So in some sense I am impressed that like you did not get disillusioned about AI doing that. So can you tell us about your story? Your AI awakening?
My AI awakening? Yeah, absolutely. It started in a kind of dumb way because I was very young and naive. So it started by me watching the first Iron Man movie and deciding that I want to do robotics. And at least I want to try. I want to see what it's like. I didn't view it as like maybe a big commitment but I applied to all of those universities in the US to do robotics and eventually all of them except MIT rejected me. I think it's because MIT didn't have the English exam and my English was terrible. So I was happy to go there. But yeah, I went to do robotics and obviously I was as you said a little bit disillusioned by it. I was kind of thinking about dropping out. The only reason I didn't is I got a little bit interested in deep learning as a distributed systems problem really. I just had a friend who was like really into deep learning and I was like whoa oh you could do this data parallel thing and distribute it over machines. And I was like thinking about it a little bit and then I started reading papers. I reimplemented Deep Q-Network which I already found somewhat enticing. It was like wow I didn't expect something like this can work. And obviously RL is a very kind of eye-opening formulation because so generally you just have some environment, actions and rewards like it feels like you can do anything and even slightest bit of progress there is great. But then we had AlphaGo and that was just like okay wow right like that was just so eye-opening. And then I knew I wanted to do AI. Yeah, I worked briefly at another AI startup called Vicarious that no longer exists and I wanted to get into OpenAI as quickly as possible. Especially like we were talking with Jakub and there was like one more Polish person that I really respected whose name is Filip Wolski and I knew that all of them planned to go. So it was like the obvious place to go to.
Okay. So actually like let me actually dig into it more is because I do remember like I knew Jakub. I didn't I think know you at that time yet but just like kind of and I remember like one day exactly like Jakub comes to my office saying okay Alexander it's time to do AI and I'm starting to apply to all of these kind of companies doing it. And I remember he was like asking oh you know I have these two offers like one is from some place actually I don't remember its name I think it no longer exists and then there was another place OpenAI and you know asking okay so what do you think where should I go. Of course for me this was like completely random names like I said I don't know but like I think you said that okay there was this guy Ilya who was kind of talking to you and kind of you enjoy it so that was your decision but like again 8 years ago like now of course OpenAI is a powerhouse of course you want to come here but eight years ago it was completely not that. So what was like what was the OpenAI effect? Was it just a bunch of Polish guys going there? Was there something more?
So actually Wojciech has been trying to draw, co-founder of Dropbox, has been trying to draw me into deep learning even before he co-founded OpenAI. I think maybe he had this vision of recruiting some of the people he knew or heard of from programming competitions or math competitions. And you know, I think a big part of what drew me to OpenAI specifically was that well some of the people that I knew like Filip Wolski or Marcin Andrychowicz were there. And those are people I wanted to work with. And of course, you know, the interview I had with Ilya and Alec Radford and that was a great experience. So yeah. So once I got to meet the people of the company also Jay Tang, I was very drawn to it.
Okay great. How about you Szymon?
Since I think AlphaGo and the steady buildup of like there is something exciting about deep learning, I knew I wanted to work on that. I initially wanted to work at DeepMind but they asked a bunch of theoretical machine learning questions that I knew nothing about so I got rejected. And then I've heard about OpenAI which sounded like very very serious so obviously I wanted to go there. And yeah, other people going there also had an effect. The fact that like the funding was serious like that kind of conveys some level of like we're actually doing this. I do think what did not draw me to OpenAI was definitely this mission statement initially because just building AGI sounded honestly a little bit BS to me initially. I was just more excited about the bleeding edge technology but I didn't believe in like super fast timelines back then. Obviously very different now. I remember one of the early experiences at OpenAI was we went to this offsite at SpaceX actually. And I think Sam walked out in at the center of the stage and he asked like hey you know like I will ask you questions about your timelines for AI like and if you think it's five years or less raise your hand. Oh this was Elon okay maybe it was Elon asking this question. But the funny part for me was that like at the beginning whoever it was they specified like by the way there are some SpaceX people in the room and please don't raise your hand. That was and I remember I think we looked at each other then and we like kind of smirked under our noses because like it felt like completely arbitrary. It felt like nobody understands like the actual timelines and like whether the SpaceX people vote or not it won't affect the actual conclusion because it's random noise.
Well, there is some kind of group thinking effect as well. But yeah, actually let's talk about this. So, okay, you accept the offers to join OpenAI. I think Szymon was a little bit earlier than Jakub but it was essentially the same time. So then you get there and you know one thing I know and I even remember from like my visits is this was a very different place than it is right now. So what was your okay you come in there and what is your impression like what do you like what you might not like you know how did it feel to be at the early OpenAI and you know how did the trajectory of the company look like you know before the GPT kind of paradigm kicked in.
Yeah, it definitely I had a bit of a different perspective and I think also Szymon to some extent from the kind of general view because we came in as you know recent converts to deep learning. You know I didn't have the sort of longstanding belief of Ilya in particular but also many other people who have been working on this for a while. I came as someone who has been you know until recently kind of thinking longer term frames and really focused on fundamentals and have been forced to you know update my beliefs based on empirical evidence, AlphaGo in particular. And still I wanted to take that empirical mindset and that somewhat skeptical view to understand really like why the systems work. And I think this drove us sometimes into a bit different direction. And I think it also enabled some very very productive collaboration with folks who are focused on thinking far ahead. The first big project that we worked on together was maybe slightly contrary in that way in that a large part of focus has been going into developing new reinforcement learning algorithms and we teamed up with Szymon to develop infrastructure to just very naively scale up the.
So the particular project that we were focused on that we were trying to solve was solving the game of Dota 2. So that was a project that started shortly before I joined the company and kind of my first day Ilya asked me if I would like to work on it and I was very excited to work on it because you know it's kind of a natural follow-up like playing a complex video game is a natural follow-up to teach you to play complex board games. And you know we weren't thinking that like oh like just naively scaling up the current algorithm is going to work very well on this project but we wanted to take the approach of kind of understanding why it fails. So you know this skeptical view and the thing is it didn't fail, right? It just kind of worked and we just ended up just like.
Yeah. So what we thought will be is like let's hit the barrier and then we can like have some more information to like go back to the drawing board ended up just kind of being a two-year process of scaling things up and of course solving a lot of problems along the way but largely around understanding you know what are the bottlenecks of scaling.
How what was your experience here? And by the way, just like about Dota, I do remember I think at first it was like a big secret what you were working on, but eventually like I was enlightened and I remember you were showing me just like one, you know, one of this, you know, one of the characters just going on an island and kind of fighting with another character. So that was the beginning of just like Alexander this is really cool and so on. I just saying oh you know like okay I trust you. So anyway, what was your perspective, Szymon?
Um, yeah, by the way, to just like respond to your anecdote, I understand why it's difficult to look at this and appreciate why it's real. For me, I actually had to play Dota for about 500 hours before I could even like understand what's going on.
I still don't understand what's going on. So anyway, yeah, your impression when you joined OpenAI like the early days.
Early days. Yeah. So I think Jakub covered the kind of technical challenges quite well. So I will just talk about the vibes. So the vibes was I think it was basically imposter syndrome the company. I think there was like a bunch of like very famous researchers there and I think they were all kind of mutually feeling inferior to each other. So there was a little bit of a very often the lunches were quiet and like people were like very thoughtful about like what they're going to say at lunch. And very different to now. I think it's very different to now. Yeah, I think people now kind of well but still I think when you join it can be a little bit overwhelming at times but I think people are definitely much more relaxed on average. And you know there was some charm to those times too because like you know everybody was like really spending a long chain of thought in their head to say something smart at the last. So it was like entertaining to hear some of those things. Yeah. But it is definitely I think the vibes, the kind of awkwardness really reflected well in also our technical path because it was also kind of like awkward and we didn't really know what we were doing and we were like probing in a bunch of different directions and I think over time we learned to build focus.
Yeah, that's definitely something that I remember from my you know visit to early OpenAI where like everyone was working on some like a little bit wacky idea you know I remember John was showing me like the browser of the websites you know there was like the visionary strategies there was a lot of things I think Dota was one of the few projects was really like long-term and very kind of focused but there was a lot of people exploring figuring out you know I guess until Alec stumbled upon well stumbled or just discovered GPT regime like where I think there was a shift where okay this is the thing we are scaling and kind of there starts to be this convergence it was like yeah very much of searching. So can we talk a little bit about this like so you know between then and now like how does your day-to-day you know experience look like when you are working on a like what do you do most of the days.
So by the way you said I just want to address because you said Alec stumbled upon the GPT. Right, I think that's like you know Alec and Ilya have been looking in this direction for a while and I think Alec has been kind of working on this at the time I think like kind of niche direction of thinking about sequence language models and actually like at first it wasn't a GPT regime I think it's often confused.
So please tell us what the real story is.
Well from my perspective the kind of the big jump was the sentiment neuron paper which was actually a recurrent neural network so it was actually before transformers were published where Alec, Ilya and Rafa had demonstrated that if you train a model on a lot of reviews, it actually can pick up the notion of sentiments whether the review is positive or negative even without any supervision on that. And I think that was like a moment where I think before then a lot of kind of the discussion around natural language modeling has been around like you know grammar and kind of and it hasn't really gotten into semantics that much. And I think that was the point where oh well if we just model this without any supervision whatsoever we actually get some understanding of meaning right? And I think that was like the moment where oh you know there's going to be something there.
I see so okay so kind of the insight was that instead of like I think a lot of work on kind of understanding language was kind of trying to leverage the grammar rules and so on and here you just kind of try to kind of infer semantic meaning purely from data and it actually works. So one of the by now repeated lessons of deep learning. Okay, thank you for correcting the record you know that's again that was a very important moment in development of AI but now going back to your day-to-day so how does you know building AI on the day-to-day basis look like?
Well, I think day-to-day you're looking for bugs. You're trying to understand, you know, what are the kind of simple things you're missing. It's a very unique field, right? Because we are like on one hand like most of the things we do is it's just kind of simulation and we're running software that we write ourselves and you know these deep networks have full control over the architecture and over how we optimize them. At the same time, we don't fully understand the principles on which the networks optimize. And so even though we are kind of in full control of the setup, we really are studying a natural phenomenon. And this means like careful experimental setup to be able to understand, you know, how different changes to this process might lead to very different outcomes is critical. And it's very easy to make a mistake also because the networks as Szymon would put it the networks really want to learn right and even if you have like some mismatches in your setup like they are going to learn. So it might be very hard to understand if you're not modeling something entirely correctly.
So what you are saying here is essentially that like sometimes even if you make some silly bug kind of the network will overcommit and you will not even notice maybe that something went wrong. So it's not like you know you make one bug and everything crashes so you know okay there's a bug to look for. It actually seems to work. Maybe it doesn't work as well as you would hope. So that makes the kind of there's a lot of the silent bugs that kind of are there. Is that what you're saying?
Um yes. Yes. And this is something that you know day-to-day is something you have to think a lot about and also to think about you know maybe taking a longer term perspective like how do we construct the training process? How do we design our experiments in a way that teaches us how to you know actually build this whole foundation so that we can experiment on it for the future.
What is your perspective Szymon here? You don't need to fix bugs, you just never make them.
Yeah, fixing bugs is important. I mean fixing bugs is a theme that kind of reverberated through like the entire tenure of my stay here. I mean my actually first blog post that I wrote that was published from OpenAI's website was about bugs basically like that was in the context of deep learning. There was like some features that disappeared because of our color processing and there was all sorts of stuff like that. Actually my favorite collaboration we had with Jakub was when we it's not quite a bug but like we would see like neural networks sometimes spike randomly and we don't understand why. And there was like one day where we decided like okay let's really understand it and like we like literally looked at every single individual math model and eventually we saw something there and like deeply understood like what was happening there. Since then we hired a bunch of more people at OpenAI who are really great at debugging.
Okay. So actually like let's talk a little bit because you work together a lot. So how does this power duo kind of what is the dynamics here? How do you work together? Like what is your what is the secret to the success that you know for eight years like you really work very closely and have some great things to you know to show for it?
I would say like it certainly evolved over time. I think for the longest time what we would do is we would just like we kind of try to maybe like look for the aspects of our projects where it's not obvious like how it fits into our current organization or like project structure and try to fix those or maybe sometimes those would be problems that like it's a little bit hard to convince people that are important. For example like the dataset story there was when we started looking at pre-training our dataset batches weren't quite IID.
Okay so this is very technical thing so just like can you kind of you know for the listeners that are not all you know into like pre-training of models like can you explain it a bit more simply.
Well so the crux of the matter is that when you train those neural networks they obviously read a ton of text. There is like at this point like terabytes of data. And it's sometimes just hard to handle this much data just engineering wise. And that leads you to sometimes cut corners where you try to like see like oh I really want to think of this data as just like a sequence of independent little chunks so that each chunk is easy to process but that was like fundamentally wrong and it violated some deep mathematical assumption that deep learning relies on and that I maybe won't explain here but for the people in the know it's like the identically independent distributed data assumption. And you know to solve it was just like a hardware engineering problem. So we spent like few days designing a proper design and few days implementing this and lo and behold we actually noticed that this did have real impact on learning that we did. It didn't look like a major impact but we saw like some modest improvements and importantly it moved us from the kind of regime where we think of data in a little bit of a vibes based way to a little bit more principled way and that's like one less thing to worry about and that's always great.
So these are the type of projects you like to like kind of get it together but like how do you work? Do you pair code? Do you like one person kind of has an idea, the other one is implementing and debugging it and do you alternate? Like how do you work together?
Very often the most productive kind of collaborations we had would be Jakub would walk around the office or apartment or something and like really think deeply about how should we like study the phenomenon and I would just like and sometimes you know obviously it's a frustrating process to both of us because sometimes the next steps are not clear and I would be more like let's jump in let's like really get data and let's let's kind of gather more bits to feed into the processing machine up here. I'm pointing at Jakub's brain for people listening to the podcast. Yeah. And I think it worked out very well. Obviously that's just like first order approximation like Jakub is also a great engineer and I sometimes think about research too. Sometimes not too often. But I think that would be like the first sort of approximation of how our collaboration worked.
Yeah, I think something you know a lot of the time the hard part of solving a problem is like actually kind of believing that it can be solved. Yeah, Szymon has this wonderful optimism and eagerness to yeah to like get the bits, get something going and yeah, I think this is something that I've always admired and I think he's really enabled a lot of the things that we did. Yeah, it's something I'm always trying to learn from him.
Yeah, he is fearless. So shifting gears a little bit. So okay, you mentioned this kind of naming the timelines and also like this term AGI that I think at first especially Szymon was kind of not really vibing with. So how is it now? Okay, so first of all like what do you think will be the slope of the progress curve for the next couple of years? It was pretty steep so far. Do you think this will continue to be so accelerate like you know slow down like what are your thoughts on that?
The big focus of ours for the past two years or so has been getting to this reasoning paradigm.
Could you explain what this paradigm is?
We've spent time scaling up
With pre-trained models, then with models like GPT-5, GPT-4, we've gotten to really compelling, knowledgeable models that form the basis for ChatGPT. And with all that knowledge and intelligence, when you ask them to think about something, they would benefit from verbalizing a chain of thought.
So chain of thought is just like some kind of representation of their thinking.
Yeah. But the thing is it wasn't really their thinking. So when you would ask GPT for the base model, or to a large extent GPT-4, to solve let's say a math problem, I think is a very natural example here, they would largely emulate how they think a human might approach this problem because that's how they are trained, right? They are trained to predict what a human would say. This might look like thinking, and in some sense it is thinking, but it is not their thinking, right? It's not how we think, right? Like we think to solve the problem, we don't think like how might someone else think about this. And this is a very important difference, right? In particular because the way GPT-4 thinks is quite different, right? The way its brain works is quite a bit different from how a human brain works. And so we were pursuing this goal of teaching them their own way to think. And because of bugs, it took a lot of bug fixing, yeah, and it took some insights from a couple researchers around OpenAI. Eventually, we got to a point where they started working and we actually saw these models start coming up with their own ways of thinking. They were still in English, they were still legible to us, but they were different from where they started. That was a big moment for us. And I think since then we've seen, like, I think we have released publicly things like o1 preview and o3 recently, and I think we are making progress in making this kind of a useful experience, but I think we are still quite far from where this is going. And so I expect that progress to pick up over the next few years.
Okay. So it will even accelerate what we already have in terms of slope.
