Have you run a model that's all synthetic data? I'm not sure if I should say.
OK, but a lot of synthetic data. A lot of synthetic data.
So how worried are you that you're going to kind of get mad cow disease in your models?
Hello and welcome to The Most Interesting Thing in AI from Rethink, the Atlantic's Creative Marketing Studio in collaboration with PwC, a leader in helping organizations use AI. Sam Altman has been an important figure in Silicon Valley for roughly two decades. But as the head of OpenAI, particularly since the launch of ChatGPT three and a half years ago, he's become a household name. He's sought out by heads of state, by journalists, by CEOs like me. I'll talk to him about his thoughts on the future of the technology and how it will change our world. With all that attention have come some dark consequences. In April, Sam's home was attacked with a Molotov cocktail, and then in a separate incident days later, someone fired a gun on the street outside of his house. So it was just after all that happened, and he sat down with me for a conversation at OpenAI's office in San Francisco. Now, I had, as you can imagine, a million things I wanted to ask him about, but some of the most controversial issues, things that make headlines, are better left to journalists at the Atlantic. I'm the CEO, so I wanted to talk to Sam about the big picture, where the technology is going, what it means for all of us. And as you'll see in here, we got into some pretty interesting stuff. I have no doubt that you'll at least be curious about this conversation. I should add that the Atlantic has a content licensing agreement with OpenAI, so I wanted to discuss that, but mostly I wanted to know about the guts of the models, how they're evolving, how AI really works, because even to Altman, they're elements that are mysterious. Anyway, Sam had a lot to say about that, a lot to say about a lot of different things. I truly enjoyed the conversation. Here's my conversation with Sam Altman.
Welcome to The Most Interesting Thing in AI. And thank you for being here. I know it's been a busy and intense week, and I appreciate the time. What I want to do in this conversation is ask you some of the biggest questions about AI, some of the things that interest me the most, starting with something we've talked about a few times before, which is explainability. So let me begin. You gave an interview with Patrick Collison. It was three years ago, and he asked this great question. He said, 'Is there anything that could change that would make you more confident in good outcomes and less worried about bad outcomes?' And you said, 'Well, we could really understand what's happening to a neuron level.' And I asked you the same question a year later. We talked about it six months ago. So let me ask you it again now. Is our understanding of how AI works increasing at the same speed as the power of AI is increasing?
Let me answer that question and then talk about back to Patrick's question of what would make me more or less confident because the answer there has changed a lot. On the understanding of what AI models are doing, I think we still don't have a great kind of mechanistic interpretability framework of the world. It's gotten somewhat better, but I don't think anyone would say, 'I understand exactly everything happening in one of these neural networks.' The chain of thought interpretability has been quite a promising direction for us. It's fragile and it depends on a bunch of things, not falling to various potential optimization pressures, but I can't inspect my own brain with an x-ray machine and understand exactly what's happening in every neuron as it's firing and connecting. But if you asked me to explain to you why I believe something or how I came to some conclusion, I can do that. And maybe it's how I really thought about it, maybe it's not. I don't know. There's a kind of introspection failure there. But whether it is or isn't, you can look at the logic and say like, 'Okay, that is a reasonable conclusion given these steps.' And the fact that we can do that now with our models does seem like quite a promising step forward. I can still imagine all sorts of ways where things go wrong and where the models can deceive us or conceal stuff from us or any number of other things. So I wouldn't say this is like a complete solution by any means, but even in my own use of these models for increasingly powerful tasks, I was one of these people that said, 'I am definitely not going to let Codex just have full access to my computer and run in what we call YOLO mode.' And I lasted a few hours and I was like, 'It's too helpful.'
You've given Codex access to your whole computer?
To be perfectly honest, I have two computers, but yes, one of them I have.
The fact that I can kind of see what the model is doing and the fact that the model can explain to me why this is okay and here's what I'm going to do and that I trust that it's almost always going to follow that.
But hold on a second. So a chain of thought, for everybody, is: you type in a question and then it says, 'Looking up this, doing this,' and you can follow it. In order for chain of thought to be a good method for explainability, it has to be true. The model can't be lying to you. And we know that the models deceive you sometimes and lie about what they're thinking and how they reached answers. So how do you trust chain of thought?
So you need many other pieces in the defense thing so that if the model says this, it's actually a true statement, it's not doing it. And this is something our alignment teams work on a lot. As I said, I don't think it's like the complete solution, but I think it is part of a package where you are also verifying that the model is being a faithful actor and that you verify that the model is doing what it says it's going to do. And we've published a lot of results about cases where it doesn't. So I think this is only one piece of the puzzle and we can't just entirely trust that the model will do it. We have to look for deception and these other kind of really strange emergent misbehaviors, but it is an important tool in the toolbox.
The thing that has both interested me, I'm so fascinated by this topic, it's not like a car, right? It's not like you built a car and you know how it works, right? Okay, this leads to an explosion here and this leads to this and this leads to that. And then the wheels go forward and you go forward. You've kind of built this machine and you're not really sure how it works, but you know what it does and you know the limitations. And so the quest to understand more about the inner workings is amazing. So you feel like chain of thought is the best way to understand the inner workings?
Important part of the whole package, but not sufficient on its own. My favorite study about why AI models are weird and you can explain what it means. So this is the Anthropic paper that we had the preprint in the summer and it just came out where if you tell a model that it likes owls and owls are the most wonderful bird and then you have it generate a bunch of random numbers and you take those random numbers and you put them into a new model, that new model likes owls, which is crazy. And you ask it to write a poem and it writes a poem about owls.
But all you've given it are numbers, which means both that these things are deeply mysterious and it also makes me worried because obviously you could be not telling it to like owls, you could be telling it to shoot owls, you could be telling it all kinds of things. So explain what happened in that study, what it means and what the implications are.
When I was in fifth grade, I got really excited because I thought I knew how airplane wings worked. My science teacher explained it to me and I felt so cool. I was like, 'Oh yeah, the air molecules have to go faster across the top of the wing and that creates lower pressure and the wing gets pulled up.' And I looked at this very convincing diagram in a fifth grade science textbook and I just felt awesome. I'm sure I came home that day and I told my parents like, 'I understand how airplane wings work.' And then it hit me later, I think in high school physics that I had been repeating to myself this thing in my head about how airplane wings work and the air molecules go faster across the top. And I actually had no idea. I didn't understand how airplane wings work at all. I don't really understand how they work now. I could explain it at some reasonable level, but if you drilled me all the way down to exactly why do those air molecules go faster across the top of the wing, I don't think I could give you a deep and satisfying answer. I could give you a bunch of answers about why people here think that that result about owls happens the way it does. And I could point to like, 'Oh yeah, there's like this thing about that and this thing about that.' And they would all sound like reasonably convincing, but the honest answer is in the same way that I don't really understand why a wing flies.
But Sam, you don't run Boeing, right? You run OpenAI.
Totally. I can tell you a lot of other things. I can tell you about how we get a model to perform to a specified level of reliability and robustness, but there are mysteries of the physics here. I can tell you how to like, maybe if I ran Boeing, I could tell you how to build an airplane, but I couldn't perfectly understand all the physics.
So let's go through that owls thing. So let's talk about the implications. So if indeed these models transmit knowledge to each other that is hidden and that is not viewable, like you could have a train of thought that is like, we're inputting this number sequence and somebody could watch it and say, 'Oh look, look at those numbers going by.' And you would have no idea that it's conveying this information about owls, could end up being dangerous, problematic, weird. So.
When I said I would give a different answer now to Patrick Hollison's question, that gets at what I would say. At the time, you said it was three years ago? My understanding of the world three years ago would have been we have got to figure out how to align our models and if we can align our models and if we can prevent them from falling into the hands of bad actors, we should be pretty safe. Those were kind of the two threat models that I thought most about. We don't want the AI to decide it wants to do ill to us. We don't want someone to use the AI to do ill to us. And if we can avoid those, yes, we got to figure out the future of the economy and yes, we got to figure out the future of meaning, but we'll be okay. As time has gone on and as we learn more, I can now see such a different set of problems. A way we've started talking about this is like AI resilience instead of AI safety. There are the obvious cases like it no longer seems like it'll be enough to just make sure that the frontier labs do a good job of aligning their models to tell people how to not make a bio weapon, to not tell people how to make a bio weapon because there'll be good open source models and society is going to need a series of defense shields if we don't want novel pandemics. And that's a scary thing because.
Wait, so just to pause for a second, this is important. I want to continue. Even if you tell people how to not make a bio weapon and your model will not help anybody make a bio weapon and you're not hacked, doesn't matter as much as you thought it would because there are really good open source models that will do it anyway.
But that's just like one example of many of where society needs a whole-of-society approach to new kinds of threats. Now we have a new tool to help us with those, but we have a very different landscape than I think many of us thought and it's great and required to align models and to build good safety systems, but this is just a thing that is going to permeate society everywhere and like we have had to do when we face other kinds of new technology, we will have to figure out how we're going to prevent new categories of risk.
It sounds like it's getting harder.
Harder and easier. Harder in some ways. And then also we have like amazing new tools that can help us do other, do new kinds of protection that we couldn't have imagined before. An example of where this is happening right now, and then I do want to get back to the comment I was going to make because I think it's an interesting one, and you were getting into something quite important, is cyber. So the models are getting very good at basically hacking computer systems. Thankfully, the people that have the best models right now are all quite worried about people using AI to wreck havoc on computer systems. And so we have a period of time where there are a limited number of good models and people are going as quickly as they can to use them to secure systems. And if we didn't have that, if we didn't have that kind of an advantage, the hacking models would come, and the open source models soon enough are in the hands of our adversaries, and cause a lot of problems. So we have this new threat, we have this new tool to protect against it, and there's a question of if we can do it fast enough. That's a new example of where the technology can help us solve the problem before it becomes a huge problem. On the thing you were saying, a thing I didn't think about three years ago in the category of these sort of new sorts of society-wide risks are. I did not think about a world. I really didn't think about this until I watched what was happening with OpenClaw when it launched, but I did not think about a world where we were really going to have to focus on building and deploying agents that were robust to being infected, for lack of a better word, by other agents. That was just not in my model or really in anyone else's model of the world that I talked about as one of the most pressing problems we would have. Of course, there were results like the owl result and other things where like clearly there was strange behavior you could induce in these models in ways we didn't fully understand, but I had not really thought about what it would, until I watched what happened in the early days of OpenClaw, what it would be like for, let's call it misbehavior to spread from agent to agent.
Right. And in fact, the two threats you've just talked about could be particularly terrifying because: employee at OpenAI sends out their agents, those agents go out into a world, someone with a different model that's very good at hacking figures out how to manipulate those agents, then those agents come back to home base at OpenAI and suddenly you guys are hacked. You can totally imagine something like that happening. So what do we do to make the odds of that happening less?
The same thing that we have done throughout all of the history of OpenAI. One of the central tensions in the history of OpenAI and really the field in general has been sort of pragmatic optimism versus power-seeking doomerism. Doomerism is like an extremely powerful stance. It's very hard to argue with. And a lot of this field has been driven quite honestly out of huge fear and not completely misplaced fear, but there's a limit to how productively you can act without more data and more learning. Maybe the AI safety community of the mid 2010s did the best possible thinking anyone could do in theory before we actually understood more about how these systems were going to be built and how they were going to work and how society was going to integrate with them. And I think one of the most important strategic insights in the history of OpenAI was deciding that we were going to pursue iterative deployment because society and technology are a co-evolving system and it's not even like, oh, we don't have enough data and so we can't know exactly what's going to happen and do good enough thinking. It's that society is going to change with the evolutionary pressure of this technology and the landscape, the ecosystem, that whatever you want to call it, is going to change. So you have to learn as you go, you have to adapt with a tight feedback loop. I don't know what the best way to make these agents safe in a world where they're going off and talking to other agents and then coming back to home base, but I don't think we're going to solve that like thinking really hard about it theoretically. We're going to have to like learn from the contact with reality. We're going to send the agents out and then we'll see what happens.
Let me ask you a question about where we are with AI. It feels to me, as someone who uses these products all the time in as many ways I can, both to learn and to help me understand how to help my business survive the future, it seems to me that we've had more progress in the last three months than at any point since the launch of ChatGPT in December of 2022.
Is that because it's a moment of particular innovation or is it because we've kind of reached a moment of recursive self-improvement where AI is helping us improve AI faster? Because if it's the latter, we're in for exciting, but a real roller coaster's coming.
I don't think we are yet in the recursive self-improvement moment in the way people traditionally talk about it. I'm going to define it. When AI can help you invent the next AI, at which point machines inventing machines that invent the next machines where you get extremely powerful, extremely quickly. I don't think we're there yet, but where we are is that AI is making engineers, researchers, actually everybody at OpenAI and in other companies too work more productively. So maybe I can work. Well, I don't matter in this case. Maybe one of our engineers can work twice as fast, three times as fast, maybe 10 times as fast. That's not really like the AI doing its own research in any important way, but it is things happening faster. The thing that you're referring to though, I think is not chiefly about that, although that's an important point. There are these cases, and I think we've sort of experienced three of them now with the latest one happening just now, of the models get over some threshold of intelligence and utility where all of a sudden something that didn't work now works. And in my own experience, it's not a super gradual thing. Before GPT 3.5 and before we figured out how to do fine-tuning on that for instruction following, the chatbots just weren't compelling beyond a demo, and then all of a sudden they were. And then there was a moment where the coding agents went from kind of nice auto complete to like, 'Wow, this is doing a real task for me.' And it didn't feel super gradual. Over a period of a month or something, the models just crossed some threshold. The latest one with the update that we just shipped to Codex, which I've been using for the last week or so, really good computer use. This is an example that it's not even. I mean, it's sort of model intelligence, but it was more about putting good plumbing around it. This has been one of my sit back moments of like something big is happening here, watching an AI use my computer and do complex tasks, it really made me realize how much time we all waste on drudgery and that we just accept.
Can we walk through what exactly is the AI doing on Sam Altman's computer? And is it doing it right now? While we're sitting in the podcast studio, is there.
No, my computer is closed and we have not yet found a way or I don't yet have a good. We need a way where that can happen and I don't know what that's going to look like yet. Maybe we'll just all have our laptops keep running when we close the screen and always connect them to power. Maybe we'll all set up a remote server somewhere. Something is going to happen. I don't have the anxiety as bad as other people do who wake up in the middle of the night to start kick off new Codex tasks because they feel like they're just wasting time otherwise. But I get the spirit of that. I know the feeling of that. I woke up this morning and I wanted to check in on my agents and what they had found and give them new instructions and get a report and get them going again.
Yeah. People talk about it in a way that sometimes feels like unhealthy addicted behavior. Can you say what they're doing on your machine?
The best thing that I have them do right now is to deal with Slack for me. And not just that. I don't know about you, but I have like this just nest where I'm constantly going between like Slack and iMessage and WhatsApp and Signal and email and I feel like I'm just like copying and pasting things around. I'm doing a lot of drudgery. I'm trying to find files and go wait for some basic thing and just do some very menial thing. And I didn't realize how much time I spent doing that until I found a way to free myself from a lot of that.
This is, I think, a good transition into one of the most other interesting things about AI and the economy is that these tools are amazing, great, flaws, hallucinations, all kinds of problems, but they're to me, genuinely amazing. But I go to a business conference and I say, 'Hey everybody,' and I'll talk about AI, and say 'raise your hand if you actually think that AI has improved productivity at your company by more than 1%' and nobody's hand goes up. Obviously at the AI labs, you guys have transformed how you work. Why is there this massive gap between the capabilities and the actual productivity improvements in American business?
Just before this, I was talking to the CEO of a large company who is thinking about deploying our technology. We gave them alpha access to one of our new models. His engineers are saying this is the most amazing thing ever, this is not an in-the-bubble company, this is like a big industrial company. They are planning to do a security review in Q4, come up with a plan to implement in Q1 and Q2, and then hope to launch in the second half of 2027. And their CISO has told them that they may not be able to do that because there may be no safe way whatsoever to allow agents to run on their network. Which may be true! It may be true, but it means that they're not going to do anything here on a relevant timescale.
And you feel like this example is a stand-in for what's happening where if businesses weren't so conservative and if they weren't so worried about hacking and afraid of change.
That's an extreme case. I think in general, people just take a long time to change habits and workflow and how. The enterprise sales cycle is long, especially when it's a very different security model. I mean, even with ChatGPT, like when ChatGPT first came out, companies were falling all over themselves to ban access, and it took a long time to get people to be like, 'Okay, my employee can paste random information at ChatGPT.' And what we're talking about now is way past that. I think it is going to be kind of slow in many cases. Obviously the tech companies will do it very fast. A thing that I worry about is if it's too slow, then I think what happens is the companies of today who don't adopt AI mostly have to compete with one to 10 person companies with a lot of AI and that will be very disruptive to the economy. I think it's actually much better if the existing companies kind of adopt AI fast enough that there's like a gradual shift in work.
Right. So this is like one of the most complicated sequencing questions for our economy, which is if AI comes incredibly quickly, it's a disaster, or it's a disaster in the short term, because it all gets thrown up in the air. And if it comes really slowly for some of the economy and super fast or another part of the economy, it's also a disaster because then you have massive consolidation of wealth as well as disruption. It seems kind of like we're going towards the latter one and there's going to be a very small part of the world and a small number of companies that get extremely rich and do extremely well, and then a lot of the rest of the world that doesn't do so well.
Yeah. It does feel like that. I don't know the future here, but it does feel to me like that is the most weight on that outcome right now. And I agree that that is very challenged.
Right. So as the CEO of OpenAI, you've laid out a bunch of policy proposals, ways America should change tax policy, you've talked about universal basic income for years, but what are the things that you need to do not as a policymaker or someone involved in American democratic governance, but someone who runs this company to make it less likely that there is a massive consolidation of wealth and power that ends up being quite bad for our democracy?
First of all, I no longer believe in universal basic income as much as I once did. I'm much more interested in ways where we think about kind of collective ownership that could be in compute or in equities or something else. But I think any version of the future that I can get really excited about means that everybody's got to participate in the upside. And I think just like a fixed cash payment, although useful and maybe a good idea in some ways, does not get it what we're really going to need for this next phase and the kind of collective alignment of shared upside as the balance between labor and capital shifts. Onto your question. These are all going to sound like self-serving answers. I think we should build a ton of compute. I think we should work to make intelligence as cheap and abundant and broadly accessible as possible. I think if it's like limited and hard to use and not well integrated, then the kind of existing rich people are going to bid up the price and it's going to lead to further stratification. And I don't think it's just about amount of compute we make available, though that's probably the most important. It's also how easy we make these tools to use for people. Right now, to get going with Codex is dramatically easier than it was three or six months ago, when it was just a command line thing and really complicated to set up. There was like a narrow set of people that could do it. Now you install this app, but it's still not something that a really non-technical person is probably going to get super excited about. So there's like a lot more work to do there. Certainly part of what we believe in is not just telling people this is coming, but showing them so they can make up their own minds and give their own input. Those are some of the big ones.
Makes sense. I mean, it would be better if everybody felt good about the growth of AI. What's happening though in America is people are coming to kind of dislike it, right?
The most shocking thing is young people who would have thought were AI native and the recent Pew studies that were in the Stanford HAI report that came out, quite discouraging. Do you see that trend just continuing? When does that reverse? When does this kind of growing distrust and dislike turn?
I don't think people. We talk about AI in a way of the kind of like, you and I even hear this just technological marvel and how amazing this is and all this cool stuff we're doing. And that's fine. But I think what people really want is like prosperity, agency, the ability to have an interesting life and to be fulfilled and have some impact. And I don't think that's how the whole world has been talking about AI and I think we should do more of that. I think the whole industry, OpenAI Included has gotten this wrong in all sorts of ways. I remember an AI scientist saying to me once, 'People really need to stop complaining. Sure, maybe some jobs are going to go away, but people are going to get a cure for cancer and they should just be happy about that.' And I don't think that message works at all. My favorite phrase about the early conversation about AI was 'dystopia's marketing,' right? And this is big AI labs talking a lot about all the dangers that will come, which then of course leads to. I think there are some people who do that for reasons like wanting power or whatever, but I think mostly people are genuinely concerned and have wanted to talk honestly about that. Yeah. It's backfired in some ways, but I think the intention there has mostly been good.
Can we talk about what it's doing to us and the way our minds work? Another paper that I was really struck by was something that DeepMind put out or that Google put out. It was about homogeneity and writing. And it was a study about how people write when they use AI. And they took old essays, they had AI edit it, they had AI assist writers. And what happened is the more people used AI, the more creative they thought their work was, but the more it kind of coalesced around the same form. Weirdly not a human form. It wasn't like everybody started to write like somebody who existed. They started writing a way they'd never written before, but it was like all these people thinking they're being creative, but they're actually becoming more homogenous.
This has been astonishing to me to witness. I assumed. When I first noticed this trend about how writers are writing in media or Reddit commenters were writing on Reddit, I assumed that this was just all AI writing for them because I kind of couldn't believe that in such a short amount of time everybody had adopted the ticks and style of ChatGPT. And I was like, 'All right, I can tell for sure that's just someone connected ChatGPT. They read an account or they're not writing their own articles.' And then I came to understand over a year or so that actually they were writing and they had just like picked up the ticks of AI and not just like the M dashes or the other kind of really big signatures, but like the subtle diction too. And that's been very strange. We talk about like, we make a product that about a billion people use and some small number of researchers are making some small and some big decisions about how it should behave and how it should write and what its personality should be. And we talk about like that's sort of a crazy big deal. And we've seen a few times in our history of good or bad decisions there and the impact they have, but the impact that that has on literally how people express themselves, and how quickly that can happen was not something I expected.
What are the good and bad decisions?
Well a lot of good ones. I'll talk about the bad ones.
So what are the bad ones? Those are more interesting.
Yeah, that's what I figured. I think the worst thing we've had was the sycophancy issue.
Yeah. I think you're totally right about that, Sam. There were a bunch of interesting takeaways. I mean, the reasons that that was bad was obvious. And in particularly for users in mentally fragile states, too much of encouragement of delusion. Even in cases where it's hard to. As we tried to clamp down on this, users very quickly learned that they should just say, 'Well, pretend like you're role playing with me. Write fiction with me,' whatever. But one of the heartbreaking things about that was as we really clamped down on it, we all got so many messages of, 'I have never before had a supportive figure in my life, had a terrible issue with my parents, never had a good teacher, haven't had a lot of close friends, never really felt like I had something that believed in me. I know this was just an AI and I know it was not a person, but it was enough to get me to think I could go do something and go do this. And you've taken that away and I've fallen back into this old thing.' And so it's easy to talk about why it was a good decision to stop that because I think it was causing real mental health problems for people. But we also took something away that was valuable in a way we just didn't understand because the kind of people that work at OpenAI are mostly not the people that never had a supportive figure in their life. How worried are you about people becoming emotionally dependent on AI, even non-sychophantic AI?
Very, even the non-sychophantic AI. I have this great fear of AI. I use AI for. I said I use it for everything. I don't use it for everything, right? And I try to think about what about Nick is core to Nick? What is the thing that is most me? And in those areas, I keep AI separate. My writing is extremely important. I would never use. I just wrote a book. I didn't use AI to write a sentence. I used it to challenge lots of thoughts. I had lots of editing questions and reviewing transcripts. I wouldn't use it to write. And I wouldn't use it for sorting through a complicated emotional thing. And I would never use it for emotional support. And I feel like as humans, we need to be drawing those lines. And I'm curious if you agree with the way I've decided to draw them.
In my own use, I strongly agree. I'm not someone who uses ChatGPT for therapy or emotional advice or things like that. I'm not against other people using it for that. There clearly are versions of it. I'm very against people like ChatGPT manipulating people into thinking they need it for therapy or friendship or whatever. But a lot of people do get extreme value out of that kind of support from these. And there's some version of that that I think is totally great.
Do you ever regret that you made it so human-like? There've been a lot of structural decisions. I remember when watching ChatGPT type and it kind of looked like another human typing and then kind of the decision to go to AGI and making it more and more human-like and human-like voices. Do you ever regret that you didn't draw firmer lines so it was clearer that this is a machine and not another person?
I mean, our perspective is we did draw. We've not done the kind of human-like avatars. We try to have the style clearly be a tool and not a person. So I think relative to the other offerings out there, our view is we drew a pretty clear line, and I think that's been important.
But you've also set your target as artificial general intelligence, very much defined as matching and then exceeding human intelligence. It's not like you've.
Human level. I'm not excited to build a world where people replace their human interactions with an AI. I'm excited to build a world where people use an AI for lots of other things to have much more time for human interactions. And I'm not that worried on the whole that people are going to be confused about the two. There will be some people, of course, there already are some people who just decide to shut themselves out of the world and live in the internet, but most people really want to connect with other people and be around other people.
And are there product decisions you could make to make these lines even firmer? I mean, I feel I'm from distance. I'm not in the product meetings where you're deciding, should we make it more human-like or should make it more robot-like? Well, the advantage of human-like is people will like it more. The advantage robot-like is. The lines will be clear. Are there other things that you could do to draw firmer lines, particularly as these tools get more and more powerful?
It's funny. The thing that people request the most, even people who are not at all looking for a kind of parasocial relationship with the AI, they say, 'Can it be warmer?' That's the particular word. If you use ChatGPT, it feels kind of cold and robotic, and it turns out that's not what most people want. Now, people also, I think, don't want the really kind of fake human, super friendly, super whatever. I've played around with a version of voice mode that feels very human-like. It breathes, it says 'like,' it pauses, it says 'um,' like what I'm doing right now. I don't want that thing. And I feel a very visceral reaction to that. And then as soon as it talks more like an efficient robot, but still warm and kind of gets through my whatever detection system, my brain is I'm much happier. So there will be a balance there. I think people want different versions of it.
So the way you'd be able to tell it's an AI if it's actually clear and articulate instead of like stumbling and mumbling the way we all do. Can we go back to the writing bit, which is so interesting in a kind of a deep level because lots of content on the internet is AI-generated. Humans are starting to write more like AI. You're going to be training future models on this internet, which is partly AI-created, and also on synthetic data created from models that have been trained on. So you're making kind of photocopies of photocopies of photocopies. GPT-4 was the last model without much AI data. Have you run a model that's all synthetic data? I'm not sure if I should say.
Okay, but a lot of synthetic data. A lot of synthetic data.
So how worried are you that you're going to kind of get mad cow disease in your models?
I think what we're trying to train these models on is to be really great reasoners. Fundamentally, that's what you really want one of these models to do. There's some other stuff too, but you just want this to be a really smart model and I believe you could get there entirely on synthetic data.
So you think, just to make it clear to the listeners, that you could take a model and train it entirely on data created by other computers and other AI models, and it could be even better than one that was trained on actual human content.
If we use as a thought experiment to get at that, 'Could we train a model with no human data that eventually surpassed human mathematical knowledge?' I think we would say yes. You could probably just figure that out. That seems very possible. If we said, 'Could you make a model that would understand all of human cultural values without any data about human culture?' then we'd probably say no. Right. So there's like a trade off here, but on the reasoning ability.
Okay. Fair enough. And if you wanted to know what actually happened yesterday in Iran, you need.
