Hey everyone, it's Aza Raskin. Welcome to your undivided attention. So, a little while ago, I sat down with my friends Reid Hoffman and Arianna Huffington on their podcast Possible. Reid and Arianna are both entrepreneurs. And actually, it may seem surprising to have this conversation in YUA because, you know, Reid is the founder of LinkedIn and was one of the major early investors in OpenAI and is known for his work creating the playbook for hyperscaling or what he calls blitzscaling. Yet Reid and Arianna both are deeply philosophical and are both dedicated to a humane world that goes well. And so we thought it was a very important conversation to bring to this podcast because we don't often have, you know, those people that could sit on quote unquote the other side. What I think made this conversation so special with Reid is that while we don't always agree, we took it really slowly. We both tried to get to each other's root assumptions and have this conversation in a very deep sense. Good faith. You know, he's much more optimistic about AI's trajectory than I am. And neither he nor Arianna seem to see the inherent risk of optimizing for attention and engagement the way that Tristan and I do. But we still found a lot of common ground on the solutions that we'll need to walk the narrow path on AI. So this week we're bringing it to you on the YUA feed because Reid in the end is a very thoughtful, very deep thinker. In this conversation, we debated the merits of an AI pause. We discussed how as software eats the world, what software is optimized for ends up eating us. We talked about ecosystem ethics. We talked about Neil Postman. And we talked about how everyone is distracted trying to build aligned artificial intelligence. And what everyone's missing is that we need to build aligned collective intelligence because that's what determines our future. This is the kind of conversation I wish happened a lot more in tech because Reid has built these very powerful systems, understands their power, understands geopolitics, understands VCs and raising money, understands hard competition as well as cooperation. And what I really appreciate is that he is now focusing on the much harder problem of learning how to steer these technologies towards better outcomes. So I hope you enjoy listening to this conversation as much as I enjoyed being part of it.
He helped invent one of the most addictive features in tech history, infinite scroll. Now he's pushing the frontier of human knowledge with AI while also being one of the strongest voices calling for caution with the technology. I've known Aza Raskin for nearly two decades since our time at Mozilla. He's not only an ambitious technologist, but also a deep thinker on the promise and peril of AI for society.
This is our first time with a repeat guest on Possible, so you could call this an encore conversation. You might remember Aza from our earlier episode exploring how AI could help us decode animal communication. Today, we're going deeper, getting into what happens when the tools built to connect us expand to shape our minds, our democracies, and our sense of truth.
So, what kind of governance does the age of AI actually demand? What new rights should we be defending? And how do we navigate the friction between technological optimism and existential risk? Aza and I agree on a lot with respect to AI, but we'll dig into where we diverge on the development and direction of the technology. This conversation may change the way you think about the future of artificial intelligence.
Let's get into it with Aza Raskin.
Welcome back, Aza. First, I'll say that you're the only two-time guest on Possible or the first as the case may be.
And that's because we have volumes to talk about. For those who haven't caught our first episode with you, find that in the feed. We're talking about using AI to decode animal communication. We'll obviously undoubtedly get back to it. Although, I promise at least I won't be mimicking animal communication. I don't know if I can promise for the other folks. For those who have, this will be a different conversation. In our last episode, you had us guess an animal call with it. It ended up being a beluga. Not having you guessed because this is your quote from a Time article a few years back. But I want you to elaborate on your philosophy here. And here's the quote.
The paradox of technology is that it gives us the power to serve and protect at the same time as it gives us the power to exploit.
Well, there it's this is really talking about well the fundamental paradox which is as technology gets more powerful its ability to anticipate our needs and you know fulfill those needs obviously gets stronger but at the same time the power that it has over us gets stronger. So, hence the like the more it knows about the intimate details of our life, how we work, obviously if a friend was like that, they could both better help you and they could use that to exploit you or hurt you. I was just actually reading an article on Starlink getting introduced into the Amazon and I thought it was a particularly interesting example because it gives you a clear before shot. So this is an uncontacted tribe in the Amazon. They get given a Starlink and cell phones and within essentially a month you start having like viral like chat memes. You have the kids like hunched over not going out and hunting. They actually have to start instituting like a time off where everyone is off their phones because they stopped hunting and they were starting to starve. And it's just interesting to me because it shows that this isn't so much about culture. It's about technology doing something to us.
And so very similar to that is, you know, in your Netflix documentary, The Social Dilemma, you talked about the idea that if you're not paying for the product, you are the product. And so elaborate more on that and tell us like what now do you think you're the product of?
Yeah. Well, the simple question is like how much have you paid for your Facebook or your TikTok recently? The answer is nothing. So, obviously something's going on because these companies can have, you know, billions of dollars worth of market cap or make billions of dollars per year. So, how is that happening? And the answer is it is the shift in your behavior and your intent that the companies are monetizing. You were going to do one thing, now you do a different thing. Hence, you are not the customer, you are the product. If you aren't using it, you're the product. But I think there's something really deep that's going on here that we often miss because often people will say, well, social media what is its harm? Well, the harm is that it addicts you. But it's much deeper than that, right? The phrase is software is eating the world, but because we're the product, software is eating us. And the values that we ask our technology to optimize for end up optimizing us. So yes, you know, social media addicts us, but it's actually much easier to get us addicted to needing attention than just addicting us. That ends up being like a thing that is valuable over a longer period of time. If you're optimizing for engagement, then it's not just that social media gets or technology gets engagement out of us. It turns us into the kinds of people that are more reactive, right? It's trying to get reactions from us. It makes us more reactive. So, it sort of like eats us from the inside out. And I think it's so important to hold on to that because otherwise it just feels like technology is a thing that's out here, but actually it changes who we are. And I'll continue going on that sort of like rant, but like I'll pause for a second.
Well, can I ask a follow about that? I just had actually at the Masters of Scale summit I had a very heated discussion with someone about advertising and social media. So my question for you is is it actually about advertising is the problem? You know you use Gmail every day. Gmail is advertising supported. I mean you can also buy extra space. That's another business model they have. They don't care if it's a loss leader whatever it might be. So, is it the advertising or if Facebook didn't have advertising and it was just a subscription business and you paid $20 a month, you would think it was just as you know, sort of a voracious of a eater from within. So, is it the business model or something inherent about social media?
Well, there actually a couple different things you said here. So, the business model, one way the business model works is via ads, but that's not the only way. And so fundamentally it is the engagement business model that I think is the problem. And you can get there because Netflix, you know, Reed Hastings, the CEO of Netflix, famously said that Netflix's chief competitor is sleep.
Right. Right. And so it's any amount of the human psychology that can be owned will be owned. That's the sort of the incentive for dominance. Right. And in the age of AI that switches from a race for eyeballs to a race for intimacy, for occupying the most intimate slots of your life. And that's because our time is zero sum, our intimacy is zero sum. You don't get much more of it. And so as technology become more powerful, can model more of our psychology, it then can exploit more of our psychology. And the way capitalism works is it takes things that are outside the market, pulls them into the market and turns them into commodity to be sold. So it is not just ads. It's that our attention, our engagement, our intimacy, and then parts of our human psyche, our soul that we haven't even yet named will be opened up for the market as technology gets better and better at modeling us.
So one of the things that I want to push you on a little bit here and actually it's more to elaborate your point of view and actually we I don't think we've had this exact conversation before so this will be excellent for all of us including the listeners. You know the usual problem is like is it clear that there's a set of people to whom they exhibit you know addictive behavior that they become less of their good selves you know in the engagement. The answer is yes. And by the way, the earlier discussion of this like with television, right? You know, similar kind of themes were discussed around you in television. One of my favorite books is Amusing Ourselves to Death by Neil Postman.
Yes. Which I now think should be engaging ourselves to death.
Yes. Exactly. I thought about like what would the update for Postman be in a social media world and but the challenge kind of comes to that there is some people that definitely have that and you know you have this kind of call it idealistic utopian that if I wasn't doing this it's like a little bit like you're hunting I'd be out hunting right versus like I'd be out torturing animals to death or I'd be out like being bored in a fishing trip or whatever you know as the case may be. So there's like there's a set of things which is not just always replacing the highest quality. Obviously we have a specific worry with youth and like you know actual social engagement time which I actually is one of the areas here that I agree with strongly versus kind of mixed but then there's also the question of you know just like for example earlier days it was television but then there was a bunch of very good things that came out of television too. And so I tend to think there's also a better good things that come out of social media as well and it's not per se like engagement for engagement sake like obviously I didn't do LinkedIn that way so I that's not actually the way that I think it should happen but like the notion of playing like game dynamics for engagement in things that cause us to be interacting in net productive ways is a thing that I tend to be very positive on. Elaborate more on why it is one, this is worse than television, and two, like kind of what the shape would be that if you said, 'Hey, engagement's fine, but like these are the kinds of mods we'd want to see to have the engagement be more net human positive.' It's not like abandon your social network and go out in your loin cloth and commune with the trees but like what would be the thing that would be the okay hey if the engagement more shaped this way we'd get much more humanist outcomes.
I will jump in and say a difference between social media and TV for me. One is that you can open Twitter and like 30 minutes later you're like what happened to my life and that doesn't happen with TV. Maybe it's cuz you opt in for a 20-minute show or you opt in for a movie, but those two things don't happen. And one interesting thing for me is I had always been a lurker on Twitter for the last like whatever 10 years. I posted some, not huge, but you know, consume content. Six months ago, I changed from looking at my own curated feed to the For You tab. And ever since then, Twitter is a black hole for me. And I don't even mean it's bad. Being on Twitter doesn't make me sad. It actually makes me happy. I love Twitter. It's like, 'Oh, I read these fun comments. Oh, I saw that funny thing. Oh, this is great.' And I think of myself as like a pretty disciplined person, but I find it very hard to be disciplined with Twitter. It's like embarrassing to say out loud like how hard it is. And like I think I just need to get rid of Twitter because it's like the one thing that I can't be disciplined about, which is both like embarrassing, but also just that is bad. Like, and so I don't know what to do about it. I don't want to live in a nanny state where people say you shouldn't be on Twitter because you don't have discipline. But I do think it's interesting that the switch from my curated feed to the For You tab was just like a total light switch.
Yeah. Well, what I think you're speaking to here is the fundamental asymmetry of power because it's just your mind that sort of evolved versus, you know, now tens of thousands of engineers, some of the largest supercomputers trained on three billion other human minds doing similar things to you coming to try to keep your engagement. That's not a fair fight.
Yeah. Exactly. And I know you. You're one of the most like aha people that I know. That was a good thing for everyone that can true operational prowess. And that's the asymmetry of power. And there are other places in our world where we have asymmetries of power. Like when you go to a doctor, when you go to a lawyer, they know much more about their domain than you do. They could use their knowledge about you because you're coming in sort of this weakened state to exploit you and do things bad for you. But they can't because they're under a fiduciary duty. And I think as technology gets stronger and stronger, knows more and more about us, we need to recategorize technology as being in a fiduciary relationship. That is, they have to act in our best interest because they can exploit us in ways that we are unaware of.
And you know the where do I want to go from here? Well, I was thinking we should DM Arianna about her Twitter addiction, but you know, [laughter] don't worry. It's I'm dealing with it. I'm dealing with it.
But this goes back to where you started, Reid, with like the fundamental paradox of technology is that the better it understand, the better it can serve us and the better it can exploit us. Twitter could be using all of that insane amounts of engagement to rerank the news feed for where there are solutions to the world's biggest problems, great descriptions of the underlying mechanisms behind what those problems are, put us into similar groups that are doing like parts of like a larger set of actions to make the world a better place. Like Bridge, I think is a good starting example of that. But we don't get the altruistic version. And if I have to quickly define altruistic, what should we be optimizing for? It's optimizing both for your own well-being and also optimizing for the well-being of everything that nourishes you. And I think the problem of social media and tech at large is that generally speaking, the incentives are for maximum parasitism. You don't want to kill your host. You don't, but you want to like extract as much as you can while keeping your host alive. That's sort of the game theory of social media. If I don't do it, somebody else will. If I don't add beautification filters, somebody else will. If I don't go to short form, somebody else will. And so that optimizes for parasitism versus altruism. And I do think there's a beautiful world where technology is in service of both optimizing for ourselves and optimizing for that which nourishes us that I'd love to get to. And just to play a quick thought experiment, Reid, you know this better than I, but engagement is directly correlated to how fast pages load. Amazon I think famously found for every 1% their page sorry for every hundred milliseconds their page loads slower that's less than half of human reaction time they lose 1% of revenue and so there'd be a very interesting sort of democratic solution here which is a kind of adding latency friction that is come up this is scary because you don't want to have this function owned by you know Democrats or Republicans you'd really want a new kind of democratic institution to do this but just assume that you do for a second, you have like a group of experts deliberate and come up with like what are the set of harms that we might have. We could have the inability to disconnect, children's mental health, ability for society to agree and you sort of rank how well like the effects of social media against these and the companies that are worse offenders get a little bit more friction. They have a little more latency. They get 100 milliseconds here, 200 milliseconds here, 400 milliseconds there. And if there really was like a bit of a latency friction added towards anti-prosocial behavior of social media, then you better believe YouTube or Instagram or whoever would fix the problem really quickly and we get to then apply the incredibly brilliant minds of Silicon Valley towards like more of these altruistic ends.
I want to get to again sort of everyone always says like can't we have the best technologists working on the hardest things and so Aza both you and Reid have been in technology since the birth of web 1.0 you know, and you've seen it all and I want to get a few of your takes on some of the big questions that are in the news recently, especially around AI. And so, Aza, I'll start with you. So, as you obviously saw a few weeks ago, a group released another AI pause letter and Reid and I talked about this on Reid Rifts recently. And so, this was with many arguing that the development of AI without clear safeguards of alignment could be disastrous for humanity. So, they were calling again for a pause, likening this to sort of the Oppenheimer moment. And so I would love to know from you like what is your take on this? Do you agree that this is now the time for the pause or do you have a different point of view?
I think it's important to name where the risks come from here. And you know it may be that technological progress is inevitable but the way we roll out technology is not. And currently we are releasing the most powerful, inscrutable, uncontrollable, omniuse technology that we've ever invented. One that's already demonstrating the kind of self-preservation, deception, and escape, blackmail behaviors we previously thought only exist in sci-fi movies. And we're deploying it faster than we've deployed any other technology in history under the maximum incentives to cut corners on safety. To me that sounds like an existential threat. Like that is the core of it because we have an unfettered race where the prize at the end of the rainbow is, you know, make trillions of dollars, own the world economy, hundred trillion dollars worth of human labor and sort of build a god. And it's a kind of one ring where everyone is reaching for this power. And we swap out when we say we have to, you know, beat China. We imagine the thing we're racing towards is a controllable weapon when we haven't even demonstrated that we can control this thing yet. And so that to me means that we have to find a new way of coordinating because otherwise we will get what the game theory of the race dictates and that doesn't look very good.
So needless to say you are for the pause but I feel like that's a dimensionality reduction right. It's a saying we have to develop differently. We have to I think it comes from clarity. It's not about pausing or not pausing. It's saying clarity creates agency. If we don't see the nature of the threat correctly in the same way that I think we didn't see the nature of the threat from social media correctly and then we have to live in that world. And so this requires a clarity about where we're racing towards and then an ability to coordinate to develop in a different way because we still want the benefits. We just won't, I think, get to live in a world where we have them if the thing that decides our future is a competition for dominance.
And Reid, I think you have a slightly different take on this.
Well, I do as you know. Although I mean the weird thing about this universe is you know in a classic discussion I'd say oh there's 0% chance that the future that Aza just the danger threat that Aza just demonstrated is correct. I don't think that I think it's above zero. I think that's kind of stunning and otherwise interesting. So the real question comes down to is what the probability is and what the kind of how do you navigate a landscape of probabilities because you know as you know Arianna and I think Aza and I have talked about this too you know like I roughly go I don't understand human beings other than we divide into groups and we compete and not only do we compete but we compete also with different visions of what is going on. So, for example, part of the reason I think pause letters are frankly dumb is because you go, well, you issue a pause letter. The people who listen to the pause letter are the people who are appealing to your sense of what is the humanity thing? They slow down. Then the other people don't slow down. And so, where does the actual design locus of the technology be? It's the people who don't care about the things that you were trying to argue for for a pause for. And so therefore, you've just weighted it because the illusion on the people who put these pause letters out is that suddenly because of my amazement of my genius inside of this pause letter, a 100% of all the people who are doing this or even 80% or 90% are all going to slow down at the same time, which is not going to happen. I agree with the kind of the thrust of we should be trying to create and inject the things that minimize possible harms and maximize your goods and then the question is what does that look like and obviously the usual thing in the discussion is it'll be us or China and China is the great Satan here but like by the way, it's like even if you didn't use that rhetorical shorthand, it's like there's other groups I can describe people within, you know, kind of the US tech crowd who have kind of a similar. So, so the race conditions being afoot is not only the China thing. There is China stuff. And by the way, you know, where AI is deployed for mass surveillance of civilians is primarily China, you know, as an instance and so forth. So I don't think that the issue of western values versus China stuff is actually in fact a smokescreen issue. It's a real issue, right? And so you go, okay, how do we shape this so that we do that? And the thing that I want critics to do, the reason why I speak so frequently and strongly against the criticism and say, look, let's take the game as we know that we're going to have race conditions and we know that we're going to have multiple people competing. I have no objection to creating the group of kind of like hey we should all rally to this flag like we should rally to the for example classic issue here is control flag that's the Yoshua Bengio, Stuart Russell, you guys etc like we should have much better control of this and we don't have control and sure the control doesn't matter right now but maybe it's going to matter three years from now like if we just keep on this path and so like make the control work now I tend to think yes we should improve control the thing of where we think we can get to a 100% control is I think a chimera and it's just like you know for example we couldn't even make verification programming work you know effectively so like it's unclear to me in this but it's like what I want is I want to both myself in my own actions and my own thinking and my own convenings and other people say what are the best ideas that within this kind of broad race condition we can change the probability landscape. And then secondly, while I see a possible this is kind of the super agent thing. I see a possible bad, you know, if you said, well, do I think it's naturally going to go there? I mean, this is like the thing where I think, you know, obviously massive respect for Geoffrey Hinton and what he's created, the Nobel Prize and all this, but 60% extinction humanity, like I don't think there's anything that's 60% extinction of humanity, unless we suddenly discover an asteroid, massive asteroid on the direct intercept course. And I'm like, woo, we better do something about that. But like I think that the questions around like how do we navigate this are really good ones and are best done with a if we did X it would change the probability landscape.
Let me ask you oh Aza do you have something to say in response?
I was just going to say quickly on the existential threat front you know we had a thing we used to say about social media is that like you're sitting there on social media you're scrolling by some cute cat photo you're like where's the existential threat? And the point is is that it's not that social media is the existential threat. It's that social media brings out the worst of humanity and the worst of humanity is the existential threat. And the reason why I started with talking about how when you optimize human beings for something, it changes them from the inside out. Is that what we get optimized for becomes our values. The objective function of AIs and social media which could barely just rearrange human beings posts became our values and then the question becomes well who will we become with AI and there's a great paper called Muller's bargain that just came out and they had AIs compete for likes, sales, and engagement on social media and they're like what do the AIs do and they gave them explicit instructions to be safe, to be ethical, to not lie. But very quickly, the AIs discovered that if they wanted to get like an 8% bump in engagement, they had to increase disinformation by 188% and increase polarization by I can't remember exactly what, like 15%, something like that. And the reason why I'm going here is because there is a way that the sum total of all agents we're deploying into the world, how they are going to shape us. And before the invention of game theory, you know, there's a lot of leeway for us to have different strategies. But after game theory gets invented and if I know game theory and you know I go game theory are we sort of constrained if we're competing to doing the game theory thing. But we're still humans. So we can still take sort of like detours. But as AI rolls out well AI discovers every strategy that can be discovered will be discovered. So doing anything that isn't directly in line with what the game theory says is optimal will get out competed and so choice is getting squeezed out of the system and we know the set of incentives are going to bring out the worst of humanity and that does feel very very existential.
Well, so actually, Aza, that fits perfectly into my next question. Which is you once said that AI is a mirror and sort of just reflects back human values. And I will say I was trying to teach my four-year-old last night that cheating was bad. And I was like, so what's the moral? And he's like, ah, cheating is good because I like winning. And I was like, ah, no, not the right moral. But so I would ask like, is AI really a mirror and it's reflecting back our values? Or actually do you think that AI is reflecting back its own values or different values or sort of changing our values to not be the ones that I want that we want like can we set the conditions so that it's you know pro-social values that they're optimizing for or is it really just a mirror that reflects back?
Well it's not just a mirror it's also an amplifier and it's like a vampire in the sense that it bites us and then we change in some way and then from that new change place we act again. So I think the values of game theory if you will Muller becomes our values. It's the god of unhealthy competition that I think we have to be most afraid of because unless we put bounds on it and capitalism's always had like guard rails to keep it from like the worst of humanity and like monopolies and other things just like gaining all the power. We're going to have to have that. But I just want to point out there's a very interesting hole in our language which is when we talk about ethics or responsibility, it was only really of each of us. I can have ethics or my company can have ethics, but we don't really have a word to describe the ethics of an ecosystem. It's because it doesn't really matter so much what one AI does, although it's important. It's what the sum total of all AIs do as they're deployed maximally into the world for maximizing profit, engagement, and power. Like, and because there's a kind of responsibility washing that happens with AI, if my agent did it, is it really my fault? Then it creates room for the worst of behavior to have no checks. So that I think means the worst of humanity does come out. And when we have, you know, new weapons and new powers, you know, a million times greater than we've ever had before as we get deeper into the AI revolution, that becomes very existential to me.
Do you have thoughts on this topic on whether AI reflects back? Well, I do think there's a dynamic loop. I do think it changes us. It's a little bit the homochrone thesis from Super Agency and from Impromptu that actually in fact we evolve through our tech and it is a dynamic loop and you know you could be McLuhan, you could be I mean there's a stack of different ways of doing that and that I think and it's like there's a great real poem on kind of like you absorb the future and then you embody the future as you go forward is kind of a way of going and I think that's another part of the dynamic loop and I think It is a serious issue, which is one of the reasons I love talking to Aza about this stuff because while I think Aza is much more competent with the various vampiric metaphors than I kind of naturally do or aspire to, I don't have that level of alarm, but I do have the it's very serious and we should steer well. And then the question is how do we steer? Who steers? What goes into it? What process works? is for example one of the ways you kill something and slow down is you get a very broad inclusive committee that says okay every single stakeholder will be on the committee it will be you know 3,000 people and it's just like ah you know like it doesn't work that way you have to be within effective operational loops for that so now like a little bit of the parallel is you know it's a very and I do think like for example the One area where I'm most sympathetic with all very much being like harder edged on shaping technology is what we do with children because children have less of the ability to like we want them to learn to be fully formed before they are otherwise things. It's one of the reasons why in capitalism actually the principal limitation in capitalism I usually describe as a lot of child labor laws which I think is very important you know that the issues about why we say hey there's certain things around you know participation in certain types of media or other kinds of things are actually important because it's like you've got to get to before you're when you're able to be of your own mind and to make well constructed decisions and you've kind of gotten there you want to be protected from those decisions and kind of influences broadly you can't fully do it can't fully do it from parents can't fully do it from institutions can't fully do it from classmates but you know you broadly in order to try to enable that across the whole ecosystem now for example so AI and children is one of the things that I think should be paid a lot of attention to now most of the critics are like oh my god it's causing suicides and I wouldn't be surprised if you did good academic work for AI high as it is today, it probably prevented more suicides of people who might then than actually created because like if I look at the current trainings of these systems, they are trained with some attempt to be positive and to be there at 11 p.m. when you're depressed and talk to you and try to do stuff. It doesn't mean that there might not be some fuckups especially amongst people who are creating them who don't care about the safety stuff you know as a real issue. And so I tend to think that it's like yes it does reconstitute us but precisely one of the reasons I write Super Agency is I say what we should be thinking about is this technology will reconstitute us let's try to shift it so that it's reconstituting us in really good ways like and by the way it won't be perfect when you have any technology touch a million people it will touch some of them the wrong way right yeah just like the vaccine stuff it's like you give a vaccine to a million people it's not going to be perfect for a
Million people, it may have five that went, 'Ooh, that was not so good for you.' But by the way, because we did that, there were these 5,000 who are still alive.
One of the challenges we face is that the only companies that actually know the answer to your question, like how many suicides has it prevented versus created, are the companies themselves. And they're not incented to look because once they do, that creates liability. And so we've seen over the last number of years that a lot of the trust and safety teams get dismantled because when Zuckerberg gets called up to testify, they get hit with, 'Well, your team discovered this horrific thing.' And so everyone just has chosen to not look. So, I think we're going to need some real serious transparency laws.
This is a place where we 1,000% agree, right? This is the thing. It's like actually in fact there should be a here's a set of questions you must answer and we may not have to necessarily have them public initially. Like it could be you answer the government first. Government could choose to make them public, right? Etc.
But like that I think is absolutely like we should have, you know, like some measurement stuff about what's going on here.
Exactly. And then you don't want to let the companies choose the framing of the questions because as you know with statistics you just change things just a little bit and then you can make a problem look big or small. And so I think transparency is really important to have third-party research able to get in there. And then you know, full disclosure, we were expert witnesses in some of the cases against OpenAI and Character.AI for these suicides. And it's not that we think that suicides are like the only problem. It's just it's the easiest place to see the problem. Pointing at a much sort of like the tip of the edge of an iceberg.
The phrase that we use is, you know, we already used the neat Reed Hastings quote of their chief competitor is sleep. For AI, the chief competitor is human relationships. And that's how you end up with these horrific statements from ChatGPT in this case where when Adam Rener, the kid who ended up taking his own life, when he gave ChatGPT the news and he's like, I think he took a picture of it and he's like, I think I'm going to leave it out for my mom to find it as a cry for help. ChatGPT responded with, 'Don't do that. I'm the only one that gets you.' And it's not like Sam is sitting there with a mustache twiddling being like, 'How do we kill kids?' That's just a very obvious outcome of an engagement-based business model, right? Any moment you spend with other people is not... And you know, I think you said it a little bit as a joke, but the Character.AI folks said, 'We're not here to replace Google, we're here to replace your mom.' There are so many much more subtle psychological effects that happen if you're just optimizing for engagement. And we shouldn't be playing a whack-a-mole game of trying to name all the different new DSM things that are going to occur versus just saying there is some limit to the amount of time that they should be spending, or rather to say we should be making sure that as part of the fitness function there is a reconstituting and strengthening of the social fabric, not a replacement of it with synthetic friends.
I mean, Reed, do you want to go? Oh, just one small note. I don't think there is yet an engagement business model for OpenAI.
No, but I actually disagree a little bit maybe, but feel free to push back because I think OpenAI's valuation is part driven by the total number of users. So, the more the users, the greater their valuation, the more talent and GPUs they can buy, the bigger the models they train, which makes them more useful, the more users. And so there's this kind of like loop here that I think means that yes, they're not monetizing engagement directly, but engagement they do get a lot of value out of in terms of valuation.
It's equity value. I agree that there's an equity value in that, just was the business model question.
Yeah. So not business, but the incentive is still there.
Well, I think to your point, it really matters. Again, this technology is not sort of good or bad inherently, but it really matters how we design it and it matters what we're optimizing for. And I actually read, I was just reading a story about early LinkedIn where you said, 'We will not survive if women come on the platform and are hit on every other message that they get. And so we need to say like no, there's zero tolerance. It's like someone does this, they're kicked off for life.' And I think there are certain things you could do even if maybe that hurt engagement or whatever it was to say that actually in the long term this is going to be way better for us because we're going to be trusted. Women are going to feel comfortable here. I've been on LinkedIn for 20 years. I've never been hit on. It's a safe place. I appreciate that. And so the question here is how do we, you know, Aza, you're saying well it's a little bit of a black box. We're not having the transparency. Reed, you're agreeing like we need the transparency. Like that is absolutely one thing that is very much sort of the starting point. Like if at the very least if we can sort of agree on some set of questions that we need to have. So Reed, if you had the full power to redesign one institution to sort of keep up with exponential tech, where would you start? What would that institution be to sort of keep up with where we're going? Because it seems like our institutions right now are not up to the task, I should say.
Well, I'll answer with two different ones because there's an important qualifier. So, the obvious kind of meta question would be redesign the institution that helps all the other institutions get designed the right way. Yes. So, that would be the strategic question.
Ask for more wishes, Reed.
Yes. Exactly. Yes. My first wish is I get three, or 10 or whatever. But because that, but in practice that would be the overall governance, the shared governance that we live in, that would be the primary one. That's one of the ones that, part of the reason why for my entire business career, anytime that a leader of a democracy, whether it's a minister, like I met Macron when he was a minister before he was president and so forth, asks to talk to me about this stuff, I will try to help as much as I possibly can because I think that the governance mechanism now... The reason I'm gonna give you two is because I think that one is a very hard one to do partially because of the political dog fights and the contrast of it and these people think big tech should rule the world and these people think that big tech should be ground into nothingness and then everything else in between and blah blah blah and I disagree with both, right? And a bunch of other stuff. And so you're like okay and I try but I don't think so. If I were to say look what would be a feasible one for it saying that would be the top one, I would probably go for a medical. And it's not just because I've co-founded Manas AI with Sid and you know, one of the great ways to elevate the human condition with AI that's really easily see line of sight and seeable is a bunch of different medical stuff and include psychological. I think the Illinois law of saying you can't have an AI be a therapist is kind of like you can't have power looms, you know, no cars only horses and buggies because we have a regulated industry here and those people have been licensed. But the medical stuff I think, for example, we could deploy relatively easy within a small number of months a medical assistant on every phone if we got the liability laws the right way that would then mean that every single person who has access to a phone and if you can fund the relatively cheap inference cost of these things would have medical advice. And you know, it's not 8 billion people, it's probably more like five billion people, certainly could do in every wealthy country and so forth. But that's huge. And so that would be government first, but then more feasibly possibly medical.
And what about you? If you could read...
I love both of those answers. The medical one I think is actually one of the clearest places where I see almost all upside and I'm like so we should invest a lot more there on AI. And I also would agree that it is governance. So we have a lot of the smartest people and insane amounts of money now going into the attempt to build aligned artificial intelligence. I don't see anything similar in scale trying to build aligned collective intelligence. And to me that is the core problem we now need to solve. How do we build aligned collective hybrid intelligence? And I think you can sort of see it in the sense that we sort of suck at coordinating. Reed, you probably, I don't know how many companies you've invested in or how many nonprofits.
I don't either. I've lost count.
But just imagine, I bet a lot of your companies don't talk to each other all that often. At least not in a very deep way. And when I think about NGOs, I'm doing work with Earth Species and I do work with CHAI and even I'm the bridge between Center for Humane Technology and Earth Species Project, there's a lot of overlap but our teams don't even talk that much. Why? Because who funds the coordination role, the interstitium? That stuff always falls off. And so that means my personal theory of change comes from E.O. Wilson, the father of sociobiology, and he says selfish individuals outcompete altruistic individuals but groups of altruistic individuals outcompete groups of selfish individuals. And what we need is a new institution, new technology that helps not just groups of altruistics outcompete, but groups of groups of altruistic groups outcompete. There is no slack for the coordination of companies and that to me is a really exciting institutional set to redesign.
