Evolving Prosocial AI: A conversation with Peter Fenton and David Sloan Wilson

Thank you. It's a genuine pleasure to be invited, truly, to one of my intellectual heroes, having read all your books over the years. Yeah, it's a real delight to be part of a conversation with you.

Well, thank you for that. The history goes into the random chance events of being born and raised in the Silicon Valley. My dad was a CEO for the bulk of my upbringing and gave me a window into the world of entrepreneurship, and particularly that community around Stanford and how there was something different here in the ecosystem, which we'll talk about as we relate it to AI, that allowed it to be maximally adaptive to new technologies. And I was lucky enough to go to Stanford as an undergrad, and I was pretty avowedly anti-commercial interests. And I found myself in rapture, as one does as an undergrad, with the world of the mind, and particularly in the world of philosophy. You have this perch from which you can dive into so many different fields and start to wrestle with the questions that are at the center of the work being done in those fields, and nowhere more than in my case, philosophy of science. And in my junior year, I had a professor, Peter Godfrey-Smith, who was my adviser, pull out this thin little book that was my version of poetry, and it was Elliot Sober's book on the philosophy of science. And within that book, there were a number of topics which are still being unpacked around units of selection, questions of altruism, things that are at the core of the philosophy of evolutionary biology, which is his most recent book. And in that time, which is in the early 90s, I was also interested in computational systems. And Tom Wasow was my other adviser, who started the symbolic systems program at Stanford. And it was clear even then in the early 90s, and it's very in vogue to say I was interested in neural networks before you know electricity or fire was, but in that case, I was quite interested in neural networks. And I took a class with David Rumelhart on optical neuronal processing, and the biological models were actually really effective at understanding computational models. And these worlds were not so separate, even though if you'd listen to the dogma coming out of Marvin Minsky and the East Coast regimes, I think there was a real hostility towards the use of neural networks because they just didn't have any of the simplicity and heuristic top-down functions that were attractive at that time and quite frankly more effective at generating computational success.

Indeed, more special-purpose. And it was a sense of, I don't know, the bubbling up of Californian mindset of loosely coupled systems collaborating together versus the command-and-control hierarchy that you find in the MIT culture.

Yeah. In particular, artificial intelligence are now at the epicenter of, I think, not just the Silicon Valley but quite frankly the human species.

What happened next is, you know, instead of getting a PhD, which I had probably thought about, dreamed about in fleeting moments of potential hubris, I went to a management consulting firm, Bain. Well, it's one of those things, I had two job offers coming out. Nobody wanted to hire a philosophy major back in 1994, except for Bain, and I had an offer from Morgan Stanley and I believe sort of one from Goldman Sachs. And it's sort of appalling, I look back at my life to think I would have gone from these high-minded pursuits to debasing myself as a consultant for a couple years. But yeah, I mean, it had its positive. It turns out philosophy is a highly portable skill. One might argue that it's the effective talent of BS. I found it to be more applicable for deconstructing problems into logical components that you could then apply rigorous thinking against. And so two years at Bain, and then I had, through my dad's work as an entrepreneur, been exposed to the world of venture capital. And I always thought that that would be the ultimate dream job, which is to work with entrepreneurs at the very beginning of the life of a company, serve as almost a co-founder, a partner to these people on the wild adventure of building something from nothing. And the Silicon Valley, you know, you can take this crazy alchemy of one or two people can imagine something that is in less than a decade worth, you know, forget the money piece of it, billions and billions of dollars, but really, really have a meaningful impact on a human scale. And that gravity well in the Silicon Valley of just getting to entrepreneurship and working on it, to me, venture capital was sort of the best route other than sort of founding your own company, which being a philosophy major, you're, you know, ill-equipped to do much of anything. But I say that jokingly.

Yeah. Except you can do everything. And venture capital, you know, I think it was a comment my dad made. He says, 'You seem to me to have like your peaks and valleys,' which is a broad way of saying my many deficiencies would be hidden in the path of venture capital and whatever strengths I did have would be amplified. And so to get into venture in that time, it was important, I felt, to work at a company, a startup, to really experience it firsthand before you start to make the decisions to invest in entrepreneurs and work with them over the years. And so in the mid-90s, I joined a startup. I was a product manager for about a year and a half, two years, and that was my launchpad into venture. And I've been doing that for, you know, since '98. So coming up on 30 years. I've probably worked with over, my two venture firms, I've worked firstly at Excel Partners and then went to Benchmark, where I've been for about 20 years. My guess is 30-plus companies. I don't do the numbers on this, but over half a dozen of them have gone public. I'd say over 10 of them have gone through a billion in revenue. And you know, I've landed right now at the center of AI. And that, you know, we have this ecosystem where the next big thing courses through the Silicon Valley's veins, I think, more cleanly than anywhere else in the world. So I joined the venture business when the internet boom was in its peak fever pitch in 1999. So I was sort of at the tail end of that. And then the next big wave was social mobile, call it, you know, 2007, '08 to about 2012. I was lucky enough to be involved with the Series A investments at Uber, Snap, Instagram, Twitter, where at Benchmark we backed those companies at the very beginning. We then, the next wave we went through in the industry was crypto, and in that case, it really wasn't much of a Silicon Valley phenomenon, even though there were a number of companies like Coinbase based here. But these cycles in venture tend to be labeled with one technology, and that was sort of from like 2013 till about 2020, 2022. And then of course the next big wave, and quite frankly the biggest wave any of us have seen in our career, started in earnest in 2022 with the launch of ChatGPT. You could speak to computers, you could have a conversation with them, you could imagine all these new possibilities, which is, in our sense, just at the beginning. And the Silicon Valley, and venture capital in particular, have proven over time to be the most responsive to these trends. And so the likelihood principle of like the next trillion-dollar market cap company when a new technology segment comes online is high probability, as in like 90% likely in my experience, to be based in the Silicon Valley. And we can get into why is that the case versus, you know, New York.

Well, I think part of it is the evolutionary biology model of being adaptive as a system is highly applicable to the nature of the Silicon Valley. And there's certain things, I'll just point out a few variables that are conducive to the adaptability as a system. The capital resources are set up to seek the asymmetry of if you put a dollar in, you can only lose a dollar. If it works, you can get a thousand or 10,000, and you know, the best venture funds are generating 30 to 50 times. So there's a risk-comfortable capital ecosystem. There's a high degree of sharing of know-how and expertise amongst the participants in the Silicon Valley.

So that high degree of sharing, by the way, is also embodied in the laws. And so we can't enforce non-competes in California, but in Boston and in Massachusetts and in the East Coast, non-competes are highly enforceable. So the idea of cross-pollination, of transmission of the ideas and know-how, is deeply restricted outside of the Silicon Valley and it's highly conducive here. There's the strong identity of the Silicon Valley of entrepreneurship. So our heroes are not financiers. Our heroes are not movie stars. Our heroes are the founders of, it's Jensen at Nvidia, it's Larry and Sergey at Google, it's Mark Zuckerberg. And so there's this aspirational sort of underlying ecosystem. And on top of that, you have a bunch of efficient flows of information around, and this is the functioning of the ecosystem, hiring people. The nature, some systems that I've worked in feel like, okay, you want to keep everything closed inside of your network. The Silicon Valley has this default of if someone asks me, one of my competitors asks me about an employee, I'll tell them everything that's of value and use because it's not, you know, to my standpoint, it's an ecosystem ethos, an ethic, a culture that you support others because you never know one day that may be you. Our heroes, people like Bill Campbell before he passed and Ron Conway, are notorious for having multiple competing interests because they collectively just want the ecosystem to flourish. So as you unpack it and you pull in the threads of, okay, what are the things that are also distinctive, there's this responsiveness of the Silicon Valley to disruption, and that to me is like most interesting, which is how is it that by 2022 when you launch ChatGPT, fast forward three years, almost, you know, when it was November I think of '22, and so fast forward three years, we probably have no less than 500 startups in the Silicon Valley that are premised on the existence of LLMs that would have been inconceivable to found in 2021. So the pace and the magnetic pull of the entrepreneurs towards the disruption forces is unlike anything I've seen. And part of it also is that we have an immigrant-based, you know, most of the people we back, I'd say over 95% of the entrepreneurs in the history of Benchmark have come here not just from the East Coast but from around the world. And that sense of aggregating talent, all of this leads to, you know, I think you could apply your, it was Tinbergen's principles of function, history, mechanism, and development around the Silicon Valley as the ecosystem, and you discover, I know there have been books written about just that, that the system has not yet been repeated anywhere else. And I think the mistake many people think is they apply one variable, oh, it's access to capital, or it's changing regulation, or it's, and I think they miss that the degree to which it's adaptive as a system. And you know, part of that is something that we as a system go through manic cycles. I don't know that this is well publicized, but you know, our mania, which we have right now, creates bubbles, and bubbles are required in many ways to move innovation at a pace that would be severely restrained if we were more tempered. And so people always say, like, is it a bubble, Silicon Valley? And but that's, yes, of course, that's the nature of what we do. And if it wasn't, then we wouldn't have this sort of, it goes back, by the way, I think to the gold rush in California, that in San Francisco there's this sense of levels of excitement that are totally unsustainable but all-consuming. And you know, there's an adage right now that a lot of our companies are sharing, which is 996: 9:00 a.m. to 9:00 p.m., six days a week. The only other place in the world where that's really happening is in China. And I think in both cases there's a sense of the totality and intensity of the opportunity is sublimating all the other things that are in your life. You know, one of the non-well-documented things is that San Francisco, as much as it is a city, you arrive and you're like, where's the Silicon Valley? Like, where is it? You go to New York, you can find the financial district. You go to Hollywood, you see the Hollywood Hills. In the Silicon Valley, you don't see any of that because it's all internalized. And there's this inside of the buildings, inside of the offices, inside of the cultures that are being created, a monomaniacal focus, which comes at the cost, by the way, of other forms of diversity in the ecosystem. But the Silicon Valley just feels like it's possessed by itself as opposed to these externalized, you know, any the outputs of all this is of course the products that we are delighted by and use. The depressive cycle which comes after the manic cycle will happen. I don't know if it'll happen in '27, '28, or '29, but it's just the nature of this world. And I was there in the post-bubble internet from 2002, '03, and '04, and it felt like nuclear winter. It sort of felt like that after 2012 and '13 when social mobile had run its course in its primary sort of ascendancy. And we'll go through it again, but at this time, you know, and we'll get into the AI implications, the stakes are higher because...

Indeed. And I think the effects, like, it's the single biggest capex in the history of humanity, buildout of AI infrastructure. And it's pushing into our lives in a way where we co-evolve with this technology, but at a scale and pace we've never seen. Over a billion, you know, monthly active users now at ChatGPT, and the pace with which it's penetrating our daily lives is unprecedented. And so it raises a whole bunch of interesting questions around, you know, the health of that evolution.

Well, I appreciate the framing. And I think the power of evolutionary biology to help us understand adaptive, not just as sort of always aligned, but maladaptive versus adaptive towards the high functioning and flourishing of an ecosystem, is a grand opportunity to take our thinking up a level to begin to understand the implication of these technologies. And we don't need to go far back in time to identify where you have pathologies that come out of the optimization of a system in a non-regulated direction. Social media has left a scar on a generation of children, and the algorithm made contact with humanity, and the algorithm won. If you look at the side effects, and I know this has been popular with Jonathan Haidt's book on the anxious generation, but you know, the idea that we co-evolve with these technologies and that they become predatory in ways that are completely logical if you're trying to drive attention, and attention is a vehicle from which you can monetize, then you prey on the human's weaknesses to fall into those traps. And I think that the consciousness we now have at the implications shows up, and many of the famous executives we know who run those companies don't let their children use those services, and they understand we've borne witness to how they can erode our sense of well-being. So you know, I think this idea that it's laissez-faire has been a bit of a Silicon Valley mantra, no regulation. And then you have the second model, which is top-down. You know, my partner Bill Gurley has a post about a long talk he gave at this All-In conference about why is the Silicon Valley Silicon Valley, and he gave you the number of miles between the Silicon Valley and Washington DC. And is that we built independent of the ecosystem of getting connected to regulatory capture, and we all know that regulatory capture has corroded major industries, healthcare being the principal example of that, which is why a COVID test costs 10 times in the US what it costs in Europe. So there's these two competing models, as you said, laissez-faire on the one hand, top-down on the other. But there's a third way, and I think that the evolutionary biology gives us a really good example concretely in everything from multicellularity and the work that I'm not familiar with Lynn Margulis, but understanding how is it that our mitochondria formed a contract with the cell to be able to, bacteria bonded to form a nucleus to have, so we have cellular life. And I think the same sort of sets of questions apply in our ecosystem. And the simplification that I think is most potent, and this is actually applicable in our companies, is what you said, which is that selfishness beats altruism within groups, and altruism beats selfishness between groups. So how do we put evolutionary pressure into the system where we have between-group competition towards a higher functioning level? And I think that is one of those insights when you start to work on the alignment question of AI, where the idea of, I think, naive idea of having top-down regulations that are coming from a bureaucratic entity, or it's, you know, okay, those people are the doomers, and then you ostracize that whole generation of people. And then there's been, you know, if you look at the press, the people who write about this, they hyperfocus on anyone who's trying to regulate as propagating violence against entrepreneurship because it's the sorts of things that have destroyed innovation in healthcare and in education. And it's, you know, what's really behind it, and this is the narrative, what's really behind that is the perverse incentives of the companies in the lead, OpenAI, Anthropic, Facebook, etc., to block innovation from the startups that can topple them. Is that part of it? I mean, undeniably that's happened in regulatory capture in certain industries. On the other end of the spectrum, you have laissez-faire, maximum force innovation, and with that bottom-up model, you invite, you talked about cancer, I mean, you invite the sort of entropy in a system where the self-interest of any one party can destroy the overall organism. And we can look for concrete examples. Obviously, we know that if you weaponize AI without any control on it, it could, among other things, you have instrumental convergence, you have all these sort of the paperclip examples, all these stories about what happens when the system runs amok. And so we find ourselves in this polarity between the laissez-faire crowd and then the regulatory crowd. And I think we have an opportunity to sort of go to this third approach, which is bubbling up in the work you've been doing. And I think you don't need to look past the human body or even an organ to see that there are multiple levels from which you can create alignment towards something that's functioning, or if you don't have it, it's dysfunctional and it destroys itself. And so I think the question in front of the world for AI is really to say what can we take around alignment in highly functioning, both pro-social systems but also flourishing ecosystems, and map it into the AI project, which is an organic ecosystem. It's not a single machine operating against an objective function. The most compelling thinker I've spoken to on this in our actual ecosystem doing work on this is Emmett Shear. Emmett started a company called Softmax. I don't even know if it's a company, but he has, he's already launched a competition on an alignment competition where you have teams, and the way the teams come together, they're agents, and multiple agents are on a group. And I think this is a useful sidebar if you bear with me. The group then competes to create a certain outcome, which is, I believe, it's an object that requires multiple parts. And so you have multiple teams competing to create the largest output, which is the number of, and I think they're hearts, they're like, there's a composite manufactured object. And what that shows you, if you have an individual in that team, an agent that's hyper-optimizing their self-interest, it destroys the group. And it's everything from the water slider example you've used in your history to, you know, playing Monopoly as teams versus playing Monopoly against multiple players, and you have these emerging properties that come out of an aligned system around cooperation. And the implications from the programming standpoint are profound because the black box that is most of AI now has a new function that it can start to train against, which is being cooperative, being pro-social, understanding in a way that simple self-optimization can't catch you. So, it's just the beginning. I think we're starting to see that if you apply systemically the idea of a complex adaptive system as a system and not as a system of agents that are maximizing their own internal objective function, the outcomes are transcendent. They're elevating the entire project beyond what I think is sort of the extremes of laissez-faire randomness, which is of course we know going to breed self-interest to the point of creating cancer, or this top-down rigidity, which is, you know, ultimately something I think appropriately falls into the, we can't even know the problems we're trying to align against, much less prescribe the solutions in a way that are effective. So setting up the AI ecosystem so that it is conscious of, insofar as the system can be conscious of, the sorts of design principles and the aspects of a high-functioning system, requires the leadership of the industry, and I think it also requires deep understanding of highly comparable proxy systems in evolutionary biology. And we're just now starting those conversations, I think, in an effective way.