It really is my pleasure and my honor to welcome Jill, who I think needs no introduction in this setting, but in case you haven't read anything in the last 10 years, she writes for The New Yorker. The first time that I read a long essay by Jill was in The Atlantic about the Happy the Elephant case, and I think that that was one of the reasons why we first connected. Jill kindly invited me to teach a session in her class on rights of nature at Harvard that she teaches with Jim Salzman. And the first time that I showed up in her class, I thoroughly enjoyed the experience. But Jill asked me one question towards the end of that session. She says, "Do you think that the Silicon Valley types are a threat to the protection of rights of nature? Does technology have an impact on ecology?" And I said, "Sure." I think that that is going to be a mess. One year later, she invites me back. And I asked her, "So, what are you writing on?" And she says, "I just completed a book manuscript on technology and ecology and the societal impacts of AI." So, this is how Jill works. And we're so grateful for that. In particular, of course, what brings us together today is this fantastic book on the artificial state. And to me, there are two types of good books. Some books offer very useful information. You learn a lot about the world through them. Definitely this book does that. It tells the story of the ideas behind techno-optimism, technocracy, and the like. And then other books offer a completely new perspective on the world. Once you see the subject matter in a certain way that the author invites you to see it, you cannot unsee that way of seeing the world. And that is the type of book that I think The Artificial State is. So we're very happy to have Jill with us. And the first question, I know that you've gotten this question at every podcast interview opportunity and every event, but I have to do this. So why The Artificial State, and what is the artificial state?
Sure. Well, thanks so much for having me, Cesar. It's an honor to be at this magnificent gathering. What an incredible group of scholars and papers. So I think of the artificial state as a possible successor to the liberal democratic nation state. We often take for granted the constitutional democracy that here in the United States we celebrate the 250th anniversary of its first founding in 1776. But it's a fragile novelty historically. And to a significant degree, I think here in the United States and in many places around the world, we've abandoned some of our commitments to liberal democracy and surrendered our obligations as citizens in liberal democracies by participating in what I chose to describe here as the artificial state, which is essentially a digital infrastructure owned by corporations that is largely automated, in which at this point in history there are more bot thoughts than humans. So if we think of the public sphere as an essential component of a liberal democratic state, a free and open public sphere where we can deliberate as equals democratically, gain information, express our views, learn how to be members of civil society, that I think for many people that no longer exists, that you know the majority of people around the world get their information from social media, where again there's more bot traffic than human traffic. And those platforms are owned by corporations that have programmed the machines that are automating discourse to ends that have nothing to do with the common good. There are other ways that I think we see the incursion of the artificial state. I mean, like I just a quick example I would give is the degree to which major tech companies present themselves as substitutes for a liberal democratic nation state. So for instance, when Facebook was pressed by Congress to consider its influence on Brexit and then the Trump election in 2016, its response was to found its own Supreme Court. Sam Altman at OpenAI has talked about how he thinks an AI president would be better than a human one. Anthropic earlier this year released a constitution for Claude that was written by Claude. And I think we can all agree that a Supreme Court that is founded by a corporation is not a judiciary in any meaningful sense. An AI president has nothing to do with the system of government that we have chosen to rule ourselves by, and a constitution written by an artificially intelligent machine is not in fact a democratically legitimate constitution. And yet, but yet we sort of we accept these incursions.
And why is a state what the artificial state? I'll read from page 10 of the book where you say that the rise of the artificial state is inseparable from the fall of the natural world, the artificial state, the factory farming of humans, the sorting and segmenting, the isolation and alienation, as if humans were becoming to machines what animals had become to humans. I love that sentence, but it also that sentence also points in the direction of something bigger than the narrow definition of a state really. It's an artificial society, artificial culture. So, could you say more about that?
Yeah. Yeah. And there I do mean artificial state in the broadest sense, like the natural state, right? As opposed to an artificial one. I thought a lot about, I read a lot of science fiction for writing this book, which we may or may not get to, but I was really interested in the degree to which science fiction writers for more than a century and a half have anticipated the rise of what I call the artificial state in stories that all involve the fall of the natural world. So there's a kind of a very arresting story from 1909 written by the English writer E.M. Forster. It's called The Machine Stops and it comes out the year before his very well-known novel A Room with a View. And I always think of The Machine Stops as Room Without a View. And it's sort of imagining a near future in which all humans live in essentially cells. Everyone lives in their own room and they never leave the room and most of them have never been outside of the room in the entirety of their lives. So they're all kind of pasty-faced and they're terrified of the natural world. They have their food delivered by the machine. Their entertainment is available on something like a screen. They can communicate with their friends by a screen. They kind of, you know, they have DoorDash. They have FaceTime. And they have one book in their room and it's called The Book of the Machine. And it's just about how great the machine is. And our main character, her son, who she only communicates with by machine, is trying to convince her that the machine is going to run out of energy because it's extracting its resources from the natural world and those resources are finite. So the machine is going to stop and then everyone's going to starve because no one knows how to feed themselves. So her son's trying to convince her to leave the room and go out and escape to the natural world while there's still something of the natural world left that humans could live off of, and she fails to, and you know, kind of everybody dies. But it really conjured for me this picture, this notion of factory farming, right? The machine is actually using, or you know, we've all seen The Matrix, right? It's that story. But it's a very, very old, very, very old story, and it's always about the inevitability that the artificial state is an unsustainable one because the machine is fueled by the resources of the natural world that are themselves finite.
I'm so glad that you got us started on sci-fi because we were definitely going to go there. The first time that I ever heard anything about Jill's work on tech was when I listened to this fantastic episode of the X-Men podcast. When was that? Five, six years ago?
2021. So way before Elon Musk, it's all about Elon Musk's ideas of the future and ideas of the present as being taken from science fiction books. But I know, and Jill barely disguises or hides her indignation at the misreading of science fiction. It's not just a reading but a misreading of science fiction. And the book has plenty of additional ideas about how not just Elon Musk but other industry leaders really got many of these ideas and got evangelical about that after reading sci-fi books in their adolescence. So could you say more about the connection between science fiction, the misreading, and the way that they talk about the world because it certainly sounds weird to many people these days.
Yeah. I mean, what's so striking is, you know, most of these men, if you think of, you know, Jeff Bezos, Elon Musk, Peter Thiel, Zuckerberg, part of their self-fashioning is they're, I think, quite self-consciously kind of man-boys, right? Like there's an aesthetic around that. And I don't mean to be contemptuous of them about that, but it's an aesthetic, right? So they wear sneakers and jeans and t-shirts and hoodies and baseball hats and they're 50, 60 years old, right? It's unusual, but that's part of the shtick, right? Like, aren't we charming? We're still boys. And part of the we're still boys is we're still jazzed up about the comic books and the superhero movies and the video games and the Dungeons and Dragons and the science fiction reading of our boyhood. And there's a real clinging to that, right? Like, and I mean, maybe that was more charming when they were quite young men, and I think it doesn't age especially well. Like, I guess what I would say, like, I don't mean to be contentious. I really don't. Like, it's not surprising that someone who goes off and founds a bunch of companies and is an engineer and an entrepreneur hasn't gone back and reread the H.G. Wells that they read when they were 12, when H.G. Wells read as fun adventure stories. But a grown-up who reads H.G. Wells, you know, who reads The Time Machine or War of the Worlds will likely observe that H.G. Wells was a socialist and an anti-imperialist, and The Time Machine is the socialist manifesto and War of the Worlds is an anti-imperialist manifesto. But that's not how, you know, these are not men who I think are avid readers of literature in their adulthood, right? So they have this kind of world-building view from their childhood in which these stories are taken quite literally as instruction manuals when really they're cautionary tales. You know, don't go and colonize another planet. It's wrong to just take the land of some less powerful people, right? Or, you know, Elon Musk's touchstone is Hitchhiker's Guide to the Galaxy, which is, you know, the great Douglas Adams kind of soap opera space opera from the 1970s. It is specifically an indictment of apartheid. Musk grew up in South Africa. You can kind of understand how as a boy that passed him by. He still makes jokes about Hitchhiker's Guide today. You know what I mean? Like, so I think it's worth, partly just defensively, I want to say, like, they don't get to decide what science fiction is. But it's one thing to be someone who reads those books and, you know, you imagine yourself as Tony Stark from the Iron Man stories, but it's another thing to become Tony Stark from the Iron Man stories. Like, they are, and we all know boys and girls who as kids really love world-building stories, but we're not trapped in their world, but we are trapped in the world that these men are building.
Somewhere in the book you say that for people who talk about the future so much, they have a very thin understanding of history, that they do not know the origin of the very ideas that they tout. And one of the huge contributions of the book is the genealogy of some of the terms like technocracy and even with interesting personal twists like Elon Musk's grandfather having had a role in the technocracy movement. So, could you speak a little bit about the history of these ideas?
Yeah. So I've been interested in the technocracy movement in the 1930s for a long time because it's just fascinating to me how much strength it had in the early 1930s at a time when, you know, Americans were looking around the world and seeing democracies collapsing all over the place, and a number of Americans were really attracted, of course, to communism in the 1930s. There was significant appeal of fascism to the United States in the 1930s. Real concern that democracy would not survive these competing ideologies. We tend to overlook the technocracy movement, which had really a sizable appeal both in the United States and in Canada. The movement was led in the US by a guy named Howard Scott, and in Canada its national chairman was a guy named Joshua Haldeman through an organization called Technocracy Incorporated. The technocrats of the 1930s did not believe in democracy. They believed that a modern industrial economy could really only be run by an elite cadre of scientists and engineers like themselves. There would be no politicians, no elections. There would be no prices, no money, there would be no banks. The technocrats would assign numbers to all humans who would live in a technate, and they would be happy that they were living in a technate because everything would be run efficiently. Their output as laborers would be measured in ergs, a unit of energy, and they would get back the resources that were proportionate to the output that they put in, in a kind of closed energy system. Joshua Haldeman, Musk's grandfather, you know, went by one of these numbers. The technocrats had themselves a kind of fascistic aesthetic. They wore identical gray suits. They drove identical gray cars in what was called the gray fleet. The movement fell apart in the United States after FDR took office and delivered the reforms that he called the New Deal. In Canada, Haldeman remained quite attached to the movement even after Canada outlawed technocracy in 1940 as subversive of the existing Canadian government. He tried to come to the US. He had been born in the US. He was born in Minnesota. He was barred entry at the border because technocracy was so threatening. There was also possibly like just a passport issue. It's kind of unclear. He was briefly imprisoned in Canada for spouting technocracy. He then becomes the leader of a different party, this anti-Semitic party called the Social Credit Party, and kind of wanders around through the fringes of Canadian extremist politics until 1948 when South Africa announces the policy of apartheid, and he decides to move his family to South Africa in 1950 with his daughters, one of whom becomes Musk's mother. And in South Africa, Haldeman becomes a very ardent defender of the apartheid regime. He self-publishes a bunch of conspiratorial tracts defending apartheid, denouncing banks and universities and newspapers and films as part of a worldwide Jewish conspiracy. He's opposed to vaccination. And I think what's interesting, I mean, Haldeman dies when Musk is just a young child. Musk is not responsible for the views of his grandfather in any way. I'm not even sure how familiar he would have been with them. But what's really interesting is you go back and read the tracts of the technocracy movement from the 1930s. They're very similar to the manifestos of like Marc Andreessen and the sort of techno-optimist manifestos that have been coming out of Silicon Valley for a number of years now. And as a historian, I think this is a really important connection to excavate because one of the things that's so startling to me about the modern technocrats is that they present themselves as visionaries, right? I mean, wasn't Altman just on the cover of Vanity Fair as like a far-seeing prophet of the future? And no doubt the machines that they were inventing are very forward-looking, but their political, economic, and social ideas really all date to the 1930s, and they're the worst, most failed, most badly failed, and most discredited ideas of the 20th century.
Speaking of the future and of time passing by very fast, in the couple of months after your book was published, so much has happened, right, with the Hugging Face security incidents and also with The New York Times telling the story of how Anthropic engaged this group of religious leaders from around the world because there is serious concern among Anthropic founders about the well-being of Claude and whether the moral education of Claude could learn something from these very, very old traditions. When I read that piece and I listened to some of the interviews around it, I immediately thought of your book and how some of these worldviews have come out into the open. When you see those reports, what do you think? How do you feel about them?
Yeah, so I wrote the book last summer in something of a fugue state with a sense of real urgency. I mean, I was writing it for these lectures I had to give at Yale last year, the Tanner lecture. So I had to write the book pretty fast, but I was also just ingesting huge amounts of material about this world and trying to make sense of it. And I would tell people what I was working on, and people were somewhat incredulous, like, is that stuff really going to be happening? So it has felt somewhat fateful that a lot of the things that the book is talking about are, even just a year ago, it wasn't the case that everybody was talking about these things. Even the kind of the rogue AI, the existential catastrophic, you know, x-risk stuff that's, you know, has been very commonplace in many small corridors for a long time now is kind of out in the public. But I also, I would also say when I wrote the epilogue to the book, I kind of, I talked about how there was growing opposition to data centers, which I just made up at the time because I thought by the time the book got to galleys that might have started, like, and it would just save me an editing pass in reviewing the galleys, and that turned out to be right. That in exactly the way the science fiction stories, the parables of the artificial state always imagine, right? Like that child of the person in the room in the E.M. Forster story is like, we have to go out there and we have to, you know, we have to protect the natural world because the machine is going to destroy it. I hadn't really actually, I thought that would happen, but I didn't anticipate happening at the scale at which it's happening. And I think a thing I hadn't really occurred to me was I think we do, even people who are very tech-phobic, have a sense that like the machine isn't just a cute little iPhone that you keep in your pocket that you have, you know, your picture of your toddler and your puppy on and you can talk to your spouse any time of the day. Like that, it's, it's that you, that's just a thing you're, you're tethered, but it's tethered to something bigger. And that data center stuff where you're looking, that, you know, these sites that are 900 football fields large, I think renders visible and material to people something that had been invisible and immaterial, that is the scale of the resources that are powering what it is on your phone, you know, even separate from AI, just the degree to which you, you have surrendered some piece of yourself to a machine of persuasion. So I think there's a, in, you know, even on top of the kind of the NIMBYism or the, you know, the energy concerns or the environmental concerns, I think there's a kind of a great sort of unease, a kind of like a howl of misery that some of that reaction to the data centers is about that I hadn't, I hadn't really anticipated just what a sound that howl would make.
I promise that we did not coordinate the back and forth, but I was going to ask you about the natural world because one very encouraging feature of your book is that you analyze the impact of AI and these ideas about the world not only on human societies but also on the natural world. And I love how towards the end of the book you mention the beings who surrounded you as you were writing the book in that fugue state in your farm, the chickens and the animals all around you that keep you tethered to this material world as opposed to that world of silicon that Silicon Valley revolves around. Now you go farther than saying that the natural world matters and that we should be worried about the impacts of technology on the natural world. You also say that the natural world sets limits on what the artificial state can do, and this is one of the reasons why it's probably doomed to fail. So could you say more about that?
Yeah, you know, I guess I was, as you were talking, I was thinking a little bit about maybe just to tie back the science fiction piece to this. You can watch this clip on YouTube of Bezos a few years ago. I don't know, maybe three or four years ago, he and his now wife, maybe then fiance, go up in Blue Origin's rocket with William Shatner, who of course famously played Captain Kirk on Star Trek. And Star Trek is, you know, Bezos's touchstone, is, you know, to Bezos what Hitchhiker's Guide is to Musk. And William Shatner, he must be in his 90s by now. He's quite frail and looks frail. And the footage is of when they finally safely land, you know, which is so miraculous, and they've gone and done this orbit. And Bezos and his girlfriend are sort of, they're popping a champagne bottle, and it has a kind of like frat party vibe, like, "Yo, we did," you know, they're celebrating. They're relieved to be safely returned to Earth, but in a kind of, I just kind of brassy, honestly. It's kind of gross. But their celebration, and Shatner comes out and is, you know, he's older, so he's a little shaken, sort of staggering a bit, and he's trying to express something. And in the kind of glorifying of the achievement that Bezos and his girlfriend are celebrating, it gets a little bit lost, but you can hear a little bit of it. And then Shatner goes on to write an essay about it he published subsequently that it really, it was so thrilling. He'd always wanted to go to space, and it was incredible to go up in this rocket and look back at the Earth, and he thought it would be like a revelation, you know, of the kind of the power of human technology, but he was totally surprised that it was an experience of profound grief and sorrow for him looking back at the planet. It left him weeping at how much damage humans had done to the oceans, to, you know, that he sensed the, and this is, you know, you read about this, astronauts themselves, there's a kind of sublime spiritual reckoning that a lot of people have when they go to space, that it just filled him with regret and concern and grief. And it's, it's kind of a measure of the two different ways of thinking about where we kind of are at the moment. And I think, you know, when you hear from someone like Bezos, it's okay that we will run out of land and resources on this planet to run the kind of compute that we would need to achieve escape velocity from the Earth because we will put the data centers in space, right? There's just a kind of, we will then, you know, we will now militarize and commercialize space in order to make it possible. And so for a lot of, I mean, just to be clear, when I talk about the artificial state, to me it sounds like a dystopia, but when they talk about it, that's what they're trying to build, right? That's their, that's the fantasy, right? That we will, we will so wholly maximize the extraction of resources from the natural world on the planet Earth that it will make it possible for us to escape.
And why is it doomed to fail?
Because those resources are finite. I think that is the lesson of every piece of science fiction that engages with this. And I also, I really, unless it devolves into an utterly totalitarian system, I just don't think humans who still have a voice over their own affairs and the future that their governments are building will agree to it.