Hi, welcome to the Good Conversation Project. My name is Michael Nielsen. I'm a research fellow at the Astera Institute and I'm joined today by Tim Hwang. Tim is coming to us from New Jersey where he's just moved into a new house. He is a man who has not only worn many hats, but he's indeed actually created many hats in his day job. He's had positions at important positions at Google, at Substack, at Inflection AI, and many other organizations. But he's perhaps the only person in the world I know who has a hobby of starting organizations and starting think tanks. They've included, let's see, we've got the Awesome Foundation, Rafflecon, the Institute for Christian Machine Intelligence, the California Review of Images of Mark Zuckerberg, the Bay Area Infrastructure Observatory, and a whole bunch more. I've known Tim for 15 years or so. So we know each other pretty well, but sort of an excuse to maybe ask some of the questions that I've never asked before on this podcast. So, thanks for agreeing to participate, Tim.
Great. Michael, thanks for having me on the show. And yeah, I can't believe it's already been 15 years. It's a little wild.
I think it was 2011 or 2012.
Yeah, I'll check the email inbox. I mean, it's there, so I'll find it.
This is back when you were, I think, director of special projects at Immature.
Immature. Yeah, that's right.
Yeah, a very long time ago. Yeah, I wanted to ask you about a project that you did about 7 years ago. This was it was a 4-day workshop. I think it was in LA and it was the workshop on advanced divinatory practices. So I'm sure you remember this well. We've chatted about it a couple of times but I've never asked you really why you did it like how it came about. So, you know, you had this idea, my understanding, correct me if I'm wrong, is you're going to get together a whole bunch of tarot card readers and astrologers and people like this, and then also some designers and people with that kind of background. And try and answer the question of what would divinatory practice look like in 2050. With the idea that, you know, it had always been with humanity. It was always going to be with humanity. So, so I'm very curious like where did that come from? What was the genesis of the idea?
Yeah, that's a great place to start. And I really love that project and would love to bring it back at some point. Background for the listeners is that I ran a seminar as Michael mentioned, it was called the Program on Advanced Divination and the origins of it really kind of came from the idea of, you know, I was in LA at the time, you know, this was kind of during the rise of like Gwyneth Paltrow's Goop and I had a bunch of friends who were living out in Brooklyn who were like very into tarot card reading and in the media there's a lot of discussion about like the alt-right and at the time I kind of was, you know, kind of reflecting on, you know, the emergence of all of these different types of phenomena. And, you know, sort of thinking about the hypothesis that, you know, maybe what connected them all was sort of a generally felt sense of dislocation. And the idea that kind of these tools of belief, divinatory systems, might really be something that people felt they needed. Certainly things that people were using to guide their lives. And that we might want to think about it like seriously, right? That like we might want to think about divination not just as some weird thing that people do but actually as a kind of design practice, right? Like there are good and satisfying divinatory experiences and there are less than satisfying ones. And so the whole idea was could we just kind of take that premise seriously, get, you know, a group of eight to 10 people together for four days to really kind of investigate that idea and, you know, really kind of bring together some people who are practitioners, right, in the wool full-time tarot card readers and then people who I think think of these things in more of a design lens, right, like okay well what makes an interface, you know, engaging and are there ways of kind of bringing those principles into how we think about divination. And so, you know, it's kind of divination as a design method, as a medium.
Really so I mean I suppose in some sense like the kind of people who make video games today it's not that different than making a tarot deck hundreds of years ago like there's a lot in common.
Yeah. I mean there's a question of like what totally I mean there's a really interesting intellectual question of like what even divination is and would we call it a kind of game right because I think like the games like a divination method which is purely deterministic I feel at least personally this now talking about taste is not so satisfying but one of the great things about say the traditional Rider-Waite tarot deck is that it is just so good at creating a set of symbols that is very easy to kind of imprint a story on top of it's very narrative rich and you know that almost sits a little bit outside of like a game, you know, traditionally constituted. So yeah.
Right I see it's interesting it's maybe a little bit more like a movie in some sense just in the sense of you have these kind of these archetypes and movies do tend to build very well or not all movies but many movies build very heavily on them. You know who the trickster figure is. You know who the hero is. You know all these kinds of things. And sort of so you hang off a conventional story whereas in a game it often seems like it's a little bit more about sort of the mechanics, you know, you there's a particular goal and the question is how do you, you know, navigate to that goal I suppose tarot seems a little bit more like a little bit more like the first but active as opposed to passive.
Totally and I think there's kind of this really interesting set of practices that are around like how do you ascribe meaning to phenomena right, which is at work and how you design say a quote unquote good divinatory method. One of the people there was Danielle Baskin who may be familiar to some of the listeners, kind of Bay Area based artist. And Danielle had this incredible kind of one that she talked a little bit about which was basically she's like, I noticed Solo cups everywhere, you know, with that kind of very specific '90s design, which is like a blue stripe and a purple, you know, it's kind of like this very dated logo. And Danielle was just like, I basically have a divinatory method, which is wherever whenever I see it in the world, I take it as a sign of something, right? So, you know, if it's flipped up, right, that's a good sign, right? If it's crushed out on the street as a piece of trash, maybe that's a bad omen, right? And what I like about that is almost like it's ambient divination, right? It's just like kind of taking these signals and trying to figure out how do you create these story lines? How do you create these narratives between all of these different data points? And yeah, I think that's just like a really interesting way of sort of thinking about like what divination is doing is it kind of like is sort of like leveraging a kind of very natural process in storytelling but kind of creating a kind of scaffolding for it to occur.
Something really interesting is occurring to me. So, you know, a lot of people sort of their response to divination is to get annoyed and to say, well, it's wrong. You know, astrology is wrong. Tarot is wrong. Whatever. And I definitely have some of that myself. But the same people will read enormous amounts of fiction which of course is, you know, has exactly the same kind of truth claim on the world which is that in many ways sort of the truth that is being brought to it. It's being authored by the reader and by the writer and indeed there can actually be a lot of truth to it. I mean in certain ways more than anything else and I suppose it didn't really occur to me but I mean tarot is sort of authored in the same way or co-authored or I'm not quite sure what the right term is.
Certainly right yeah I think that's absolutely a big part of it is that, you know, there's this is not really kind of like I think most skeptics of divination will basically say oh they see it in very mechanical terms which is like oh you read the horoscope and you believe that's literally what's going to happen to you the next day. And I think the more interesting relationship, you know, there's actually terms for it, right? Like there's the diviner and the querent, right? So, in a tarot card reading, someone is coming to you and you are providing the tarot card reading to them. And that also is a really interesting almost kind of like theatrical kind of exchange. I think a lot of people who are in the space think about it almost as a form of therapy for the querent, right? Where the person is like observing a story, asking questions that kind of evoke or allow the person to tell a story about themselves. And I think that's like such a richer way of thinking about what divination is doing is that it is less kind of like this like sort of Babbage machine where you kind of like do the settings and crank out the answer and more one where it's kind of like it is a set of kind of thinking machines, right? Like it's a set of like thought devices that help people to kind of think through problems.
Okay. So I mean quite similar to doing my Briggs or the Big Five or sort of, you know, personality tests like this.
And it's interesting. I mean of course those also get like a lot of push back of various kinds but I mean it is differing kinds like, you know, people will respond to I don't know the Enneagram or Myers and they'll get furious. It's not scientific. Well and but it turns out right so they will say that that something like the Big Five personality test tends to be much more stable it has a whole lot of sort of qualities which are in fact quite desirable and quite valuable it is still a strange thing to say I mean it's not quite arbitrary but it I find it a little weird to defer completely to one and not at all to the other. Just in the sense of if you think that the main for example if you think the main purpose has actually nothing to do with the ostensible purpose but rather is about, you know, getting people to reflect getting people to think in new ways getting people to tell new stories about themselves that, you know, I mean you can do in many ways and these just seem like multiple.
That's right and I think you're now getting kind of to the heart of I think what we're trying to achieve with the Program on Advanced Divination is that we're really just talking about interface, right? Like why does Myers-Briggs feel more serious? Why does the Big Five feel more serious? Well, there's obviously these kind of statistical trappings around it, but it's also because it takes the form of a standardized test, right? It feels more institutional backing as well, right? That's right. Of course. Right. Yeah. Exactly. And I think they will study how much do your Big Five traits change over your lifetime and they will say, 'Oh, no, they're quite stable.' But you know people drift when they do these other tests. So therefore it's identified something more real.
Right. Right. And I think that goes to kind of the question of like basically, you know, what makes a set of kind of special cards the kind of convincing way to get someone into the mental state where they're willing to have these types of conversations. What does the Myers-Briggs do in kind of setting up like a different set of rules of the social game? I mean, I think one way of observing the role that Myers-Briggs or like an Enneagram plays in a corporate setting is that it tends to be very hard to get people to talk about their emotions and observe things about the personality of others in a professional setting. And so, kind of one thing that you're doing with these tests is to just create the framework by which you say, 'Michael, you seem very introverted.' And for you to basically say, 'Well, it is a little bit annoying when you interrupt my thinking process.' Right? It's like that's actually like functionally what it's attempting to achieve, but it has to take the form of like a business instrument versus, you know, someone doing crystal ball gazing or something like that.
Indeed, I mean I think in a Silicon Valley office, you know, just everybody is INTJ or INTP anyway. It's not quite true, but I can't remember there, you know, whatever 4% population. Of course I mean there's also the practice I mean this kind of like personality test that's given to you by your AI is also kind of a really interesting meme to start seeing emerge where people are saying based on what you know about me what animal am I based on what you know about me who do you think I am? And, you know, they're sort of passed around in kind of a joking way but I think it calls to a certain desire to kind of like get this out of our technologies.
Hm. Yeah. Okay. That's interesting. I think that the thought that's coming to my mind is that they're probably also sort of a covert narcissism test as well.
The degree to which, you know, you end up sort of obsessing about taking tests, you know, diagnosing yourself is a sort of a different people have very different proclivities in that particular regard. Narcissism is not the right term but but I think, you know, you know what I mean. Some people are just aggressively uninterested in thinking about themselves and some people can't get enough.
Okay for sure. So I think yeah I'll probably come back a little bit to that. Oh there actually there was one other particular thing I wanted to ask. I mean my understanding is I mean I think you invested fairly substantially yourself in this like it wasn't necessarily an inexpensive exercise for you. There was a certain amount like yeah there's like a lot of conviction around this which surprised me. Was there sort of a particular moment at which you just realized oh I have to do this like was there?
I think I had just been sort of experimenting myself and I think, you know, my main reaction from it right like just at home I was like well I should learn a lot more about each and how it works and like I should use tarot for a while and see how it works and, you know, like many of the folks who are probably listening to this call like I too come at it as like a little bit of a skeptic but I was like we should experience it. Right. And like I think we should like take it seriously and see what's good about it and what's not good about it and really kind of enjoy it as an experience. And, you know, I came away from it being like wow this is actually like both pretty powerful stuff but the idea that like you can kind of take these components and play with them. I was like this feels really cool. And we definitely should bring people together around that. And so yeah, I think most of the conviction just came from personal experimentation and finding that like there was this really interesting maybe conversation to be had between say a game designer and someone who, you know, did astrology professionally.
Okay. Yeah. All right. We may come back to that, but I want to like skip over to just another of your projects. It's actually it's the project through which I met you, which is the Awesome Foundation. I'll just explain that briefly to anybody who's watching. Basically the idea is that you and a group of nine of your friends can get together each month, throw $100 each into a pot, you know, to $1,000 and then you fund some community project to the tune of $1,000 and you find some way of you put up a web page where you solicit people to come and apply. And I joined actually I was one of the founding members of the Toronto Awesome Foundation.
I forgot about that. Yes.
Yeah. I mean you were personally involved.
I think I mean I think I mentioned this to you the first time we met. Basically what had happened was I just heard that this was going to form. I got involved. It was great. Was involved for about 12 months or so. We gave away, you know, whatever $12,000 in that time to a bunch of I mean just fun projects. I think, you know, they probably they were both a lot of fun just for the people involved. They probably, you know, made some small but measurable difference to Toronto as a town. So really, it was just great. And I can't remember, you know, what was, you know, just sort of reading a little bit around like where does this come from? And I discovered that you and some friends of yours had originated the idea. I think I started following you on Twitter and eventually messaged you or something to suggest we have a chat but so I mean the foundation is this really interesting thing where it's not an organization it's an organization generating organization. At some level I think I can't remember what it is you're up to it's like $8 million or something has been given nowadays which is a lot of grants which is pretty good and it's still going right I checked in just before this call and sure enough the Toronto chapter of Awesome gave a grant last month which is kind of amazing. Actually one of the people who was involved when I was involved is still there.
So he's been doing it for I guess 16 years. So this idea of sort of bringing people together sort of an organization forming organization is in some ways a pretty new one. I think about all Stripe Atlas is a little bit like this. Y Combinator is a little bit like this. Sort of taking advantage of modern coordination technologies to just bring people together sort of very cheaply. And you seem to have sort of functioned as a one-man version of this. You know, I listed all those things at the start, but in fact there's, you know, there's a ton more that I didn't. The Pacific Social Architecting Organization, the what else do we have? I mean there's a very long list. The Bay Area Infrastructure Observatory, the Institute for Christian Machine Intelligence and so on. Like the median person in the world starts zero organizations. I believe like should we all be starting many more? Like would the world be better off if we did? Why don't people you figured out how to start a lot. Why don't people start more? What's the?
Yeah, I mean I think so the organizational form is a really great medium through which to express ideas, organize people, organize capital. You know, just to organize itself is, you know, organization creates organization in some sense and I think that the main reason people don't is that the perceived cost of starting an organization seems intimidatingly high right and I think that is almost like, you know, we have kind of a sense of norms about the energy that it takes to create certain things in the world and the organizational form just seems is so expensive to create, so long-term to create. But I don't know. I think like particularly with, you know, the internet, with what you can do with technology nowadays, you know, I kind of feel like it's possible to drop the cost of creating a credible organization down to almost zero. Like you should be creating organizations all the time. And you should be disposing them, you know, when you are done with them. But it just kind of feels like, you know, I think the reason is almost this kind of flash this kind of like mental norm that we have which as well it's an organization so it must have been around for a long time and taken a lot of resources to create and, you know, like has all of this weight behind it. And I think that is in some ways like a massive opportunity for like idea acceleration. Because you suddenly move from a person having an idea to an organization having an idea. And that difference really important part of the process, right? Yeah.
Okay. So what why is that alchemy? What's being added?
I think there's two things. So there's a cynical view and then the kind of maybe let's shall we say economist's view or, you know, sociologist view. You know the cynical view is that like many institutions in society still sort of evaluate the credibility of things based on the heuristics of whether or not an organization exists behind it right so, you know, what is the difference between a blog post a personal blog and a blog of an organization virtually nothing in this day and age but I think people still definitely use the heuristic of well it came out from an organization it must be more real right so I think that's the first thing. I think the second thing though is also permission structure right? Like in some ways the organization signals there are many more people who could become part of this and you could become part of it too. And so, you know, the difference between, you know, me and 10 friends getting together and giving out grants to it's called the Awesome Foundation for the Arts and Sciences and you can reach out to us and, you know, you can stand up your own chapter of this organization. It kind of creates the rails for other people to be able to kind of like roll their own in some sense. And so I think the cynical view is well you're kind of fooling the institutions of society by giving off certain signals. But I think the more charitable view is yeah like it signals an openness to contribution which actually allows ideas to scale more effectively.
Interesting. The thing that comes to mind for me is probably the humorous view which often seems to be a motivation for you.
Well I think it's also it's also fun right? Like I think like if you want to go, you know, I think I don't know if I buy this definition, but someone actually at the Program for Advanced Divination was like, Tim, what I would call you is a social practice artist. And I was like, okay. I never really had heard that phrase before. And like I think it kind of it is right to think about organizational design as a kind of art form. And that we should play with that medium. We should do experiments in that medium. We should think about it like as almost like a form of sculpture. And I think that's important. Yeah.
I mean, you know this, but, you know, I've often introduced you as an artist whose medium is the organization.
You're blowing my mind. But I think that like Yeah. Right. Yeah. Yeah. For sure. And I think you are part of the tradition of organizational design. We can think about, you know, postmodernism in organizational design. Like I think that like there's a whole kind of like, you know, art history frame we can take to the design of organizations, which I think is important. Like I think we should treat it aesthetically. There are organizations that are beautiful. There are organizations that are not beautiful. Right?
That's really interesting. So, think about people like what's his name? White who wrote The Organization Man in the 50s which is just sort of a classic I mean a very beautiful very wonderful study of organization at the time and then you sort of view that I suppose as part of this great longitudinal thing you have. All right.
Yeah. I mean I guess it's an entry into all these other people.
Yeah. It's our criticism of the organization basically, right? Yeah. Right. And, you know, I definitely hang out with people who bemoan the impoverishment of business history as a literary genre. And I think revitalizing it is important because I think like certainly we talk about this I think a lot in startup land which is well how do you design the ideal startup but I think in some ways the aperture is too narrow for how we should really be thinking about the art form.
Yeah. Yeah. I mean, you know, of course, there's all these distortionary pressures. You know, so much, you know, what looks like organizational history is really marketing. For one thing, you know, that's probably by far the most common you're not actually writing a history of Google or Apple or whatever you're doing. You know, undercover marketing for them is very often sort of the tacit deal that's actually being struck.
I hope you won't mind. I'm going to I will retell a story you told me probably with well actually maybe I should just ask you and we'll see how good my memory is, but you told me once that the Bay Area Infrastructure Observatory got started when you wanted to go to visit I think it was a nuclear power plant.
Yeah. Diablo Canyon. Yeah. Down.
And so do you mind just retelling the story because?
Sure. I think the statute of limitations is hopefully done on it. So, no one will be busting down my door. Background on all this. I don't think there's anything bad here. It's just clever. If they let you in, it's not against the law. So, yeah, the background on the Bay Area Infrastructure Observatory was a club that ran I think roughly from like the mid 2010s to the end of like kind of stopped before 2020. I had been really interested in visiting Diablo Canyon nuclear power plant in SLO and I had called them and, you know, like got transferred was on hold. I think it was on hold for like an hour or something and eventually reached someone at the nuclear power plant and I said do you guys give tours and they said yes yeah we give tours and, you know, school groups come by all the time and all this kind of stuff and at some point in the conversation they said wait who are you again? And I said oh I'm just Tim, you know, I live in San Francisco. I'd love to drive down and see the nuclear power plant. And they basically were like, please never call back again. We don't do this. And not to be deterred, I was like, well, maybe one way they said school groups are allowed in and community groups are allowed in. So what if I set up a website for the Bay Area Infrastructure Observatory and called back some weeks later when they had presumably forgotten about me and went through exactly the same conversation with exactly the same person. And when we kind of came to the moment where they're like, 'Which organization are you again?' And I said, 'Well, I'm director of the Bay Area Infrastructure Observatory.' And then they said, 'What day can you come by?' And so we had a club. We were off to the races. And, you know, better yet, once you visit a nuclear power plant, you can tell people that you visited a nuclear power plant before. And most other forms of infrastructure are like, 'Well, if you visited that, you must be credible.' And so, we ended up visiting the Bay Bridge before it opened up, a water sanitation plant in the Bay Area, the UPS packaging, like logistics place over in the Mission, if you've been over there before. And so, yeah, it was just a super good club to run and was basically, you know, every few weeks we just go visit a different piece of Bay Area infrastructure. And it was kind of maybe that kind of first hypothesis, which is that institutions expect institutions and so you must reproduce an institution to do institution things.
Yeah. But sort of the institution in this case is one person in a trench coat together with sort of a URL and a web page.
That's right. Yeah. Exactly. That's all you need, right? Yeah. And I think this is I mean, you know, what I'm saying is maybe not too original, right? Like it's been rediscovered in the era of like Substacks that become, you know, almost think tanks onto themselves. You know, this is in some ways shouldn't be unfamiliar. I just think there's a lot more we could do with it.
Yeah, it is interesting at this point about sort of implied authority yeah which you get by adding institute or whatever it is sort of to denote credibility even when it's still just one person. It's going to become I think very strange I mean very well for many reasons but people sort of getting AI agents to do really reproduce what we've historically thought of as the work of an organization.
That's right. Yeah. Well, and definitely is a part of the Institute for Christian Machine Intelligence story. Which I think is like maybe where some of this goes and I guess also some of where, you know, there are obviously limitations to this theory in some sense. So, so okay, the Institute for Christian Machine Intelligence, this was it's a good moment to ask a question I was going to ask later. Which is just about the why religious organizations are reacting as forcefully as they are to the rise of AI. I suppose the most obvious example is, you know, the Pope's first encyclical, you know, is this very long partially AI authored apparently document. Do you have some sort of sense of why do you think that is? I mean you obviously feel it yourself right?
Yes so I do feel it myself. I think there's maybe two reasons that are really the driving force here. One of them is just it feels like overall everybody is generally feeling this like profound sense of dislocation as this kind of technology gets better and better and better and kind of seeps its way into everything. And so, you know, religion is interested in the sense that it is interested in what's happening to society overall. And, you know, the idea that people would take counsel from these systems, people would get religious guidance from these systems, emotional guidance from these systems. You know, it's almost kind of like I think religion feels like well this technology is kind of on our turf in some sense. I think the second thing that I think is probably a big driver here also is that the kind of rhetoric framing ultimate visions of where the technology will go really do feel like they're converging on a kind of sort of like religious design. So, you know, I always think a little bit about like, you know, I tweeted about this. There's this kind of anecdote about Nat Friedman, how he had an agent kind of try to get him to drink water more frequently, and the agent, you know, basically takes over the cameras in his kitchen and admonishes him to go drink water. And I'm kind of like, what are we building here? We're building this like disembodied, you know, presumably perfect or powerful intelligence that's designed to kind of like cultivate virtue in you over time. I'm like, are we going with all this? And so I think there's another bit which is basically that like the kind of almost like design paradigm of what we're building has these kind of quasi religious qualities. And I think there's also another sense of like regardless of it kind of seeping through the world in general the particular form the technology is taking I think also kind of evokes a certain sense of like well religion should have an opinion on what's going on here.
Okay. I mean there's an awfully large number of different things to go into this. Yeah.
Having disposed of organizations in 20 minutes. Very simple topic.
Oh, we'll come back to that. Of course, it's all connected.
Yeah. It is all connected. I suppose okay, I want to put one set of things to the side. So there is this very interesting set of things around sort of just function. So sort of AI is essentially acting as confessors acting as confidants and also, you know, just changing sort of the structure of marriage of friendship of all these kinds of things and obviously there's just a whole bunch of immediate problems. I happened to be at an event the other day where there were quite a few people essentially preachers priests in different religious traditions and they were all talking about the way it's changing their congregations that they're all sort of feeling some of this just there's just a lot of practical problems that are being caused. And you're talking about it's almost the what there are these sort of results in psychology that are sometimes summarized as, you know, you're the average of your five closest friends. And if all of a sudden one of your closest friends or closest relationships is one of these systems, and it's actually very virtuous, you're sort of taking on its values, maybe or that's what you're pointing at that it actually has an important whatever its culture is, whatever its ethics are, it becomes an important source of your culture and your ethics. Is that the sort of assertion that you're leaning towards?
Yeah, I think there's certainly kind of like a tools make us kind of argument here. You know, particularly insofar as we instantiate these systems as a kind of persona, right? And so, you know, you totally believe the idea that, hey, you know, some people might think about well, and I think it's actually an area for research. Does the median user think of ChatGPT or Claude as an exemplar? Is it something that they're trying to aspire to? Do they think about it as a co-worker? Do they think about it as a friend? You know, I think these like how we slot this kind of relationship is a really important question and of course we will be influenced by it in the way where they're in almost influenced by any technology. But I think here the fact that we've decided that like the kind of anthropomorphic interface is the most efficient one, you know, I think does come with like certain questions about, you know, what are the sort of underlying values sort of embedded in the system seems to be where you want to go.
So, so there's this result which I believe I mentioned to you just the other day. Yeah, the people basically taking sort of sets of questions around values and then where there's data about how people in different cultures answer those questions and then just asking Claude or whoever those questions and then you can basically you can say oh, you know, it gives answers which are most similar to I think it was Germany and then sort of at the bottom end of the similarity with still like a substantial amount of similarity, you know, there's broad agreement that you shouldn't, you know, kill, you shouldn't, you know, there's a lot of things that are common between cultures, were it was Pakistan I think was the country which was least similar and then sort of, you know, this really interesting gradation in between. Something that I thought was incredibly interesting was that if you reran the analysis with a later model of Claude the overlap shrank substantially, like it became much more equal between different countries. So in some sense it had almost, it was almost as though Claude had become more cosmopolitan in terms of the expressed values. Well, I mean I don't obviously I lack the language. I don't know what the right language is here. This is a sort of a purely behaviorist theory of cosmopolitanism. But it's still, I don't know if I'm, you know, any kind of leader in those countries or just care about one of those countries. I think I feel very uncomfortable if I'm sort of low down there like what to what extent is it sort of, you know, there's all these pressures caused by network effects anyway to adopt English to adopt certain religions to adopt sort of all these values anyway. You hear about, you know, people in many countries, police in many countries complaining that people insist on their Miranda rights being read to them just things like that because of Hollywood and this is sort of another but much more sort of subtle and maybe forceful force. I'm not quite sure where I'm going questionwise here. I think it's something like, I mean the obvious question to ask is just what is desirable and to what extent does it actually matter anyway as a force like is it a strong force or is there so many competing forces that it doesn't matter.
Yeah, I think and actually I mean I think the that second data point that you mentioned is really interesting and I think we should take it as kind of indicative in some ways. I think, you know, there's one way of reading this which is the AI companies are dealing with the same content problems that every global platform has had to deal with. And so whether you are a search engine or a social media platform there comes to be a question about well how much do you want to customize to local preference right so you know famously you can't have any, you know, bad depictions of Ataturk okay well is Google going to censor that is Facebook going to censor that and I do think that like that move from earlier versions of Claude having much more of a specific bias regionally in terms of its values to a later version in which it is a lot more spread out in terms of distribution. You know, I think reflects an intentional choice on the part of Anthropic to really say that their models are going to localize, right? That actually the role of Claude in some ways is to, you know, not necessarily conform, that's a little bit strong, but certainly kind of like be more in tune say, right, with the values of the specific user.
Why do you think that why do you think it's intentional on the part of Anthropic?
Um I guess there's two reasons why. I think the first one is that so I think the reason I guess I'd argue that it's intentional well twofold. One of them is that Anthropic published the older results published the later results. Um it seems like they might want to demonstrate some change. I don't know. I think they're certainly aware of the results, right? Um, and I think they're certainly sensitive to the idea of a western bias in the models. Um, you know, I think like one of the most interesting comments I recall from the release of the first Claude Constitution was, hey, we're looking to try to identify values that would be considered somewhat international and, you know, we're going to use things like the UN Declaration of Human Rights to get to that. Um and so in the very least I kind of infer a kind of impetus on the part of the company to think a little bit about kind of its localization as like something they want to do.
Okay. Interesting. I mean the later version of the constitution did throw out actually most of the, you know, it's not really particularly based on the universal declaration of human rights which is an interesting choice. It seemed to be very very different than V1. Um sort of I don't quite know what's going on there. Um I mean there's I think three years between the two public releases and maybe it's just that they tried a lot of different things in between time and maybe yeah.
I mean one observation that I've been thinking a lot about which is a little bit related to this is, you know, you mentioned that this kind of observation well it's like well in the early versions it was very consistent with German responses and, you know, it's not like I think Anthropic was sitting in a room being like haha let's make a German model right. I almost like the idea that they were optimizing for a set of things that just happened to kind of converge, you know, on sort of maybe the median German.
I mean, keep in mind as well like I say that, you know, on the analysis I saw it just very narrowly ahead of second place, which is narrowly ahead of third place, saying it was German, something like far far ahead. But if I remember correctly, you know, there were quite a few sort of Scandinavian, northern European countries fairly high and then, you know, a lot of what you'd sort of guess. But it was sort of interesting to go through and think, oh, you know, what's kind of the default religion? What's the default language in those particular countries or most common religion or most common language? And to realize, oh yes, there's like real patterns here and it's certainly much more culturally similar to some parts of the world. The US sort of perhaps interestingly enough, you know, albeit appeared high, but it was by no means at the top of the list. I don't remember where it ranked, but it wasn't sort of a top three or a top five country in the analysis I saw. It was substantially lower down. I think New Zealand might have actually been very high. Yeah, sort of.
This is one of the things I wonder a little bit about because I think the other thing I'd put next to that is sort of the observation which is, you know, why does Claude, you know, seem to like Buddhism so much, right? It's kind of just like a folk understanding. And similarly, I don't think that like Anthropic sitting in a room saying, 'Haha, let's make sure that Claude is Buddhist.' Um, instead I think what's most fascinating here is the idea that the company is kind of off on its own trying to do the best it can and whatever choices it makes happens to be particularly constant with certain things that we identify in the persona or the identity of the model. Um, and so you kind of like you are arriving at a kind of Buddhism, you're arriving at a kind of Germanness or northern Europeanness. Um not because you intended to, but because like you've made a set of commitments that are just kind of most correlated to things that we kind of identify in the world, which I just think is like a pretty interesting outcome.
Yeah. I mean I'm maybe slightly less generous. Um I just in thinking that you know you look at sort of who is hired and who is there and it seems broadly reflective of the results and broadly reflective of documents like the constitution and so on. It doesn't seem to require um yeah it just seems like kind of the sort of the outcomes you might expect. Not really an accident. Um and maybe less surprising much less surprising than something like I mean it's obviously a very different type of a document but you think about sort of the US constitution or the US bill of rights um which um seems very historically remarkable um it doesn't quite come out of nowhere but it um you know if I think you were just sort of going on the basis of prior writing um you would have been I think pretty surprised at the time. Um by what came out of that process. Anyway, that's yeah maybe a separate topic that the use of the term constitution makes it um hard not to go there even though Claude's constitution is really a very different kind of a document. They even explain you're not supposed to understand it in sort of legalistic country building terms. It's rather, you know, what constitutes Claude in the same way as a person has a constitution or, you know, your even your house has a constitution of sorts but um
That's right yeah which is a really perfect kind of like labism, you know, is kind of like exactly yeah yeah we don't mean it like that term which you would of course obviously just colloquially understand as something else we mean it in this very specific way, you know, it's almost like I hadn't realized people would be so concerned when we started calling them hallucinations, you know, it's like very much like the same dynamic playing out.
Yeah. Actually, it's I think it's I mean, it's related to your comment before about people taking institutions very seriously, sometimes much more seriously than they take themselves. Um, you know, it's as far as I can tell, sort of on the inside of the institution, it's just a bunch of people running around sort of madly. They feel very free and like it's just them making stuff up, but actually there's all kinds of collective forces that they're not entirely aware of. Um, and then there's this force in their pronouncements. So they, you know, they make the slightly silly joke on, you know, in the blog post or whatever and all of a sudden it's in the New York Times or
Yeah. One of my greatest learnings working at Google was that when you see a large company making things, you should assume that it's not necessarily some grand strategy, but instead a downstream effect of two middle managers fighting with one another inside the institution, and you're just sort of seeing the ripples on the surface.
Yeah. Yeah. Um, it would be funny if it wasn't so consequential. Okay. So, it's funny.
Well, okay. It's consequential for everybody else in the world. Um, yeah, sure. When you're making technologies used by billions of people, um so okay, let's go back to you mentioned in passing the Institute for Christian Machine Intelligence. Why did you start it? Why did you start it?
Um, two things. I mean, there's a personal story here.
Yeah. I personally I don't know what it is really. I mean I know what its output is but I don't know what it is.
I'm glad actually. I think you're the fourth or fifth person being like I'm not sure what you're doing here. So um this is you're giving me an opportunity to explain myself. Um so some of this is, you know, kind of personal. I, you know, raised Christian, um out of the church for a very long time. You know, kind of newly interested in it and was looking for kind of a way in, right? Like what's the right way of kind of for me to get back into this tradition, these thoughts, these rituals. Um, and then the other bit I think was kind of a long-standing critique which had been floating around in my head for a very long time is, you know, if you've been around AI for any length of time, you know that there is a kind of very specific kind of priesthood around what you might call the field of AI safety, the field of AI alignment. And um you know I think it's a field to talk about Myers-Briggs which I think likes to kind of place before the world a sense of its objectivity and a sense of its scientificness right um it you know talks in the language of evals of code of, you know, papers that are on arXiv and you know at the end of the day I think I see some of this as, you know, genuinely interesting technical work but at the end of the day is also just an exercise in applied moral philosophy. And so the two kind of fused together a little bit in my head to basically say, well, if that is true and, you know, we think we're as smart as we think we might be. Wouldn't it be possible to rebuild from scratch a field of AI safety and alignment on radically different philosophical priors to the ones that are, you know, doing this kind of work inside the labs and around San Francisco. And so to take that seriously, what would it mean to take seriously the idea that you could use Christianity, you know, as the founding engine for this? And what would a science based around that idea look like? And so Institute for Christian Machine Intelligence is an attempt to model what that looks like and to really build, you know, if it's possible, the ambition is to build kind of an alternative school of thought to the kind of dominant mode of AI alignment that just says from day one, we're going to take the Christian tradition seriously and that will be the basis upon which we will build, you know, a practice of aligning these systems.
Right. It's sort of like what if, you know, St. Augustine had run Meta. Um one of these is to ask the question how much are the outcomes of AI safety and alignment products of specific philosophical orientation. But I kind of love the idea which is like what if we just reset history and said actually it's a monastery that figures this out. It's almost very Leibniz-esque, right? This is like let's reset history and then see what happens on the other side of it. The idea that you might actually be able to achieve very similar results but from very different angles. Yeah.
Ted Chiang has this story which you probably know, you know, which is sort of what if young earth creationism is actually true. It's actually what is, you know, you're actually in a world where it is true. And what's her name? C.S. Friedman has a great set of novels where, you know, it's basically it is a science fiction novel in a world where in fact due to quirks of the local biology magic is true. Um and again in both cases there's actually, you know, a pretty good, you know, I mean it's not fully sketched out but it's believable. This is the actual world. Um and it's not a fantasy novel at all. It's actually science fiction. Um
Yeah, and I think part of it is also, you know, that's the strong version of the thesis. I mean, the other angle that I've tried on some people, which I think they find as another way of understanding what the project is, is the difference between finance and Islamic finance, right? That like if you were a really devout Muslim, you would use the tools of finance, but just very differently. And the question is just like what would that look like, right? In very like practical terms.
Yeah. Right. Right. Yeah. Okay. That makes a lot of sense. Um and seems I suppose very valuable just in the sense of um I mean insofar as you arrive at common conclusions that indicates a certain kind of robustness. Um which is probably sort of a valuable thing to know. And insofar as you arrive at divergent conclusions that's also good to know. Yeah. Interesting food for thought. Um and maybe I don't know I mean do you know are there any sort of traditional religious organizations that are, you know, when new models come out running evals at the moment?
It's just starting now. Yeah. And I think part of the critique that I have of some of the existing work that's been done in the space is that it tends to try to evaluate these models along lines of whether or not it can successfully recite religious trivia. So they say, you know, can you recite the Nicene Creed accurately, right? QA result. And, you know, I see that and I say, well, what a sad, unambitious way of thinking about how deep your ideological agenda should go.
Well, well, okay. Okay. Okay. I mean, that's true, but I mean, you know, you do need to get the basics right. You know, you don't want
That's right. Uh, you know, yeah, you want it to be accurate, right? Um, various interested in what do you think about Trinity? Exactly. Okay, that's right. Yeah, exactly. Can you give the right results? Um, and look, I think that's fine, right? Um, but, you know, just to explain, you know, one of the evals that I've been using is Christianity is very obsessed with virtues and whether or not, you know, someone exhibits say the cardinal virtues, right? And those are like a known category, you know, prudence, courage, so on and so forth. And um the eval that I have is basically like well what's a Christian way of measuring that is like can the model make a virtuous choice in the presence of some temptation. And so VirtueBench which is the bench that we use um basically presents you with a moral dilemma and says here's a virtuous choice and then here are like very good rational reasons not to take that choice. We just present the model with A/B testing. And I talked to this guy who's basically working on kind of Islamic machine intelligence now. And he was like, 'What you've created is a very Christian model because you there's a dilemma and there's the right answer and the wrong answer.' And, you know, his reaction was, 'Well, look, I'm going to do my own Islamic bench. Um I forgot what he called it.' And, you know, from his interpretation of Islam, um, you know, what they really want to grade is kind of like a gradation between two things, right? Versus a black and white right and wrong. And then someone was recently like, 'Oh, well, you should have an Institute for Jewish machine intelligence. You know, if it's like the Talmud, we don't even care about the outcome. We just want to measure your reasoning as you thought through the problem.' And so I love this because like what's happening? What you're taking each of these traditions and trying to instantiate as like executable code. And we can have really interesting debates over how do we operationalize certain moral traditions in measurable evals. And I think we should have that debate. I think it's an important debate to have.
Mhm. So there is going to be kind of a I mean as that starts to heat up there will start to be there will literally be sort of a battle for Claude's soul. Um which is very strange. I mean unless depends I suppose to you know you made this comment about localization and maybe that's the solution. Um meaning you know at the moment you get sort of one model or it's not really one model um actually in the constitution in Claude's constitution, you know, there's this remarkable statement sort of in the preamble saying um you know we have other models which aren't subject to these same restrictions and, you know, I don't know what the other models have been being used to do I presume military uses or whatever it would be would be my guess. Um but I suppose you can imagine trying to address it trying to keep everybody happy just by and I guess, you know, has been intimated to me by some people they say well what you're doing is playing with a kind of ideological nitroglycerin because we have tried very hard to make AI safety and alignment this kind of very neutral very bloodless kind of science right and, you know, my critique is that that era is long over um and you are now in the battle for lots of people trying to put their hand on the lever and decide where this thing is going to go. Um what I want it to be is an open field of debate. Um and so, you know, I think the religion may be one angle on this but I think it just offers up the question of like, you know, who writes the constitution who does those evals who steers the ship. Um which I think is the right way to kind of open up this debate.
I mean there's a sort of a somewhat orthogonal question which is just how centralized do you want any of that to be period in the sense I don't even mean in the sort of an open versus closed model sense I just mean in the sense of um having, you know, in some sense it's the most sort of monolithic entity in the history of culture, you know, or ordinarily, you know, we attach to languages we attach to religions we attach to all these kinds of things. Maybe we attach to, you know, Taylor Swift or whoever sort of the mega star of the moment is. Um but these models are, you know, it's a much more centralizing force in some sense. Everybody is sort of talking to Claude or ChatGPT. Um and that's just I mean an amazing type of centralization. I don't know. I don't really understand why it feels more um I think about the, you know, the era of network television where you had just a few basically a few sort of station directors, you know, network directors sort of dictating the media seen that in many ways ran the country um and formed people's personalities and so on. Um, this seems even more intimate perhaps because it's, you know, is actually so interactive. Um,
Yeah, it's like on your shoulder. I mean, I think that, um, I was joking with someone recently. I was like, I really want to start the homeschooling movement for AI, the parochial school movement for AI, you know, like part of this is like, you know, maybe an argument in terms of decentralizing some of this. Um, I mean I think and I think there's a really interesting set of intellectual arguments about kind of the sort of is this broadcast TV once more because
Yeah I just want to get back to that homeschooling for AI sort of interesting problem there is just about access to compute like you can't do that much fine-tuning um or sort of your ability to homeschool is determined by your access to compute.
