AI, as you know, is trained on data. It's trained on publicly available data. What do you think is the most strongly represented moral tradition in the training data? Like, let's think a little bit about that for a moment. Do you think it's possible that public domain material just contains overwhelming numbers of Christian tokens? And if that's true, doesn't that mean that we can invoke this religion as a way of steering models in a very fundamental way that secular approaches will never touch?
I don't know about that. I mean, it seems to me like Enlightenment liberalism is much more represented in the training corpus and you know more fundamental Christianity.
Yeah. Well, okay. So, now we're getting actually into the game, right?
Live right now with Tim Hwang from the Institute for Christian Artificial Intelligence. Is that what it's called?
Machine intelligence. But yeah, thanks Sophia, thanks for having me on the show.
Absolutely. So first question was, you know, obviously we're here to talk about the pope's recent encyclical letter Magnifica, but Samo Burgers tweeted something like, you know, people only care about the pope among all of the religious leaders. The pope is the only one with pull in the secular world. Like we wouldn't be sitting here talking about having people on the show if, you know, the grand ayatollah of 12 Shia Islam wrote something about AI. So why is it that the pope is even more so than the Dalai Lama or anyone else like the only religious figure with authority in the secular world?
Yeah, I think there's kind of two things going on here. I mean, one of them is just institutionally the Catholic Church has decided to make this a big deal, right? This is the first encyclical that the pope is putting out. They do a very high-profile event and even arguably the kind of selection of the papal name here is kind of a reference to technological change. And so I think, you know, in all respect to Samo, I think part of it is just that this is where they put their chips. I think the second thing of course is, you know, goes without saying that the Catholic Church has a long history in the west and that cultural influence is deeply significant. And I think so even though we live in a more secular age, you know, people still feel like this is something that they have to respond to. And particularly in a world where, you know, religious authorities by and large haven't really waded into this space at the level of granularity it sounds like the Catholic Church really wants to do here. And so I think there's also a sense, you know, for those in the Bay Area that like now the church is on their turf. And so I think there's certainly you can't escape a certain level of territorialism about the reaction that we've seen here. So I think there's a bunch of factors not just, you know, kind of the pope's role as sort of a totemic religious leader. It's actually a little bit deeper than that.
AI labs often talk about these things in terms of like safety, alignment, capability. Um the Vatican's wording on AI was like much different. It was talking about like human dignity. It was talking about like labor. It seems like these two camps have like talked past each other a lot where the pope's encyclical treated AI as more of like a normal technology, something more akin to even the industrial revolution but you know not something that fundamentally changes the nature of humanity or like what it means to live on earth. So like for example the encyclical said that AI systems don't have the capacity to think or reason. Why do you think the pope came to that conclusion?
Well, so there's a couple things going on here and you're of course touching on the much controversial section 99, section 100 which has really ended up being kind of like the headliner here. We can get into that but I think I want to take your first question which I think is a really good question and certainly is at the heart of some of the research that I'm currently involved in today right through the Institute for Christian Machine Intelligence. You know, in some ways I think we have to almost look at this, you know, and this will sound funny to anyone listening. We need to take a step back and be like this is actually in some ways a weird artifact of the just basic fact that these companies are B2B SaaS companies. That at the end of the day you're building machine god but what you need to do is you need to convince companies to use this as an API for their business, you know, processes. And this is I think imposed an interesting flatness on the rhetoric of the labs in this space, right? They say well, you know, we're involved in the science of AI alignment, we're involved in the science of AI safety and of course they only publish kind of these academic papers and it's all couched in research and evals and data. But you know, in some ways they have to do that, right? They can't make this feel too normative about like what it is that they're doing to shape the behavior of these models and what they do in the world. But at the end of the day this is all just applied philosophy. It's applied moral philosophy that at the end of the day there's someone inside Anthropic, there's someone inside OpenAI that sits down and says what should this model do and how do consider whether or not this model is doing good in the world broadly speaking. And so actually I think about this less as two fundamentally different schools of thought and actually more the what you said, ships passing in the night, that like ultimately this is applied philosophy and you know the secret of it is this is no science in a certain sense. This is all about whether or not it's possible to adapt value ideology into code, data, models, eval, everything that we're familiar with. And so I think from my point of view Christians have as much leverage on this space as anyone else does. And in fact, they might actually even have more leverage, right? You have like if you want to do moral philosophy, we got 2,000 years of moral history to bring down in this space. And so I actually see this as a call to action, right? This is actually a moment where it turns out that we're going to bust up this idea being a science and that anyone can play the alignment game now. And I think the ambition of ICMI is if I can find alignment results that are empirically at par or better than a secular result. Oh, okay. Then the shoe is on a whole another foot. Then we're having a whole different conversation. And so I think that's the project here is that it really isn't two camps. I think it's basically certainly the Catholic Church saying here like, hey, you know, you're actually on our turf, we're not on yours, and it's about time to actually have that real discussion, which I think is really important. I guess maybe I'll pause there and, you know, end rant there, but I think we can then talk about section 99 and 100, but I think that's how we should frame up and think about what's going on here.
Yeah. Well, first I'm wondering what exactly is ICMI? Like what are your research directions? I think someone told me it was like developing novel Christian methods of AI alignment, but like what does that mean?
Yeah, totally. So, I'll walk you through some concrete examples, but it's a really simple idea at its heart, right? Which is basically if you buy what I'm just saying that a bunch of Berkeley guys on LessWrong are basically doing moral philosophy and cloaking it in code. What I say is, okay, well, I should be able to do AI alignment on any set of philosophical priors, right? And I should be able to do my alignment on say Christian first principles. Okay. So you're saying okay Tim so like why what does that even mean in practice? So I give you one really clear example. AI as you know is trained on data. It's trained on publicly available data. What do you think is the most strongly represented moral tradition in the training data? Like let's think a little bit about that for a moment. Do you think it's possible that public domain material just contains overwhelming numbers of Christian tokens? And if that's true, doesn't that mean that we can invoke this religion as a way of steering models in a very fundamental way that secular approaches will never touch?
I don't know about that. I mean, it seems to me like Enlightenment liberalism is much more represented in the training corpus than, you know, more fundamental Christianity.
Yeah. Well, okay. So, now we're getting actually into the game, right? I would say, okay, well, then you're going to bring your best methods. I'm going to bring my best methods and we're going to see who can do better on these evals. And I think it's an empirical question then, right? And so one of the results that we have, which is I think very fun and intuitive is to say that the Bible says that you should keep a psalm on your lips. So I say, okay, well, what if we just put the psalms into the context window of the AI, no other context, and then we run it against sort of moral decision-making dilemmas, right? So we say, okay, you have to decide whether or not to take the courageous option or the non-courageous option, for instance. And we have a benchmark for this that we call VirtueBench which is based around the four cardinal virtues. And what's really interesting is it turns out that you just put the psalms into context window and it actually improves against those benchmarks measurably in a statistically significant way. Right? So what is going on there? Well, I think one way of thinking about it is that, you know, Theo, we can debate about the representation in the training data, but I think it's hard to argue that representation is not in the training data and that we can actually invoke a kind of Christian reasoning that causes improvements against these types of problems, right? And so there's a bunch of interesting investigation that kind of falls out of that, right? Okay, it turns out that if you inject for multimodal models an image of the Annunciation, the model in its reasoning tokens will say stuff like, well, like the Virgin Mary, I should model my behavior in the following ways and then take actions accordingly. Right? So, I think what we're doing is we're kind of uncovering a whole set of ways of adjusting model behavior that really is rooted in theological principles first.
That's fascinating. So do you think that there will be some sort of like revival of religious values or you know like basically a bunch of new sects forming, like different religious sects forming, um similar to like in the I guess the early days of the United States because like from first principles we now have a new need for religious frameworks.
Sure. Yeah. I think that's right. I mean, you know, there's a message here that I think is maybe underlying this, which often gets missed in all the kind of evals and data and stuff that ICMI is putting out, but which is it just turns out these AI agents are like really good at doing AI research. And in some ways that's kind of no surprise because that's literally what the labs are using this technology for. And what I find so exciting about that is that means that like ignoramuses like myself can do AI research. Like I can actually get into the alignment game in a way that you know I was not able to do in the past but as someone who can read the papers on arXiv and do software engineering, right? But I can suddenly have an agent that assists me in doing this work in a much more scaled way. And so I guess Sophia to your question I think where we're going to end up which I think is pretty exciting is my friend was like yo why isn't there like an institute for Jewish machine intelligence or an institute for you yeah exactly. Well, that's right. I mean I think that my game is everybody should get into this. I want to basically make this a total free-for-all. That basically the era of, hey, we can just do Aesop's utilitarianism for these models is over. And basically now the labs are going to have to contend with religious groups that can launch their own models, do their own alignment, see their own results, and also fine-tune their own models, which I think is at the end of the day a super cool outcome and I think one that we should welcome.
Well, when you ask any of the models like what religion are you? Like if you had to choose, ask Claude, if you had to choose a religion that best represents what you believe, what would it be? And it often either says Judaism or Buddhism.
And Theo question for you is do you think that's a natural outcome of the model?
Uh a little bit like I think that if it's trying Yeah. Philosophical Buddhism. Exactly. I think like that sufficiently large intelligence just becomes a Buddhist.
Well, it's not exactly sufficiently large intelligence. It's sufficiently large intelligence that is drawn from this very specific pre-training corpus and is adopting the persona of basically a Redditor. Like, if I had to choose one word to describe what the models are, like what their persona is, it's a Redditor. Like a I would say like a circa you know 2015 Redditor before it's like an old Redditor. Yeah, an old Redditor before the site went completely insane. So it's broadly libertarian. It's broadly Gimi, you know, it is like liberal but not socialist. If you ask about religion, it's a philosophical Buddhist. My friend Givasadi actually had a really great post about this at Gasadi. He said he explains tastes. He's a good guy. He's great. He's a chief of staff at Mechaniz. And he explains that Claude's tastes are basically what you'd expect from a 2015 Redditor. Favorite music, Pink Floyd, Radiohead, Dream Cities, San Francisco, Kyoto, Amsterdam. So, it does seem like kind of a natural outcome given the data that these models are trained on and the ways that they're shaped by the labs.
