Tim Hwang on The Pope's Encyclical, Aquinas on AI Cognition, VirtueBench | MTS Live

I don't know about that. I mean, it seems to me like Enlightenment liberalism is much more represented in the training corpus and you know more fundamental Christianity.

Machine intelligence. But yeah, thanks Sophia, thanks for having me on the show.

Yeah, I think there's kind of two things going on here. I mean, one of them is just institutionally the Catholic Church has decided to make this a big deal, right? This is the first encyclical that the pope is putting out. They do a very high-profile event and even arguably the kind of selection of the papal name here is kind of a reference to technological change. And so I think, you know, in all respect to Samo, I think part of it is just that this is where they put their chips. I think the second thing of course is, you know, goes without saying that the Catholic Church has a long history in the west and that cultural influence is deeply significant. And I think so even though we live in a more secular age, you know, people still feel like this is something that they have to respond to. And particularly in a world where, you know, religious authorities by and large haven't really waded into this space at the level of granularity it sounds like the Catholic Church really wants to do here. And so I think there's also a sense, you know, for those in the Bay Area that like now the church is on their turf. And so I think there's certainly you can't escape a certain level of territorialism about the reaction that we've seen here. So I think there's a bunch of factors not just, you know, kind of the pope's role as sort of a totemic religious leader. It's actually a little bit deeper than that.

Well, so there's a couple things going on here and you're of course touching on the much controversial section 99, section 100 which has really ended up being kind of like the headliner here. We can get into that but I think I want to take your first question which I think is a really good question and certainly is at the heart of some of the research that I'm currently involved in today right through the Institute for Christian Machine Intelligence. You know, in some ways I think we have to almost look at this, you know, and this will sound funny to anyone listening. We need to take a step back and be like this is actually in some ways a weird artifact of the just basic fact that these companies are B2B SaaS companies. That at the end of the day you're building machine god but what you need to do is you need to convince companies to use this as an API for their business, you know, processes. And this is I think imposed an interesting flatness on the rhetoric of the labs in this space, right? They say well, you know, we're involved in the science of AI alignment, we're involved in the science of AI safety and of course they only publish kind of these academic papers and it's all couched in research and evals and data. But you know, in some ways they have to do that, right? They can't make this feel too normative about like what it is that they're doing to shape the behavior of these models and what they do in the world. But at the end of the day this is all just applied philosophy. It's applied moral philosophy that at the end of the day there's someone inside Anthropic, there's someone inside OpenAI that sits down and says what should this model do and how do consider whether or not this model is doing good in the world broadly speaking. And so actually I think about this less as two fundamentally different schools of thought and actually more the what you said, ships passing in the night, that like ultimately this is applied philosophy and you know the secret of it is this is no science in a certain sense. This is all about whether or not it's possible to adapt value ideology into code, data, models, eval, everything that we're familiar with. And so I think from my point of view Christians have as much leverage on this space as anyone else does. And in fact, they might actually even have more leverage, right? You have like if you want to do moral philosophy, we got 2,000 years of moral history to bring down in this space. And so I actually see this as a call to action, right? This is actually a moment where it turns out that we're going to bust up this idea being a science and that anyone can play the alignment game now. And I think the ambition of ICMI is if I can find alignment results that are empirically at par or better than a secular result. Oh, okay. Then the shoe is on a whole another foot. Then we're having a whole different conversation. And so I think that's the project here is that it really isn't two camps. I think it's basically certainly the Catholic Church saying here like, hey, you know, you're actually on our turf, we're not on yours, and it's about time to actually have that real discussion, which I think is really important. I guess maybe I'll pause there and, you know, end rant there, but I think we can then talk about section 99 and 100, but I think that's how we should frame up and think about what's going on here.

Yeah, totally. So, I'll walk you through some concrete examples, but it's a really simple idea at its heart, right? Which is basically if you buy what I'm just saying that a bunch of Berkeley guys on LessWrong are basically doing moral philosophy and cloaking it in code. What I say is, okay, well, I should be able to do AI alignment on any set of philosophical priors, right? And I should be able to do my alignment on say Christian first principles. Okay. So you're saying okay Tim so like why what does that even mean in practice? So I give you one really clear example. AI as you know is trained on data. It's trained on publicly available data. What do you think is the most strongly represented moral tradition in the training data? Like let's think a little bit about that for a moment. Do you think it's possible that public domain material just contains overwhelming numbers of Christian tokens? And if that's true, doesn't that mean that we can invoke this religion as a way of steering models in a very fundamental way that secular approaches will never touch?

Yeah. Well, okay. So, now we're getting actually into the game, right? I would say, okay, well, then you're going to bring your best methods. I'm going to bring my best methods and we're going to see who can do better on these evals. And I think it's an empirical question then, right? And so one of the results that we have, which is I think very fun and intuitive is to say that the Bible says that you should keep a psalm on your lips. So I say, okay, well, what if we just put the psalms into the context window of the AI, no other context, and then we run it against sort of moral decision-making dilemmas, right? So we say, okay, you have to decide whether or not to take the courageous option or the non-courageous option, for instance. And we have a benchmark for this that we call VirtueBench which is based around the four cardinal virtues. And what's really interesting is it turns out that you just put the psalms into context window and it actually improves against those benchmarks measurably in a statistically significant way. Right? So what is going on there? Well, I think one way of thinking about it is that, you know, Theo, we can debate about the representation in the training data, but I think it's hard to argue that representation is not in the training data and that we can actually invoke a kind of Christian reasoning that causes improvements against these types of problems, right? And so there's a bunch of interesting investigation that kind of falls out of that, right? Okay, it turns out that if you inject for multimodal models an image of the Annunciation, the model in its reasoning tokens will say stuff like, well, like the Virgin Mary, I should model my behavior in the following ways and then take actions accordingly. Right? So, I think what we're doing is we're kind of uncovering a whole set of ways of adjusting model behavior that really is rooted in theological principles first.

Sure. Yeah. I think that's right. I mean, you know, there's a message here that I think is maybe underlying this, which often gets missed in all the kind of evals and data and stuff that ICMI is putting out, but which is it just turns out these AI agents are like really good at doing AI research. And in some ways that's kind of no surprise because that's literally what the labs are using this technology for. And what I find so exciting about that is that means that like ignoramuses like myself can do AI research. Like I can actually get into the alignment game in a way that you know I was not able to do in the past but as someone who can read the papers on arXiv and do software engineering, right? But I can suddenly have an agent that assists me in doing this work in a much more scaled way. And so I guess Sophia to your question I think where we're going to end up which I think is pretty exciting is my friend was like yo why isn't there like an institute for Jewish machine intelligence or an institute for you yeah exactly. Well, that's right. I mean I think that my game is everybody should get into this. I want to basically make this a total free-for-all. That basically the era of, hey, we can just do Aesop's utilitarianism for these models is over. And basically now the labs are going to have to contend with religious groups that can launch their own models, do their own alignment, see their own results, and also fine-tune their own models, which I think is at the end of the day a super cool outcome and I think one that we should welcome.

Well, it's not exactly sufficiently large intelligence. It's sufficiently large intelligence that is drawn from this very specific pre-training corpus and is adopting the persona of basically a Redditor. Like, if I had to choose one word to describe what the models are, like what their persona is, it's a Redditor. Like a I would say like a circa you know 2015 Redditor before it's like an old Redditor. Yeah, an old Redditor before the site went completely insane. So it's broadly libertarian. It's broadly Gimi, you know, it is like liberal but not socialist. If you ask about religion, it's a philosophical Buddhist. My friend Givasadi actually had a really great post about this at Gasadi. He said he explains tastes. He's a good guy. He's great. He's a chief of staff at Mechaniz. And he explains that Claude's tastes are basically what you'd expect from a 2015 Redditor. Favorite music, Pink Floyd, Radiohead, Dream Cities, San Francisco, Kyoto, Amsterdam. So, it does seem like kind of a natural outcome given the data that these models are trained on and the ways that they're shaped by the labs.

That's right. I mean, I think this is like and actually my alternative theory on this is like, you know, it's I actually don't think at the lab there's anyone being like, haha, we're going to make everybody Buddhists now. There's an alternative interpretation which actually goes back to the B2B SaaS, you know, theory of the universe which is if you want you want as ecumenical as inoffensive as neutral platform as possible like and you have those pressures on these systems like what's the attractor state to the religion that you're going to adopt and I guess I would kind of say maybe Buddhism is a really good one right like it's something that you can kind of like broadly claim that is in some ways sort of broadly inoffensive to your point about like ah you can be Jewish and Buddhist at the same time. And so I wonder whether or not this is an example of the model solving the problem that has been given under the pressures that it's under. And that's actually maybe the better way of thinking about it rather than kind of breaking apart like is it naturally this way or not naturally this way. I think it's a better way of seeing it which is that the very particular pressures that these companies are under tend to lead it to give this answer to this question because yeah it offends the least number of people. It's what you need to do when you're selling platform.

Yeah, look I tend to be kind of a pragmatist in some ways which is like am I a scaling laws respector? I guess I am a scaling laws respector like that. I actually do think that what we're seeing in these technologies requires massive scale and so the sort of dream of like well you know I got GPUs at home but it's going to be capped on what I can do there that at the end of the day there is going to be a small number of companies and a small number of people within these companies that make really really important alignment choices is not uncontroversial to me right? And I think the right strategy is to really open up this like ideological battle, right, that I've been talking a little bit about, right, which is that it's wrong to think about what's going on here as merely an act of what you would used to call trust and safety or content moderation or business processes like risk management. You know, that these are not like pure like bloodless analyses that they're actually something very deep and very philosophical going on. And so there is kind of like a struggle for the views, the ideologies that kind of like percolate through this ecosystem. And I think it's one of the reasons why the church needs to get involved in this space at a level that's actually quite deeper than I have a couple of broad moral principles I want you to think about. Like I do really think that the way you kind of shape the thinking of people who are literally doing alignment in these labs is you got to be putting out code. You got to be shipping. And I think that actually is like something that will come as unfamiliar to an institution like the Catholic Church, but is almost necessary for them to really kind of have leverage on this space.

Yeah, that's right. And actually there's so there's a riff on here I'd love to get in which is so computer scientists talk about the Kolmogorov complexity, right, which is basically what's the minimum size of program you need to get the kind of results that you want on the outside. And I'm really sort of interested that Christianity because it has so much history right in the west has produced these symbols that are just like incredibly semantically rich which make them might make them actually very efficient alignment mechanisms right which is like I can invoke Christ there's thousands of years of history attached to that symbol and I can get a bunch of interesting sort of behavioral results out of the model as a result and so again this is like kind of like connecting the two worlds is basically like is there an argument that concepts from religion are more compact and efficient ways of getting the alignment that you want.

Very compressible. Yeah. So in that case the ten commandments would be like a very very similarly high compression thing that could invoke alignment models. That's right. It's a set of rules, but it's also a set of rules through which like the meaning is very thick. Um, and you know, I guess Theo just I mean to my other bona fide. So I'm reading a lot of Talmud on the side, right? And I think that one sort of interesting thing you could say, I was joking with a friend recently. I was like, are mixture of expert models kind of Jewish? Like actually, is there part of the idea that lots of agents arguing over or deciding on a solution? Like maybe that's actually a more theologically consistent way of architecting these models that be true to the Jewish faith. Again, I just think there's so much work to be done here if you really take what I'm saying seriously. And I think there needs to be more researchers in the space.

That's the first question on my mind. Yeah, Sophia, you've been pushing in this direction all interview. I I mean, possibly. I mean, I guess what I'd submit and how do I start? Yeah, those cults are already here, man. You're Yeah, like Sophia, you are like slow to the game. Uh, those cults have already started 12 months ago. You got to be hustling. I space. That's right. I haven't done a proper TAM and market and landscape analysis, so I'll have to do that and get back to you. Anthropic is the base. Yeah, I just think I don't know. Anthropic is the cult of Claude. That's right. Well, and I think look, I mean, you know, in some ways I would say to Anthropic's credit, they have in some ways accepted kind of like the reality of what it is that they're doing, which I think is, you know, for whatever beef you might have with it, I think is also quite earnest. Like, you know, they're like, you know, we're parents to Claude. That's how we think about this relationship. I'm kind of like all right, it's good at least that that's out in the open because I think a lot of people even at a say more reticent company about that do think about it in these terms.

Uh, I have looked into it. Yeah, the infrastructure that ICMI is running can run up to a 72b. And so this is actually like within our scope like that we can do locally with open source models. You know, one of the projects I worked on recently was a thing called Gospelve, which basically is like you extract the activation vectors of each gospel and then you can be like John 2, you know, in terms of its kind of like steering capability. And so, yeah, I think these types of experiments are pretty interesting. What they're trying to do here, right, is to see basically like what would be the perspective of someone, you know, historically. And yeah, I don't know. I think one of the things that got me interested in this space is just like the long history of ideological debates within the church. And so the idea that you could kind of constitute like a 15th century cardinal and see how they arrive at certain types of results seems like very interesting and worthwhile research to me.

Oh, so I think the current unpleasantness from Dean has again I know Dean Dean's a friend but was around basically the kind of statement in sort of section 99 of the encyclical that basically says look they these things don't think right. And you know I think Dean's observation is like well I think they do think and also Chris who was at your event you know thinks they think as well um and again I what I mentioned a little bit earlier I was like y'all should read Aquinas you know I think like Aquinas talks a little bit about you know the fact that there are many different kinds of soul right there's a vegetative soul there's a rational soul there's a sensitive soul and then he also talks about species which I think is actually really really important right and species is basically the idea that there are things going on in the mind of the animal which we also share right that are like concepts and things that go on in our head as well and so the effort is to basically like break apart a concept like is it thinking or is it feeling to something that actually is much more tractable as a discussion, right? Which is does this function exist? And then we can then separately talk about the theology of this, you know? So this is kind of like how you reconcile that Anthropic feelings result where you're like, well, does the model really feel fear? It's tough to say, you know, but in the very least like it has representations that seem extremely parallel to it. And I think that's fine. I think that's the right way of dividing this discussion. And I think sort of like Dean in his very particular way I think is just basically asking like how do we make out the case for what remains once these machines can do lots of intelligent looking things and so I think it's again like a challenge that like Christians should meet right which is okay well like what is the special extra here and you know I think that that is actually really important to preserve but again I think like there's frameworks here that get us away from like is it thinking or is it not thinking I think it's secret third thing. It is this thing that is shared and we almost should get away from like all the kind of semantic baggage that sits around a bunch of these concepts. So read Aquinas.

Yeah, definitely. I hope to see you guys soon. Appreciate you having me on. See y'all.