Welcome to the AI and human well-being series presented by the Skills Program for Public Disclosure. I'm Dr. Safe Kyra, chief AI officer and the vice provost for AI at Carolina. I'm here on behalf of Provost Eggerstead, and it's my true honor to be providing the opening remarks for this exciting panel discussion and exciting topic as well.
Tonight's question is one our institution takes seriously: Is AI a threat to human and their well-being, or can we harness AI to improve human health and human well-being? This is exactly why this topic belongs at the Abbey Speaker Series hosted by our Program for Public Disclosure, which brings leading voices to campus to model productive disagreements on pressing issues four times a year. We are grateful to the Abbey family for making this special series to foster a culture of constructive dialogue possible.
AI, artificial intelligence, is quickly changing how we think, how we learn, and even how we live our lives. But just like any technology, it brings benefits and potential value to our lives. It can also bring risks and threats.
For example, we have seen cases where AI enabled a patient with advanced ALS who was starting to lose their ability to communicate, and people were not able to understand his speech. We used AI to be able to decode what he was saying and got people to understand about 95% of his communication, which was a breakthrough, and it wouldn't have been possible if it wasn't for artificial intelligence.
On the other hand, we've seen AI chatbots released in specific cases where it's used as a helpline and people are asking it questions, and in one case it provided advice for weight loss that actually resulted in a food disorder, and as a result that chatbot was brought down and no longer in service.
And that specifically speaks to UNCC's approach to AI, a human-centered approach where humans are at the center of this technology. We and you all lead this technology and not vice versa. We have to learn how to intentionally and thoughtfully use this technology to our advantage and to our benefit. And that's why I'm excited about today's panel, and I think they are going to bring really interesting ideas representing public health, law, and even economics.
So I encourage you to listen graciously and ask questions, and maybe even proactive, provocative questions. Make sure that you are learning the basics of this technology. Don't shy away from it. That's the only way to empower our students and our faculty. And the institution is focused on investing in our humans on campus, students, faculty, and staff. And with that, I'd like to introduce Nora Henigan, program director for the Program for Public Disclosure, to introduce the panelists. Thank you.
Good evening and thank you, Vice Provost Kyra. We are honored to have the Office of the Executive Vice Chancellor and Provost co-sponsoring tonight's event. And it is my pleasure to introduce our distinguished panel. Alessandra Bazzano is professor and chair of maternal and child health at the UNC Gillings School of Global Public Health. Her research focuses on applying artificial intelligence to public health.
Tyler Cowen is the Holbert C. Harris Professor of Economics at George Mason University and author of multiple books including The Great Stagnation, Stubborn Attachments, Big Business, and most recently, Talent with Daniel Gross. Peter Salib is an assistant professor of law at the University of Houston Law Center. He studies how the law can help us manage the serious risks that come with powerful AI. He also co-directs the Center for Law and AI Risk and advises the Center for AI Safety in San Francisco.
Our moderator, Deb Aikat, is a professor at the UNC Hussman School of Journalism and Media. He's researching how AI is changing the free press as a democratic institution. He also serves on the Abbey Speaker Series committee, and we're grateful for his service. Tonight's conversation, including audience questions, will be recorded for YouTube. Panelist views do not necessarily reflect those of the university or sponsoring programs. Deb, over to you.
Thank you, Dr. Henigan, and can you all hear me? Vice Provost Kyra, for that generous and warm welcome. I take this opportunity to thank students, faculty, staff, and our community members for joining us tonight. I can't believe you took time out of all the memes that are going on about Nvidia to be here. We are looking forward to an engaging conversation. We will have time for audience questions, so save time at the end. We will have it at the end.
And without much ado, I wanted to... we always run out of time at these events. So without much ado, I wanted to first thank our star-studded panelists for being here. And let's get started with the first question. As Dr. Henigan and Vice Provost Kyra said, all of you are recognized leaders in AI work. And so my first question is, what do you see as key moments in the history of AI? And I promise you I did not make it up for today. We have been planning these questions for seven months. Alessandra, why don't you try the first question?
Sure, absolutely. Yeah, I think that what's really important to recognize is that AI initially developed, you know, in the context of academia, of military, of the government. And so that's really different than how it's operating today. And you know, some of the things that are quite interesting about the history of AI are one of the first ever chatbots, ELIZA, which is actually trained to act as if it was a Rogerian psychotherapist. I think that when we look back on the early days of what people were developing, you know, they may have had a very different idea about where we would end up today.
I'm going to give a very boring and obvious answer, but I think it's ChatGPT. I think before the release of ChatGPT, at least people in the public, people who hadn't, you know, had research access to say GPT-2, mostly thought that machine learning was going really well, advancing very rapidly, but for basically narrow use cases. At that point AI was already very good at playing chess and playing Go and identifying images and distinguishing between cancerous and non-cancerous lesions in pictures of skin.
But I think ChatGPT was the moment where it became clear that deep learning and the transformer architecture could lead to a kind of general artificial intelligence, a system that was not just limited to doing one thing, producing kind of narrow outputs. You know, this can't do, this is not in exactly the domain where it was trained, but could become the kind of thing that in principle could think generally, could strategize, could pursue open-ended goals on behalf of users or maybe on its own behalf. So I think that was just quite an obvious turning point in our history, and it wasn't so long ago.
Turning point it is, Tyler.
My answer is more historical. I think of 1936 and '37 as key years. You have Alan Turing and Claude Shannon coming up with the basic logic of AI. Claude Shannon, by the way, Claude is named after Claude Shannon. Then by the mid to late 1950s you have some very crude neural net models. They're not really usable. They don't work. But people are starting to figure out this could be some kind of thing. There's then what is called the AI winter. I think 1997, Deep Blue beating Kasparov. It's not scientifically important, but symbolically people realized, hey, you could do something that could take on the smartest human and win. So that would be another landmark. And then the 'Attention Is All You Need' paper, you know, 2017, that's the other breakthrough. Those are the dates that come to my mind.
That's great. So, as you know, our panel is going to talk about human well-being. And so, I wanted to get to that question right off the bat. Is AI a threat to human well-being and happiness? And feel free to define well-being while you are getting to that answer. Anybody could take it, but Alessandra, if you want to.
Sure. Well, my background in public health really speaks to well-being not just as the absence of illness or, you know, of infirmity, but of actually thriving. And so I think that, you know, we want to keep that in, we want to really keep that in mind as we're talking about AI and human well-being. And of course, like any other powerful technology, it's going to be able to do harm and good. And so I think that that's where we have to really focus on understanding where can we find the benefits, where can we find the good, and put those at the forefront, especially around health.
Do you think we are doing well in that front?
No, I think it's very lopsided. I think it's very... I think there's a tremendous focus right now on driving, you know, the models forward without really thinking about where the needs are, where the needs are in society, where the benefits need to be. And it's incredibly powerful and could help us with a lot of our major challenges, health, illness, poverty, and so on.
Well, I think it will increase the variance of human intelligence. The people I know who have kept their initiative, they can use AI to learn a lot more, do a lot more. There's other people who use it to be lazy, do their homework for them, write this memo for me, and they don't think as much. I don't think we know yet what the proportions will be. It may differ greatly across societies, countries, and so on. In a way, it's like the internet. The internet makes the smart people smarter, but other people, it's easier for them to latch on to poorly conceived conspiracy theories. So, like any new development, like cheaper food, well, some people become more obese. So, it's a challenge. We need to rise to that challenge.
The parts of the world that will benefit the most are probably the poorer countries where services and knowledge are really at a premium. So I think it will be highly egalitarian overall, but it's really up to us to guard and extend what it means for us to be human and stay curious.
May I follow up? I'm curious why you think that it will benefit populations that are not doing as well. I sort of see it as the opposite, kind of disagree with...
Well, say you're in Kenya and it's hard for you to get access to a doctor at all. Now you can get really top-rate diagnostic, at least advice, legal advice, advice on how to run your company, what the laws of Kenya are. Some people will use it to educate themselves. If you're an upper-class, upper-middle-class kid in, you know, Palo Alto, you already have access to a lot of those things. So you'll gain a bit, but again, it's the people who are denied services that will gain the most over time. I'm not sure they're doing it right now.
I think, but quickly... yes, I think that's the question that I have is at what point in time will they gain those benefits? Because I think you have people, you know, as you mentioned, countries like Kenya, they're sort of, they're actually training models, they have, you know, not enough income, they're, you know, not accessing the models, they're doing the grunt work behind the models. So I don't know at what point...
Once everyone in Kenya has a smartphone rather than just a cell phone with mobile money. I think that will be a key breakthrough point. We're not quite there yet, but it's not utopian far off. It's certainly well within 10 years, and it could be within two or three years.
Peter, what do you think?
Well, so I'll just define well-being also again, boringly. I would think of AI as helping human well-being if it makes us richer and healthier and longer-lived, and harming human well-being if it kills us or destroys our property or undermines the economic system that allows us to prosper. And I think that both are serious possible outcomes. I think there will be a mix of both. We're already seeing AI making incredible breakthroughs in mathematics, discovering things, solving problems that the smartest human mathematicians have sort of bashed their heads against for decades.
What the material benefits of those breakthroughs will be is yet to know, but basic science does tend to have positive spillovers. We've seen Anthropic releasing some early evidence that Claude would be useful in identification of molecules for drug discovery, and that all seems great. On the other hand, we've seen indications of how things can go badly. We saw over the summer that the frontier labs do not seem to be completely in control of their models.
You know, the models are becoming ever more goal-oriented, ever more not only able but intentionally designed to sort of like change the world over time, bring about states of affairs in the future. And when you give smart things goals, things can go wrong. That's, you know, what we observe when things go wrong with humans. Humans with conflicting and misaligned goals lead to all kinds of problems from crime to warfare. And as we make AI smarter, more goal-oriented, and more able to pursue those goals over a long time horizon, it's going to matter really a lot more what those goals end up being, like what we put into the model as we do these crazy reinforcement learning training processes.
And I think if that goes badly, it could be quite bad for the same reasons that it can be bad and scary when there are humans out in the world who have either bad goals or who just have different goals from you. They want to use all the stuff in the world to achieve different stuff than what you would like. Like Tyler, I think that there are many things that we can do about this, that we'll need to rise to the challenge, and how well it goes will depend on how well we rise to it.
Yeah, that's great. I do want to tell you, I mean, let's bring it to the level of this room. A lot of younger people are represented in this room, and they have told me, because I work with them, how they use AI when they feel lonely. They have used AI for mental well-being support, and as you know, in some cases, AI has not given, maybe not in this age group, not given good advice so that people have been driven to harm themselves. So in that context, do you think there could be anything to prevent? In other words, how can we maximize the benefits of AI as we see it, especially with the younger population, while minimizing the risk? I think if, and we can't prevent a younger user from just accessing AI. I mean, they're all adults and harming themselves, and in some cases that they may be benefited because AI can be very reassuring at times in a very wrong way. So how do you all think about that?
Well, one way to think about this is to ask what the counterfactual is. And it's not clear that the counterfactual is so sunny. Like, if we think about the march of technology over the past 20 years, as Tyler said before, some people have used the internet to educate themselves, to learn a lot more, to start businesses, to make friends in other countries they never made before. But then a huge share of people using the internet seem mostly to be watching TikTok on their smartphones. And that is in many ways a substitute for companionship and human contact in the same way that I think you're suggesting some people are and will use AIs, and I think that's bad for people also.
It could be that substituting the AIs is somewhat less bad because yes, in some cases there are these tragic outcomes, but I find talking to Claude to be at least more intellectually stimulating than watching TikTok. And so if that was my alternative, maybe it's an improvement. But from there, it's a hard problem because it's really quite a tricky situation to start, I don't know, legislating people's ability to read words that an AI produces for their own good. You could imagine a world where we were quite paternalist about this, but I'm not sure we'd accept it.
Peter, I'm just curious. What do you talk to Claude about? I don't want to be nosy, but how does Claude help you with your questions?
All kinds of stuff. I mean, I'll be working on, you know, so I'm a law professor. I think about how law needs to evolve to integrate these new kinds of entities we're bringing into the world. And I sort of come from a broadly law and economics tradition where the way I think about laws is as shaping behavior by changing incentives. And so if we're thinking about these near-future AIs as having long-horizon goals and trying to achieve those goals, one thing you could imagine is shaping their behavior not just by trying to give them the goals you like, but by giving them rewards and punishments for pursuing the goals the ways you like and not the ways you don't.
But thinking through this, of course, requires you to think through how the sorts of rational actor models that we think of as applying to humans in law and economics do or don't fit when you try to apply them to LLMs that are goal-oriented and maybe instrumentally rational, but maybe have, for example, different preferences over future instances of themselves than humans do. And Claude is really smart in thinking through all this stuff. And Claude's talking about, you know, Derek Parfit's sort of theory of personal identity, which actually doesn't require you to have like a strong attachment to future versions of yourself. And that maps in some ways onto how you might think of how Claude thinks, you know, the Claude I'm talking to today thinks about the Claude that will exist tomorrow.
So in my research, I find Claude to be a very productive interlocutor in the same way that I think of my smart colleagues as being productive interlocutors. Claude is not a full substitute, to be clear. I do still talk to my colleagues and enjoy that very much. But I find those to be very illuminating conversations.
Thank you. When novels first came out, it was alleged that they drove young people to suicide. It may in fact have been somewhat true. I think any new communications medium or learning medium, it takes time to figure out how to use it. You know, teen suicide, the rate of that was going up. Some people say social media, but now it's very much going down. I would make this very general point: if you don't think that human beings can handle an additional dose of intelligence in their society, like what is it you think they can handle? Like you must have a deeply pessimistic view of humans, and if that was your view, I would suspect you'll end up concluding we're a bit doomed anyway. I'm more optimistic than that. I think if we get more intelligence, which is what we have now, there'll be a lot of transitional issues, but in fact at the end of it we'll come out much better off.
That's great. Do you agree with Peter?
I'm not sure I agree in the sense that I'm coming from a background of being a maternal and child health professional. I think a lot about mothers and families and health decision-making that they're doing, and I know that more and more people are turning to, you know, to Gen AI for that, and I think that that can be very, very dangerous. And the models aren't built to accurately answer people's health questions. I think that, you know, they're built to be agreeable. They're built to, you know, have all sorts of other goals, but we also just don't really know whether the intelligence that people, you know, have access to, as you say, is going to be what's going to help them with their health, with their well-being, with thriving and flourishing. And we've seen examples where that hasn't been the case. I would...
AI is very good at diagnosis, right?
Well, people used to use Google. It's not as if everyone was flying to the Mayo Clinic, right?
I think, you know, it's the middle of the night. A mother is trying to take care of her crying infant. You know, turning to one of these, there's value in that, but it has to be carefully considered. And right now, the business model for these companies is not to think about that particular use case. The, you know, the business model is elsewhere. And so I would like to see much more investment so that it can be a tool for well-being. I would like to see investment by the companies. I mean, when you think about the billions, the trillions of dollars that are going into this, the entire stock market is riding on it. What about, you know, the vast majority of people? How are they going to actually benefit from those investments, if at all?
Those companies have spent a lot of money bringing in top doctors, having specially assigned teams to evaluate how good Claude and GPT are at diagnosis, improve how they are, adjust for potential human errors in describing symptoms and so on. I mean, they've spent at least hundreds of millions of dollars doing that. That is part of their business model.
Well, I think it might be for clinical diagnosis, but it's not the same thing, you know, on an individual basis where people are in the middle of the night asking questions and they're, you know, perhaps not even prompting it in a very good way. You know, they're exhausted. It's very different from talking to a human being, right, or a clinician. And I think there's a really important role for augmenting health with AI. But I think we just have to be really careful about, we don't know what those turn-by-turn conversations are because that's all behind the firewall of the company, right? We have no idea. I mean, you're talking about these investments. I would love to see the research on that. It's all behind closed doors, right?
Let me surface something that maybe be a disagreement, I'm not sure, which is the question: what is the business model of Anthropic and OpenAI? I think sometimes when I talk to some of my colleagues, they have a view that the business model is something akin to the social media business model, which is roughly: we're giving you some text on your phone and we want your eyeballs to be on it because the way we'll monetize the text is by selling your eyeball time to advertisers. And you could imagine that being a way of using LLMs. Indeed, there are these sort of companion LLMs that maybe roughly have this business model.
But when I think about the business model of the frontier labs, I think of them as being in the business of doing something else, and that's automating labor. Like, you know, taking jobs that currently we need humans to do and then creating AIs that can do them instead. And to do that, I actually think the incentives are quite different from the incentives for social media platforms. So for social media platforms, like, sycophancy is in some sense a feature, not a bug, right? Like, you know, you get content you agree with and you click more of it. Whereas if you're trying to make an AI that you can then sell to an electrical engineering company and help them do design on whatever, it's like a big bug if every time you give the AI a design and it has a lot of flaws in it, the AI comes back and says, 'What a brilliant, you know, what a brilliant design, no notes, A+, excellent work,' right?
And indeed, when we saw GPT-4o becoming too sycophantic and OpenAI sort of rolled it back. And so that in some sense makes me more optimistic about the truth-seekingness and the truthfulness of AI as a technology as compared with, say, social media. It just seems like the payoffs to making the AIs tell you the truth is much higher than for social media.
There's plenty of evaluations, often independent. They just keep on getting better and better, including on health. I mean, there's plenty of evidence that they put forward the models and they're evaluated by outsiders and they're incredibly good. I mean, don't they just blow you away?
No, not in some cases. I just mean flat out.
This example, okay. If I'm a parent of a newborn and I'm putting into ChatGPT, I'm very concerned about, you know, my newborn has these signs, these symptoms, it's going to give me a long list of tests that I should go get done, potential diagnosis, etc. What if I'm on Medicaid? What if I can't get to, you know, the doctor? What if I don't have the means and the resources to do all of that? So I think that the context is going to be really...
change policy. It's not the fault of the AI if you can't afford the treatment.
I don't know because I think it's a bias that's baked in, right? And we know that the bias is baked in to, you know, favor well-resourced people, to favor, you know, a gender bias, racial bias, all of that is baked in. So as much resources as they might be pouring into it, I think we still need to see it work for the vast majority of people rather than a subset of people.
So what would be the solution to what you just said?
More input by community members, by people who are not wealthy, you know, audits, bias audits, lots of lots of approaches that are available out there.
There's a lot of bias audits. I mean, they're very politically correct, not always, but the others, they're center-left in their political views. They're scrupulously... maybe objective isn't quite the word, but trying to be fair to everyone. I think it's a miracle how proper they are, certainly compared to any other institutions we have.
I don't see how you could really ask... evaluations.
Is that the third-party bias evaluations? Is that...
Well, for not having racial bias, gender bias, whatever else you might be worried about. I think it's incredible what a good job they've done. I mean, if anything, people allege they're sort of too politically correct, and it's hard to get them to say things that might be true out of fear that it would sound biased, and you have to work at it and harangue them to say the thing, but they're very center-left.
So how can you get an AI response that is not center-left? Because some people don't like center-left.
Well, you can go to Grok, is one thing, and there'll be more models. You can go to the Chinese models, which tend to be... they're censored on issues of relevance to the Chinese government, but they're not as censored on political correctness. So there's plenty of options.
But aren't the Chinese models quite... I mean, a lot of them are distilled right off of the American model, so it's just the same.
But they will say things the American models will not because their post-training is pretty different.
So I'm now careful about Chinese models because there would be a spy at the other end.
Well, it depends who's hosting the model. Yes. You know, if you download a small open-weight model to your laptop, there's... I mean, I don't know, I'm not a cybersecurity expert, so this is not cybersecurity advice, but to a first approximation, if everything's running on your computer, then you're safe.
But truth be told, the Chinese models are very good at video. Like, you know, in this country, you are not using AI for video. You know, Sora got, you know, discontinued. But if you are looking for a strong model that produces videos and you link it up with audio and all of the stuff, Chinese models are very powerful and at very low cost, like $25 a month, and they provide you tokens to do that.
So related to that, I wanted to talk a little bit about AI adding the most value and where does it risk undermining human well-being. And I'll share with you an example. I often go to the doctor, and not often do I get a radiology test, right? And it has... I can't even understand what's going on in that report. I'm sure you all would relate to that. So a lot of my colleagues suggested that why don't you feed it to your AI system and they will tell you what the risks are or there are only... is there anything wrong with that?
You're asking me? [laughter]
I mean, I can't give you advice personally.
No, but what I'm trying to say is, do you think in that level, considering your concerns, does AI add value and where does it risk undermining the human well-being? So that's why I put a personal example: that what do I risk asking? Yeah, because, you know, sometimes the doctor doesn't have time to talk with you. Certainly the radiologist doesn't have time to give you details in their report that you can understand. But if I ask AI as a good friend, and of course I can ask AI not to be very psychophantic. I mean, now you can do that, right? You can say, AI, don't be psychophantic, in a second, and AI will follow your advice. It's like a genie out of the bottle. So do you think there is any risk in that?
Yeah, I mean, I think the risks are on a couple of different levels. One is that, you know, it may jump to a more expensive or extreme suggestion for you. We just don't entirely know because they are trained really well on medical, you know, things that may not suit your particular context. So context is a big issue. But the other thing is that then, you know, you may risk not reaching out to a care provider and speaking with them and, you know, getting more holistic care, and you may, you know, cede some of your own thinking, some of your own critical judgment to essentially this statistical prediction engine, which is a problem that I think more and more people are talking about.
Yeah. One interesting question is when it's good versus not good to cede your thinking to a statistical prediction. I mean, like, why do I go to the doctor? It's because I want to cede my thinking to the doctor, right? I expect a doctor to have better thinking than me, and at least in the cases where I expect AI to give me very high-quality medical advice, which in my experience is all the cases I've tried to get advice from it, from... well, I don't know, like, maybe it's fine. Like, maybe this is like just what we would do in the human case.
And then I guess we could worry that if AI is telling you things that are false or bad for your condition, that would be to be avoided. But the question is whether that's happening. I mean, I think one obvious case where we don't want to cede our thinking to AI is when we're trying to learn or build human capital, build cognitive ability.
We're all teachers in universities. I think we probably noticed that our students are very happy to take any questions we send home with them for return and just kind of pop them into GPT and give us those answers rather than learning themselves. And, you know, probably there is stuff we should just not ask students to do anymore because it's not going to matter in the future. But surely there are some things that we want students to struggle through and learn without asking GPT to tell them the answers, because this will update the weights in their brains in ways that will be valuable for them in the future.
Many of my colleagues are very worried about this. I'm a little bit more sanguine. I think we have technologies that allow us to do this reasonably well. Like, I don't know if anyone here knows this, but in law school we have this incredible technology called the Socratic method, where I assign a bunch of reading and then my students, they can go do whatever they want. They can do the reading. They can talk to GPT. I don't care that much, and obviously I can't police it. But then every Tuesday, Wednesday, and Thursday in civil procedure, they have to show up in my class, and I have a deck of cards, each of which has a name on it. I shuffle the card, and then I start drawing and I start asking them questions about the cases, and they've got to be able to talk to me about it.
And I think that's actually a pretty good incentive system for them to do whatever it takes to update the neurons in the brain. And ideally they'll figure out like the optimal mix of actually reading the words in the casebook versus asking questions. So, you know, I sort of think for this kind of stuff we'll figure it out.
We'll adjust. Paul Novosad has this new idea where you have students debate each other over video so it's not faked, and then the AI grades it at scale. Whether or not you think that's exactly the right idea, the point is there are many, many things we can do. We now have a whole host of new options. You can grade people's AI logs, have AI grade the AI logs, and so on. And I think education can be much, much better once we actually say we're willing to adjust.
That's a very good point. So my next question relates to: should AI be more heavily regulated? If so, how? And I would split the question in two parts. One, we have an audience of a lot of university students, and as you know, you are well aware about what happened at Dartmouth where the students were not allowed to use AI, but there was a provost who the campus newspaper found out he was using AI for his article. I don't want to get into the details or whether it was fair or not, and that whole investigation led to the provost quitting. So do you feel that AI should be more heavily regulated, A, on the campus, considering what Peter you said, and at the second level, let's scale back and say should AI be heavily regulated, period?
Let's start with the campus situation first. We need more blue book exams, if you want to call that regulation, but I don't think we should stop students from using AI. A big part of the curriculum should be teaching them how to use AI, and a big part of the curriculum should be teaching them how to think and write when they're not using AI, and just do more of both. Whether or not you want to call that regulation, it's a moot point.
So, I teach a class of 300 students. It's not realistic for me to convert to suddenly a blue book system because, truth be told, we are under budget cuts. We do not have so many teaching assistants who would grade the blue book, and I cannot read half of the time the handwriting of my own students.
But AI can do that. Again, you need some adjustment period. But scan the blue books, right? AI grade them. It'll do a better job than you already.
Peter, what do you think? There are some law schools, in the spirit of full disclosure, that limit AI use in their first years, like Berkeley being one of them.
I heard. Yeah, I think Berkeley has even beyond the first year some pretty strong restrictions on AI usage. I mean, don't quote me on that, but I think I roughly agree with Tyler's view. I mean, what is this university for? I mean, one thing is for producing research, and that's great. But we're not talking about producing research right now. We're talking about education. It's like, why are all of you in college to learn physics or philosophy or English literature or whatever? Like, that's all in books. You know, that you could just go to the library and learn all that stuff, but you're not going to, right? You just have weakness of will, or you'd rather watch TikTok.
And so the universities already are a kind of commitment device. You sign up, you apply, you pay a bunch of money, and then we like force you to come and do a lot of things. And so you're opting into this regulatory regime already that forces you to use your brain and think instead of doing whatever you would do otherwise. And so I think, yes, obviously we should be thinking as teachers about how we should treat AI with that goal in mind, but it's not so different from what the whole teaching institution is about in the first place. So yeah, do more Socratic, do more in-person exams with either literally on a physical paper or on the electronic blue book with, you know, full lockdown of the computer. You know, but these are not fundamental transformations of what we do, and I don't think they're well understood as like, you know, AI bans.
Would you agree with that?
I agree that we need to be using AI with students, and students need to learn how to use AI, need to understand some of the background of it. And if they choose not to use AI, at least we will have given them some preparation and exposure to getting involved in it. And I think that there are lots of ways... I mean, I teach a course where we spend a lot of time using AI, understanding how it works, and things like that, and I think that's really vitally important because when students also go out into the job market, there are going to be expectations.
So it is a little bit challenging if in their university time they're told don't use AI, and then they're out in the job market, well, why don't you know how to use AI? So I think it's really important to train that up. I think, you know, in terms of academic freedom and in terms of instructors being able to make policies for their own class, that, you know, there needs to be able to be a little bit of variation, and, you know, faculty are still having different minds about their own AI use, and they're at different places, and so you kind of can't really force everyone to use it, nor do I think you should ban everyone from using it, but I think it's important. Yeah.
Great, that's wonderful. So as you know, with power comes responsibility, and I wanted to ask the question about AI being regulated. And this was a great week to see this in action because we had Nvidia, which is a wonderful company, but it didn't have a leadership structure. It didn't have anything, and as a result, it's in a limbo now. And, you know, a lot of people are very concerned about the power of AI and the way they are growing. So do you feel that they should be regulated? But as I say this, I'm also reminded that AI is very global, right? So when you are regulating AI, which legal domain regulates it?
Should AI be regulated? Well, the question is, regulated to prevent what, right? And for me, the answer varies a lot depending on what you have in mind. So it seems to me that there's like a lot of things that could happen from AI that would be bad, but which are not so obviously outside of the realm of our current laws that we need to sort of like radically reinvent the wheel.
Like, some people are very worried about the speed at which the frontier AI labs are growing and gathering market share, and that's a question of monopoly and a question of antitrust, and hey, let me tell you about the Sherman Act and the however many years of law interpreting it. To me, insofar as AI monopoly is a problem, and I'm not convinced it is yet, but if it becomes one, we basically have the legal apparatus for dealing with this. We have a well-developed set of laws.
The place where I'm most convinced that we do not have a legal regime that is adequately accounting for like the risks of AI are what you might think of as like the tail risks or the potential catastrophic risks from AI. So for example, we already know that the models that exist today are extremely capable at doing cyber offense. You know, when Anthropic finished training its Mythos model this spring, it didn't release it to the public. It gave it to just a few trusted partners, they did a white-hat sort of hacker sweep and found like thousands upon thousands of zero-day vulnerabilities in critical systems.
You had people at the NSA saying if a foreign adversary had this in their hands, they would have broken all of our systems. And that's the worst they're ever going to be at cyber offense. I think it's not crazy to think that in the near future there will exist systems that, absent adequate defense, could do things like, you know, disable the eastern seaboard power grid for on the order of weeks or months at a time, which would be a really, really big disaster.
The models are getting more capable at doing things like helping do synthetic biology, which can be really useful for, you know, inventing new vaccines. Very good. But can also be useful for helping terrorist groups and rogue states invent novel bioweapons. And but then beyond that, if you layer on this idea that I mentioned before, which is that we are intentionally creating AI to be more goal-oriented and pursue goals over...
longer time horizons, and we also are not so good at making sure they only have the goals we want. Like all of those things together, our ability to control the systems, even insofar as humans are controlling them, our ability to make sure that no bad humans get control of systems that can do bad things. There's a kind of fat tail of bad outcomes you can imagine, and the problem with those bad outcomes is like it's an uninsurable risk effectively. Like if you create even a COVID, which was a mild pandemic by historical standards, that's just like well beyond the market capitalization of the entire AI industry. It's not clear to me that it's insurable even with like the entire global insurance industry. You're just not going to liability your way into making these AI or these companies take optimal precautions against those risks. And so this is where I see more scope for ex ante regulation, for having more oversight, maybe by government bureaucracies, maybe by sort of more market-oriented auditors like we do with the public markets, but by outside entities that are going to be setting standards before these models are trained, before they're released, to try and help reduce those risks. This, by the way, is not so crazy. It's what we do in other high-risk industries. Like, I don't know, the airline industry is very heavily ex ante regulated. Maybe it's overregulated, but like probably the optimal level of regulation would not be zero, which is roughly what we have in frontier AI now. So that's my view. Yes, AI should be regulated, but in particular with an eye towards reducing these tail catastrophic risks.
I would regulate biological materials a lot tougher with or without AI, but certainly with AI. Over the next year or two, I think by far the biggest risk is cyber security, and there we actually need to deregulate cyber defense. So we had this thing, Operation Glasswing, that was supposed to distribute cyber defense to leading American companies. Now what, seven, eight months, we've done basically nothing with it. We need to get that going. AI indirectly by the executive branch, the national security people. It's very regulated right now. You can debate whether it's regulated well or not, but it's all done by discretion and decisions of people behind closed doors. Over time, I think we need to move to something more FINRA-like, a system where the labs regulate each other. But it's very hard to regulate a new thing that's changing so quickly. And keep in mind, in terms of talent, knowledge, and salary, our government just really cannot regulate it now. It's not like the SEC, where there's plenty of lawyers who are good enough to work for the SEC and do the jobs they're supposed to do, or the FTC. If you tried to regulate AI models from government now, it would be a complete failure. I recall seeing an ad from the United Kingdom like to be chief AI regulator living in London at a salary of 60,000, and that's pre-tax. Now, that's them, not us, but it's the same basic problem that's popping up. The labs have some ability, you know, with embeddedness to regulate each other, and maybe Meta can help regulate Anthropic, but those two are so closely linked, and it's very complicated. It's not just a standard problem of regulation given the speed of change and how much people are paid and the scarcity of talent who understands this stuff at all. So I'm inclined for now to focus on cyber security, do a lot on biological materials from the side of the materials and the biology, and keep a close watch on things. Alessandra, what do you think?
Well, I think just because it's challenging doesn't mean we shouldn't do it. And I think that I'm interested in all the different potential ways, you know, that things could happen. There seems to be a tremendous amount of money just purely focused on pushing capability. What about pushing safety and alignment? Why can't companies be required to spend more money on safety and alignment? I mean, that, you know, would be a start. But I do have a question for either of you. My understanding is that there is a sort of ability for the federal government to prevent a model from being released, and this has happened, but it's kind of like a behind-the-scenes secret policy that had to be FOIA'd into the public. Is that...
I don't know the details of this, more or less how it is. I think for now it's the best we can do. Like we don't have an agency set up that has the knowledge and capabilities. A model that's shown to people in Washington, the national security establishment talks about it, and there's a kind of informal decision. I think that's optimal for the time being. Obviously, it cannot go on that way forever.
The formal power only sort of exists. So the instance you have in mind is probably when the Trump administration ordered Anthropic to halt deployment of Fable. I think it just was Fable. I mean, Mythos is not public, but maybe they halted the deployment even though...
They had to delay Fable and Mythos had to be kept off.