Thanks everybody for being here. Thanks to the Murphy Institute for creating this convening, and it's just great to have everybody together. So, the title of this talk, no one's taking responsibility for that, unfortunately. So, 'The Good, the Bad, and the Ugly AI' — like, it's not even pronounceable. I thought Nick came up with it, but Nick says that I came up with it, and Nick is almost certainly right because Nick... but it came out of a discussion that the two of us had to try to put a framing around what should... how should we kick this thing off? How should we convey like what's going on in AI? And as Hoo pointed out, there's these sort of interesting conflicting aspects to it, right? That's like really, really cool and really, really scary at the same time. So I thought it would be great to just kind of run through the things that are on my radar and use that as a jumping-off point to talk a little bit about the underlying technology, a little bit about the societal implications, and a little bit just kind of entry point into our discussion after that. So, yeah, next slide, please. Oh, no. I say that, but I'm saying it to myself. Next slide, please. There it is. All right. So, all right. So, what's the... So, people know, right? This was a... there was a movie with the title 'The Good, the Bad, and the Ugly,' and I took some of the images from that. The good in this case is referring to the benefits of AI. And for those who don't know, those who have not been part of this field for as long as folks like Nick and I, the term AI is really old, right? So it dates back to the 1950s. So this was a topic that people had been studying for a really long time, you know, going... we're going on 75 years. And over the years, there have been lots of benefits, lots of software systems that have been built under the banner of AI that have had positive impacts on society in a lot of different ways. But the recent stuff is particularly interesting. There's a lot of amazing progress and amazing impacts that have been happening. So I just want to, I don't know, remind people or introduce to people some of those examples. So one of the ones that I think is really, really neat is DeepMind's AlphaFold system. So this is a system that allows users or people, scientists, to predict the 3D structure of proteins. This has been a long-standing open problem. People, scientists knew that proteins, like the genetics in your cells and so forth, they actually specify proteins. Proteins are these actual biological things that do work in your body. But we didn't really quite understand how to go from the genetic sequence to the actual physical form of it. And so people worked on it for a really, really long time. There was a competition to try to figure out who can make the best predictor, and this group out of DeepMind put together their solution to it, and it was remarkable. It actually... people talk about it as solving this 50-year-old scientific problem. It is now being used by researchers to solve all sorts of scientific problems in health and in biology, and it's super cool. And it is built on top of this idea of deep learning, of machine learning systems that can actually take in tremendous amounts of data and then produce sometimes actionable information from that. So AlphaFold, super cool. I probably should have put up the... one of the Nobel Prizes last year actually was granted to the people who put this together, right? So this... people across... this is not just an AI thing. The people across disciplines have recognized this as a really important contribution. But what people I think are most familiar with when they think of the... when they say AI, often what they mean is ChatGPT, right? So they mean this sort of notion of a chatbot that you can have these interactions with. And it is remarkable, right? It was for many, many, many, many years people have been trying to build computer systems that can actually process language, that can do things with words, and the advances have been undeniable. Computers can now support language tasks. You can actually interact with these systems as an interface in a way that we just couldn't before. The systems were so freakishly brittle, right? People would work on a system for two decades, and then the first person to sit down and interact with it would break it because it just wasn't good. And now that's not the case anymore. You can actually have a very long interaction with these systems before you succumb to AI psychosis. Sorry, I mean, before you realize that you... you reflect on the fact that you're actually interacting with a machine. And so this graph on the slide here is just tracking a number of different challenge problems that the field has been using for many, many years. Like the graph goes back to 1998, but actually a lot of these problems pre-exist those. And with a line on the graph which represents human-level performance on these same tasks, and the tasks are things like, hey, here's a sentence, where's the verb, you know, things like that on one side, but then also much harder problems like, here's a description of a problem that I want you to solve, write me a computer program that can solve that problem. So there's a range of tasks, and in all these tasks now, it is the case that systems... they started off kind of slow, and then they actually ultimately crested above human-level performance. And now the more recent benchmarks are created on the far right side of the graph, like they're around for like a year or two before they're actually achieving human-level performance. So it is really quite remarkable the progress that's been made, particularly in this kind of natural language space. At my home institution, Brown University, we've got mathematicians who are actually employing the same technology to solve longstanding mathematical problems. So the understanding the behavior of the Navier-Stokes equation. I'm not a physical scientist, so I actually haven't spent a lot of time thinking about Navier-Stokes, but it is basically equations that describe how fluids behave. And it is an open mathematical problem as to whether or not those equations can actually blow up. And I don't mean like literally you write down the equation and it explodes in your face. But like if something followed the rules according to the equations, would they actually behave like an infinite amount of energy created in a finite amount of time? It would be nice to know if that... if either the equations are broken or actually like water can explode. Like these are things that people wanted to resolve, and they're now very much on the edge of being able to solve this problem. They've made tremendous progress over the last few years solving subproblems, building up to like we know what happens in 2D now, but like fluids are in 3D, so we need to understand 3D. But that progress was just not possible without the support of these computer systems behind them. So that's all good. What about the bad? There's a lot of the bad. So part of the bad comes about because this isn't just a technology. This is a technology that's being deployed on a massive scale over the entire planet all at once. And so the fact that this all happened so fast. Sometimes I like to describe ChatGPT as a lab leak because it's a really cool technology that needed a lot more study, and then suddenly it was like everywhere, right? They kind of lost control of it, and now it's endemic, right? Now it's just a thing we have to live with. But there was an opportunity to actually think this through in advance. They didn't take that. They just released it on us. So there's various bads. So one example of the bad is electricity usage. So because of these large-scale data centers that are being built and fed with electricity, the costs of electricity are actually rising. And if you look at the average electricity cost in the US, it was flat for a number of years. It kind of crept up again in 2005, leveled off in 2010, and stayed more or less flat until around 2020, 2021, where it's been now rising really, really rapidly since then. And the reasons for this are complicated, right? I don't want to like say like AI, this is all your fault. But you know, buildings have become more electrified. Electric vehicles are now much more of a thing, and that's actually drawing more and actually driving demand and therefore making costs rise. Industrial reshoring, just a lot more stuff happening within the US. But if you add up all those, what you're left with is like 53%, and that's probably mostly AI. So about half of this cost increase we can... we might be able to actually lay at the foot of AI and say, 'Huh, we didn't ask for that. It would be nice if we could continue to pay for electricity.' So that's one of the bad. Another bad that I think about a lot, and in Hoo's introduction he kind of nodded to this a little bit, that there's all sorts of ethical and legal issues. One of the ones that actually I don't know, sort of gets me the most or disturbs me the most is the notion of intellectual property. These giant AI models, these systems that are being built, are trained on and can produce material that violates copyright. And so the discussions around this are really weird and complicated. I've heard a... I had a talk to a lawyer who defends one of these big AI companies who's like basically admitted, 'Yeah, yeah, no, they should have never done that. But now that they're doing it, we have to let them keep doing it. Like you just have to.' And it's just such a weird argument to me. Like we should probably have a real discussion as a society about what's okay and what's not okay. And in some sense the horse has gotten out of the barn. Like we... it's difficult to now roll back the expectation. So the two examples that I surfaced on the slide here, one is on the right-hand side, that's Shrek, if you have not seen the movie. And the other movie, and also there was another movie, I think there's like seven movies altogether. That's the actual Shrek as copyrighted by Universal. That's an output from a system called Midjourney, which can produce images just from verbal prompting. And it's like, yeah, they clearly trained on Shrek, which they probably shouldn't have, and they can clearly produce Shrek, which they probably shouldn't. Similar things are happening on the text side. So this is a quote from The New York Times, and edited in something that ChatGPT produced when it was asked to produce something on this topic. And it's basically this, and I just took a little segment of it. The rest of it is almost verbatim from The New York Times. And like, is that okay? That doesn't seem like the way I was taught when I was coming up as a young AI person. That's not okay, right? You can do it in the research setting, but you really can't do it at the commercial level. And this is absolutely happening. So this is of the bad. All right, but what about the ugly AI? Well, so the ugly AI is the fact that a lot of these things actually cut both ways. They're actually kind of good and bad at the same time. And what makes that ugly is that now all of us are in a position where we have to navigate this extremely complicated boundary between what's okay and what's not okay. Like we're all responsible for that now, and that kind of sucks. So I think of it as being ugly. And I didn't... I wanted it from the... I took the word 'the ugly' from the poster, and I tried to change it so that it said 'the ugly AI,' and it was bad. It was ugly. And so I was like, 'Oh, should I do it? I think I'm going to do it.' So I put it into Google Gemini, and I'm like, 'Can you make this says the ugly? Can you just make it say the ugly AI?' And it was like, 'Sure.' Boom. I'm like, 'Oh, that was actually really good.' So that's good, but like also feel a little dirty because you know, it's clearly their font, their rendering, like their words, but like it's on my slide now. All right, so some of these attributes have both good and bad sides. And so let me just run through a couple of them with you. So these are some good things. I think good things. We can debate that, but these are... I think these are good things. One is that it's really helpful for people to have access to information and information retrieval. On the topic of information retrieval, these chatbots can actually do really sophisticated question answering. So they can help you get the information that you need much better in many ways than having to retrieve a document and then read the document and find the piece of it that actually answers your question. And this is a long-standing problem in computer science, like predates all this generative AI stuff. There was a text retrieval conference called TREC that had a track on question answering. Like there was originally the conference started as let's just find documents that are relevant, like a newspaper article or whatever, that are relevant to your query. And then it's like, well, could we actually give them the answer? Sometimes you just want the answer. That's now trivial, or some version of trivial, in the sense that you can just type it in and you get back an answer. Sometimes you get them back in these really long paragraphs that talk about way too much stuff. I just want the answer. But nonetheless, you can query these systems to actually get them to give you the answer. Academic support. So this can be used as a study aid, and I've been working with a lot of undergraduates talking to them about their AI use. And one of the things that they really appreciate is the ability to take subject matter that they're struggling with, like leading into the exam, and have it generate quiz questions or have it have a conversation with them about it. And it really does seem to help them internalize the material better. Good. In the workplace, right, this technology is starting to be used and has a great deal of promise for executing repetitive tasks, things that we have to do, but it's very tedious, like copying information from one spreadsheet format to another. Like, this is not what human beings exist to do, and yet we're like these cogs in the machine. To the extent that these tools can help us do that, that's awesome. This is maybe not a problem that everybody thinks about every day, but Erdős was a mathematician. He solved a tremendous number of problems. He actually interacted with people all over the world, collaborating on mathematics. Some people here, you probably even know what an Erdős number is. It's sort of how distantly related you are in terms of your publications to Paul Erdős. And one of his gifts was actually coming up with really interesting questions, some of which he couldn't answer. And then he left them. And so there's, I think, on the order of a thousand unsolved Erdős questions that are out there. And you can just ask a chatbot like, 'Hey, here's, you know, Erdős open problem number 413. Give it a go. Tell me what you come up with.' And so there, people who track this sort of stuff have seen publications where they're like, 'We found solutions to Erdős's open problems.' And chatbots themselves can actually be really helpful in terms of social support. People are using them to kind of talk through their problems and, you know, practice certain kinds of interactions that they could have with people in a really safe environment. These are good things, but every single one of these has like a dark other side. So let's do the bads. Yeah, sure, it can answer questions for you, but it also hallucinates, right? It actually... this is now a term that I feel like everybody knows, but it's just a weird... it's a very weird term, relatively new term, but the notion that it can very confidently give you an answer to a question that is completely made up, right? Right. And so now we see these almost daily newspaper articles of like, oh yeah, this lawyer was supposed to do a thing for his client, but then he got fined by the judge because all the references in the document that he wrote were completely made up by a chatbot, right? And the lawyer didn't check it. The lawyer didn't actually write it him or herself in the first place. Like, this is a problem at a societal scale, right? That we just as a group of people, we don't know how to deal with that. The system that just looks you right in the eye and says, 'I can tell you the answer to that,' and it's completely fabricated. Academic support. Well, anybody who's affiliated with the university is on the front lines of this. We don't ask homework questions anymore. We're actually giving prompts, right? You can take the statement of the homework that we give to students and just feed it into a chatbot, and the chatbot's like, 'Happy to help you with that.' And it will just produce like often a decent answer. Not generally a great answer, but often a very decent answer that a student who is pressed for time might simply say, 'Great, I'm just going to hand that in.' Right? This is a problem. This is a problem that's hitting not just in the kind of AI space, but every place that people are assigning essays or any kind of homework that's not, you know, make this with your hands. It's... this is undermining the way that we have structured education. Sure, this technology can help us in the workplace, but isn't it also going to like replace us in the workplace? And then what the hell do we do now if we can't actually work for a living? The first paper that came out where they say they found solutions to the Erdős problems, they literally found the solutions. They actually just identified existing literature that had the answers that nobody had put together before. Like somebody solved a mathematical problem that Erdős didn't know that they solved it, or it happened since then and just the dots hadn't been connected yet. So it's like, okay, that's helpful, but that's not what we thought you were doing. We thought you were a mathematical genius. Instead, you're like a really good librarian that has value, but it's not the value that was claimed. And I already alluded to this, but this notion of using chatbots for emotional support, we're seeing some really weird things happen in that context. One of them is people spiraling, getting sort of deeper and deeper into a fictional world that they think is the real world, and they kind of lose touch with reality. And this is... I've met people who are in the thick of this. I think I've become a little bit of a magnet for them because my title is associate provost for artificial intelligence. So people think he knows about artificial intelligence, like I'm having this experience with it. We solved everything because that's the form of this sort of delusional state that people get into, is that between them and the chatbot, the chatbot reinforces their beliefs to the point where they start to think, 'Oh my gosh, I actually cracked like a long-standing Erdős problem.' But like, long problem, long-standing. The people come to me, they're like, 'I solved AI, like I solved the hallucination problem.' I'm like, 'No, you are the hallucination problem.' But like, it's this horrible thing. Like, very... and I don't mean to... I'm not mean to make fun of anybody. Like, it really is horrible, and people are just not prepared for this. I think we're going to get to a point where we have some defenses against it, but at the moment it just gets in and it does problems. So I want to talk a little bit about how we can negotiate the border between we've got this, the good, we've got the bad, like what's happening in the middle. How do we actually stay on the good side and stay away from the bad side? So I'm going to touch on each of these, and I don't know if this is the right audience, and I don't know if this is the right way of presenting it because I've been experimenting with this, but I'm going to tell you a little bit about chatbots and how we have chatbots and what they are because they're programs. They're computer programs, right? They're just computer programs. We've had computer programs for a long time. They're just computer programs. However, they're computer programs that people didn't directly write, and that's weird, right? So they're very complicated, like a trillion-line program. They're bigger computer... bigger programming systems than people have ever created, and people didn't write them. Instead, the systems themselves were kind of... some people say grown. They're like, what do you say in a garden? Like they're sort of cultivated, right? Like it's not a thing where people write down all those individual lines. It sort of just happens. So, all right. So, and if you don't have an appreciation for that, I'm going to try to... I'm going to try to hit you with this, and you can just tell me that it didn't work or if pieces of it worked. Let me know. But okay, so at core these systems are giant neural networks. A neural network, again, is just a program, and it's a particular kind of program that converts blocks of numbers into other blocks of numbers, right? That's all it's doing. It's just transforming numbers into other numbers. Okay? And I say blocks of numbers because it's not just one number. It's not like you put in 17 and it's like 53. It's actually just huge numbers of numbers. And you can represent with numbers anything that you have on a computer, right? So like text is just numbers at core. Images are just numbers, like the brightness values and the color values at each position in the grid, which is the image. Sounds are just numbers, right? We just digitize the frequency strengths at each moment in time. We turn them into numbers. So blocks of numbers is already a pretty general thing. We can convert really kind of any block of numbers into any other block of numbers. So, and programs can do that, and humans have written lots of programs that do that. So that's fine. But these are weird programs. They were not written by people. Instead, what people do is they define a goal. They're like, 'I want a program that converts blocks of numbers into other blocks of numbers like this. I'm not going to write the program, but I'm going to just... I'm going to wish it, right? I'm going to say I want a program that does that thing.' So an example, a really simple concrete example would be the block of numbers that's in the input are images, like small black and white images, and then the number that comes out is just a zero or one. Is that a cat? Right? Right? So this is just a simple simplified version of what a chatbot is doing. How can you make a thing that takes a block of numbers that is a picture and say yes or no, that's a cat? So what we do is we collect tons and tons of examples of images that are cats and images that are not cats. And we say, 'Hey, program, write yourself so that this one, if that was the input, you'd say yes, it's a cat because it's a cat. This one, you'd say no, that's not a cat because it's a screwdriver.' Something something else like that. And so we have thousands and thousands of these images, and we let the program just try to solve that. So the program initially is just random. It does what it does. It guesses wrong on almost everything, or maybe half the things, because it's either a cat or it's not. And now we say, 'No, we want you to be better at that. How can you change the way the program works to make the output more like what you want?' So it turns out, and I will not... there will not be a quiz on this. That's calculus. So what calculus tells you is how small changes in the state, in a situation, in a number get translated through a formula to be a different number. And so that's what's at the core of being able to train these models, is we set up the problem by saying we want the cats to be recognized in this way, and then we let calculus kind of push that information back through the program to do the thing that it was supposed to do. All right, that's all I'll say about that. No, it's not. I'm going to say more because that's just the beginning, right? This thing that actually takes blocks of numbers and produces other blocks of numbers is what's created to become what's known as a large language model or an LLM. So you've probably heard that term. Some of you may even know what that is. But we're talking about using this network to do a very specific task. And that task is given context, given a string of words like the middle of one of Walter's books. So we've got the whole book up to this point, and then we want to guess what's the next word that Walter would write in this book. And so the output is now a probability distribution over what that next thing is. Now, Walter actually chose a word. There's a real word in there, but the system is trying to learn to guess that. And so we hide the actual answer and we make a guess, and then we tell it, 'I'm sorry, that's... it was actually the word Kentucky, and you didn't get that right. So, little penalty for you.' And so the system over time is trying to learn how to do better and better at that task of predicting just the next word, which is basically just autocomplete, right? We had this on our phones 20 years ago, 15 years ago, 10 years ago. And that's what that neural network is trained to do on, by the way, unimaginably large amounts of examples, right? It's not just a book. It's not just Wikipedia. It's everything that these companies can scrape off the internet, even things that they probably shouldn't, to try to learn to solve this particular task. Okay, but that's just a piece of the puzzle because that's now wrapped in a wrapper. So we go from being able to predict the next word to a thing that can actually tell stories. And the way that we do that is we let it predict the next word. We take that word and we put it back in the context and say, 'Okay, well, predict what comes after that.' And if you've ever... I used to play this with my kids all the time, that some people call this the one-word story, where you go around the room, everybody adds one word to it. That's the game that this network is playing. It's just picking a word and it's like, 'Oh, now I'm in a new situation. What should I say next?' And then it adds another word to that, and it can just keep doing that indefinitely. We can just keep asking it to predict the next word. But these models, instead of just doing that, are actually trained not to produce not just the next word as it would appear in the internet. That's where they start. But now we ask them to actually produce the next word as if you're somebody that's answering a question, and they're given a lot of data, a lot of practice actually answering questions, and are trained to do a better job at that. And what you get out of this is something that can actually have a conversation, that can actually process language and output often very sensible, reasonable responses. But it doesn't end there because there's another layer that wraps around that that's called a reasoning model. And that's to try to teach the system not to just produce the kinds of outputs that people accept as like, 'Oh, that's a reasonable answer,' but to have it actually get answers correct, like to do things like math and programming where we have a concrete correct answer and we see what it produces, and sometimes it produces like a nice story, but it's actually the wrong answer. So an extra level of training happens to try to get it to be more often producing...
Correct answers. So that's it, except for wait, there's another thing, which is now there's these agentic systems, which is just another layer wrapped around all of that to try to organize the system so that it's not just producing this kind of one-step conversations, but acting over time, solving sequences of problems. And so the outer system is kind of prompting the inner system about what problem to solve next.
And these are really neat. Again, the middle gets retrained a little bit to be better at this overall task. But one thing that's really neat about these is now they can impact the outside world. And so in my field of computer science, everybody's using these systems now to help them write programs. It's like it's rare now for people to sit down and write a program by themselves. They'll call up one of these agentic systems. They'll kind of wish the program into existence, and then the people who are doing it right will look at the response, look at the output of this, and say, "That's not exactly a good answer to this. Let me tweak it. Let me try to make it better."
All right. So there's a whole field now, well, I'm not going to get into it, of mechanistic interpretability, of trying to understand what it is that these systems are doing. So between the way that we build these systems and the way that we understand them, hopefully things like hallucinations, we're going to tamp them down. Hallucinations are kind of built into the architecture of these systems, though. So they're not going away, but maybe we can catch them better. We can work with them better.
All right. So AI and education is a topic that I end up thinking about a lot in my current role. And I hear lots of people say lots of things to try to calm me down, but every single one of them has a retort. So, you know, oh, don't worry, it's just like we used to have, we used to all be freaked out about calculators in math class, and now it's all fine. It's like, yeah, except that this is a thing that's in all of our classes and sort of integrated into the desk, right? You can't even avoid it if you try to.
People say, oh, you know, the students should know not to do this because just like you don't bring a forklift to the gym, right? People know, people have seen this sort of analogy before. It's like you don't bring a forklift to the gym because you actually have to lift the weights to get the benefit. Using a chatbot to answer your homework for you is like subverting that. It's like bringing a forklift to the gym. It's like, yeah, except the forklift is already in the gym and you get credit for whatever it lifts. So like students are not going to stop doing this just because we tell them that.
Generative AI output should be F-level work. Like we should just give people harder problems. There's some value to that, but like the headroom maybe is starting to vanish. Like there may not be harder problems that are still solvable that the chatbot can't do. Maybe harm reduction, maybe we can try to maximize the benefits and minimize the costs. Sure, but it's sort of like if for things like drunk driving that we have these mechanisms in place, it took us a long time to develop those. It's like beer just got invented three years ago and now we're all trying to figure out how to deal with that.
So one of the things I've been arguing for is trying to set up a baseline on my campus to try to have better norms around what's okay and what's not okay. And this is, I'll try this out on you. The thing that I think is reasonable is to use the analogy that the system is kind of like another student in the class. And the things that you shouldn't ask another student in the class to do for you, you shouldn't ask the AI to do for you if you're a student.
So, for example, which of these is okay to ask a friend to do in the class? Answering questions about the material from the lecture, like you heard a lecture, could you go to another student in the class and say, "I didn't understand how those things connect." That's probably okay, right? I think that's okay to do with chatbots as well. Outlining a response to an assignment, like I was given an assignment, I had to do it, but like I'm not going to start on it. I'm first going to ask one of my friends to do an outline and then I'm going to use that as my starting point. That doesn't seem okay, right? And people are doing that with chatbots now. And I don't think that's okay. I think we have to get the message out that that's not okay.
Or even worse, writing a first draft of the assignment response. People are doing this with the students are doing this with their chatbots. They're like, "Well, I'm going to edit it afterwards." Yeah, okay. But the hard work of actually organizing the thinking already happened. You're not getting the benefit.
Looking over a response to check for muddled thinking. Like I write my whole essay and then I take it to another student in the class and like, can you just read this over and tell me if there's like spots that are messy or like confused? That seems, I think that's okay. And I think that's an okay use of a chatbot. Acting as a study buddy, posing quiz questions. I think that's great and people are definitely doing that as well.
All right, I'm getting really low on time, so I'm not going to say much more about chatbots at work. I'd love to talk about this and maybe it'll come up in the Q&A about the way that we're changing the way we're trying to teach about programming now that students can wish programs into existence. Yeah, there's a Family Guy reference. So that's a thing that happened.
What's happening on the math side is actually way more interesting than I kind of led on in that intro, that people actually are solving really hard, really interesting problems, but the way they're doing it is so fascinating. Like sometimes the mathematicians will credit the AI, but when you look at what actually happened, the mathematician is doing an incredible amount of deep thinking to get to the point where the AI can actually be helpful. And I think that's a really important thing to keep in mind. And math is way more advanced than the sciences as far as these systems actually helping people do their professional work. So it's not taking over this really heavy lift intellectual work at this stage.
Working a lot on this, chatbots and emotional support. We have a lot of folks on campus who are studying this. More to say on that another time.
And so here's the last word. So I asked Gemini, help me understand how I could talk about the theme of this AI moment in the context of the good, the bad, and the ugly, which I don't think I've even ever watched. So I probably should not have been referencing it, but it was very happy to give me an answer. And it said in the movie, a gun is an equalizer, but it doesn't have a moral compass. It amplifies the intent of the person holding it. Similarly, AI is a dual-use technology. It can be the good, solving complex diseases. I don't know what it means to solve a disease, but okay. The bad, autonomous weaponry or deepfake fraud, or the ugly, chaotic, biased, or unpredictable outputs.
The lesson is that the tech itself won't save us. The character of the operator matters most. And I don't think that's actually what the movie was about, but I do like that message. So I thought it would be sort of interesting to include that. So that's my tour for you of the good, the bad, and the ugly. And I look forward to getting to interact a little bit more coming forward. Thanks everybody.
