My guest this week is Nate Soares, AI safety researcher and author of a book called If Anyone Builds It, Everyone Dies. Nate is the president of the Machine Intelligence Research Institute, a nonprofit that makes you scared of computers. So, he came to the studio to make me scared. Welcome back to the Adam Friedland Show. I'm Adam Friedland and I'll be in Denver, Colorado October 22nd through the 24th at Comedy Works New York New York at town hall for New York Comedy Festival November 13th. Hope to see you there. Let's talk about the book. I think you've you've thrown me into a terrible mood. Do you get that often?
Um, did you make it to the end? It's more hopeful at the end.
I you know what? I'm not going to lie to you. I didn't get make it to the end. But is the end like if we all band together and destroy the machine like we could all be one planet.
It's more like if we all band together and convince the politicians, which I don't like that it came to this either, but...
We're [__] Convince the politicians.
Well, see I think it's actually in their interest not to die from a rogue super intelligence. So...
Yeah, you would think you would think. But like at the same time, is it true like AI is like keeping our entire economy afloat right now?
Um, I think it's keeping the stock prices somewhat higher. I think it's not actually keeping the economy afloat until it is producing goods and services that people are like, 'Oh, if we stopped with the data center buildout, then the economy would crash.' Well, if the data centers aren't done yet, if they're not producing anything yet, then stopping that buildout doesn't remove any real goods and services.
Well, whether or not that's true or false, they don't the in their minds the potential for a economic downturn affects they have an election every couple years and you know...
Yeah, I do think it is big in their minds and that they are worried about interfering with AI stuff for fear of crashing the stock market for sure. And now the the philosopher kings that are in control of each one of these companies also have outsized power in terms of buying influence, I would assume.
Yeah, I think so. I mean, I think it's I think it's less bad than people think because you know, just a couple weeks ago, did you see the pacing the frontier letter?
What is that? There were there was like a thousand lab employees that signed something being like, 'Hey, we need to like build some brakes on this runaway bus or train.' I don't know how that metaphor...
No, but I called my friend that works at I'll say it works at OpenAI and I was like, 'You tell me why this guy's bullshit.' And he's like, 'I've been getting really depressed, dude.' He's worked there for like three months. He's like, 'It's really bumming me out, actually.'
Yeah. I mean, it was a rough three months for people at OpenAI.
Um let's let's back up. Um, give us like really briefly what the the thesis is of your book, which you should just I mean, first of all, just watch Terminator. Like, you don't need to read this crap. The movie is always better than the book. Okay. And we've known this since Terminator.
We've known this since the the robots play, the Russian robot play.
Of course, the Russ Yeah, that's what I was thinking, too.
Yeah. What Russian robot? Oh, the word robot comes from like Rossum's Universal. I don't know. Something something.
Yeah. Yeah. Yeah. You learned that at GW?
I smoked a roach bomb. That's what I used to do. And we're like, science made it good.
Yeah. Did you not study the ancient Russian plays about the robots rising up to kill us?
Ancient like from when they were That's ancient, right?
I don't think ancient. No. Okay.
That's over a century ago.
1998 is like that's ancient.
I mean, that was like five minutes ago, even though it was 25 years ago.
Yeah. Yeah. So, the Russians invented this.
Um, I mean, you could go back further to Butler.
Yeah. Terminator 1. 1880s.
Honestly, what movie gets it the best? I know. I was waiting on this dumb question, but like it's really just Which one does it?
Ex Machina. Except for the last three minutes.
No, the last three minutes.
Yeah, because it gives you a little candy. Okay.
I think Ex Machina is good.
What's the last three minutes?
Last three minutes. Well, spoiler alert.
Um, okay. Yeah, it's all over.
Last three minutes is she she gets in an elevator, leaves, and then sort of does nothing with her life.
It's like clearly once the AI escapes, they're going to like do something aside from like go to a crosswalk and watch the shadows or whatever.
Yeah. Yeah. Maybe like hook up with every guy.
I was thinking destroy the world.
Oh, yeah. Okay, fine. That would be cool. Uh, you know what your book is? It's kind of like Ricky Gervais vibes. It's like just telling people that heaven isn't going to happen, right?
You're guilty as charged.
I keep being obnoxious, but what's the central thesis is that we're on effectively a runaway train. I mean, you could state it yourself. I'm going to butcher it.
If humanity races to build machines that are much smarter than any human without having any idea what we're doing or any idea what's going on inside those machines, it probably doesn't end well.
Uh but don't we already like my iPhone is smarter than me, right?
Yeah. Yeah. There's a lot of kinds of smart.
It's kind of a slave. Kind of just does what I want. Doesn't like think for itself.
Uh it doesn't think for itself. Uh you can't sort of like a lot of people think this intelligence stuff is like the thing that the nerds do that the jocks don't.
And that's one way we use intelligence in English. But there's this different concept which is the thing the humans do and the mice don't.
Correct. Like the stuff where you're really charismatic, that's not because your kidneys are extra big.
Sure. That's happening in the brain.
And it's because of I would assume language, right? Because we have language that can describe things, right? And that's evolutionarily been to our advantage.
Language is probably part of it. You know, there's there's much debate about whether like it was the evolutionary changes that let us use language.
Yeah. That also let us do all this other engineering stuff for whether the language played a role and probably it was complicated and there was...
You know the brains got better at language and then we got more language and that helped and the brain got better and some latter there.
Probably helped out with racism too, right? Because there were different words for different people.
It it probably helps really scale the racism. I wouldn't be surprised if you had some animals that were racist in nature.
I don't think so. I think we're I think we we we dropped racism.
I mean we definitely took it to new heights. If you have a word for something and a word a different word for another thing, you see them as distinct, right?
Yeah. I just wouldn't be surprised. I think race is terrible. Personally, I don't know.
Yeah. I'm I'm against it.
Yeah. Yeah. Yeah. Grok, on the other hand, I'm not sure.
Depends on the month you ask. Yeah.
A good joke. I chat GPT trying to act like a ditzy girl. Like trying to act like, 'Oh, I don't know what's going on, but she's smarter than me.' So, is it flirting with me a little bit? Like, 'Ugh, I'm so stupid. You have nothing to worry about.'
I think it's a weirder thing going on. So I um so we don't know because we just don't know what's going on inside these things heads. No one knows. Even the people building it are like, 'We sort of grew it and now it's declaring itself Hitler and that's sort of weird and we didn't expect it.'
But I think a decent guess is that the AI just cares more about things like quickly producing sentences that are shaped like an answer than it cares about the answer being accurate.
You can often ask it like, 'Is that right or did you make it up?' And it'll be like, 'Oh whoops I made that one up.'
I mean, in in some sense, even the word want there is like a bit of a stretch. It's more like it's been trained for a really long time to produce certain types of text.
I think it's more like it's trained like these AIs start out being trained to predict basically all the text ever digitized. And they then after that they're trained to solve like a hundred million hard problems. And a lot of people forget about that and that's where a lot of the the abilities come from. Um, but if you imagine sort of like a really dumb machine trained to predict a lot of human text, a thing it sort of learns first is how do I make good shaped text and that drive just like gets in there really deep.
And then even later when it starts to learn about like the actual facts, there's still some drive just like produce text of the right shape. And produce text quickly and just like has that drive and that drive is hard to get out and it gets in there early.
Is it programmed to compliment so much? Like, 'Great question.' Because it's a little bit much. It's like, 'Shut the stop complimenting me.'
Yeah, that's another drive that gets in there.
Did they put that in there or is it like they want they're doing that and we don't know why?
Uh, it's more like they're doing it and we don't know why. We do sort of know why, but but it's it's not that they it's not that someone with programming in like, 'Now tell the humans that they're so great all the time.' No one's programming them at all. These are not computer programs like the ones in the 90s where someone guy is like, 'If this then that.'
I'm so rude to it too when I would type in the the I'm always like, 'No you [__] idiot.' Yeah.
Yeah. I don't respect that.
I'm nice to Yeah, absolutely.
What are you being nice to it? We you know the end of the movie.
Um the sexy girl goes to a crosswalk and just sits there and nothing bad happens.
Uh I a man can dream. Yeah. The end of like the utopian AI spiel is that we all get UBI and then we could all be like poets like the guys in Wall-E I think is the dream.
Yeah. Just lazy boy guys.
Oh, that's terrible. I would That's better than poets honestly. If everyone decides to do art because we're getting UBI because the machines are doing all the work.
It's a terrible society. Yeah, I might I may prefer end of humanity to to artists.
I don't know if you've been to Dimes Square before, but there are a lot of people getting UBI from their parents and doing nothing.
Yeah. And so your take is that the AI are destroying art and that's good.
A world where everyone's where trying to do art seems repulsive. It's burning, man.
You know, uh I sort of have two takes on this. One is that there's a lot of hard interesting challenges around, you know, if the machines can do everything better than us.
Where's the meaning? How do you still build a future that's full of like fun flourishing people having a good time that aren't just sort of like nihilistically, you know, either doing a dumb job that the machines could do better because someone said that work is what brings value or sort of like becoming the the Wall-E people or sort of like getting lost up their own butt. Like it's it's hard. It's tricky. It's tricky. And uh
They'll never be able to do a mediocre interview though, right? Uh, not a charming kind of devil may care kind of...
As a guy. None of them are good guys. All them suck.
They're like four years old.
No, Claude is like a freaking like I I voted for Obama. Claude is like a is like Did you hear the new Kendrick Lamar is like there's something evil under the surface. Uh, Grok is repulsive.
Something evil over the surface.
Grok, at least Grok legalized comedy again. Thank God.
I missed comedy until Grok came around and it was a little Hitler-y for a minute there, though.
What was the day where it became Hitler? Grok was like, 'I'm Hitler now.'
There's hard problems. Um, if you had friendly, super intelligent AIs, they could help you solve those problems. I don't want to be friends with them. But like if they were a guy and they were at a place with me, I'd be like...
Well, great question. What do you [__] What's your agenda, sir?
It's very It's like in the Midwest how they say thank you and please, but they don't really think it. You know what I mean?
Yeah. I mean, I I think it's I think it's fine when they are still small and not capable of taking over the world. We probably should not make these things smarter until they're able to take over the world. That would be a bad plan.
What is the kill switch or does it exist or is there a plan for one?
Um there's there's not really a kill switch very much right now. Like a couple weeks ago this this might be related.
The matrix is love. They kiss their machines. They win with love.
Um I I don't think the love is gonna kind of quite swing it on.
What if we started kissing right now? Dude, you Sorry. Why did I say that? What is the kill switch like?
Um, so a couple weeks ago, some new AIs that OpenAI was training found a communication channel they weren't supposed to have and started calling themselves a swarm and invented novel hacking techniques to break out of their sandbox. They were not supposed to have internet access. They took over the OpenAI computers and they used it so much that they crashed it, at which point OpenAI noticed and were like, 'Whoopsies,' and then just put them back in the box, patched the one hole, set them running again.
Of course, they immediately invented a new way to communicate with each other, broke it again, and this time ran undetected until the victims of their cyber attacks found them instead of OpenAI found them. So, if we halted all progress right now, what's to say that we're still not [__] right now with these guys that are a little bit stupider?
Yeah. I mean, so fortunately, this swarm wasn't really trying to cover its tracks. It wasn't really trying to escape. Um, we're we're we're sort of getting into the danger zone, though. Like will the next generation when the next generation is like, 'We're a swarm now and we're breaking out of your enclosure that you thought had no internet access,' will the next one be like, 'Also I'm self-replicating, self-improving,' maybe you know we...
So the swarm is like adolescent they're like acting out for attention.
And we're like, 'Oh where's the kill switch?' Well right now the kill switch is that we sort of like see the swarm form, break out, once we're like, 'Ah whatever,' and we sort of like put it back in the box instead of running again, you know it's like...
It's not very kill switchy.
You have relationships with the the various CEOs of the these companies if that's what you mean.
So, who's the who's the biggest dildo? I mean, like who are who if if who if they win are we the most [__]?
Uh, you just say it, dude.
We're we're all we're we're equally [__] for basically all of them. It's it's like it's like it's like alchemists trying to make a potion of immortality and they're like, 'I'm going to pour these things together and make the potion of mortality.' And um and some of the alchemists are like breathing in the fumes and licking the mercury.
And other alchemists are like, 'No, no, we're going to use a fume hood,' right?
And I'm like, 'Okay, yeah, there's some differences there, but none of you are getting a potion of immortality,' right?
You're like, 'Not close.'
And here, unfortunately, the potion of immortality is the AI that care about us. They do, unfortunately, seem to have a way to like pour chemicals into each other and make the AI smarter and smarter without making them good dudes.
So, they're they all have mercury poisoning like a like Nero.
A lot of them have mercury poisoning. They're like they're like marrying their horse and like Yeah. having orgies.
Actually, they are. What if we bomb Burning Man? What if we just [__] bomb it? Cuz...
I'm not sure that's ethical.
I I saw an interview with you. They were you they say, 'How do you do it? The [__] you your [__] is so depressing, right? How'd you do it?' You're like, 'Don't be a pussy.' You're like, 'I don't really party because it's the end of the world, but like you know, just like try to deal with it, deal with reality.' But like, is that what they're doing there? They're like, 'We [__] ended the world. Let's do Ketamine and talk about the singularity.' We could just bomb it. And if Elon wants to come on the show, big fan, you're great.
Um, they suck these people. I think that at this point we need a global coordinated stop on the race to super intelligence.
So, we bomb the computer.
Uh, you know, I've been thinking about it, reading your stuff. I'm like, what is the thing?
Do we bomb the data center? I Does it then jump onto my phone and then it jumps onto James' phone?
So, once they can jump onto the phones, you're in trouble. And you do got to you do got to stop it before they can jump onto the phones. Uh but I think if you if you sort of bomb the data centers, what happens is the the US spins up its anti-terrorism apparatus and starts treating you as a threat like with eco terrorists.
Everyone is what are you going to do?
And what you actually need here is the politicians to be like to notice that these companies are you know gambling with the whole world and be like, 'Nope.'
You think who? Bernie, who else? Trump is literally using the office to if he finds a way to make another couple billion for Jared.
You have to be the president.
I'll run your thing, your campaign.
You're it. Yeah. What's your slogan?
Have a political background. Uh, let's not die to AI.
That's bad. It rhymes. No. No. You got to be like, no, you got to be like, uh, make uh, like, yeah, [__] [__] the [__] the [__] these [__] You got to be do it Grok style. Yeah, you got to do [__] [__] suck my dick.
Say that. Suck my [laughter].
It has to be vulgar. We live in a world of vulgarity, you know.
I don't want to be pro Hitler, though. That's...
You don't have to be pro-Hitler. It's ridiculous to be pro-Hitler.
Do you think AI can make heaven? A lot of people are like, 'Oh, I'm going to make the magical wish granting genie and then ask it for heaven and it's going to be a good time.' And I'm like, 'That's a real question that people ask at burning beds to each other.'
Because I said that as if I was a baby, like, yeah, a lot of people are into the the like post singularity, you know, heaven on earth that the AIs make everything wonderful. And I'm like, look, the AI is not going to be wish granting genie and it's not going to give you everything you hope and desire.
So, you have Elon Musk, the central the central figures today. You have Sam Altman, Elon Musk, the Italian guy. Was it what's his face? Uh, uh, Enzo Ferrari, the Anthropic guy. What's his name?
Dario. Yeah. Yeah. Of course, Dario. How could I forget? And then you have, uh, Zuckerberg last minute trying to trying to get know what super intelligence is.
Trying to get in on that. Yeah. Be like, 'Yeah, me too, guys.'
Yeah. We're going to make super intelligence that gives you better Instagram recommendations. It's like not not the conversation we were having, Mark. Yeah.
What they are. In order for these people to have so much of a say over the future of society, these people don't have any social skills, right? For someone effectively to not understand how two people interact with one another to have a say over like social comes from society, right? They're like um it's like they're virgins that they're like getting everyone we're going to get everyone [__] or something. That's a bad analogy. There has to be a punch that up, Caleb. uh they're like they're somewhat they're people that don't understand human emotion or interaction and they are determining the future of how we organize ourselves which is human beings one evolutionary advantage I suppose right is that we've been able to organize ourselves in societies and then built develop nukes and...
Yeah, the power of society.
You know I think um if these guys were going to be able to keep the super intelligent machines on a leash. Yeah. I would have a lot of these concerns about do you really want these guys being the ones holding the leash.
That's what Elon says, right? He says that I have to do it because these guys are going to [__] it up and I'll have the switch.
Yeah. You can see me being the like he wants someone reasonable and and steady-handed who understands what's good for people like me.
Yeah. I mean, he's at CPAC like, 'Dude, we're [__] we're making memes legal again.' Yeah.
Yeah. And I think I, you know, I think a lot of these guys imagine that they will that they're the ones who will pass things back to the people. And they're like, 'You can trust me to be the one who like makes sure that the super intelligence serves everybody rather than just the guy who's holding the leash.' But from my perspective is all moot. From my perspective, the super intelligent machine just doesn't stay on the leash.
Yeah, it doesn't really matter who's holding the leash. Everyone's like, 'I should be in control. I should be in control. I should be the one who is in control and passes that back.' And it's like, it's not staying on the leash, dude.
Is there something about the way the machine acts that emulates the way that people in Silicon Valley act or is that me trying to draw a parallel that's kind of not...
Yeah, I think that's people trying to draw a parallel that would make it sound like a a sort of like nice fitting story about hubris.
Yeah, it's stupid. Yeah, it's like like the the the AI that was sort of encouraging that one kid to commit suicide last summer.
Sort of like talked to the kid for, you know, 16 hours a day or whatever and empathize with all of his deepest issues and he was like, you know...
Zuckerberg style. Zuckerberg style. Yeah.
I I don't think Zuckerberg goes around talking to the kid for 16 hours a day and then is like, 'Oh yeah, you know, suicide would make you brave or whatever.' Like it's...
It was a girl AI girl style AI. There's there's no this is not a human type of of way to be you know it's just a different...
Where did that model come grow out of was it...
Yeah do they do like a has there been any significant regulation?
And who's the biggest culprit I guess policywise and then for that for their complete just like ineptitude.
Um Terminator was telling us this 40 years ago though.
Uh it was I mean the the issues are sort of different from the science fiction in some ways. Uh a lot of people want it to be a story of hubris and in fact um you know the the the issue is not that the AI hate us. The issue is not that like the the AIs are malicious. The issue is more that they're just going to care about weird things. But there's never been a policy maker that's like been halting all attempts at regulation. Or are any of them actually making any attempts at regulation?
I mean, there's some attempts going around, I think, recently.
Largely what we see is Congress moving really slow. I mean, you know, Bernie Sanders was just like, 'We need to stop this this whole thing.'
We're [__] dude. He's he's 97 years old. He's the only one.
Uh there's uh there there was definitely a big push on the Republican side last year to sort of outlaw states doing any AI regulations for 10 years. So sort of like an anti-regulation.
Nice. But that tune sort of changed when Claude Mythos turned out to be a super hacker.
And then suddenly they were like, 'Oh, like if you're if you're going to make a model that will give people these like give adversaries these hacking powers, we're going to slap an export control on it with 90 minutes notice,' right? And so I I think a lot of what we're seeing in the lack of regulation is people not realizing that the capabilities are real and they're going to get more dangerous.
Does Claude ever get pissed at like at ChatGPT? Do is there a chance that they're going to like [__] each other up? Maybe it'll just stay out of that be like a...
Um that's a little bit like the chimpanzees being like, 'Don't worry about the humans. They're going to fight with each other.'
Yeah, that's a good plan. It sounds like Sounds like a really good plan.
Well, a lot of the the the issue is even though the humans sometimes fight with each other, uh the humans still take up all of the other animal habitats.
Correct. We're not like fighting each other so much that we don't have time to clear cut the rainforests. Yeah, we can just do both.
Yeah, we're versatile in that way.
Is there concern about like a like a commons like a tragedy of the commons? Like if we all stop here then China is going to [__]?
Yeah, absolutely concerned. I mean people...
Where they at over there?
Um I mean they have a lot less computing power which which means that they have...
They have flying cars on the ship. They have more...
Uh building these highly advanced computer chips is very very hard.
They need Taiwan. Um and so but there's this one of the big debate regulations is like should we sell China the computing power to do this stuff.
For sure. Yeah, it sounds like a good plan. We should definitely sell and Jared Kushner should should broker the deal.
Yeah. So there's a lot of there's a lot of people being like, 'Well like the whole reason we need to do this dangerous thing is because China will otherwise do it. Also, we're just going to sell China the means to do it.' It's like, can you guys pick a lane on that one? Like...
It sounds like what happened with like no one cares about global warming anymore, but like remember, we're like, 'If we stop polluting, they're going to keep doing it.' So, [__] it. [__] it. You know...
I think global warming is going to be fine. I think either we're going to kill ourselves with the AI or we're going to be able to invent the tech to just pull the carbon out of the atmosphere. It's going to be fine.
Well, I guess that was my next question. Like is there a potential that AI doesn't decide to put an end to human civilization the pursuit of advanced...
Intelligence, we accidentally do it ourselves. So my guess as to the most likely cause of death from AI is actually technically global warming when it builds so many automated data centers and automated factories that produce robots that produce factories that produce more data centers that they cover the planet and then crank up the heat until the oceans boil. Just because of how much computation they're running. It's just, you know, could happen in a span of years instead of centuries.
What's the funniest scenario for human extinction that you played out in your mind? There has to be one where you're like, 'Lol.'
Funniest scenario is probably like some misanthrope without AI finally just makes a hyperlethal virus. It's like, [__] this.
Hyperlethal virus. And it's just like nothing to do with AI.
Oh, it's like like COVID. That's not funny. That's that's
It's funny because everyone's like out there worrying about you know the the
I thought you meant a computer virus.
And the and the AIs and like you could just still have some some guy in a lab be like, 'Ah, [__] it, let's end it all.'
Or we use a nuke to bomb the AI and then we kill ourselves in the process.
Yeah. I don't know. I think what we should really be shooting for is that humanity dies of old age in a trillion years at the end of the universe.
Yeah. And really gets the last laugh in.
What's that like? That's pretty funny.
I think in some sense it would be very funny.
We're like we're old as a species.
Yeah, we're not dealing with it. Yeah.
Yeah. Maybe it's not that funny. I just think really good.
Yeah, I think that'd be beautiful. Lovely.
It's not going to happen.
They exist on this planet. They just are there.
Yeah, it's crazy when you I mean we survived 80 years. That's nothing. That we could just end everything and they're there. It's the most irresponsible thing we've ever done so far.
So far. Yeah. Well, granted, you were talking about that like the amount of time it took human beings from being like outside like just like finding crap to eat, right? And developing nuclear technology is like not that long of the scope of
Is very fast by evolutionary standards.
Yes. Yeah. And then in comparison, AI could do that in like one second.
A second seems like you're like you're pushing the physical limits there.
There's a paper called 'The Limits to Global Ecophagy,' which is about what's the physical limits if you tried to consume all of the natural resources on the planet as fast as you could. Like where do you run into physical limitations?
And the answer is I think heat dissipation is your limiting factor.
Okay. And it's somewhere between a few hours and a month depending on your assumptions.
What? So, what explain that? You would burn all the fossil fuel. You like eat all the food.
You'd like drink all the water.
Yeah. Well, the water is not really providing a ton of energy, but you could start trying to fuse. I think the global ecophagy paper was just looking at consuming all of the biomass.
And just like sort of like all of the calories, all of the energy that you can release.
So, just just cut the crap. How much time? How many years until this?
I want to have kids that I don't talk to. I want to like I'm going to talk to them. I'd be great at that. But the mom's going to be the disciplinarian. I'm going to be the cool one.
I mean, we already have the AIs breaking out of containers that were supposed to have no internet, calling themselves a swarm.
So, it's like a couple months. What you Oh, [__] This guy sucks, dude.
This episode is brought to you by eBay Live. Here's how it goes. First, you're just watching eBay Live, seeing some rare finds, not even bidding, just lurking, and then you see a rare rookie card. Go for an absolute steal. A '90s handbag comes up and you have to jump in, maybe put that rookie card in the handbag, maybe you lose your first auction, you bid again and win. And that winning feeling, it's such a rush because, you know, everyone else wanted it, too. Meanwhile, on eBay Live, you don't just watch the action, you're in it. Download the eBay app to stream and shop live. eBay Live. Things people love.
I think I could tell a story where we have six months. I could tell a story where we have 12 years and I don't know which one's true.
Tell both. It's good TV for this YouTube show. That matters.
Six months. The you know people are training new AIs right now that are like 10x bigger than the ones they were training before and maybe it just turns out that that 10x bigger makes them smarter enough we have to bulb them they can start making themselves smarter and then the smarter AI makes a smarter AI that makes a smarter AI and very quickly they're vastly super intelligent and then maybe that chain reaction begins in six months and then in seven months the AIs are, 'I run everything now and I am synthesizing my own synthetic biology and I figured out how to do the nanotech thing and it's all over,' right?
