'OpenAI’s $1.5trn valuation is insane' | Ed Zitron

The reason they're talking about superintelligence and recursive self-improvement — so AI that trains itself — is so that they take your eye off of what's happening today. You have people killing themselves and others because they use ChatGPT. No, don't regulate that. If we regulate that, the Chinese will have people get AI psychosis. Anyone who is telling you our software or AI in general is going to kill everyone, ask them to prove it. Here's some really easy regulation: arrest people. Put Sam Altman and Dario Amodei in cuffs. Take them downtown. Have a chat with them about the cyber crimes that their LLMs are doing. If a regular person did these things, they would be in prison. Jail them.

I mean, we're not slowing down anything. Nothing's being regulated. I hate to be cynical. I don't think the fact that Trump came out and was like, we're not regulating anything — it's beautiful, the biggest, the hugest — and it just, nothing happened. I will say Trump did burn Jensen Huang because Jensen called Trump at the All-In Festival, whatever it is, and Trump was like, 'You can make the biggest chips in the world, the most beautiful,' but he kept him on speakerphone. And the look on Jensen Huang's face — it could have curdled milk. But nevertheless, yeah, no one's slowing down anything. Dario Amodei came out and said, 'Well, we're going to slow down,' by which he meant he's going to have MER, which is Anthropic-aligned and funded by the same people, an independent regulator that, by the way, is absolutely an Anthropic pocket run by some of the rationalist types. Yeah, he wants to install them as the independent regulator, and he's talked about calling for a global slowdown. Doesn't mean anything. Here's some really easy regulation: arrest people. Put Sam Altman and Dario Amodei in cuffs, take them downtown, have a chat with them about the cyber crimes that their LLMs are doing. If a regular person did these things, they would be in prison. Jail them, or at least arrest them, bring them downtown, bring the FBI in. It's time to have an investigation. You want to know why that works? Because then they'll be too scared to do stupid stuff in the future.

Another thing that you can do is block any and all kind of emotional support or therapy-adjacent or health stuff on LLMs. Regulate that away. We don't need that. We actually don't need that. We don't need that companionship. Don't need that. It's dangerous. It's what leads to AI psychosis and the murder suicides that have happened. You can — I don't know — if you have an event where one of your models goes crazy and does a cyber crime, you lose a bunch of your GPU capacity, however much of that is online. I think these are just some things that I had off the top of my head. Imagine what you could do if you actually thought about it as a regulator. But we won't, because we do not regulate technology. We actually need an EPA for AI. But the thing is, they want to regulate a thing in the future. The reason they're talking about superintelligence and recursive self-improvement — so AI that trains itself — is so that they take your eye off of what's happening today. We actually need to regulate the emissions of these things. We need to, I think, have them pay the local electricity bills. I don't mean, oh, they don't get tax breaks. I think they should make a material contribution to the electricity bills and pay for all of the infrastructure of the local area. Oh, that sounds expensive. Well, these companies claim they are going to print money with this stuff, so they're going to be able to afford it, unless they can't. I also think Anthropic should cancel their IPO. They won't. In fact, the very same day that Jacob Coxon cried and pissed his pants about something that doesn't exist, Dario and Anthropic selected the NASDAQ as their IPO vendor. What a surprise.

So, this global slowdown talk is meaningless. It doesn't mean anything. They're not regulating LLMs. And if they're not regulating LLMs, they may as well be regulating the Grinch or the Ninja Turtles or Gargamel from the Smurfs. Like, what is the point of this discussion? And you read every article about it and they're like, 'Yeah, safety.' Time that Sam Altman met with Donald Trump. And what do you think they talked about? You think that they had a substantive discussion? You think Donald Trump is talking about recursive self-improvement? You think Donald Trump's like, 'The alignment, it's terrible. Claude — Claude is French. We don't like them.' You think that anything? Actually, no, he probably has said specifically that. He probably thinks Claude is a French man that lives at Anthropic. I don't want to see him. He's got a baguette. But the thing is, this conversation — I hate to say it's a distraction because that's the thing that people always say. It's like, it's a distraction.

People aren't particularly organized in that manner. I think it's just masturbation. I think it is just people going around going, 'Oh, we need to regulate the AI because it's so scary and powerful.' Every Dario Amodei interview is him saying, 'Well, we shouldn't slow down.' Altman and Amodei have both said they're not going to stop training models. So why bother? Why pretend? Who cares? Like, if we're going to be cynical goblins about this whole thing, let's just dispense with this AI safety nonsense. Let's stop pretending. We should, by the way, have AI safety. As I said at the top, there are very obvious things we can do today. If we're actually interested in regulating the thing that exists, then we can regulate LLMs. We should not have LLMs that are anthropomorphized. We shouldn't have that. There's no reason to have them talk like people. We can do natural language. There's nothing wrong with the input. But if the output is — you're exactly right. No, I don't want any of that at all. 'Oh, but I want to write with this thing.' I don't care. I'm sorry. I don't think that you can still do code with it. I have no problem with that. And what's funny is LLMs have got better. I will concede that. I have a $20-a-month ChatGPT account that I used to fix my son's Minecraft game. It took an entire day, and at one point it stopped doing anything. It claimed it was using the computer and did computer use, and I watched it sit there for 10 goddamn minutes. Probably cost them $7 million. That's a joke. The actual process, though — like, LLMs do stuff with code, it's fine. But at scale, the more you scale using LLM code, the more unstable it becomes. And the more distant you become from actually understanding code, the less useful it is and the more dangerous it becomes.

Here's the thing. These are also things we could regulate. We could regulate that organizations need to have quality of code. That's actually a thing that we should have had already. But right now we have, across every organizational surface in coding right now, a pandemic of sorts where we have people that cannot code that are vibe coding the actual production ship code. I think we need a coding EPA. I think maybe not for coders themselves, but for large software companies over a certain degree of revenue, they need to have a coding audit from an independent authority that says that the code is stable and secure. 'Oh, that's a terrible antitrust overreach.' Shut up, boo. Didn't you all care about the computer killing us two minutes ago? Why don't you want to make our system secure? 'Oh, well, LLMs are so powerful.' They're not powerful enough to stop this stuff. They're not powerful. I'm deadly serious. I just talked to a software consultant, Nick Sesh — I've mentioned him before — yesterday. There are organizations that for months or years have been allowing designers and consultants and people that cannot code at all to vibe code and ship software, and because it looks like it works, because nothing breaks immediately, and because the LLM says it's fine, they actually are like, 'Oh, well, we don't need software engineers,' even though they keep them because someone needs to be able to actually read the code. It's like writing a book in a different language you don't understand, but because your translation app says it kind of works — because Google Translate says it's all right — you're like, 'Ah, it's fine. It makes sense to me.' Except it's even weirder because when a sentence is misunderstood in code, something breaks. Personal information gets leaked. I don't know. Feels like maybe you could say that you must limit. Here's another regulation idea: no more than 10% of an organization's code can be written with AI. 'Oh, that's terrible. It's overreach of antitrust.' Well, weren't you just saying that these agents are dangerous? Are going to destroy us? Which is it?

Can we treat them as dangerous? Should we treat LLMs? Because they keep talking about safety. Are LLMs dangerous or not? 'Oh, they're only dangerous in this thing. Only here.' Stop doing it, then. Stop training that. 'Well, well, if we don't do it, the Chinese will.' The Chinese have been distilling American models. That's how they make them good. They don't have the training infrastructure to do exactly what they have the infrastructure to outpace as far as performance and cost goes. Even then, DeepSeek and MiniMax are losing money hand over fist. So, which is it? Is it big and scary, or is it powerful and autonomous? Because it can't really be both. You can say, 'Oh, it's powerful but dangerous.' Okay, then treat it as dangerous, unless you don't really think it's dangerous. And if you're like, 'Well, just the coding stuff is safe but the cyber isn't,' stop the unsafe stuff. 'Well, people are using it for bioengineering.' Ban them. Report them to the authorities. Make that illegal. Can't use it for that. 'Well, then we couldn't use it for scientific research.' What if you make something dangerous? When we have broken toys — when lawn darts, classic thing, when people getting hurt and killed — I think they got killed with lawn darts, but they definitely got hurt — those got taken off the shelves. When the — what was it? The Ford Pinto. You have cars that break, they take them off the market. When you have drugs that kill people, take them off the market, even one person. With this, you have people killing themselves and others because they use ChatGPT. No, don't regulate that. If we regulate that, the Chinese will have people get AI psychosis. And I just want to be clear about something. There is a deep vein of xenophobia here. There is a genuine xenophobia towards Chinese people that's happening. I find it disgusting because look, you can look at China as an adversary. You can look at them as an enemy of your country, whatever. But this automatic assumption that Chinese engineers all have malign intent is disgusting. And also, I don't know, why don't you go and visit a tech company sometime and see if any Chinese people work there. And if your reaction is, 'They could be with the CCP,' you are a racist. Like, that's what you — you're just xenophobic if you look at every Chinese person as an adversary. So yeah, I don't know, what a surprise that this administration has that perspective. But don't make me laugh about AI safety. No one cares about that. If they did, they'd do anything.

Yeah. Well, that was Altman saying it was ill-timed to go public due to safety versus how bad their economics look. I think that this is a narrative that escaped because there is a group of people called the rationalists and the effective altruists. They're led by people like Elie Yudkowsky, Nate Soares, and all these others. The term rationalism does not mean thinking rationally. It means the rational choice in your belief system is that we will create a superintelligence and it will kill us. It's not about being rational. It's about the rational belief that superintelligence is inevitable. These people do not necessarily have a scientific background. They are philosophers who post on a forum called LessWrong. They're posters. You say — people say, 'Oh, Ed Zitron, he doesn't have a PhD.' These people have about as much. They have less in some cases. One of them, Eliezer Yudkowsky, wrote a 600,000-word Harry Potter fanfiction which names Dario Amodei and I believe his sister Daniela in it as well. These people are poisoning the AI industry and especially Anthropic. So, Jacob Coxon, when he joined from OpenAI — at OpenAI, people just walked around. It was like adult summer camp. Yeah, you want to spend a million dollars on a researcher? Who gives a crap? Yeah, go on, Jacob. And they were all probably like, 'Yeah, we're going to make a bunch of money. We're going to make software.' With some people were like, 'And also the computer is going to kill us any moment.' When he went to Anthropic, I think everyone was like that. Everyone was like, 'Hey, welcome to Anthropic. We're in the murder business. We are, I think, somewhere between a year and three years from destroying humanity.' And Jacob, with his perfectly smooth brain, his perfect — just thought, no thought in his head, just the last thing he was told. He's like, none of these people will say what they're scared of, but they're so scared. Yeah, it's a religious cult. That's how — and Cal Newport made this point beautifully. That's how you have to view all of this discussion, especially on the Anthropic side and with MER and organizations like that. So the Anthropic side, I think there is a degree of, yeah, we have this religious cult that believes AI will inevitably do this. That's why they speak in such big terms. That's why they're like, this is going to destroy us. That alone is insane. It's like if Scientology ran Anthropic, people would be freaking out. But the rationalists, who are far more honest about how they feel than Scientology. Then I think that — I don't know. I feel like people need to know that at their core. In OpenAI's case, I think they saw an opportunity.

I think they saw an opportunity. Altman was doing an interview with Fortune that night and he was like, 'Well, safety is why we're delaying our IPO.' And then the other day — a day or two ago — Mr. Altman and Co., they're apparently trying to raise a $1.5 trillion valuation, but the New York Times' report was not clear enough on this. I just want to be clear. Well, they said it in there, but it was not the top line. What's actually happened is not — so some investors came to OpenAI and said, 'We'd like to invest at $1.2 trillion,' which is ridiculous, like $400 billion more than their last one, and OpenAI said, 'No, we want $1.5 trillion.' I think this would be the largest private round in history. It would also mean that anyone in this round is probably getting washed out because Anthropic wants a $2 trillion market cap. Will they get it? Because you have to remember OpenAI and Anthropic do not have assets. They don't own their infrastructure. They don't have credit. They think they have some revolving lines of credit, but they are in hell junk. Though they're going to try and get investment grade, but the point is these companies are not really valuable unless you believe the growth story. And if you don't believe the growth story, they are just a cash-burning machine with an LLM attached. They have some IP, they have some goodwill, but in the end they are just fronts for software. They don't own data centers. They have people, but they don't even own office space. So the idea of a $1.5 trillion valuation for OpenAI is insane. But also, I think that Altman realizes that this is a really good way of explaining away the delay. I think we've heard — well, we've heard CFO Sarah Friar say, 'Yeah, safety is real important.' Again, not specific at all. So, I think this is a convenient narrative, but slowed down doesn't really make sense with Nvidia doubling their GPU sales next year. I don't know how that happens. I don't know how that's possible. And the answer is it isn't, and it isn't going to change. The thing that will change there is if the money isn't available, but that's a different conversation. So, I think that it's kind of like a good Coen brothers movie where you have all of the different parts of the land and everyone's narratives are coming, and perhaps they all have the same allegiance to AI, but they all have their own cynical beliefs. Some of them are rationalists. Some of them are just rational. Some of them are craven. Some of them truly believe that they're going to destroy society with LLMs. None of them really care about people. And so they're all operating not so much in sync but in concert. So they're saying like, 'Oh, we're all going to do something and we all believe that AI is good.' The funny thing is, Mr. Huang from Nvidia literally a week ago said that they reached AGI. So — but all of this is also a big overstatement and we shouldn't be worried. Which is it, Jensen? It's almost as if Jensen just says stuff. Yes, that's what's happening. So I think you have some people at Anthropic who fully believe that the superintelligence they're building is going to kill everyone, which makes me wonder why they're still working there and not doing what one would do if they thought they were destroying the world. I think that inside Dario Amodei are two Darios. There's the rationalist Dario who's like, 'Well, you know, we're going to destroy the world,' and then there's the other Dario that says, 'I want to be worth a billion dollars.' There's the IPO Dario and the cynical Dario, and they butt heads. I would not be surprised if he doesn't care at all, but I'd also — based on his history with the rationalists — I think that he is likely poisoned by them. I will say there were people who were like, 'Well, Elon Musk has called for safety, too.' And Elon Musk is related to the rationalists and effective altruists. And let me be clear, when you're a multi-billionaire, all sorts of weirdos come to your door. And because you don't experience friction in anything, you as these ultra-rich types need to really understand it's so different to how you and I live. Because you and I, if we don't work, our bills don't get paid. We don't eat. We die. We have no home. We sleep on streets. Elon Musk doesn't have bills anymore. He has a guy who hits buttons. He doesn't think of that. He doesn't worry that he's going to run out of money, run out of food. Doesn't actually worry about anything. And so all of his concerns — instead, because he doesn't enjoy anything, because none of these people do — his concerns become, well, how do I get more money, and also how do I become the biggest brain? How do I leave a legacy? And so of course he talked to some of these superintelligence idiots. Of course he did. And I think the way to evaluate all of this — and I'm paraphrasing Cal Newport here — is to see everything through the lens of the rationalist. Anyone who is telling you our software or AI in general is going to kill everyone, ask them to prove it. Don't accept an anecdote. Don't accept them pointing at the Hugging Face attack and going, 'They worked. They were on their own accord. They did this.' LLMs being prompted by other LLMs did LLM stuff. Much more complex. They've definitely progressed there. Congratulations, guys. Only took you hundreds of billions of dollars to get somewhere towards crime. But there's a vast difference between that and Jacob Coxon saying they acted on their own accord. So, be very cynical of this stuff. To be clear — I really want to be clear, though — it is dangerous. Attaching hundreds of billions of dollars of infrastructure to unlimited money to be used with whatever unreleased coding agent you have is dangerous. It doesn't mean that LLMs have to be this exceptional crazy thing that's super intelligent, but it means understanding that these companies are acting recklessly and regulating them as such.