In confidence. Sax told me he's a little broken up about the empty nest kind of thing. And by empty nest, I mean his bulldog Moose is coming to live on the ranch. Has the moose landed?
The moose is landing next week. We're giving him a 3-month trial. If he's on his best behavior, he stays. He's going to be great. He's so well trained.
What's going on? Moose is our bulldog. Yeah. And Moose is loose.
He needs the space. He needs a ranch. He's a very athletic bulldog. This dog is not a couch potato. He's very active and he just needs the space to run around.
Why don't you just buy a ranch and give him his own ranch?
Why buy a ranch when a bestie has one already? A bestie who loves bulldogs. But no, he's a spectacular specimen of a bulldog. I mean, this is a stunning bulldog.
Yeah, he's a beautiful dog. Great bulldog. This is a trial adoption, a Calacanis adoption cycle.
Oh my god, this is sad. You know these bulldogs resist training like Jason, you know, they are the most stoic, loyal, but stubborn individuals. They will overweight, stoic, stubborn, overweight.
This is a spelt dog. How dare you? How dare you? Moose is a specimen.
In terms of personality, this is founder market fit. I mean this is like owner pet fit. Founder bulldog fit. So what Moose would do is he would jump on the couch and just kind of lay there. So we'd say, 'Moose, no, go down.' So we put him down. He jumps right back up. We take him down. Jumps right back up. I mean, he can go on all night doing this. So finally, you just give in and let him sit there. It's literally J Couch. It's literally them. Now it's his couch. Now it's his couch. It's literally J.
Rainman David and it said we open source it to the fans and they've just gone crazy with it. Love you.
All right everybody, welcome back to the number one podcast in the world. We got a banger episode for you today. All the topics that you want to hear about in the world. And with us, our amazing crew, our chairman, dictator from Pio Hapatia, enjoying the final moments of summer. How you doing there, brother?
Well, I come back in a couple of days.
You get a little melancholy there, brother? You a little sad?
I mean, I think everybody should have a two to three week vacation in August. I think it should be mandatory around the world. I think the world should stop and we should all go on vacation.
Yeah. Well, you proved that to the audience by not showing up for the show for a couple of weeks.
Yeah. It's the best time to go on vacation. I agree. I just literally got back from a little... Take your ass off through the end of July. Take two weeks off with your family. It's nothing better. I just did a little visit down to Mexico City, which was absolutely delightful. Rome is amazing. Have you been there before, Jim?
No, but I've heard it's incredible. I heard the food scene in Mexico City is bananas.
It was bananas. And the startup scene there is amazing. It's kind of like if Spain and Williamsburg, Brooklyn had a baby, that would be Mexico City. It's super hip, super fun. Lots of young people, lots of energy.
Hey, speaking of young people with lots of energy. Hey, Freeberg's back with us in his cell working throughout August to get incredible things done like our AI summit and the All-In Summit. Freeberg, you gave up your summer once again for the all-in crew and for the audience. How do you feel? You resentful or you feel good about your decisions?
I live for the work, Jal.
You do. How's Aho doing? It seems like you're doing a lot of work there. You got your head down. Just give us a little update on the company.
I appreciate BS. There's a lot going on. We're very busy. I actually had a really cool test tube here that we had a customer visit today. I'll go find it. The world's first true potato seed. And we are ramping up production. Lot going on. It's pretty exciting.
Yeah, sounds thrilling. Saxs, any thoughts there on potatoes and just how thrilling this exercise has been?
What are we talking about?
We're just talking about... Oh, Freeberg had a thrilling summer. He's got a new potato in a test tube that he's working on.
Wait, did we invest? Is that in... What's going on with that?
Yeah, you're a shareholder. I'm in.
Okay. All right. Well, okay. I care about you in... No, you did not me an investment. I didn't get... Wait, is this a sore subject?
No, it's a sore subject for me. When this thing becomes worth 10 billion and I didn't get to slide in a little 250 slice. Not even a slice, huh? Free bird. Look guys, a slice for your friends. They want value added investors. Jal, what's your value ad?
How about spreading disinformation on this podcast?
All right. Well, since I'm not on the cap table, good luck with your mutant potatoes. Disinformation. Mutant potato.
Oh, we'll get through it, folks. We'll get through it.
Hey, Zach, you and I did a little interview with a certain senator. Give us an overview.
Yeah, we did a great interview with Senator Eric Schmidt from the state of Missouri who previously was attorney general and wrote a book about his experience as attorney general. He brought the lawsuit Biden v Missouri which exposed the Biden administration's censorship apparatus that they ran through social media companies. This was the government's side of the Twitter files and the discovery he brought when married with the Twitter files that Elon released really paint a picture of how the government was in bed with social networks to basically censor Americans' free speech rights in favor of their point of view on COVID and many other issues. So great conversation.
Yeah, it was spicy and it's everywhere where you can find allin. But we got this big event coming up. The summit is coming folks. Here we go for the fourth time in just a few weeks. September 7th to the 9th. I think the 8th and 9th are the main days. Man, this lineup is stacked. Alex Karp, Vlad from Robin Hood, my guy, Energy Secretary Chris Wright and I are going to have a great discussion part two. Wow. Uber CEO hitting an all-time high. Circle CEO, just an incredible lineup here. And Freeberg, it looks like we've got some great partners.
Yeah. First of all, the event is going to be very expensive to put on this year. We actually have in addition to the great speakers, we have Diplo headlining one of the main parties, the Diplo. Yeah. Gary Richards and Dylan Francis. And so we have a huge set of blowout parties every night. It's going to be awesome to put all this on. They pay for the event. We want to thank the sponsors. So Solana, big shout out. They were a sponsor last year and they're stepping up as a big sponsor this year. Big shout out to our friends at Solana. They're amazing. They support everything we do. They're doing over 100 million transactions a day. It's unbelievable what they're getting done over there. Great team. And they're going to do a full open bar for us Sunday. Juice bar, coffee, the works. Classy team over there. And we don't just have like these typical sponsorships of events. What we do is we ask our partners, hey, make an activation, make a place for the besties and for the fans to hang. And they've all created these amazing experiences. And there'll be matchas and espressos and craft beers and sushi and all kinds of great stuff. And to our friends at OKX, they're going to be coming back. They're building an onstage. Got a matcha bar coming. All these great things by our partners. Google Cloud's back offering 350K in credits to Google Cloud for startups. What mentions they are over there. Google's crazy. Look at this, man. They're putting out a mini campus at the summit and yeah, 350K in cloud credits. Athletic Brewing's back. They're doing the scholarships again. We have a couple of scholarship slots left. Please don't apply for the scholarships if you have means to buy a regular ticket. But if you do need a scholarship, please do apply if you're an up-and-comer. Allin.com/summit or allin.com/ yada yada yada.
Did you guys see this? What's going on now? Apparently, I was in the new South Park episode. The whole thing was around the sick offensity of AI. That's you.
Do I say anything in the show or is it just I'm just in the background?
No. They had a line of people giving tributaries to the president, you know, based on the Apple gift that Tim Cook gave to our amazing president. It's a funny episode. You know what I like about the administration? Like JD actually retweeted all this stuff and you know had it's good fun. We get to mock our civil servants.
That's the vice president to you.
That's what I said. My guy JD. What did I say? Suddenly on a first name basis. My guy JD. Hold on a second. Let me just say hi to JD here and I'll just get his feedback. I'm going to invite him to the Diplo party. Hold on. Your biggest nightmare is here. Me and Chris Wright also talking directly now. I'm directly plugged in.
Oh yes. Chris, right? I'm bringing him a special gift to the summit. You're going to love it.
All right. We got to talk about AI. AI mainet hit a bit of a detour this week. All over the, you know, three or four days, AI stocks were down across the board because of this MIT study that went viral as well as Sam Altman's comments about a bubble and Zuck instituting a hiring freeze in AI after going on a complete blitzkrieg. So, let's get into it. Act one, Monday, Fortune dug up a generative AI study that MIT published last week or last month, I should say. In that study, MIT found that 95% of Gen AI pilots are failing to make it to production because of employee resistance, poor quality output, and the most interesting problem seems to be resource misallocation. According to the study, 70% of Gen AI budgets are going towards things like building sales and marketing tools, which have poor ROI. The highest ROI was found in back office optimization like automating tasks that cut back spends on various departments. Basically these pilots aren't working. Chamath, this is what the study found. They evaluated 300 AI implementations and interviewed 150 leaders across 52 companies. You've been grinding it out with your own software company now called 8090. Does this align with what you're seeing on the field?
I think what I would tell you is that I think the first wave was just a lot of boards who read the words AI somewhere in an article and then went to a board meeting and turned to the CEO and said, 'What's your AI strategy?' And then the CEO turns around and sends that down into their or eventually hits the CTO's desk. And I think the first wave is mostly people just spending money because they had large existing budgets. And so they were like, let's just go and try a bunch of different things. And I think now we're going through the sorting function of realizing that there's a big difference between probabilistic software and deterministic software. That's probably the biggest reason why you're seeing so many failure modes in sales and marketing. It's very hard to codify sales and marketing into a set of heuristics that never change. But back office processes, why they're so good and a great target for AI is ultimately you have so many people that have been hired to deal with edge cases, right? I think that's what people do in most companies is they're in charge of a process and they're dealing with edge cases. And I think that you can get extremely high rates of accuracy if you implement AI correctly in back office tasks. I think the real question is like what happens to all of this revenue that has been generated. You're seeing companies generating $50 million of ARR in a matter of months and then raising huge rounds. I think what we haven't seen is whether there'll be any sort of either logo churn or dollar churn as new companies come in with even cheaper solutions, the foundational models move up the stack and just absorb capability, or things just don't work and they get abandoned. All of that churn happened in social. I remember when I was in the middle of helping build Facebook, we went through that whole cycle. There was seven or 8,000 social companies and within six years there was five of us left. It happened in SaaS when I was investing in SaaS. There's a couple of very early and important successes like Yammer which Sax started which I was very lucky to be an investor of, but then it took many years for the handful of winners to get really sorted out and I suspect we're about to go through that same cycle in AI. So I think that article basically paints a very accurate picture. There's been a lot of trying and experimentation. We now need to go through a sorting and a cleansing and then we'll rebuild from first principles around the... and not surprising to our Sultan of SaaS David Sachs because the sales and marketing departments, they're very promiscuous when it comes to new tools. Getting a great lead, closing a sale, you can directly connect it. So we always see them test stuff out. Doesn't surprise me that we'd see sales and marketing go after this first. But what do you think about the brittleness of this revenue, the churn, Sachs? Are we going to see a lot of these companies rocket up to 100 and come back down to 50? Is this something you're seeing or your firm, which I don't know your status at the firm? Maybe you could tell us how that's working out in terms of your intelligence there, but what is Craft seeing on the field there?
I think we're seeing a lot of interesting AI applications being developed, but it's still very early days. And I think that over the past week or so, there was a correction in sentiment towards AI, but I think it was a healthy correction. I don't think this was the beginning of a bust cycle or something like that. I still think that we're in a boom. I still think we're in an investment super cycle, but I think there was a healthy dose of skepticism applied to some of the more fantastical claims that have been made about AI. And I think this is why you saw there was like roughly what, like a 10% correction in public AI stocks. And there was that MIT report that said that 95% of projects and companies are not making it to production yet and so forth and so on. So I feel like we're getting in the weeds a little bit here and what we should be talking about is just sort of where we are in this AI super cycle.
And where do you perceive us at?
We're in the experimentation phase. We're in the pilot phase.
But this issue around probabilistic versus deterministic makes it hard to trust this software. Is that what you think the key issue is?
Well, let me tell you why I think that this correction is actually healthy is that after ChatGPT launched at the end of 2022 and then throughout 2023, the dominant narrative in AI was that AGI was just two to three years away. And everyone kind of had their own definition of AGI, but it was kind of this idea of smarter than human super intelligence and kind of magic AI. AI would be able to do everything. And as a result of that, you kind of had both utopian and dystopian narratives really proliferated. And so, you know, you started getting this like job loss narrative that within a few years, 50% of knowledge workers would be out of jobs. You got this rapid takeoff narrative that basically the leading AI models would be able to turn their intelligence towards improving themselves towards recursive self-improvement and therefore within a couple of years the leading models would basically achieve super intelligence and leave everyone else in the dust and then capture all the value of humanity. And then based on that narrative, which again it was the same underlying narrative that fueled both utopian and dystopian or doomer takes on AI, you got I think a huge backlash which has already been forming where you have a thousand bills running through state legislatures right now and you have all this AI safety legislation. You got bills like in California the SB 1047 which would have applied tremendous amount of new regulation to AI. So you saw this policy backlash happen as well. And it was all based on these fantastical and kind of magic views of what AI was going to do in just the next two to three years. And I think that the reason why this recent skepticism is healthy is because I think it's rebutting all of that. And it's showing that, you know, AI is a powerful tool. I mean, I definitely think it's a new and important form of computing and it is going to unlock tremendous value in the economy, but it's going to take us a while to get there. I mean, you can't just tell the AI, you know, be a sales rep, be a customer service rep, and kind of throw it over the wall and expect that it's going to replace a human. It takes a lot of prompting and iteration and validation to make the AI work, to make it generate business value. And if we were on a path towards rapid takeoff, then what you would see is that the leading AI models would be increasing the distance between the top one or two models would be increasing the distance between the rest of the models. And instead, what we're seeing is a clustering of model performance around the same performance bench. Incremental, right? They're incrementally the progress to be a little bit more incremental. It's more evolutionary rather than revolutionary. And I think this really crystallized around the launch of ChatGPT-4o where a lot of people were expecting GPT-5 to be this huge breakthrough. Sam Altman was sort of teasing this concept by posting photos of the Death Star that the idea that this model was going to blow everybody else away and the reviews ended up being very mixed and then we saw that on the performance evaluations. It's not that the model didn't represent progress, it just fell short of these lofty expectations that have been created. So Freeberg, let me get you in on this just to... sorry I've been kind of long-winded here, but just let me just sum this up, which is please... I think that what people can now see is that we're not in a loop of recursive self-improvement. We're seeing that there are a handful of great model companies, but the development of this technology is going to be a more normal technology race. It's not like the leading players just all of a sudden achieve AGI very quickly. And as a result of that, I think because it is a more normal technology race, I think we can apply a more normal logic to it from both an investment and a policy standpoint. And I think that a lot of the narratives that were hyped up about imminent doom or imminent utopia, depending on what side you were on, were just massively overhyped. And this is why I think it's just a very healthy thing.
Revenue. I mean, they have actual revenue because they have subscriptions and they appear to have the dominant position in the consumer space and it is a replacement for a lot of people for search, which is the most lucrative franchise on the internet. And this is one of their applications. So I actually think that you can make that case pretty comfortably. But can we could we go back to a point you were making before, Jason, about in the survey that the attempts to just apply some sort of generalized AI model didn't work very well like 95% of the time it didn't succeed in these large enterprises. But if they used a more specific vertical application or vertical model or an SLM approach which is a smaller specialized model then it showed much greater success. I mean that makes a lot of sense to me is that in order to drive business value there's a lot of what I would call last mile problems, right? LLMs need context and so you have to first of all connect to all of your enterprise data sources and you have to prompt them in a very detailed way in order to get to a good answer and then you have to validate that answer to make sure it's not a hallucination and then you need to iterate on it. And so this idea that you're just going to have one super intelligence that just figures all this stuff out, it's just not the way it's playing out in the real world. You're seeing again a lot of very specific business problems that have to be solved. But I think that this is a great thing ultimately for the ecosystem because it implies that you're going to get lots of vertical applications and lots of specialized models that capture value in lots of different markets. And that's actually the way that we're going to drive this throughout the economy as opposed to it just being one foundation model eating all the value. So I think this is a very healthy thing for the ecosystem.
Yeah, it makes total sense that the vertical systems would feel more deterministic because they have a tighter problem set, a tighter data set to actually come to an answer. Whereas the probabilistic, just asking an LLM to come up with a business plan for you, it just feels like it could be 80% correct, 90% correct as opposed to 99% correct, which the vertical ones are actually getting very good at getting the correct answer.
