I witnessed almost daily people that are either in government or even friends of ours who say we have to win the AI war with China, and I don't know what that means. I can't imagine an instance where we control all the AI and they don't have any — like, it's already too late. It's too late and they're smart. The reality is that we just need to focus on running our fastest race. We need the Teslas, we need the OpenAIs, we need rockets that land themselves, we need all of this. But to think that they're not going to have BYD building great cars, or they're not going to have DeepSeek building great models, or they're not going to have rocket companies that copy us and can land themselves — that would be naive. It's remarkably naive.
Bill, it's good to be with you. Good to see you. I mean, we're in this — wait, should we tell them that Steve Ballmer gave us this man cave? It's a private cave. The truth of the matter is the hardest thing about this pod — I love this pod — but you and I getting our schedules to match and actually getting together. And so I've gotten a ton of feedback. I've seen on Twitter people like, 'When are you guys going to record the pod?' First, thank you for the audience encouraging us to do this because I love doing it. Know that we would love to do it more, it's just a little challenging to get together and to do it. We have an ongoing dialogue, I think pretty much 24 by 7, correct, about this stuff going on in the world, and then occasionally we get to get together and share it with you all.
I thought maybe today, Bill, we'd kick it off with Grok 3. So we're now like 10 days out since Elon and his team unveiled, in pretty record time, an unbelievable model. And so maybe you can just help us zero-base where you thought — what you thought when the model came out, where it stands in the rankings, then we have a conversation about the impact and what it means.
You know, we've talked about this in the past, but everyone in the ecosystem was super impressed with how quickly they built the Memphis facility, exactly, and how big it was. And it was the largest contiguous cluster in the world. And there was a lot of chatter about that ahead of time. And I can remember some of the investors there saying, 'This will prove that pre-training still has headroom, because this will be the biggest cluster ever trained on.' Correct. And you can decide what your expectation was after that.
That kind of laid in the things that happened — I mean, I think the generic way of saying it is like it went right up near the top of all the benchmarks, right? On some, not on others. Some people argued about whether the reasoning component, or did they cheat or overtune to a benchmark. But I don't think it matters. I think the biggest positive takeaway is there's a new player in the model market. And we had often — a lot of people said this — a sport of Kings, there's only going to be so many players. There's a new one in the market that invested what they needed to do, that has access to capital, has a data asset that they argue is important and special, and was able to get at the front of the race, let's just call it that.
Right. And you're looking at — we're looking at this artificial analysis that just shows there's this clustering here in the upper right. DeepSeek kind of got up there a couple weeks before you had Grok. What's interesting is they all seem to be coalescing in an impressive way around the top of these benchmarks. When we say 'they all,' I mean we're really only talking about five or six players who have a chance to be in this game at this point.
Correct, correct. And I saw people who interpreted Grok's fast rise as proof that pre-training still got legs, and to me I kind of had the opposite reaction, which is I felt like they just slammed up against the ceiling that's holding everyone in, right? Although, again, an incredibly capable level.
No doubt, I'm not — I just, you know, I've said this for a while, I've been concerned that the way an LLM works and the way it's optimized, that building bigger clusters and more parameters won't buy you much. And whether I said it or not, Ilya said it, Andrej said it, other people have said the same thing. And so to me this reinforced that point. I was expecting, if there were pre-training headroom, I was expecting this to go through. Now I will qualify — this was their first run. Maybe there were some tricks they didn't know. They could very well back up and do another run on that same large cluster and maybe shoot past these people, or maybe these benchmarks aren't the exact right thing to be looking at.
Well, I would say a couple other things. Number one, it's not just a pre-trained model. They also have an inference-time reasoning component to the model that's incredibly capable. We have this benchmark chart that I tweeted the other day, and I compared it to kind of the search index benchmarks that we all used to track. The benchmarks are one thing, but the reality is, how do we feel when we're using the product? And what I will say is Grok 3 rocketed to the top of all app downloads on the iPhone charts. At least my Twitter thread was full of people having great experiences, showing those great experiences on Grok 3. So it had a personality and an interaction with people that I think people were enjoying.
Number one, it just has to be capable enough, right? And it clearly crossed the threshold of being capable enough. Now the real question shifts to: can they leverage the X platform, which reaches a massive and important audience, to really drive that? And what I would say the early indications — to me when you compare it, for example, to how Meta has used Meta AI, like as incredible as I think Zuckerberg and Meta are and the advancements they've made, I have not particularly been impressed by the productization of Meta AI. It's basically just a search box stuck at the top of Instagram or stuck in my WhatsApp thread, and when I'm on it I never intend to be there, it's not direct content. Whereas on X, they figured out — the first thing they did is they put that button at the bottom of the app that clearly distinguishes it as its own standalone application. They launched a standalone application, they're using X to drive those app downloads, and now I just opened up my XA today and it said, 'Hey, go out and try the new voice for Grok 3.' So to me, the execution on the product side to drive consumer use has been pretty damn impressive, and took them to the top of the charts.
And only DeepSeek and Grok, of all the others, have shown the ability to break into the top 10 on the App Store download.
Exactly. And so I think while we all have a fascination with where did they get to on the benchmarks, my own sense at this point in time is this is going to be one of these battles kind of like search was, where you have the five or six players. And it's going to be — just out today literally as we're about ready to go on, OpenAI has released ChatGPT 4.5, and they've kind of hinted in this presentation as to ChatGPT 5 or 6, I guess that was shown on a screen. And if you look at 4.5, one of the important distinguishing elements that they're pitching is it's more human-like, it gives better answers, more concise answers, etc. Not a big breakthrough on evals, although there are some improvements in the early looks against the evals. But I think ultimately we're going to measure the success of these things by how many people are using it.
Well, let's — I think one thing to be good for the audience, I know you've said it in the past, but you're an investor in OpenAI, and I think you have a theory about their prowess in the consumer market and their lead in the consumer market. So why don't you reiterate that.
Yeah, well, I mean, I've showed this chart before, right, that in the search wars we had Google and Yahoo and AltaVista and Lycos and Ask Jeeves and Excite and Infoseek, and by the way they all did pretty damn good on the benchmarks, right? But the reality is that didn't get them to any value creation, because ultimately all the consumers aggregated around Google. So the real question is, does that same pattern play out of win-or-take-most in consumer around AI? It did in search and it did in social, but it's not necessarily a follow-on that it will in AI, because I'll stipulate — X has an incredible installed base that they can market into, Meta has an incredible installed base, Google has, and it's existential for those companies in order to market to those consumers. So I don't think it's going to be win-or-take as much. I don't think we're going to see a 99% monopoly here, but I do expect that we're going to see 70 or 80% share go to the winner.
Now if we look at the numbers — yeah, the numbers — if we look at the numbers today, I think last week Sarah reported that OpenAI has crossed 400 million weekly average users, and that that's a user number, not a paid user. The number of paid users is a fraction of that. I think they also reported last week something like 11 or 12 billion dollars in expected revenue this year. So you can reverse-engineer your way into kind of what percentage are paying for that. But more importantly, I think the number — monthly average users — must be somewhere in the order of magnitude of 700 to 800 million monthly average users. And there, you and I followed consumer for a long time, there's this magic number around a billion. I mean, I already think they're near at escape velocity, but at a billion monthlies you can funnel all of those folks into weeklies and then you funnel the weeklies into paying subscribers or people who are consuming advertising. So what I have seen is everybody else catch up on the benchmarks. What I have not seen is people catch up on the consumer velocity.
Let's handicap some of the other players a bit. Who do you think is closest from a user standpoint? Is it — probably you'd have to count the Gemini searches in the Google Search, right, to get to a number that's close to OpenAI.
Yeah, I mean, listen, I think — let's just start with Google, okay? So there's been a lot of reports out over the course of the last couple weeks, we have public companies now reporting that are reporting their Google organic clicks are down 20 to 40% year to date. So the question is why — why are Google's clicks down so much? Because if I do a Google search today on my phone, half of the page is taken up with an AI answer to whatever my Google query is, and the rest are all paid links.
Yeah. So I think Google — I think that's the right decision for them to make. If you want to compete, you ultimately have to be willing to take the innovator's dilemma head on and really just cannibalize your product with AI. Now if they do that, and if you're one of these humans that thinks SEO wasn't dead already — which I would have declared it dead a while ago — yes, it's really dead. SEO is dead. And just — those are the free links, that was the core product that used to attract everybody to Google, and the idea that SEO is basically now gone is pretty — I think — and this is just a side, but I've been remarkably frustrated with Google's organic links for the past five years, because you go in and search for your favorite team's schedule, right, and all the ticket guys are up front now. The link you're looking for, you have to hunt.
Well, okay, so let's talk about that for a second. I mean, now the obscure link or the obscure information that you and I may be looking for may be on page three, four, or five. You and I are never going to get to page three, four, or five. What's so interesting, for example, about OpenAI's deep research — now if I launch a query using deep research, it will go to page four or five or 10 or 100 and find those obscure pieces of information. So I think the evolution of Google actually provides acceleration to the deep research projects, because I don't want to go do that deep research.
So Google, I think you just can't discount their installed base. The number of people going there, inertia will continue to carry them there. But I would say this, and you can go search for this on Twitter or anywhere else, and I know certainly with my own behavior, the amount of activity that I used to do on Google has been 80% cannibalized by ChatGPT, because there's search embedded within ChatGPT. And so I'm getting all of that information, all of those answers. So I think that they're going to be formidable. I think they're being bolder than they've been, but I think they'll have to continue to do that.
I think we've talked about this, but I think some of their assets are remarkable. I mean, you've got the YouTube data set and all the search queries over all the years. Their understanding of structured data around a lot of the consumer verticals — I mean, they built that out in airlines and things. They should be able to do those agent-type queries better, faster. Should — their velocity on product has not been impressive. Velocity on consumer has not been impressive. So they have them, they've had them for a long time. Bill, they had ChatGPT before ChatGPT. They also have Android, which is a massive asset, and they also have a browser. Both Perplexity and OpenAI have started toying with the idea of either having a browser, or in the operator case, of using a browser in the cloud to go do this work. Anyway, they have so much. I still think they have a bit of the innovator's dilemma, in that they can't — they still have to try and maintain those paid links on that page. And this chart here, the black line is Google's paid click growth plotted against the weekly average user growth at OpenAI — it's not going in the right direction.
Well, and I think it benefits from the fact that informational searches are what ChatGPT cannibalized first, not the commerce searches, which is where most of the money is on the paid link. Right. Although, again, we're going to see this out of X, we're going to see it out of everybody — the entire domain of the internet is the domain of agents. So if you think about operator as one of the first agents rolled out by OpenAI, what does operator do? It goes and it mimics me as a human going out and researching a hotel and booking a hotel or whatever on the internet. And we're in very embryonic state, I agree with you, we're not there yet, but it's very clear what the roadmap is going to be. It's going to want your credentials, and whether you give it your credentials or not is going to matter because it's searching against an —
So let's talk about Meta.
You know, I actually — one last thing on Google. Like, there was a point where Meta went public at 40, we had some trick, we had some — so I was paying attention. And Zuck, as he has many times, got woken up on mile 100%. And everyone thought he was dead, because of mobile, and he had also built an HTML5 — he didn't believe in native app. There's a whole thing that they weren't going to be able to monetize mobile. He was on the cover of Barron's magazine, right, 'The Week Magazine' — it was like Meta's dead, or Facebook's dead. But he woke up, yes, and fixed it. Can Google do that here? Is that possible? Can they have a similar — and what would it look like, and what would it take?
Well, I mean, listen, I've said publicly that Google's moat was not a technological moat with search. Their moat was a distribution moat, their moat was a mindshare moat. We googled everything when we wanted to know anything. And the only thing that could attack Google was never anything head-on — it had to be an orthogonal attack from something that was 10x better, 100x better, because it gave us answers instead of blue links. Right. That's why it was such a mortal sin for them to ever, ever allow anybody else to go first, because the only thing that could give you a trillion dollars worth of free mindshare is going first with something that was 100x better. And that's exactly what ChatGPT did at the end of 2022.
All right, go to Meta. So, I mean, again, Meta — if you had to handicap the big guys, three billion users of their product. I think they have products that are tailor-made for chat-oriented AI, whether it's Instagram and having shopping agents and co-shopping agents, or whether it's WhatsApp and just having a bunch of agents live within my WhatsApp channel. It feels natively much better positioned for AI. And we know that Zuckerberg is in complete beast mode. But I am surprised, I have to say, that we're now kind of 18 months into kind of the Llama thing, and it feels like the manifestation of it into the product was slower than I expected in 2024, back to your product point.
Exactly right. Like, Meta hasn't — and I will say, even, you know, we know he was ripped about DeepSeek, right, kind of blindsiding Llama in the release of R1. And so I would say it's not just product for them. I think they have — I heard from several inference players that you and I are friends with that all of a sudden DeepSeek rather than Llama is the enterprise open-source model of choice that everybody's experimenting with and playing with. And so that becomes a real problem for them as well. So I think 2025 is a critical year. I think they will come through. And remember, when it comes to almost all product stuff — stories catching up with Snapchat, or Reels catching up with TikTok — they've always showed up to the party late, but they are grinders and they always deliver the product.
It'll be interesting to see what they do. Who else would be in the list? I mean, Anthropic has really not been — they pretty much ceded the game on consumer. You know, there was a product announcement yesterday about they're going to be powering Alexa. But, you know, now we're stretching. And Amazon did do a big Alexa launch yesterday, correct? Pretty late in this game.
Yeah. And by the way, Alexa is not really — again, it occupies a different space in most consumer minds. It is not what ChatGPT does. And so I think to dislodge something that has the momentum ChatGPT does, you have to go at them and do better than what they do at the thing that they do. And this is why I think X — if I go through the whole list here — X to me is so interesting, because they have a platform that is the number one news platform in every country on the planet. The people who are most actively engaged are using this platform, and they go there for information. They go there for answers, they go there to engage. So I think it's an audience that's very well suited for AI.
I think the integration they've done is as good. By the way, they've done this in a very short period of time. I mean, I'm talking everything from the logo — some of the tweets will have the logo pop up and then it'll summarize or do more research. So I'm really, really impressed at the velocity of not only catching up on the benchmark, but catching up on the consumer product side. And so, listen, they're number one on the App Store, and that stands for something. Coming out of nowhere, and people said that Elon couldn't do this — I never doubted that they would catch up on the benchmarks if they got a big enough cluster, because Elon set a mission that people become messianic about, and his engineering capability to build out the cluster and do all those things — so that was never the question for me. The real question was, can he close the gap on the consumer race?
Yeah. Okay. And there, I think, the odds-on favorite there has to be OpenAI. I think they continue to widen their gap. By the way, Bill, I think they're accelerating at scale. But that's where the race is. And it may very well be that coming in second place with 20% share is a pretty good place to be.
I want to mention one more company, and then I'm going to make a guess at four ways someone could try and win this game. But the one I want to mention before I do that is just Perplexity, briefly. I will give them credit for being product-centric, to your point, and innovative in ways that the others haven't. Yeah. And kind of on their own terms. They don't have near the usage of OpenAI, so there's a question that it kind of looks like an acquisition candidate to me. I don't know if anyone can agree on price, but for one of these other players that hasn't been as successful from a product standpoint — I mean, you could — and you and I just had this conversation — you could imagine, for example, a world in which Microsoft were to buy Perplexity and now they have a consumer brand to go battle it out. We know how much Satya wants to win in consumer. Now he owns a bunch of OpenAI, and so he's got some potential channel conflict there. But I think the bigger issue at this point — now, we know with Lina out, you're probably more likely to be able to do a deal like that. But when founders are raising at 89 billion, it becomes — you take it off the table. It becomes a much, much more difficult decision for a company like Microsoft. Not saying that it couldn't happen.
And I will say this, that when it comes to punching up, being innovative, being scrappy, product velocity — the founder there, Arvind, and the team — it's been super impressive to watch. I think they made other people better. But the numbers, as we look at them today, they're really powerful but a much, much smaller player. So here we go. I have four things I'm watching out for that could potentially lead to either further lock-in by OpenAI or a window for someone to do something else. And some of them I mentioned before, but I think memory is still this thing that could just tie you to something.
And OpenAI has probably done more with memory than anyone else, but no one's really got to the place where I'm telling it to remember things, to store things, to create lists — where it starts to become like an executive assistant for you. And I haven't seen that yet. I still think that's a dimension that could really be important. Voice, we've talked about, and they're all playing with it. I think voice also ties in with device type, and this is where Alexa may have some assets. But if the voice were spectacular, I might not have to carry the phone around as much.
And by the way — yeah, you need an earbud. You need an earbud.
I will say, advanced voice mode on ChatGPT is excellent, Grok 3's new voice, excellent, and they're getting better at an accelerating rate. We're an investor in this company LiveKit that's powering a lot of this voice, and I will tell you what I see in the product pipeline is super impressive as to what's coming with voice. The third one's nebulous, but just someone could focus on a feature that no one has to date. And right now the game looks so much like, with the benchmarks and voice, like everyone's running at the same place. So that's an easy thing to say, but it'd have to be really out of the box.
And then fourth, I've just been thinking about this — no one's really thought about a network effect. And I wonder how you could make the quality of the AI experience a function of your user base. And let me give you an example of a network effect I think that is happening around model improvement. If you have 700 or 800 million monthly average users, your diversity of information in questions and answers and follow-ons is much, much higher. That data is now being fed back into the models to improve the model. And some users may have seen — I know I have — you get a pro, you get two answers, and OpenAI asks you to pick. So I think that's an example of OpenAI very actively building, attempting to build network effects in terms of the quality of the model, the quality of the answers.
But there could be a more intense form of network effect if you found a way to leverage the user base as part of the value proposition. Let me go back to your first one — memory. Because you and I have talked about this a lot. If you get memory, the switching costs explode. Exactly right. And I would argue not only switching costs explode, the conversion rate from free to paid probably also goes up, just because the value delivered.
And so I was with my 89-year-old mother last Sunday, and my mom has wanted to write a story of her life for a long time. And the reality is she's never going to sit down and write the story of her life. And yet when I'm with her, podcast style, I'll ask her questions and I'll just record it on my phone so that I have it and I could perhaps go back later. And then I started thinking about it, and I said I don't need to be the interviewer, right? Advanced voice mode could be the interviewer. And so I was sitting there with her last weekend — here's the prompt that I gave to advanced voice mode: 'I'm sitting with my 89-year-old mother tonight who wants to write her life story. I want you to interview her about her life, to ask questions about her childhood, stories, having kids, working, growing up in the depression, her love of computers and travel. Remember everything you talk about, and then compose a story of her life that her grandchildren would like to read.' And then advanced voice mode just started asking her questions. How long did it go on? I mean, my mom was really nervous at the start, but then like a little tear wells in my mom's eye, because she realizes all of a sudden that, oh my God, this could be a massive unlock. So here's the thing, Bill — advanced voice mode and ChatGPT already has memory. You can already do these things. The problem is the nature of the product — you don't know that it can do those things. So part of the challenge about designing a product where prompt is your way in is you got to help people imagine. Like you and I could have imagined in the age of internet somebody building a website that just did that thing. So I think that's one of the challenges all these companies face, and the innovation around that top end of the funnel in the prompt that can help people better get into it.
I'll give you another example — deep reasoning, which is really fascinating. They basically took the o3 series of models and fine-tuned end-to-end based upon all these browser interactions. But the more specific the prompt, the better the deep research report is going to be. So a lot of people are using o1 to help them build sophisticated prompts that they then feed in to deep reasoning. And so I think there's something in there where we're effectively using AI to get us to the point where we're better prompting. And one of the ways will be very simple — once I have this assistant and I'm having an interaction, I just say to the assistant, 'Hey, my mom wants to tell her life story, I'm not sure how to go about doing that, do you have any ideas?' And she would say, 'Hey, yeah, just use this prompt.' And I also think that there are other assets that could play a role, like a contact database, an email. I mean, I can even imagine moving my email to one that's integrated inside just because contacts is a great one — if they just cleaned up your contacts, knowing your contacts, send an email, send a text. There's a lot. And then for anyone that works with content, there's some app — all my writing and everything I've done for the past 10 years has been in Equipped, but some people use Notion. And that type of repository and how these things can interact — there's just a lot of surface area to figure out.
Well, and like, where does that story land? Where is it stored once you did it? Like, or do you have to take it out of OpenAI? You'd rather just have a place.
