I witnessed almost daily people that are either in government or even friends of ours who say we have to win the AI war with China and I don't know what that means. I can't imagine a state where we control all the AI and they don't have any. It's already too late. It's too late and they're smart. The reality is that we just need to focus on running our fastest race. We need the Teslas, we need the Open AIs, we need rockets that land themselves, we need all of this. But to think that they're not going to have BYD building great cars, or they're not going to have DeepSeek building great models, or they're not going to have rocket companies that copy us and can land themselves, that would be naive. It's remarkably naive.
Bill, it's good to be with you. Good to see you. Should we tell him that Steve Ballmer gave us this man cave? It's a private cave. The truth of the matter is the hardest thing about this pod, I love this pod, but you and I getting our schedules to match and actually getting together. I've gotten a ton of feedback, I've seen on Twitter people like when are you guys going to record the pod? First, thank you for the audience encouraging us to do this because I love doing it. We would love to do it more, it's just a little challenging to get together. We have an ongoing dialogue 24/7 about this stuff going on in the world, and occasionally we get to share it with you all. I thought maybe today, Bill, we'd kick it off with Grok 3. We're now like 10 days out since Elon and his team unveiled in pretty record time an unbelievable model. Maybe you can help us zero-base where you thought when the model came out, where it stands in the rankings, then we have a conversation about the impact and what it means.
We talked about this in the past, but everyone in the ecosystem was super impressed with how quickly they built the Memphis facility, exactly, and how big it was. It was the largest contiguous cluster in the world. There was a lot of chatter about that ahead of time. I can remember some of the investors there saying this will prove that pre-training still has headroom because this will be the biggest cluster ever trained on. You can decide what your expectation was after that. The generic way of saying it is it went right up near the top of all the benchmarks. Ahead on some, not on others. Some people argued about whether the reasoning component or did they cheat or overtune to a benchmark, but I don't think it matters. The biggest positive takeaway is there's a new player in the model market. A lot of people said this is a sport of kings, there's only going to be so many players. There's a new one in the market that invested what they needed to, has access to capital, has a data asset that they argue is important and special, and was able to get at the front of the race. We're looking at this artificial analysis that shows this clustering in the upper right. DeepSeek got up there a couple weeks before, you had Grok. What's interesting is they all seem to be coalescing in an impressive way around the top of these benchmarks. We're really only talking about five or six players who have a chance to be in this game at this point. I saw people who interpreted Grok's fast rise as proof that pre-training still has legs. To me, I had the opposite reaction. I felt like they just slammed up against the ceiling that's holding everyone in, although again an incredibly capable level. I've said this for a while, I've been concerned that the way an LLM works and the way it's optimized, building bigger clusters and more parameters won't buy you much. Ilia said it, Andrej Karpathy said it, other people have said the same thing. To me, this reinforced that point. I was expecting if there were pre-training headroom, I was expecting this to go through. I will qualify this was their first run. Maybe there were some tricks they didn't know. They could very well back up and do another run on that same large cluster and maybe shoot past these people, or maybe these benchmarks aren't the exact right thing to be looking at.
I would say a couple other things. Number one, it's not just a pre-trained model. They also have an inference time reasoning component to the model that's incredibly capable. We have this benchmark chart that I tweeted the other day and I compared it to the search index benchmarks that we all used to track. The benchmarks are one thing, but the reality is how do we feel when we're using the product? Grok 3 rocketed to the top of all app downloads on the iPhone charts. My Twitter thread was full of people having great experiences showing those great experiences on Grok 3. It had a personality and an interaction with people that I think people were enjoying. It clearly crossed the threshold of being capable enough. Now the real question shifts to can they leverage the X platform, which reaches a massive and important audience, to really drive that. The early indications to me, when you compare it to how Meta has used Meta AI, as incredible as I think Zuckerberg and Meta are and the advancements they've made, I have not particularly been impressed by the productization of Meta AI. It's basically just a search box stuck at the top of Instagram or stuck in my WhatsApp thread. When I'm on it, I never intend to be there. It's not direct competition. Whereas on X, they figured out the first thing they did is they put that button at the bottom of the app that clearly distinguishes it as its own standalone application. They launched a standalone application. They're using X to drive those app downloads. Now I just opened up my X app today and it said go out and try the new voice for Grok 3. To me, the execution on the product side to drive consumer use has been pretty damn impressive and took them to the top of the charts.
Only DeepSeek and Grok of all the others have shown the ability to break into the top 10 on the App Store download charts. While we all have a fascination with where they got to on the benchmarks, my own sense at this point in time is this is going to be one of these battles kind of like search was. The five or six players, and just out today as we're about ready to go on, OpenAI released ChatGPT 4.5. They've hinted in this presentation as to ChatGPT 5 or 6. If you look at 4.5, one of the important distinguishing elements that they're pitching is it's more humanlike, it gives better answers, more concise answers. Not a big breakthrough on evals, although there are some improvements in the early looks against the evals. Ultimately, we're going to measure the success of these things by how many people are using it.
I think one thing to be good for the audience, I know you've said it in the past, but you're an investor in OpenAI and I think you have a theory about their prowess in the consumer market and their lead in the consumer market. Why don't you reiterate that?
I've showed this chart before. In the search wars, we had Google and Yahoo and AltaVista and Lycos and Ask Jeeves and Excite and Infoseek. By the way, they all did pretty damn good on the benchmarks. But the reality is that didn't get them to any value creation because ultimately all the consumers aggregated around Google. The real question is does that same pattern play out of winner take most in consumer around AI? It did in search and it did in social, but it's not necessarily a follow-on that it will in AI. I'll stipulate X has an incredible installed base that they can market into, Meta has an incredible installed base, Google has, and it's existential for those companies to market to those consumers. I don't think it's going to be winner take as much. I don't think we're going to see a 99% monopoly here, but I do expect we're going to see 70 or 80% share go to the winner. If we look at the numbers today, last week Sarah reported that OpenAI has crossed 400 million weekly average users. That's a user number, not a paid user. The number of paid users is a fraction of that. I think they also reported something like 11 or 12 billion dollars in expected revenue this year. You can reverse engineer your way into what percentage are paying. More importantly, monthly average users must be somewhere in the order of magnitude of 700 to 800 million. You and I followed consumer for a long time. There's this magic number around a billion. I already think they're near escape velocity, but at a billion monthlies you can funnel all of those folks into weeklys and then funnel the weeklies into paying subscribers or people who are consuming advertising. What I have seen is everybody else catch up on the benchmarks. What I have not seen is people catch up on the consumer velocity.
Let's handicap some of the other players a bit. Who do you think is closest from a user standpoint? Is it probably you'd have to count the Gemini searches in the Google Search to get to a number that's close to OpenAI?
Let's start with Google. There's been a lot of reports out over the course of the last couple weeks. We have public companies now reporting that their Google organic clicks are down 20 to 40% year to date. The question is why. If I do a Google search today on my phone, half of the page is taken up with an AI answer to whatever my Google query is and the rest are all paid links. I think that's the right decision for them to make. If you want to compete, you ultimately have to be willing to take the innovator's dilemma head on and really just cannibalize your product with AI. If they do that, and if you're one of these humans that thinks SEO wasn't dead already, which I would have declared it dead a while ago, it's really fucking dead. SEO is dead. Those were the free links that was the core product that used to attract everybody to Google. The idea that SEO is basically now gone is pretty significant. I've been remarkably frustrated with Google's organic links for the past five years. You go in and search for your favorite team schedule and all the ticket guys are up front. The link you're looking for you have to hunt for.
Let's talk about that for a second. Now the obscure link or the obscure information that you and I may be looking for may be on page three, four, or five. You and I are never going to get to page three, four, or five. What's so interesting about OpenAI's deep research? If I launch a query using deep research, it will go to page four or five or 10 or 100 and find those obscure pieces of information. I think the evolution of Google actually provides acceleration to the deep research projects because I don't want to go do that deep research.
Google, I think you just can't discount their installed base. The number of people going there who will inertia will continue to carry them there. But I would say this, you can go search for this on Twitter anywhere else. I know certainly with my own behavior, the amount of activity that I used to do on Google has been 80% cannibalized by ChatGPT because there's search embedded within ChatGPT. I'm getting all of that information, all of those answers. I think they're going to be formidable. I think they're being bolder than they've been, but they'll have to continue to do that. I think some of their assets are remarkable. You got the YouTube dataset and all the search queries over all the years. Their understanding of structured data around a lot of the consumer verticals, they built that out in airlines and things. They should be able to do those agent type queries better, faster. Their velocity on product has not been impressive. Velocity on consumer has not been impressive.
They've had them for a long time, Bill. They had ChatGPT before ChatGPT. They also have Android, which is a massive asset, and they also have their own browser. Both Perplexity and OpenAI have started toying with the idea of either having a browser or in the operator case of using a browser in the cloud to go do this work. They have so much. I still think they have a bit of the innovator's dilemma in that they still have to try and maintain those paid links on that page. This chart here, the black line is Google's paid click growth plotted against the weekly average user growth at OpenAI. It's not going in the right direction.
It benefits from the fact that informational searches are what ChatGPT cannibalized first, not the commerce searches which is where most of the money is on the paid link. We're going to see this out of X, we're going to see it out of everybody. The entire domain of the internet is the domain of agents. If you think about operator as one of the first agents rolled out by OpenAI, what does operator do? It goes and it mimics me as a human going out and researching a hotel and booking a hotel on the internet. We're in a very embryonic state. I agree with you, we're not there yet, but it's very clear what the roadmap is going to be. It's going to want your credentials and whether you give it your credentials or not is going to matter because it's searching against an... Let's talk about Meta.
One last thing on Google. There was a point where Meta went public at 40. We had some... I was paying attention and Zuck, as he has many times, got woken up on mile 100% and everyone thought he was dead because of... and he had also built an HTML 5, he didn't believe in native app. There's a whole thing that they weren't going to be able to monetize mobile. He was on the cover of Barron's magazine, The Week magazine. It was like Meta's dead or Facebook's dead. But he woke up and fixed it. Can Google do that here? Is that possible? Can they have a similar... and what would it look like and what would it take?
I've said publicly that Google's moat was not a technological moat with search. Their moat was a distribution moat. Their moat was a mind share moat. We googled everything when we wanted to know anything. The only thing that could attack Google was never anything head on. It had to be an orthogonal attack from something that was 10x better, 100x better because it gave us answers instead of blue links. That's why it was such a mortal sin for them to ever allow anybody else to go first, because the only thing that could give you a trillion dollars worth of free mind share is going first with something that was 100x better. That's exactly what ChatGPT did at the end of 2022. Go to Meta. If you had to handicap the big guys, three billion users of their product. I think they have products that are tailor made for chat oriented AI, whether it's Instagram and having shopping agents and co-shopping agents, or whether it's WhatsApp and just having a bunch of agents live within my WhatsApp channel. It feels natively much better positioned for AI. We know that Zuckerberg is in complete beast mode. But I am surprised that we're now kind of 18 months into the Llama thing and it feels like the manifestation of it into the product was slower than I expected. Back to your product point. Meta hasn't... I will say even we know he was ripped about DeepSeek, kind of blindsiding Llama in the release of R1. It's not just product for them. I heard from several inference players that you and I are friends with that all of a sudden DeepSeek rather than Llama is the enterprise open-source model of choice that everybody's experimenting with and playing with. That becomes a real problem for them as well. I think 2025 is a critical year. I think they will come through. When it comes to almost all product stuff, stories copying, catching up with Snapchat or whether it's Reels catching up with TikTok, they've always showed up to the party late but they are grinders and they always deliver the product. It'll be interesting to see what they do.
Who else would be in the list? Anthropic has really not been... they pretty much ceded the game on consumer. There was a product announcement yesterday about they're going to be powering Alexa. But now we're stretching. Amazon did do a big Alexa launch yesterday, pretty late in this game. Alexa is not really... it occupies a different space in most consumer minds. It is not what ChatGPT does. To dislodge something that has the momentum ChatGPT does, you have to go at them and do better than what they do at the thing that they do. This is why I think X is so interesting. They have a platform that is the number one news platform in every country on the planet. The people who are most actively engaged are using this platform and they go there for information, they go there for answers, they go there to engage. I think it's an audience that's very well suited for AI. I think the integration they've done is as good. They've done this in a very short period of time. I'm talking everything from the logo, some of the tweets will have the logo pop up and then it'll summarize or do more research. I'm really impressed at the velocity of not only catching up on the benchmark but catching up on the consumer product side. They're number one on the App Store and that stands for something. Coming out of nowhere, people said that Elon couldn't do this. I never doubted that they would catch up on the benchmarks if they got a big enough cluster because Elon set a mission that people become messianic about. His engineering capability to build out the cluster and do all those things, that was never the question for me. The real question was can anybody close the gap on the consumer race? The odds-on favorite there has to be OpenAI. I think they continue to widen their gap. Bill, I think they're accelerating at scale. That's where the race is. It may very well be that coming in second place with 20% share is a pretty good place to be.
I want to mention one more company and then I'm going to make a guess at four ways someone could try and win this game. The one I want to mention before I do that is Perplexity briefly. I will give them credit for being product centric to your point and innovative in ways that the others haven't, kind of on their own terms. They don't have near the usage of OpenAI, so there's a question that it kind of looks like an acquisition candidate to me. I don't know if anyone can agree on price, but for one of these other players that hasn't been as successful from a product standpoint, you could imagine a world in which Microsoft were to buy Perplexity and now they have a consumer brand to go battle it out. We know how much Satya wants to win in consumer. He owns a bunch of OpenAI, so he's got some potential channel conflict there. But the bigger issue at this point, with Lina out, you're probably more likely to be able to do a deal like that. When founders are raising at 89 billion, it becomes a much more difficult decision for a company like Microsoft. Not saying that it couldn't happen. When it comes to punching up, being innovative, being scrappy, product velocity, the founder there Aravind and the team, it's been super impressive to watch. I think they made other people better. But the numbers as we look at them today, they're really powerful but much, much smaller player.
Here we go. I have four things I'm watching out for that could potentially lead to either further lock-in by OpenAI or a window for someone to do something else. Some of them I mentioned before. Memory is still this thing that could just tie you to something. OpenAI has probably done more with memory than anyone else, but no one's really got to the place where I'm telling it to remember things, to store things, to create lists, where it starts to become like an executive assistant for you. I haven't seen that yet. I still think that's a dimension that could really be important. Voice, we've talked about. They're all playing with it. I think voice also ties in with device type. This is where Alexa may have some assets. If the voice were spectacular, I might not have to carry the phone around as much.
You need an earbud. I will say advanced voice mode on ChatGPT is excellent. Grok 3's new voice is excellent. They're getting better at an accelerating rate. We're an investor in this company, LiveKit, that's powering a lot of this voice. What I see in the product pipeline is super impressive as to what's coming with voice. The third one is nebulous, but someone could focus on a feature that no one has to date. Right now the game looks so much like with the benchmarks and voice, everyone's running at the same place. That's an easy thing to say, but it would have to be really out of the box. Fourth, I've been thinking about this. No one's really thought about a network effect. I wonder how you could make the quality of the AI experience a function of your user base. Let me give you an example of a network effect that I think is happening. Around model improvement, if you have 700 or 800 million monthly average users, your diversity of information in questions and answers and follow-ons is much higher. Those questions and data are now being fed back into the models to improve the model. Some users may have seen, I know I have, you get a pro, you get two answers and OpenAI asks you to rate them. I think that's an example of OpenAI very actively attempting to build network effects in terms of the quality of the model, the quality of the answers. But there could be a more intense form of network effect if you found a way to leverage the user base as part of the value proposition.
Let me go back to your first one, memory. You and I have talked about this a lot. If you get memory, the switching costs explode. I would argue not only switching costs explode, the conversion rate from free to paid probably also goes up just because the value delivered. I was with my 89-year-old mother last Sunday. My mom has wanted to write a story of her life for a long time. The reality is she's never going to sit down and write the story of her life. When I'm with her, podcast style, I'll ask her questions and I'll just record it on my phone so that I have it. I could perhaps go back later. Then I started thinking about it and I said I don't need to be the interviewer. Advanced voice mode could be the interviewer. I was sitting there with her last weekend. Here's the prompt that I gave to advanced voice mode. I said, 'I'm sitting with my 89-year-old mother tonight who wants to write her life story. I want you to interview her about her life, to ask questions about her childhood, stories, having kids, working, growing up in the depression, her love of computers and travel. Remember everything you talk about and then compose a story of her life that her grandchildren would like to read.' Advanced voice mode just started asking her questions. How long did it go on? My mom was really nervous at the start, but then a little tear wells in my mom's eye because she realizes all of a sudden that oh my God, this could be a massive unlock. Here's the thing, Bill. Advanced voice mode and ChatGPT already has memory. You can already do these things. The problem is the nature of the product, you don't know that it can do those things. Part of the challenge about designing a product where prompt is your way in is you got to help people imagine. You and I could have imagined in the age of the internet somebody building an internet website that just did that thing. I think that's one of the challenges all these companies face. The innovation around that top end of the funnel in the prompt that can help people better get into it. I'll give you another example. Deep reasoning, which is really fascinating. They basically took the O3 series of models and fine-tuned it end to end based upon all these browser interactions. But the more specific the prompt, the better the deep research report is going to be. A lot of people are using O1 to help them build sophisticated prompts that they then feed into deep reasoning. I think there's something in there where we're effectively using AI to get us to the point where we're better prompting. One of the ways will be very simple. Once I have this assistant and I'm having an interaction, I just say to the assistant, 'Hey, my mom wants to tell her life story. I'm not sure how to go about doing that. Do you have any ideas?' And she would say, 'Hey, yeah, just use this prompt.'
