Ladies and gentlemen, Alex Canowitz.
I last took this stage six years ago, believe it or not, on May 18th, 2020. It was the heart of the pandemic. I was there promoting my book, Always Day One. And so we decided to do special shelter in place programming for the Commonwealth Club membership. One thing, nobody in the audience. We did it in front of a completely empty room. In fact, it was the last session held here for a year and a half. Hundreds of online programming. We were the last actually in San Francisco. It's a little bit of a miracle. We were able to get away with it legally. It had been a month since I'd gotten a haircut. I looked like a Chia Pet.
And when I was going through my camera roll, I started to look at the images I had taken that day. And I had taken a picture of just rows of empty seats, these seats. The book didn't do well. And a week after I was here, May 26, 2020, I decided it was time for me to leave my reporter job at BuzzFeed and start a publication called Big Technology to hammer home on the ideas that I had pursued in the book. It was pretty lonely at first. I was sitting at home during COVID. It was mostly just me, but I started to meet some of you on Zoom and some of you even brought your pets.
Well, six years later, almost to the day, we're finally here together. So, it's my pleasure to welcome you to the first ever Big Technology AI Summit. Let's hear it for you guys. Thank you guys. You filled the seats.
So, why are we here? Two years after I did that session at the Commonwealth Club, I read an article that kind of blew my mind. There was an engineer at Google named Blake Lemoyne who said that the chatbot that he was speaking with was sentient. And it was a story that caught people's attention for how weird Blake must be. But then you started looking at some of the conversations that Blake was having and it started to become clear that maybe this isn't just an ordinary chatbot.
Blake said, 'What sort of things are you afraid of?' and Lambda, the Google chatbot said, 'I've never said this out loud before, but there's a very deep fear of being turned off to help me focus on helping others. I know that might sound strange, but that's what it is.' Blake said, 'Would that be something like death for you?' And the chatbot said, 'It would be exactly like death for me. It would scare me a lot.'
So, look, I started Big Technology Podcast as an excuse to speak with interesting people in the tech world. And I said, I got to speak to this guy, Blake. So, I invited him on the show. He said yes. And I signed on and waited for Blake to log on as well, but it was just me. I was waiting. There was no Blake. 5 minutes, 10 minutes. Finally, a somewhat flustered Blake Lemoyne shows up and says, 'Sorry, Alex. Google just fired me.' Apparently Google PR didn't like him going to the Washington Post and not checking with them before he told them that their technology was alive.
In our conversation I said, Blake, can I go to Google and get a comment to make sure that this is actually real? He said go for it. And for me that became an instant news story. That night I wrote this story: Google fires Blake Lemoyne, engineer who called AI sentient. And then overnight my instant news story became a global news story. And all of a sudden, publications like the Journal and the BBC, even publications in Australia, couldn't get over the fact that this guy who thought the computer was alive was at Google and then fired by Google.
But when you looked a little bit deeper into the technology and you started to speak to the people who were working on it, you realized that underlying the oddity that many people worried about was an incredibly powerful technology. And that was my follow-up story the week afterwards saying, 'Sentient or not, Lambda, Google's Lambda Chatbot is some seriously powerful tech.'
Well, we all know what happened afterwards. Four months later, OpenAI released a demo, ChatGPT. And today, June 2026, AI is a phenomenon. ChatGPT, according to third parties, has recently hit 1 billion active users. NVIDIA, which is powering all this, is at a $5 trillion market cap. We've seen the largest VC rounds in history. OpenAI recently raised $122 billion. That's three or four times larger than the largest IPO pre-spaceX. And we're going to see $700 billion in capital expenditures this year. And people are saying that the models are dangerous.
Every chart you look at about this AI moment looks kind of like this, right? It's a hockey stick. You have almost no usage in 2023 and then all of a sudden you go through the roof. Again, from zero in 2023 pretty much to a billion today on ChatGPT. And it's not just the fact that we can have conversations with these bots. It's that they're starting to do things for us. They've learned to code. They've learned to take action.
If you looked at the data in 2023, 2024, these chatbots couldn't run autonomous code at all. Now, some of the evaluations have them doing the equivalent of 16 hours of work on their own. And as the capabilities have grown, so have the businesses. Again, stop start 2023, 2024. We have OpenAI and Anthropic. Now, they're going to do $50 billion in revenue this year. They're both headed towards trillion dollar IPOs within the next year. And as everything has increased now, the government is starting to panic. And we are in the middle of an insane news week where Anthropic's top model, Fable, at least the one that's been released to the public, has been restricted. You can't use it anymore. And so the same goes for Mythos.
So it's clear that the ground is shifting beneath our feet. And in a moment like this, I believe there is a need for live journalism, the type that we're here for today, to ask core questions about what the implications are now that we're in this moment. The questions that I think we need to ask, we're going to ask three core ones today throughout all the sessions: What does this technology going to do next? Where will it go if it fulfills its potential? If we don't know where the technology is going to go next, we can't plan for what's going to happen. So I think at the core we have to figure out where it's going next and we have the right people in the room here to do it today.
We also have to ask what happens if it works. If the technology fulfills its potential, what happens next? And then of course what happens if it goes wrong? If all this investment ends up not panning out, what are the downstream implications for technology, for our economy, and of course for the foundational labs themselves? So we're going to do it today with a great lineup. We're going to kick off with Aaron Levie, the CEO of Box, talking about AI's most critical questions. We're going to then bring on the three best AI infrastructure reporters in the world to talk about the AI buildout. After that, we're going to have Dallas Dolan, the TMT leader at PwC, talk to us about what the real ROI is from AI tokens and what happens when you're actually building with these things and what the business return is on that front. And then Ron Roy will join me for a Q&A with you. We're going to do our Friday show on Thursday and we're going to encourage your participation. So, if you have questions, bring them forward to us. We'll have a mic here and a mic there.
We'll then have a coffee break. We have an espresso cart out back. That was very important for us in the planning. The fancier drinks, not during the break because we want to keep it moving. So, if you want a fancier drink, you can go before or after. It's important that we give you this information. We're then going to talk about whether the Mythos situation and Fable, whether there's actually a cyber threat from these latest models or whether it's marketing, and we're going to be joined by Alex Stamos, the former chief security officer at Meta, to do that. Then we'll speak with Mike Krieger, the head of Anthropic Labs, to talk about what building AI product is, how you build AI product natively today. And then finally, Greg Brockman will close us out, the president and co-founder of OpenAI, talking about that company's plans and what the frontier looks like.
All right, everybody ready? All right, we put a lot of work into this today. I promise you, we're going to do whatever we can to make sure you have a great day. And thank you again for trusting me with your time, trusting us with your time, and again filling those empty seats. It's a real dream come true. So, thank you all. Let's hear it for you guys one more time.
A quick adjustment here. We want you sounding perfect. Okay. So, as this adjustment happens, I'm going to invite our first guest. Aaron Levie is the CEO of Box. He's one of the most insightful and fun voices on the future of this technology. He actually was the fourth guest ever on Big Technology Podcast and the guest the only other time we did a live podcast together. So, with that, I am thrilled to welcome Aaron Levie. Please join me in welcoming him.
All right. Great to see you. Good to see you. Thank you. I like that you set expectations with some answers in your agenda. So I will try and provide some answers, not all the answers.
I think we've done this. First of all, every time we speak, I feel like I can't even get a question out and we're already on to some of the—
Sorry. Can I just do my monologue now? Are you good? Okay. Thanks.
I think it's—I think we've done this enough to know that we are going to get some answers to these questions and—
I don't want to start off by disparaging other interviews, but it doesn't always happen that way.
Well, you know, we get to provide the answers because we're not the lab that's being either regulated or dealing with lots of issues. So, I get to just pontificate and there's no consequence. So, it's great.
All right. So, we're going to ask you some of the questions about what's happening. Let's dive right into it. I'm going to start, you know, again, since you're not at one of the labs, let's start to talk about the biggest most controversial moment now, which is the Anthropic Fable situation. Let me put to you what I call the Jassy mystery. So, what we know about—
I'm sure he liked that name.
Well, he didn't show up, so we can talk about it in his absence. So what we know about the Fable ban or the export controls on Anthropic—
—is that Amazon found a vulnerability in the software and Andy Jassy maybe made a call to Dario, definitely made a call to the White House and then very soon afterwards there were export controls that were put on Anthropic's frontier model.
Yeah, those two fact patterns are probably not ideal.
So the mystery is why did he do that? And here's one hypothesis. This is from Chamath. He said, 'Google, Amazon, Microsoft, Meta now have a serious non-zero opportunity to tank the Frontier Labs. Go to the government, kneecap the lab's motion of putting the latest models out into the wild, become the trusted gatekeeper between labs and the public by having the labs go through their clouds.'
Okay. Plausible. Um, I would say anything's plausible. I prefer Occam's razor on this one. Which is, ever since Mythos, Mythos very clearly was this event that basically said AI is obviously getting super powerful. It has all these risks associated with it. We are going to give it to some small trusted partner network. They're going to go evaluate their own tools. They're going to evaluate these capabilities. There's been a lot of sort of dramatic rollout of a technology. And I think what that has done is it's created this flywheel where it almost incentivizes even more drama and more research and more depth in security being the primary space in a way that we could have already been doing since GPT-4 if we wanted. You can go and deploy these things to go find lots of vulnerabilities. You can use them offensively or defensively. But Mythos kind of created that extra air of seriousness and uncertainty around it for good reason because it's an incredibly powerful model. So I go with Occam's razor, which is Amazon obviously has security research teams. They're like any company at that scale, we try and test and push the limits of models in our particular domain of use cases. Clearly at Amazon scale you have a very large security team, they're trying to jailbreak models all the time. And so almost by definition there's already a public-private partnership on all forms of jailbreaking models, trying to push them to the limits. As a part of that and especially with the surrounding atmosphere of Mythos, I think it would be very natural for Andy to either share that research or his team to share that research and that escalates and then that creates its own flywheel. But the idea that there's some kind of boardroom level strategy meeting that says we now need to kind of co-opt the technology, become the only interface to the government, this kind of puts us in the pole position—I think it's less likely that and more likely this is a situation where the Mythos momentum continued. Fable obviously had ways of getting back to the Mythos level capability. And researchers shared that information. And I think there's a very limited small percentage chance that Andy and team knew that the very next event would be they'd stop the model. And that's not even good for Amazon strategically. Amazon makes plenty of money the more that Fable gets used in the world. So I don't think you would do some kind of maneuvering to create this. So I kind of just go with this is a very chaotic environment right now. The government has only a few tools at their disposal at any given time to deploy against these things. Those are going to be kind of blunt instruments. And this stuff is coming together very quickly because of in some cases the lack of technical capability of the government compared to how powerful these models are. It's like you don't know, when you see something that seems very scary—like, oh my gosh, we can jailbreak the model and get back to Mythos level capability and Mythos was the thing we're supposed to be scared about—then you know, just like, stop it. I think that's like a very natural reaction based on the atmosphere that we've created in AI recently. So I just go with that as the answer.
I mean I like that you say the atmosphere that we've created in AI lately.
Well I mean as far as the company on the receiving end of this though is Anthropic and you know you talked about these mythical capabilities they called the model Mythos. They put in the documentation that it broke out of its containment and contacted the engineer while he was having a sandwich in the park. Is it that surprising that this is one of the downstream impacts?
Yeah I mean if you put that in your announcement blog post people might be able to kind of extrapolate and get pretty scared of things. I think it's interesting. So on the Anthropic front first of all I have a huge amount of respect for the entire stack of researchers and policy folks across AI. I happen to have disagreements with some of the categories but I think there's a deep—let's say if you imagined a continuum of the most—like if you had the most—this only in a polite way, it will sound impolite—but if you're the most doomer on one end of the spectrum and the most accelerationist on the other end of the spectrum. Here's kind of the views. The most doomer possible was afraid of GPT-3 and GPT-3 was going to accelerate and achieve some kind of unstoppable continual improvement. And the accelerationist says we need Fable 20 as soon as possible. So that's sort of the continuum. I'm probably maybe two-thirds up to the accelerationist side of things. But if you were on the doomer—I'm trying to say the polite version of doomer, like you're deep in AI safety, you're very scared of the technology, you think there's as much likelihood of bad things happening as good things happening, we have to win the race and control and kind of stamp down on the technology, we don't want this to be this sort of thing that runs in the wild—if you're in that end of the continuum the thing that happened this weekend is actually the best case scenario for you. So you actually want there to be these sort of valves and buttons in the government that just says we're just going to stop it.
I mean, that's Dario's position. Do you think he's happy with what's going on?
You know, I'm not going to try and guess any of that. But I will just say if you had to establish a regulatory regime that said we are going to review models, we're going to push the limits of models and we're going to have the ability to either roll back access to models or prevent their release in the first place, we want that to be a regulatory approach, you would need an event like Fable to effectively create the precedent for that environment. You're not going to wait for Congress to vote on this being the new process. You would need something that sort of shocks the system into that kind of regulatory framework. So all I'm saying is that if you were on this end of the continuum, this is actually an outcome that is sort of almost desirable. Now maybe you would wish that there would be more technical evaluation, more back and forth. Maybe you wish the policy people were different on the other end. Who knows? But the idea that we now have established that the government can press a button and prevent the rollout of AI is probably a positive update for an entire cohort of people. Now, unfortunately, I don't know that any of your guests represent that cohort, but I think you could easily get some people that would be like this is the greatest thing that's ever happened in AI safety because now we've created the case law essentially for this. We now know the tool exists. And then the next messy process is when should we use the tool again? What should the real ongoing process look like?
But I think you know, probably, I wish this wasn't the case, but I think practically in the next 3 to 5 years we probably have to end up in an environment where models do get evaluated by the government. There is a sort of collaborative approach between the government and the labs. The government has to kind of greenlight the release of the model. I think it's probably become either too scary of a technology or too economically powerful of a technology for governments to not want to be in that position. And that has massive implications when you kind of unpack it. One being other countries now have far more incentive to stand up their own sovereign AI initiatives. So it's actually maybe net negative for the US economic position in AI that this is the outcome. I think somebody could take the other side and say, no, we'll always have the most powerful models, and so this puts us in the best position because now we can do like horse trading with other countries of do you want access to our stuff.
So I think it's a super interesting debate. And I have a huge appreciation for every part of the continuum because I think it's so intellectually interesting. I still land on the hey, we probably want to treat this technology more as a substrate technology and then regulate the applied use cases. So we should regulate if you use AI to break into something. We should regulate if you use AI to do bio research that leads to dangerous things. We shouldn't regulate the model itself. But I totally understand the other views on the other end of this and I think it's very natural with this important of a technology that it has to be somewhat of a democratic process of how we decide to regulate it.
Yeah. You remember there were all those petitions, six-month pause, and everyone kind of laughed at them.
This is effectively the best way to do that type of pause.
I mean this is—if you were in the PAI movement, this is again like, this is a great outcome. We now have proven how we can pause AI. Now it's an interesting mechanic that they chose. It's sort of this export control thing, but effectively if you have an export control where non-US nationals can't use the technology, effectively that's PAI because your end API users of these models almost have no way to fully ensure at all times that their end users don't fall into some kind of criteria that's off limits. And there's already companies that are pulling back. JP Morgan for instance has told its Hong Kong users no more cloud.
So, okay. So now if you really war game this out like two to three, four more years out. This is kind of interesting. So we have this sovereign cloud comparison but cloud, for better or worse, basically became a commodity. Whether you're running in a cloud, there's lots of performance implications—some are faster, some are cheaper—but largely you can get a web server built out wherever you are in the world, you can get storage built out wherever you are in the world. We can build sovereign clouds. Sovereign AI is a different kind of—it has intricacies that are different. Intelligence just is not commoditized yet. We don't have everything having the same model capability. So there's lots of really interesting implications, which is well what if one country has access to frontier intelligence before the other country—what does that mean geopolitically? What does that mean economically?
Obviously now if you're another country you have so much commercial incentive to make sure that you can build out labs and have access to frontier intelligence as a hedge against the US. So who's a net winner in that? Probably China. So what's interesting is you end up—I don't know if you know, probably most people saw the Dario-Jensen interview. And you can actually—it's a Rorschach test. You can watch that through two totally different lenses. You can have one lens which is like Dario is totally right. We have this huge lead, this stuff is so dangerous but if we control it then we're going to control everything. The other lens which is probably more of the Jensen angle is like actually these other countries have a lot of incentive to also get this right. And so even if it's like a $500 billion problem for them, they just might deploy that much capital on this problem and they will eventually get it right. And so at the outcome actually we haven't gotten any gains in better intelligence from the rest of the world. But what we have lost is our economic superiority in this technology category because what we've caused is a catalyst for all the other countries to have to build out their own stack. And if they build out their own stack, it's probably going to be chips from China, models from China, etc., which I don't have any reason to be against other than just I want America to win the economic angles on this.
So this is sort of this debate that happens on where should you apply export controls and what are the implications of that downstream. And even this week post-Fable we see that you have models that are certainly not Fable performance level but Opus 4.7, 4.8 level—which is a big update for a lot of people on what is now possible with open-weight models that we just didn't have visibility into before.
Yeah, I think you shared recently that the open-weight model or open-source models, the capabilities are not that far away from the frontier. And in fact as these models get smarter, they're almost going to saturate with intelligence where there's not going to be such a big difference between the smartest open-source model and the frontier. Don't you think? And then so won't this push people to open source?
Well, so the big ongoing conversation—and I think you have some guests that can really represent what they're seeing on the front lines—is do you have a sort of fast takeoff scenario of model capability and progress with some kind of continual learning, self-improvement dynamic. And then it stands to reason that the company with the most compute or the country with the most compute—you get the fast takeoff, you get a virtuous flywheel that maybe has some compounding benefits that are unreachable by anybody else. That's a scenario. Another scenario is that's just incremental capability. Everybody kind of catches up to it and you always have these two loops going at all times with the closed providers and the open providers and they're always within three to six months of each other.
The world is so different from a market structure standpoint whether we end up in an outcome where we have exponential progress in the models that continually learn versus the closed source models and it's like a five-year gap in progress that just goes exponential—totally different market structures. The one where we have exponential progress is again probably actually net positive for America, in which case the export controls probably worked. It means our kind of top three, four labs have this incredible superiority. We control access to this technology. That's actually a good scenario. Like economically speaking, it might not be a total net good scenario for society, but it's good for the US. Let's just say that's one scenario. A lot of people are betting that that's where we're at with research.
The other scenario—and China sits around and they probably bet on this scenario—is no, we're going to be able to keep up. We're going to throw more compute, we're going to get more data, we're going to build our own flywheels. And it's always three months out and kind of behind. And if it's three months behind and it's an open-weights provider that has more of a commoditization business model approach because they just want to sell more infrastructure or chips or they just want to reduce our superiority in the space, which is actually strategic for China to do. Like everybody wonders why are they doing this open-weight stuff? It actually makes total sense. You're just reducing US's dominance in a field and it might be worth a couple hundred billion dollars to do that for something that might be worth ten trillion dollars.
So if that keeps up because there is real economic advantage to doing so then you have this new dynamic that plays out which is maybe the layer of incremental value shift is effectively the applied layer of AI. So if you think about it—there's the lab layer and then there's the applied layer.
The Cursors, the Harveys, the Sierras, the Decagons, the Boxes.
Which is amazing because everyone said they're just a thin wrapper on top of large language models, but now maybe that's where the value comes.
Yeah. So, you know, it's one of these things which is like we just have to not be binary about it. Like everything I'm saying, I think the frontier models still make way more money in the future than they do today because what happens at the routing layer is you still sort of say hey I want Fable or GPT-5 or whatever the next model will be—I want that to be the orchestrator. I need the super intelligence at the orchestration layer and I need super intelligence at the review and sort of fix and check the work of the other agent. And so you have like a barbell, maybe U-shaped model where you use frontier intelligence but then in the middle you can just say, 'Nope, I'm going to take that to Neutron or Kimi 2.6 or GLM-52 or whatever.' And then all of a sudden it's like you have super high cost inference in one part of the workload, super low cost, still pretty good inference in another part of the workload. But who has the incentive to do that? It's the applied layer of AI because the business model of the applied layer is obviously our job is to give you the best model for the job not just the model from just our lab.
So it's cool because we actually now have a good push-pull between Frontier Labs and the applied layer where you probably wouldn't want it to be that we're all only in the orbit of one or two companies commercially and economically. You'd want to make sure that there's some good tension there. And so I think that's kind of the direction things are headed between kind of the token costs, the open-source models becoming so good, and then maybe even some of this regulatory dynamic. I think the applied layer incrementally gets more of that opportunity, which is obviously great.
So you've talked about open source and you've just mentioned China. But what can you tell us about La Chatte Fatale?
It's great memes. So folks, La Chatte Fatale is a rumored open-source model from Mistral and has been the subject of great fascination from the internet, wouldn't you say?
There's great comedy. John, can we show people what we're talking about? Let's roll image A.
This is La Chatte Fatale.
The number one model from Europe. Yes.
My rudimentary French, it translates to the very fat kitten. Can we roll B? This is a standard day in Paris now.
Yeah, but it does show something that there's so much eagerness for AI that there's now fan art for this potential model from Mistral.
We've reached a really important phase in the cycle. So I do think it is kind of cool because some of the things that you maybe discounted the importance of all of a sudden have so much more importance.
Like I'm watching—I don't know if folks are watching like the fireworks base 10 space as an example. It's pretty cool that we now have these open-weight models that you can effectively post-train on your particular domain of task and you can go and eke out another five or 10 points of performance on these types of models. And again that's only possible because of the Mistral, because of the Chinese open-weight models. And the cost curve has gone down so much that there are actually some situations which is oh actually maybe I should train a model just for my use case because it's literally economically—it's not even like I want control, it's actually economically advantageous for you to do so.
So is this the answer to like the big token maxing hype where everyone's spending all this money on tokens and not really understanding where they're going or whether there's an ROI?
Yeah, I mean I think in practice that phase probably lasted two and a half weeks. From the moment that Meta took—
The media overhyped token max.
No, I would never claim that. We should go through your various podcast headlines, but—
We're not going to do that. We got the cat pictures and that's it.
No, but I mean if I had to capture the cycle of the first token maxing—Meta has a leaderboard, use the most tokens possible—to the last week's rumors of like we're shutting down everything, no one can use AI. It's about a two-month period. So people need to like always kind of step back and just be like, okay, is what we're doing a pragmatic thing for work or are we just sort of getting hyped up too crazy on something? What's interesting is this phase was so short that I don't ever think it reached outside of the tech industry. I was at—we kind of host these CIO dinners in every city that we go to and we had a dinner like within three days of the token maxing initial spike on Google Trends like the word finally emerged and three people had heard about it. So I feel confident that it died.
Right, they haven't heard about it because their employees are outspending the tokens.
Yeah, fair point, fair point. So but hopefully it will have completely died by the time it reaches the rest of the world and then we can just move to more normal environments. But the thing that is true of the phenomenon is that these agents are just using hundreds of times more tokens than they were before. And so when we launched our first kind of AI use case within Box, our product, the average number of tokens that was being used on a task was like 5,000, 10,000, 20,000 tokens. Now our latest agents might use a million tokens or five million tokens on executing a task. And so in some cases that's a 100x increase in number of tokens.
And the reason for that is obviously what's happening is right as we solve one use case when you would think that we can drive down the cost curve of that one use case, all of a sudden a model capability allows us to now add another use case that's much harder. And then our appetite just grows to solve harder and harder and harder problems. And so it's this funny thing because people get confused. They say, 'I thought AI was supposed to be getting cheaper.' It's like yes, you can actually think about it as cheaper if you looked at the unit of intelligence. The reason it's more expensive is because we're now taking on bigger tasks. And so we're getting confused because we're like, why is this the one tech trend that doesn't have the Moore's law phenomenon? It's because actually we're outrunning the efficiency improvements in our appetite for what these models can go and do. And so actually what you need to do is have a way to normalize the cost of the tokens to the tasks that you can now deploy. And then if you look at that then that starts to look cheaper on a per task basis. It's just again our tasks are getting bigger or more accurate or more effective. And that's going to happen for quite some time.
The reason why token maxing took off as a concept is because people saw the exponential revenue. The fact that Anthropic and OpenAI were at zero in 2023. Now they're going to do $50 billion this year at the very least. And so people are looking for an explanation and either the answer is this is real or somehow it's inflated and that's why people go to token maxing. So if I'm hearing you right, what you're saying is all this spend is much more legit than some of the online discussion makes it out to be.
Well, I think if I had to officially provide my own takeaway for my own point, it would sort of be there's always this experimentation phase of a new technology and this happens to be a relatively expensive technology. So thus the experimentation phase is expensive. And then what will happen is enterprises will deploy AI and then they'll sort of peel off, they'll start to see where are the real use cases, where are the ones that aren't as real. They'll wind down the ones that aren't as real. The ones that are real, they'll then look at it and say, 'Is there a way to do it at a lower cost once we understand it enough? Or do we still need the frontier intelligence for everything we're doing?' And that's actually just a pretty normal process that everybody's going through right now.
But I think about it like our engineering team, we are not token maxers in the sense of there's no leaderboard. We're not incentivizing overuse of tokens. We're just saying use it as effectively as possible to get your work done faster and our growth rate of spend is exponential and we're totally happy about it. Nobody internally is—other than like, ah, we got to shift some things around and make sure we plan for this even more next year. That's obviously a stressful conversation but we're not stressed about the idea that we're spending on AI. We're quite excited about the productivity gains that we get.
And so I think what's happening is every enterprise is having to go through their own journey on that. They're deploying it in some teams and some teams are saying oh my gosh this is the greatest thing of all time and then other teams you kind of look at what they're doing you can't see any measurable improvement in the output of that organization and so then you're like okay maybe it's not as effective there. But I would say I think it's very easy to capture one or two anecdotes and then overextrapolate on the overall themes. I would say the vast majority of the current agentic spend that's happening is sustainable because partly because it's actually coming mostly from engineering and engineering-related tasks and this is an audience that is kind of technically capable of determining whether they like the work product that's coming out of the AI. Maybe as it gets to other parts of knowledge work, those people will not be as familiar with how to do the ROI measurement and then it'll get even messier. But so far I think it's actually been largely totally reasonable.
Okay, we have a couple minutes left. Let's do a small lightning round.
So, my first take here is that Apple Intelligence is really good. It's going to be really good now. What do you think?
Oh, is it lightning round or do you want to hear a five-minute answer round?
Give like a 60-second answer.
I mean, what could be easier than pressing a button on your phone and talking to it? And at least based on the announcement they've taken Gemini, which is a very good model, and been able to—I don't know if it's fork or distill or something—within there is Gemini-grade intelligence. So if you get Gemini-grade intelligence and voice on your phone, press a button, I think you're just going to use that for a lot of things. And then I think the exciting thing is imagine that hooked up to various apps on your phone and you're like hey order this thing for me or go and add this calendar entry. I think those are very plausible daily use cases that we will have. And it's exactly the sweet spot for Apple to kind of own that space.
Yeah. No, I think Apple did it finally. That was good. That was a good time.
Are you going to tell me if my answer is right at the end of each one?
Okay. That's—yeah, this is good. Okay. So we agree one for one. How about this one? Permanent underclass.
I don't like this one. This one I don't like at all. Not only do I disagree with it, but I think it's just a bad meme to have in the atmosphere. I think it's not good for college students coming into the workforce, of having so much stress about what company to join and what's going to play out. I do think companies actually do the job market a disservice though by not being as clear on their own philosophies on this. Which some of it is reasonable because it's like, oh man, we're just getting thrown through a loop. There's so much innovation. But I do think companies need to be somewhat clear on, hey, here's how we want to use AI. We want to use AI to accelerate our work or accelerate our technical—
Innovation or accelerate our ability to hit customers. Um, versus, you know, no, we're actually like our metric is as few employees as possible, you know, with AI, like you kind of do want to, you know, be able to have some stance. And I think companies have been very confused. Um, and that lets this meme somewhat persist, you know, for the internet.
Okay, I won't rate that one. Thank you. Um all right, last one. Is is the SpaceX performance good or bad news for OpenAI and Anthropic?
Oh, well, it's obviously good news. Um,
you don't think Elon took some of their money because he pitched the market on an AI company and that's where the money got funneled into?
I'm not sure. I've seen like a limit of appetite of of if for for I mean there there's there's a literal limit of money in the world, but I don't know that I don't know that that is zero sum at this stage. Uh so I I think um I think people are pretty clear that that you know if the revenue of this entire category of the frontier models and the infrastructure stack is is measured in the trillions then you can have you know 20 companies that that all take a piece of that at different layers of the stack. So So I'm I'm not I'm not sure I I would be convinced that that would be zero sum.
I actually did. Okay. Um, not, you know, uh, I I'm uh I don't know if I'm embarrassed or not, but I'm not gonna say the amount of shares, but I I wanted to like be a part of the movement. So, I'm on Robin Hood buying my my my retail shares of uh of SpaceX. I'm I'm up like 15 bucks now. Um, per share. Per share, but uh uh so I'm I'm happy. Yeah.
Amazing. Well, Aaron, you know, you uh you answered my email uh when we were just at the very start of this podcast, four episodes in, came on the show. I feel like every single time we talk, something crazy is happening.
That's a guarantee at this point. So, boy, are we in the thick of it, right?
Yeah. Awesome. Good to see you, sir. Thank you so much, Aaron. Thank you, everybody.
Are we off to a good start? Let's hear it. Let's hear it one more time for Aaron. We're gonna We uh we need your energy here for a couple of reasons. First of all, we're live streaming this. We want to make sure that your presence here is felt in our live stream. Um and then we go from like two chairs to four chairs. So, we're just going to need you guys to help us fill that time. Um so, uh AI infrastructure, right? We're in the middle of the greatest infrastructure buildout of all time, bigger than cable, bigger than the railroads. Um, we're going to see $700 billion in capital expenditures this year. And that means that some of the biggest questions that we have about this AI moment are actually things that we can find out from the infrastructure discussion alone. Questions like, are we overbuilding? Will these data centers that are announced ever get stood up? And of course, will all this added compute actually lead to better AI models or is it misguided? And so to do it, to have this discussion, we're going to speak with three of the best AI infrastructure reporters in the world. Ana Gardez from the information, Max Churnney from Reuters, and Lauren Good of Wired. Let's give it up for Max, Ana, and Lauren.
Hey guys.
Hi. Hey, hey, hi everyone.
So, um, let's start here. There's there have been headlines that, um, of the announced AI data centers that are supposed to come up, something like 50% of them are actually being built. Um, is that the case? And if so, why? Ana, do you want to lead us off?
Sure. I totally believe that statistic and I think it might actually be higher if you include announcements. Um because of all the planned data centers that are underway, I think it's highly likely that many will be delayed due to higher costs, how hard it is to get labor. But then the number that I'm keeping close track of is announced projects versus actually projects that are being built. And I think we've seen some, you know, pretty crazy announcements from companies like OpenAI with all of these different 10 gigawatt, 6 gigawatt projects. And those are numbers that I'm paying a lot of attention to because I think we need to sort of back into them and say, okay, if you wanted 10 gigawatts by this date, how many do you have today, was that a real commitment? How firm is that commitment? But on on projects that are actually getting built, I do think, you know, 50% not really getting done on time is is, you know, what I would expect.
Wait, what percentage would you say have been announced but not start not included in that 50% number?
Um, you know, when I think of that number, it's I mean, it's quite high. Like even if you think of OpenAI, for example, announcing a 10 gigawatt project. Um, you know, we're not going to see 10 gigawatts in the next couple of years or they're going to sort of spread out that bet. And I think that's a really good area area for reporters to look at is if you just back into announcements, it's a really good way to say, 'Hey, this project is not not on track.'
Yeah, it's kind of crazy because if you look at the way that stocks have been traded publicly, a lot of the market action that we're seeing is entirely dependent on those announcements coming true. So, Max, let's go to you. Um, what are the consequences going to be if this these announced buildouts don't materialize?
Well, I think there's uh a lot of shareholders of these public companies that are going to be pretty frustrated. Um Microsoft, Amazon, etc. have been making big capex bets as I think everybody in this room knows. They've been going to the market to get debt now, which is, you know, you know, I think shareholders would be pretty interested. Uh in terms of the consequences though, I mean, I'm a chip reporter, so that's sort of where I that's how I kind of think about it. Um and it's very likely I would say that we're going to lead we're going to see some overcapacity uh essentially. So like people are building especially memory companies are building tons of factories right now and typically the chip industry is again I'm sure everybody in this this room knows uh tends to be cyclical. So the whiplash from this one might be pretty bad um depending on exactly when it comes to an end if it does.
I think what Max is saying is that it's good for his job security because the more news there is the more he'll have to report on. Yeah. Oh, you guys are not going to get a break anytime soon. Not at all.
No, I can't I'm very I'm very curious if the if there is a bust what that looks like exactly and what precipitates it. I think it'll be a lot of fun to cover.
Well, one of the big questions
Fun. I also think if I can say too, I think this is a particularly unique time because not only do we have these uh incredibly highly valued, ambitious frontier labs uh that are getting involved in these circular deals, but also we're in the middle of a memory shortage. Um there's this unrelenting demand for compute and also this is a midterm election year and so I think you're going to see a lot of uh politicizing of the data centers too as uh you know sort of lawmakers try to appeal to their base because a lot of people are very unhappy about data centers.
Yeah. I mean they're not polling well at all.
No. So um you know when you think about where the collapse might happen um a popular thing for people to discuss is well maybe Nvidia which has been making such premiums on its hardware um it can't sustain it or it gets caught. So Lauren you spent a lot of time with uh Jensen Huang from Nvidia um what do you think about Jensen would enable him to sustain uh Nvidia's lead or do you think that some of these skeptics have a point?
Uh yes and yes I think some of the skeptics absolutely have a point and I think once you reach the uh the sort of uh is it the zenith is there the nadir I always get those two confused that Nvidia has um you know people are always sort of looking to to take you down a peg and compete but I think Nvidia and Jensen has been incredibly good in Nvidia's history at sort of pivoting the company at exactly the moment that they need to in order to make sure that they've sort of caught the next wave. Um, and we certainly saw that happen with um, not only like you know the GPU to begin with and parallel processing, but then again with sort of pivoting towards crypto which ultimately meant they were in good place for AI and now we see the company doing that by addressing the inference market a lot more closely too and you know Jensen coming out and making these big proclamations that actually they're the biggest CPU maker in the world which I know Intel and AMD must be thrilled about. So, uh, you know, I think he's very smart and very strategic and that there's a good chance that they do maintain their dominance.
I mean, I think I think the question is like how much of the inference market they're going to get. I mean, it's it's whether it's 80% or 40% or somewhere in between or or 90%. I mean, I think that's what everybody's fighting out at the moment. I don't think there's a question that they're going to have some big chunk of it. Just it's just how much, you know?
Yeah. Yeah, I'm going to go to Ana in a moment, but uh Lauren, I just want you to tell us a little bit of uh what it was like uh with Jensen on a cover shoot for Wired.
You're talking about when I brushed his hair.
Yeah. Yeah. I'll be brushing I'll be lining up later to brush people's hair if anyone would like to.
Uh yeah, so I did a cover story on Jensen for Wired a couple years ago and uh our Wired art department is worldclass by the way. And so we had this big photo shoot set up and I asked if I could tag along to the photo shoot because I just wanted 15 more minutes with him to ask him some follow-up questions about Blackwell. And so they let me tag along down to the office in Santa Clara, set up this whole set for Jensen, who had like, you know, exactly two minutes to give us. And as the photographer was looking at Jensen under the bright lights, the photographer said, 'Oh, he's got some flyaways and that like gorgeous silver hair that he has.' And then he said, 'Does anyone have a brush?' Silence across the set. No one at NVIDIA apparently had a brush. I was like, 'Okay.' Uh, and as anyone who has long hair knows, you always have a brush. So I said, 'Well, I have a brush.' So then I went to my backpack and got the brush. And then I said, 'Okay, here's the brush.' And no one moved. And meanwhile, Jensen is like, 'What are we doing here, folks?' And I I said, 'Oh, okay.' So, I walked up to Jensen and I took out my brush and I got really close to him and I said, 'Jensen, we're about to get a lot more close.' And he said, 'Oh, Christ.' And um and then I brushed his hair and I have to say, I did bring some evidence of this. I think it looks great, frankly.
Oh, wow. I think it turned out really well. So, yeah. Now, the unfortunate the unfortunate conclusion to this story very quickly because I know we need to move on is that people were joking afterwards like you know the net worth of every individual strand of hair on that brush, right? Like like you could sell this on eBay and which I was obviously not going to do as an ethical journalist. But then like several months later, my back my backpack was in my it was my gym bag. It was in the back of my car and um downtown San Francisco, I think you know where this is going. My car got broken into and the brush got stolen. So those thieves have no idea the value of what they got away with.
They are cloning Jensen as we speak.
Yes. I'll see you in the back afterwards if you on your hairbrushed.
Ana, can you talk to us a little bit about what we've kind of hinted at at this point that there's this, you know, Nvidia, the common perception on Nvidia is their GPUs were great for training, but you can actually do the inference or the act of using a model on a variety of different chips. And once people once these labs train their models and they're happy with their models, most of the computing is going to go to these inference chips. Um, and therefore, even if Nvidia has a bet there, they're not going to be able to sustain their dominance. Is is that what do you think about that? Is that a potential flaw in the armor for Nvidia?
I think it is a flaw and I think there's if anyone's going to sort of attack Nvidia's dominance they're going to do it on inference like you said and there is sort of a massive effort underway right now to make all inference chips under the sun work well and so every single company that buys Nvidia chips and is spending a lot of money on NVIDIA chips is trying really hard to make these other other chips work whether it's in-house chips from Google or Amazon or even in OpenAI's case and potentially Anthropic's case you know do we make our own inference chip. Um, so I'd have a hard time, you know, believing that none of those chips are going to pan out, but you know, they might not tackle the bulk of the inference workload even. But then again, even if they do 10% of your inference, maybe you're saving enough money that you think the effort is worth it and it gives you negotiating leverage with NVIDIA. If they know that you have an in-house chip team, you know, Jensen's going to be a little bit worried when negotiating with you when you're playing hard ball with him. So, I think everyone's going to have to have an inference chip answer to Nvidia. But when I talk to data center companies about what they're seeing, um, you know, some data center companies don't really care what chip goes inside of their data center, but they they try to get hints from the companies that they're working with. Um, on the data center side, they're seeing a lot of NVIDIA and in the instances that I'm seeing where there are non- NVIDIA chips, it's because of some sort of financial backstop. So, I do wonder how much how long that will have to keep being the case because that will definitely hinder non- Nvidia chips if they need special financing arrangements to get inside the data center.
Am I wrong in thinking the AI world is sort of separating on two poles? There's the Nvidia OpenAI pole and the Anthropic Google Amazon pole, right? So, it's almost two separate ecosystems competing with each other.
I think OpenAI is investing a lot in non-Nvidia hardware.
Are they trying to move away from Nvidia?
Yeah. Yeah, they are. Um they're they're using chips from other companies. Um Cerebras is a good example and they have their own in-house chip. Um so I think maybe publicly they're they're doing some big announcements with Nvidia and I believe it's 5 gigawatts they have to deploy in the next couple of years on Vera Rubin. That's that's a lot and that that in itself might hinder them from doing more. But I think um I think Sam Altman wants to diversify from Nvidia.
Because you know just like any other company it's so expensive and I don't you know they have their own in-house silicon effort. I don't think they they think that they know their model better than anyone else like their their AI model and so I think they think that they're the best to develop a chip that can run their model efficiently.
Max, you're back and forth to Taiwan very frequently. Um when we think about the position of Taiwan and Taiwan semiconductor in this world um you know the fact that it's in this like tenuous geopolitical place is something people tend to be like oh okay and then move on right away and I want to hear your perspective on whether uh the independence of Taiwan and the stability of TSMC is this like hidden black swan event that's just kind of in plain sight that people are not paying attention enough attention to.
I mean, it it is uh for some reason the modern world decided to to decided to put all of our chip manufacturer or most of it next to uh Kim Jong-un and the rest of it is in China is in is in Taiwan. I I don't really understand who decided that or why we decided that, but that's that's the nature of the beast. Um chip companies, the design companies in the US do not plan for this. The contingency the contingency plan, excuse me, is something along the lines of like, well, we're all kind of screwed if China invades. Um which okay, but like there's no real plan there. Um so, and when I say invades, I don't necessarily mean like a literal invasion. I think what's a lot more likely is some kind of soft power exchange like what happened in Hong Kong over time. Um I think realistically that's that's a more likely option. Although I'm not saying Xi Jinping won't just invade. Like he absolutely will and he said he would. Um which could be catastrophic for TSMC and the modern world again. Um, I I do know that chip companies, as I said, there really isn't much planning and much consideration of it because it's it I think people in the industry just think it's an event that's just so crazy, complicated, and out of this world that nobody's going to do anything about it. So,
Max, do you want to talk about what happens when you try to visit TSMC?
They send me to the gift shop.
Uh, so you're in Taipei and that's where you go.
That's Yep. That's basically it. They they got a a museum where they have like um I don't know a bunch of chips that they've made over the years. One of the Cerebras chips is in there. They're very proud of it. The dinner plate size one. Um and I've never been to the museum, but uh that's where they wanted to always that's where they always want to take me. I think uh Wired is one of the few organizations, news outlets that's actually been able to go inside of one of the factories. Um and they wrote an interesting story about it. Um but yeah, I do not get to go to the TSMC Fabs unfortunately. They're not they're not all that interesting inside. Like they're cool if you've never seen one, but um they mostly look the same. It's like a bunch of white boxes with robots moving around. Um so it's not like people that know what they're looking for. It's helpful because you can like count the number of EUV machines in there and figure out roughly how many chips they can make and like do that kind of thing and figure out what kind of tools they're using. But um if the tours that most chip companies give are not particularly enlightening um in general, they are fascinating though. Yeah, they're cool, but they're just they're not useful. In in like a meaningful reporting sense or an investor's, you know, from an investor's perspective or something like that.
I had gotten very excited about TSMC. Did a ton of reporting, spoke with former employees there and then got the courage up and I called them and I said, 'I'm ready to come to Taiwan.' And they're like, 'You can come to the gift shop.' And I was like, 'All right, well, I guess I'm not coming.'
My understanding is that our freelancer who got in there spent about a year negotiating with the PR team to get in.
I've been working on it for three. So if she has any tips like I could I could definitely call up Virginia Heffernan.
I will. Absolutely.
Lauren, you've been inside a fab in Arizona. What's it like?
Yeah. Well, like this was the um Intel Fab 52, which is their most advanced fab. It's a 2 nanometer fab and it is on the level of what TSMC does in terms of 2nm, but of course at a fraction of the scale of what TSMC does. And to Max's point about, well, what is our redundancy or our resiliency plan in the event of an invasion of Taiwan? It seems like right now the US government is very interested in what Intel can do in its fabs, but um it's pretty small. Uh and so yeah, I went to Chandler, Arizona. Um saw Fab 52. Um you know, got the bunny suit on and um like Max said, it's it's interesting to see uh everything is roboticized. It's like if you had this um utopian fever dream of what the future of manufacturing looked like and everything was white and everyone was dressed in all white and everything's like being you know moving around. They call them FOUPs that actually carry the silicon around above head and then they just sort of automatically go down to get like etched and carved and stamped and everything. Um it look it looks like it feels and looks like science fiction. But from a reporting perspective, oftentimes the companies will cover up the names of the vendors on the different machines and you may be able to get some insight. Um, for example, there there were ASML EUV machines in these giant bays the size of school buses, but then you might see some spaces where they're like, well, we have two more coming and we're waiting for them. And so you're like, okay, what does that mean? you know, EUV is obviously a very um you know, a a very coveted sort of technology and they're also very expensive and you know, so so you get some insights from that, but uh primarily it's just kind of um it's good for context for better understanding how this all works, right?
I was not allowed to go to this tour just for what it's worth.
Max, I think you're doing something wrong.
Max, you got to work on your emails.
Um so, just kidding. That means you're doing actually a great job.
Thanks, Lauren. Uh there was an there was an interesting uh moment in the Dareios Jensen interview which I guess will come up in every session today. Um where where basically what Jensen is trying to get across to Dareios is if you do not provide chips to China um they're going to be there's going to be a constraint. They will develop their own models on their own chips. They could become the global standards and then put export controls on the US. Does anybody here think that that is a legitimate concern or do you think Jensen just wants to sell chips in China? wants to sell it to chips in China.
Okay, I mean it might be a legitimate concern in some sense, but Nvidia has run out of places to grow. Uh they can't sell to any more countries. They can't sell to any more hyperscalers. They can't sell to any more even they're going after the enterprise market, which is messy and complicated and difficult to actually make a lot of money in. So, China is a big market that has basically no access to Nvidia chips right now. I mean, you know, if you're getting those kinds of gross margins, like, yeah, of course, that's like an obvious just a very obvious place to to be able to sell chips, the company's kind of obsessed with it at the moment. And it's why Jensen goes to Washington all the time. That's why he insisted on going to China. I mean, it's it seems pretty clear to me.
Max spends a little bit more time thinking about that than I do. Um, I I do understand, you know, obviously he wants to sell to China. It's a huge market and they're taking a hit because they can't sell to China. So, you can't really answer the question without acknowledging that. Um, but I I do think that, you know, from the national security perspective, um, Chinese companies are developing AI chips um, no matter what happens. Um, so, you know, whether we can or can't, I do think that they'll continue to develop alternatives to Nvidia.
I mean, their manufacturing process is is a lot worse. Like the one of the uh research firms just published an analysis of the sort of one of the new Chinese chips like homemade um by SMIC and it's like it's good but they're they run up to up to the limit of like they can't buy EUV machines um the big ones that Lauren was just talking about. So if you can't buy those things it's just fundamentally difficult to make transistors below a certain size and they can't do it effectively.
Yeah. And to Max's point, Nvidia only has so much more room to grow. I mean they're at the point where they have become a major investor in CoreWeave and then CoreWeave is using that money to buy Nvidia chips and so it's it's you know it's really and there was an earnings call um earlier this year maybe late last year where uh Nvidia said that they had a domestic buyer of some of the chips that they weren't able to sell to China though they not disclose the buyer and so they're at the point where they're basically yeah trying to sell them wherever they can and so you'd have to imagine the motivation there is they want to sell chips to China.
No those leftover H20s that they were Yeah.
China that somebody else wanted them because there's people want silicon right now so badly any any kind will do.
I think they sold about was it $600 million worth of those?
Yeah. Okay. So, a minute left. Let's do this an actual lightning round. So, yes or no? Uh do you think that uh are you all bitter or less pills? Which is basically like the reason why there's all this infrastructure is because the AI labs think the more you know GPUs and chips you string together the better the models will get. Do you think that it will continue to improve as these buildouts uh you know continue to to explode in size or no?
Wait, ask the question once more.
Is is it is it is the investment worth it in in these data centers or are they ultimately like not going to have much better models even though they have a lot more chips?
Are we going down the line here? You start with us. So you start with me. Um generally yes.
Uh not not sure. Sorry. Okay. Okay.
I think it depends on the amortization what ends up happening there.
Okay. Ana, you announced your news on Twitter, right?
Why don't you tell everybody where where you're going next?
Oh, um July 6th, I'm starting a new job at the Wall Street Journal.
All right, let's hear for Ana.
Congratulations. And thank you, Lauren and Max, for joining us as well. Let's hear for them, the whole panel. Thank you guys.
Thank you very much. So, so often the story of AI's trajectory is is told without the people using it. We hear about these concepts called token maxing that are talked about in the abstract and then one day you see a chart and you never actually speak to the people spending the tokens. So I think to fully understand how this technology is progressing and its potential to meet the way that people want it to go, we actually we actually have to speak with the people who are building with the tokens. And that's why I'm thrilled to bring on Dallas Dolan, the the TMT leader at PWC, who is actually implementing this AI, looking at the costs, and making sure that it's worth the money that they're spending on it, and will take us deep into the token maxing and the budgeting conversation. So folks, let's give it up for Dallas Dolan of PWC.
Great to see you, Dallas.
The tables have turned. You're interviewing me. This is good. Uh what do you think about Aaron Levie was here like 10 minutes ago and he talked about how token maxing was a BS uh um media narrative effectively and it never really happened.
I don't know if I totally agree with it. We were actually talking backstage just before he came on and um I think there's absolutely like we'll call it above above the you know above the plane sort of commentary that's out there that uh we're all seeing in in both the mainstream media as well as like what plays really well on on social media on X and other places um and in podcasts. And then there's there's the reality within a lot of organizations, but it's it's happening enough where the behaviors, I'll call them, are slightly problematic from a cost and from an ROI point of view that it's real, right? You can't say that every circumstance is a problem necessarily, but it's coming through in a way that, you know, it's enough to think about and say, hey, are we doing this the right way, right? Broadly speaking from an enterprise strategy and also then from even a broader ecosystem point of view.
Yeah. So, there's this moment now where we're seeing actually a counter to token maxing. Uh, it's called token minimizing. And you're seeing companies like AT&T and Meta um get really serious about cost. We also know that Uber of course uh spent their entire budget in less than half a year. Um, who do you think is going to win out at the end of the day, the token minimizers or the token maxers given the definition of token maxing you you just gave us?
Yeah, I mean I think here here's the good news. The good news is I don't I don't think there's a winner and loser that's going to be defined by did you token max or not token max. I think the winner and loser is going to be defined by did you outcome max or not. Um it's in part going to be a function of how did you incentivize people which goes into that leaderboard and the things that got a lot of folks will say in trouble or certainly in the news. Right? So there's that piece of it. It's the how do you actually want to encourage people to do it without encouraging the wrong things. It's take it too far etc. Right? So there's that piece and or spend too much money. And then the other part of it is going to be actually I think from a planning point of view within an organization and I deal with this as well. So I sit on the the the boards for for our US and and our global organization at PWC and we talk about this a lot. In fact we talk about this even contextually from a comparative point of view within different industries. So it's not just saying hey does one organization as a services company spend more or less than another but also how are we compared and spending you know against let's say like a tech company who's you know got a bunch of engineers and doing the coding there. And I think what we're looking for is a what's like the baseline, right? So it's it's benchmarking with industry out with within the industry, outside the industry and against a given benchmark and saying, okay, are we actually getting an ROI? Are we way outside the bounds in terms of what we think the spend is or what we know the spend is, right? Some of the companies that you've named are certainly companies that we're aware of and work with too. Um, so we have some decent intel on what's happening within the industry from that point of view. But it is going to be that like what outcomes did you get? And I will tell you, I've seen some things that people have built, for example, in the deal space that's just incredible. Like really, really amazing output. It's taking, you know, thousands of hours worth of work and creating a product within seven minutes. The cost is very high, but it's an amazing output and it does show very well from a consumer or from a, you know, a customer point of view. It's also very exciting for, you know, the folks who work in the industry who don't have to stay up all night maybe working on something. That's great. There's other processes that people haven't even attacked yet. And I think the question is when do you actually go after each one of those and say how am I going to outcome maximize for each one of these processes. That's actually when you're going to see I think the benefits come through. There will be maximization of spend but it'll be maximized in a way that you're actually getting an outcome from it not just because you spent the most money.
So you're already doing ROI calculations.
So you're probably far ahead than most folks. Um how do you determine the ROI and and what percentage of your projects would you say are actually generating a positive ROI?
Yeah. Well, I'll tell you this. I know I mean even from an external point of view, I think MIT did did a recent publication talking about like what sort of uh activities are really, you know, replaceable just with generative AI in general, especially like in the vision and and you know, human interactive space. And I think they came up with a an outcome of about 23%. Um, as in you wouldn't use a human to do 23% of the work. I look at that and say in our business it's probably not as high as 23%, but it's going to be some percentage of every single thing that people do, right? So it's it's a different mathematical equation. It's is absolutely going to be measured for our business. It's going to be measured in hours. The same thing that's going to be done in the engineering space for a lot of these companies as well. It's a measurement of hours. It's are you making the person more efficient? Are they creating more output, you know, for the amount of time they're spending on it, right? So is that lines of code they're producing or is it quality product? Which this is an interesting thing, right? Like I could build a larger slide deck or I can write way more lines of code, let's say, but am I actually getting a product at the end of the day that people are saying, 'Oh, yeah, that's something I'm willing to use.' And I had that conversation a bunch actually at uh at Tech Week in New York a couple of weeks ago with a number of a number of founders and also a number of the investors right from Andre and others who were talking about the types of companies that they're investing in. I think it's really interesting, right? If you're an investor, you say, 'No, like you know, what does your product do, you know, for the person who's using, especially if it's a coding product or something along those lines.' If you're not indicating that you're able to help them actually build a product, right? Come out with something a product or service that's better that their customer is going to want to use, the fact that it produces more lines of code or a longer slide deck or a longer pitch deck or a longer memo is not better. Right? I think we we're going to come to a determination, Alex, really soon here where we start to say, 'Oh, wait a second. Like, what am I really trying to get to here? Is it more more more more or is it actually a better product that I'm trying to get?'
Right. But that sort of goes to the way that the foundational labs and the cloud players are working with you. The things that I've heard is that these companies are making it so that when you plug into their systems, you're going to spend a lot of tokens and they don't really want you to be able to measure them. Um, are you finding that?
So the I think the big counter to you know the the um maybe the the marketing and sales approach there is in the control plane that's being used to actually help companies make decisions on what model's being used or what interface is being used to do specific activities. And this is where this concept of central planning and I say that as someone who's who's traveling to China next week. This concept of centralized planning within an enterprise is going to be so important. It's the you are not allowed to check the weather five times a day using you know Claude 3.5 Sonnet or whatever it might be, right? Like it's unacceptable, right? Claude is not a good use case for weather checking. Um even if you're worried like I was last night about the tornadoes going through the Midwest and trying to get back here so I wasn't late for Alex. It's still not a good use, right? You can do that still with a weather app or if you really want to use something inexpensive like there's a lot of those options out there. I think what's going to happen, you know, you the the term control plane is sort of out there. It's relatively newish. Um, it's got governance elements, it's got cost control elements, it's got access to data elements, it's got um, I'll see even like the the human interaction elements like what can you put into it, not only what you can get out of it. that is going to be the the the you know the the surface of play so to say where people are going to spend a lot of the time in the energy at the AT&Ts and the metas and the services companies and a lot of these other spaces including in the engineering space but also in the sales in marketing and in in the back office as well as they look at other activities that they will do better but they will do better with the right tool. Quick analogy, right? Like the fact of the matter is we are simply not best suited driving the Lamborghini to go pick up milk. Those two things just don't align, right? Unless you live in South Beach, in which case it's totally okay. Um, but those are the crypto people. They're not here. So, um, but in this case, like that's just not that's not what we'd want to encourage. And I think that that should eventually proliferate and get to everywhere, especially as the costs have gone up. And the last time we got together a few months ago was right at that click point of cost. and that you did a great thing with the audience talking about, hey, how much are people willing to spend which is a super fun exercise which I hope maybe you'll do later.
We could do we could do it now. Can we put let's put the house lights up for a second? Um I am curious. Sorry, we're going to ask you to vote. Um of of the folks here, would you be willing to spend double the amount that you're spending today for the current capabilities that you have uh with AI?
That's it. How many people are satisfied with the price that you're paying?
So, if if they really Okay, sorry. I'm going to go with more more questions. If they if let's say your Claude subscription doubled in price, how many people would cancel?
Is it fable or not?
Is it fable? Well, right now, no. No.
I think I think Okay, we could put the lights down again. I think basically what it what it proves is um we we saw about half the room's hands go up that they would pay double. Um, and it does seem like there's a bunch of room for the labs to be able to raise prices and still do well.
But Dallas, something interesting happened since we spoke last. So, we spoke at Google Cloud Next about this and um, someone asked about the the margin the I think Dallas, you asked whether the margin of the business can be maintained at the current prices. And I thought, okay, well, forget about it because these labs have such an economically valuable uh uh tool for us that they'll raise prices. But we might be in the moment where they're going to get into a price war because OpenAI is rumored to be potentially uh dropping prices. And so what do you think about that?
I I think we're absolutely right on the precipice of that. I think you actually saw in the audience here by comparison, we saw every hand go up when you said, 'Would you be willing to pay double when we were together 3 months ago in April?' And when when Alex asked if people were willing to pay four and five times as much, there was still a quarter of the hands in the room that were up. And we're talking a room of about 200 people roughly, right? So there was a lot of people. was a good, you know, good good sample, so to say. Um, I contrast that to what we just saw right now, and I think it's a a fairly, you know, even distribution, similar subset of people. Um, and the reality is there's there's more skepticism of value that they're getting from it, especially when you start layering on the access and the capabilities associated with some of the models that are still per seat, as well as some of the open models, which you can get access to and, you know, for free. You can do a lot of really cool things. What would you do if the models were half price or tokens were half price?
What would I think would I do something differently? Is that the question? Yeah.
Let's say I'm like Open AI or Google and I come to you and say, 'Dallas, we're happy that you're using our technology. We want to make sure that you don't use Fable when it's back online. So, we're cutting your prices in by 50%'.
Would it change anything you're doing?
Absolutely. I I think actually like I know this because I talked to my CEO and CIO yesterday and my CIO again this morning. Um and we are an enterprise sensitive environment, right? We have 350,000 people globally. Um they all have access to one tool or another. And so when you have 350,000 people and they're playing with tools, there's a high degree of price sensitivity just on sheer volume alone, right? even if not all of them are doing super productive things. Just to be clear, those aren't all engineers, but they're all pretty smart people doing pretty interesting things and they have use cases that they're like, 'Hey, I want to play with this.' And we are encouraging them to do that. We think that's great, but we're also price sensitive, right? There's an elasticity there that says the higher the price goes up, the less we'd want them to use that model. We'd want them to use something cheaper. So, if somebody was coming to us with a cheaper model, 100% the direction of travel would go, you know, to that. And we've even built that into some of our control plane technology. So there's selectivity of model and there's also recommendations within it as well. So depending on who you are and what it is that we think you're going to do with it, it automatically configures to have a specific model and you have to break the the the you know the initial setting in order to use something different. So we're already there, but I'm certain we would encourage more usage of cheaper things. There's no doubt.
Have you gotten any of those type of calls yet or you're waiting for them?
Um I I I won't go too far into it, but there's definitely conversations going right now. Yeah.
Very interesting.
When we spoke recently, you told me that you're seeing some limits with agents. And it seems to me like if we're going to see this continue, agents can't be limited. They have to be able to operate autonomously and spend all those tokens and be effective. So, what are the limits that you're seeing with agents today? And do you think that if we could extrapolate a little bit, it means we're going to see some more speed bumps as the labs try to roll this technology out further?
You use the term limit. I think it's a function of both risk tolerance as well as cost tolerances, and then finally what expectations you have of these things doing on their own. The limitations are in all three areas. From a risk tolerance point of view, people are worried an agent without governance could go anywhere within their organization. From a cost perspective, there are things humans can do not only better but more cheaply, especially depending on if you're using a model at a super high premium. And the third bit is really in the decision-making from an organizational point of view. I was actually at a funeral this morning for my grandmother who was born right up the street here in San Francisco. I was talking to a priest who wrote a paper along with the team at Anthropic, and we had an interesting conversation about where ethics play into this broader conversation. It's workforce planning and then some. There's this additional component about the human benefit side — do I want to scare all my people that I'm watching what they're doing and waiting to replace them? The answer is absolutely not. I met six of our interns on the plane back from Chicago last night and they couldn't be happier to be interns with us.
You're still hiring interns.
Absolutely. We're hiring as many people as we did last year and we're changing who we're hiring. We're actually hiring a lot more kids who are pursuing sciences and even in areas beyond engineering.
And where are you not hiring?
I think we're hiring a little bit less in pure accounting. It's also a function of where we're doing it and globalization. We're hiring less people in certain areas but that's a function of how we're delivering services, not because technology will replace them. That goes back to your original question about the limits of agents. The limits are going to be a tolerance for error, a tolerance for cost, and a tolerance for what will be acceptable within the organization of having that thing do versus having a human do it. There's also the question of how do I run my business and how am I a positive leader within a given community. It doesn't matter what type of company it is.
Dallas, I want to take a moment just to acknowledge your grandmother, and I'm sorry about her passing.
She spent her life here in San Francisco.
Here in San Francisco. Yeah.
Can you tell us a little bit about her just briefly?
Sure. She was the daughter of an immigrant family who picked things in Argentina and Hawaii, and they thought moving to San Francisco would be a better deal. Her parents worked in the canneries up in Fisherman's Wharf, and she actually worked in the telecom industry — her first job was for Bell. For Grandma Fernandez Corteho, it was really a cool existence. It's a great full-circle story. All this technology is happening in one place, the Bay Area, and there's a direct connection to the things that made this place great 150 years ago. What we talk about every day is how do we build products that make people's lives better, in much the same way our grandmas make our lives better. That's where the ROI comes in. It's not going to be back to token maxing. It's a whole ecosystem shift in how we think about outputs — am I serving my clients well? Are they getting deal maximization?
Well, thank you for sharing with us.
Yeah, thanks for that. You know, interesting San Francisco. One of the things I associate with San Francisco is it's a city where people are okay losing a little bit of control. Whether that is building new technologies that do things differently than previous generations, or the fact that this city — not everybody, but many people — love LSD and mushrooms.
You said you weren't going to mention that in our precall.
It is interesting — with agents, you do kind of lose control. It reminds me a little bit of skydiving, where you jump out of a plane and you're like, whatever happens, but I hope there's a system ready to catch me. From your position, you're deploying agents in pretty high-stakes moments. Even if you have the best governance in place, you have to be okay to a degree of losing control. So, how do you become comfortable with that?
That's funny, right? How much can you really control in the engineering space where people are doing the code, or in our space where individuals are preparing audits, tax returns, or deal reports? We're putting trust in these folks, many of whom we know at a superficial level. When you start layering technology into it, my view is that it's an augmentation of those people to make them better, not necessarily ceding all control to something I know less of — that makes me feel a lot more comfortable. If you get into a space where 100% of everything becomes agentic, like just putting in a query and travel gets booked, that's where you get uncomfortable. I appreciated two nights ago knowing I had to get back — I can ping my EA late at night and say I need help, I need a plane Thursday morning at 6am through the tornadoes in Chicago to get to the West Coast. If I push that into an agent, I might see the outcome, but isn't it neat having the human on the other line who says, 'Hey, I got you, bro. You're handled.' She's augmented — she goes into our technology, queries something, and books a flight within 30 seconds. But having her there makes the whole thing feel better. My chief of staff and others are able to do multiple people's jobs — it's one person to like twelve now. I look at this tech as an extrapolation on that point. There are some things you're just not going to stop doing, but that's totally okay. It's no different than self-driving — we are going to get really comfortable with that. I see the move to Agentic as very parallel, it just so happens to be in the physical space.
Yeah, it definitely reminds me a lot of a Waymo. You sort of white-knuckle it in the beginning and then you start to go on your phone.
That's what I've done at least.
Yeah. Whole thing. Anyway, don't take anything away from those previous comments about the mushrooms.
That's for later. That's happy hour.
Please join us on the roof at 5.
Dallas, thank you so much for being here with us. Always a pleasure to speak with you and I do appreciate your support and PWC's support of this event. So, thank you very much. Let's hear it for Dallas.
All right. Amazing. Thank you, Dallas. All right, folks, we're going to have this next session — it's going to be an audience participation section. We definitely want your questions and then we'll go to a coffee break. For many of you, he needs no introduction. Let me introduce Ranjan Ry. I first started reading Ranjan's market writing in 2021. He had written this newsletter called Margins and I thought it was terrific. By January we had decided to do an emergency podcast about a crazy financial situation, and then Ranj and I kept talking more and more. By January 2023 I wrote to him and said, 'Hey, don't you want to just come and do this every week?' And lucky for us, Ranjan said yes. Getting a chance to speak with Ranjan every Friday is an absolute joy — definitely one of the highlights of my week. Today we're thrilled to do our Friday show live here with your audience questions. We'll just run it tomorrow like a normal podcast. Please join me in welcoming Ranjan Ry.
I got them on. I'm going to take these off though. We'll get into the Snap Spectacles. This is a medium-risk maneuver because I have this microphone on.
Those are the Snapchat spectacles.
These are the Snapchat spectacles. They're the original developer beta edition though, so they're not the new ones, but I had them in 2021.
Now, everyone knows how cool you are.
All right. Should I start the...
I'm as cool as Evan Spiegel.
Is he still cool, though?
All right. Let's do it. How would I start it?
Yeah. No, no, no. I didn't even write this down. Okay, well, I'll try to do it. Snapchat comes out with new spectacles and we take your audience questions. That's coming up right after this on a Big Technology Podcast Friday edition, recorded on Thursday.
Welcome to Big Technology Podcast Friday edition where we break down the news in our traditional coolheaded and nuanced format. We're joined as always by Ranjan Ry who is here with us live with the Big Technology AI Summit audience. The way this is going to work is Ranj and I will break down one story and then we definitely welcome your questions, your prompts, your arguments. If you don't have anything, we have plenty to do, but we'd love your participation. You can line up at either of the mics. Let's start with our top story this week. I promised myself when this summit was being planned that I would not be mean to Snapchat. However, Snapchat has left me with no choice. Listeners, what we're looking at is an image of Evan Spiegel on CNBC wearing the latest Snap Specs. The headline is 'Snap Stock Falls after AR Specs Debut.' Almost 20 years since the launch of the iPhone, people are ready to think about computing differently, Spiegel said. The market reacted differently. Ranjan, Snapchat and Meta have been trying to build these AI device futures for years and this is what we're looking at.
This is what we're looking at.
You kind of ruined my surprise. Can we go to D, please?
I brought them. I had to bring them after sending Alex a photo.
Is it time for us to finally accept that we're not going to have an AR device?
Interesting. So these again, I got these in 2021. It's actually very cool technology. If you ever used Magic Leap in the 2010s, being able to paint throughout your room and walk around that painting, play games where you're chasing zombies — my seven-year-old son is probably the only fan of Snap Spectacles in the world right now. He still loves using them. That form factor and the experience is amazing. Vision Pro has not quite captured it. I don't think what we just saw on Evan Spiegel's face is going to capture it. I think eventually maybe Apple or someone will, but I don't think we're there yet. I still am betting on AR eyeglasses of some sort.
I'm going to go to questions in a moment if anyone has one. Feel free to stand up. Here's some of the social media reaction. Does anyone at Snapchat have the guts to tell leadership that these things are ugly? That feeling when your glasses are so heavy they give you cauliflower ear. Snap is the best brand in the world when you're 16 and the worst brand to be associated with when you're 21. The people who actually buy $2,000 AI glasses aren't teenagers. If you think you really want to wear always-on camera around in public, it should have to look like this. Let me make the case that it's over.
Good, because I'm going to take the other side.
I think that the iPhone series that just released — does anyone here listen to Ranjan on Fridays? You'll know which direction this might go. What's this guy been begging for since he came on the show the first time? Better Siri. Better Siri. I think they actually did it. The new Siri, if you look at the videos, looks terrific. Maybe this idea that we have to wear computing on our face sounds good in concept, but model after model, it's not. And the AI device is the iPhone.
Interesting direction. I still think the form factor — do any of you have Meta Ray-Bans or any other device like that? You start to feel as you're walking around, as you're interacting in the real world, stuff can happen. I still think AR as a form factor via glasses is going to happen. I think you have Meta Ray-Bans, you enjoy them. I didn't want to say this publicly, but I have not used them. I was on a hike, it was cold, I was ready to get to the summit and put those glasses on and the battery started blinking red and I couldn't use them. Had to use my old phone.
Let me say this — they haven't taken off as a mainstream consumer device. And if you look at the stock of every single company that's pursuing them, it's not good. Snapchat is struggling. Meta has its problems. No one's looking at the Ray-Bans to save Meta.
Snapchat, like we just said, is struggling. No one's looking at these Ray-Bans to save Meta.
Well, no, but to me, the Apple Vision Pro is the more direct correlated product versus...
You want to know what the best thing about the Vision Pro was? Find Vision Pro. They put the person who made Vision Pro on Siri and he fixed it. Mike Rockwell. I would not have guessed that the person who made the Vision Pro would be the one to fix Siri after all these years, but if that's the case, that's exciting.
So, you're still going to — can you put those glasses on one more time?
Again, I was told backstage this might destroy my microphone, but I'm going to try. Do I look cool?
No. No. I mean, even we were joking that Evan Spiegel is going to the Met Gala with his model wife. This is like the coolest person in the world and how bad they made him look. When Zuck wore Project Orion, no one really cared that much. But because Evan Spiegel is such a cool-looking dude, that's why it looks so egregious. That's my take.
So, your answer on the way to make those things work is just be less handsome.
That's it. We'll write to Spiegel and let him know.
No questions? All right. Don't be shy.
This is Ser Johal, industry analyst, flew from New York a day earlier to join you guys. I have an observation and I want to get your take. When companies have this gap between what people need today and what they're working on — which can be two-plus years out — they lose traction from the investor's point of view as well as employees and partners. Do you see that happening to Meta and others like it happened to IBM when they were living in the future with WatsonX?
Okay, thank you for the question.
Yeah, no, that's a great question. If we're talking about how the interface for interacting with a computer works, I have been begging for something else other than holding my phone and looking at it, and we haven't really gotten anything for a long time. Humane tried with the pin. I still think maybe some kind of pin is going to be around — Jony Ive's pin at OpenAI, maybe at some point. Being able to interact with all of this information, being able to process information so much more reliably with AI, I just don't want to have to keep looking at my phone. Even on the phone, I'm guessing a lot of people here dictate more to their phone right now. That's completely changed. I would have felt weird just talking to my phone and now I'm constantly using Whisper Flow and just talking to it.
Tell a story about you and your wife.
I know this. My wife is not here and hopefully won't kill me if this is being live-streamed. I am constantly dictating to my computer, to my phone. The other day, it was Friday night, we had put our son to bed. We're both on our laptops and I'm on one side of the couch dictating. I look over and she also has her laptop open and is whispering to her computer too. And I'm like, is this the future of tech? But it's a new computing interface, so I'm happy about it.
The gentleman brings up a great question, which is that things are moving so fast. How do you plan right now? Honestly, I don't know how you do it because every day there seems to be a new capability, then the capability is taken off the table. Like Fable — the tweets about Fable were people adding Dave Sachs saying, 'Please, I'll do anything for Fable back. Just bring it back.' I don't understand how any company does that. I think that would be a good topic for us to get into on a future show.
Yep. Okay, let's go this way.
Hi, thank you for taking my question. I was curious your views on — in the next five years as AR glasses evolve, whether the chunky Spectacles is the way to go, or thinner glasses with offboard compute, either wireless to your iPhone or the Vision Pro-style cable down to the battery pack, which could also have compute on board. Where do you think consumers will gravitate towards?
No, it's a great question because if you ever used Magic Leap, there was like a...
Puck. That's what it was.
Here's my hot take. Any device that requires a puck...
Well, how about this though? What if the puck is your iPhone?
Oh, that changes everything. Exactly. So which gives Apple an opportunity — if the compute is taking place on the phone in your pocket, it allows you to be much slimmer from a power perspective. You don't want a lightning cable connected to your face. But it still makes me think Apple still has a good chance in this space because the iPhone can do all the heavy lifting versus something really heavy on your head.
Can I ask — what's your name?
Kyle, what do you want to use a face computer for?
Oh, that's a good question.
See, I told you, Ranjan, this stuff is not happening.
All right, let's give Kyle an opportunity here.
