Alex Canowitz welcomes the audience to the first Big Technology AI Summit. He recounts his last live event in May 2020 during the pandemic, where he presented to an empty room while promoting his book. After that, he left BuzzFeed to start Big Technology. Now, he's hosting this summit to discuss AI's future, outlining key questions: what the technology will do next, what happens if it works, and what if it goes wrong. He introduces the lineup starting with Aaron Levie, CEO of Box.
Aaron Levie thanks Alex and says he'll try to provide some answers, joking that as someone not at a lab, he can pontificate without regulatory consequences.
Alex comments on their past interviews and introduces the first topic about the Anthropic Fable ban, referring to it as the 'Jassy mystery'.
Aaron discusses the Fable ban, explaining it through Amazon's security research and the post-Mythos atmosphere. He describes the continuum from AI doomers to accelerationists, suggesting that government evaluation of models might become standard in the next 3-5 years, which could have implications for US economic position and encourage sovereign AI initiatives in other countries, potentially benefiting China.
Alex asks if Anthropic's Dario Amodei would be happy with the situation, given his known positions.
Aaron declines to guess but notes that for those on the doomer end of the spectrum, government intervention like the Fable ban is a desired outcome, as it creates precedent for regulatory frameworks. He adds that while it might not be good for the US economically in the short term, it could lead to a more democratic process for regulating AI.
Alex recalls past petitions for AI pauses and notes that the Fable ban is effectively a pause.
Aaron agrees, explaining that export controls act as a pause by preventing non-US nationals from using the technology. He war-games the implications: other countries like China might accelerate their own AI development to hedge against US dominance, potentially leading to a commoditized middle layer in AI.
Alex asks about open-source models and whether their capabilities are catching up to the frontier, which might push development towards open source.
Aaron discusses two scenarios: fast takeoff with exponential progress favoring closed-source labs, or incremental progress with open-source models keeping pace. He highlights the role of the applied AI layer in optimizing cost and performance across models, creating a barbell effect with frontier models for orchestration and open models for execution.
Alex asks about 'Lashon Fat', a rumored open-source model from Mistral, and shows fan art, indicating high eagerness for AI.
Aaron comments on the fan art phase, noting it shows AI's cultural impact. He discusses how open-weight models allow domain-specific post-training, reducing costs and making it economically viable to train custom models for specific use cases.
Alex asks if this addresses the 'token maxing' hype where companies spend heavily on tokens without clear ROI.
Aaron explains that token maxing was a brief phase within tech, but token usage is increasing due to more complex tasks. AI costs per unit are decreasing, but tasks are becoming larger, so overall spending grows. Enterprises are experimenting and optimizing, and current agentic spend is largely sustainable, driven by engineering teams that can measure ROI.
Alex notes the exponential revenue growth of AI companies and asks if token spending is inflated or legitimate.
Aaron says the experimentation phase is expensive for new technologies, but enterprises will deploy AI, identify real use cases, and optimize costs. He adds that his engineering team uses AI heavily and sees productivity gains, so the spend is justified.
Alex starts a lightning round, asking quick questions.
Aaron gives a brief answer on Apple Intelligence, saying it's good and will be widely used for daily tasks.
Alex asks about the 'permanent underclass' meme.
Aaron disagrees strongly, calling it a bad meme that causes unnecessary stress for college students entering the workforce.
Alex asks Aaron to elaborate.
Aaron says it's a lightning round but agrees to brief comments. He reiterates that the meme is harmful and companies need to be clearer on their AI philosophies.
Alex asks another lightning question.
Aaron responds briefly, continuing the fast-paced Q&A.
Alex wraps up the lightning round and thanks Aaron.
Aaron thanks Alex and the audience.
Alex concludes the session, inviting applause.
Alex adds final remarks about the event.
Aaron says goodbye and thanks everyone.
Innovation or accelerate our ability to hit customers. Versus, you know, no, we're actually like our metric is as few employees as possible. With AI, you kind of do want to be able to have some stance. And I think companies have been very confused, and that lets this meme somewhat persist for the internet.
Okay, I won't rate that one. Thank you. Um, all right, last one. Is the SpaceX performance good or bad news for OpenAI and Anthropic?
Oh, well, it's obviously good news. I'm not sure. I've seen a limit of appetite, but there's a literal limit of money in the world, but I don't know that it's zero sum at this stage. I think people are pretty clear that if the revenue of this entire category of frontier models and the infrastructure stack is measured in the trillions, then you can have 20 companies that all take a piece of that at different layers of the stack. So I'm not sure I would be convinced that it's zero sum.
I actually did.
Okay. Um, I don't know if I'm embarrassed or not, but I'm not gonna say the amount of shares. But I wanted to be a part of the movement. So, I'm on Robinhood buying my retail shares of SpaceX. I'm up like 15 bucks now per share. So, I'm happy.
Amazing. Well, Aaron, you answered my email when we were just at the very start of this podcast, four episodes in, and came on the show. I feel like every single time we talk, something crazy is happening.
That's a guarantee at this point. So, boy, are we in the thick of it, right?
Yeah. Awesome. Good to see you, sir. Thank you so much, Aaron. Thank you, everybody.
Are we off to a good start? Let's hear it. Let's hear it one more time for Aaron. We need your energy here for a couple of reasons. First of all, we're live streaming this. We want to make sure that your presence here is felt in our live stream. And then we go from like two chairs to four chairs. So, we're just going to need you guys to help us fill that time. So, AI infrastructure, right? We're in the middle of the greatest infrastructure buildout of all time, bigger than cable, bigger than the railroads. We're going to see $700 billion in capital expenditures this year. And that means that some of the biggest questions that we have about this AI moment are actually things that we can find out from the infrastructure discussion alone. Questions like, are we overbuilding? Will these data centers that are announced ever get stood up? And of course, will all this added compute actually lead to better AI models or is it misguided? And so to do it, to have this discussion, we're going to speak with three of the best AI infrastructure reporters in the world. Ana Gardez from the Information, Max Churnney from Reuters, and Lauren Good of Wired. Let's give it up for Max, Ana, and Lauren.
So, um, let's start here. There's there have been headlines that, um, of the announced AI data centers that are supposed to come up, something like 50% of them are actually being built. Um, is that the case? And if so, why? Ana, do you want to lead us off?
Sure. I totally believe that statistic and I think it might actually be higher if you include announcements. Um because of all the planned data centers that are underway, I think it's highly likely that many will be delayed due to higher costs, how hard it is to get labor. But then the number that I'm keeping close track of is announced projects versus actually projects that are being built. And I think we've seen some, you know, pretty crazy announcements from companies like OpenAI with all of these different 10 gawatt, 6 gawatt projects. And those are numbers that I'm paying a lot of attention to because I think we need to sort of back into them and say, okay, if you wanted 10 gigawatts by this date, how many do you have today, was that a real commitment? How firm is that commitment? But on on projects that are actually getting built, I do think, you know, 50% not really getting done on time is is, you know, what I would expect.
Wait, what percentage would you say have been announced but not started, not included in that 50% number?
Um, you know, when I think of that number, it's I mean, it's quite high. Like even if you think of OpenAI, for example, announcing a 10 gawatt project. Um, you know, we're not going to see 10 gigawatts in the next couple of years or they're going to sort of spread out that bet. And I think that's a really good area for reporters to look at is if you just back into announcements, it's a really good way to say, 'Hey, this project is not on track.'
Yeah, it's kind of crazy because if you look at the way that stocks have been traded publicly, a lot of the market action that we're seeing is entirely dependent on those announcements coming true.
So, Max, let's go to you. Um, what are the consequences going to be if these announced buildouts don't materialize?
Well, I think there's a lot of shareholders of these public companies that are going to be pretty frustrated. Microsoft, Amazon, etc. have been making big CapEx bets as I think everybody in this room knows. They've been going to the market to get debt now, which is, you know, shareholders would be pretty interested. In terms of the consequences though, I mean, I'm a chip reporter, so that's sort of where I that's how I kind of think about it. And it's very likely I would say that we're going to see some overcapacity essentially. So like people are building, especially memory companies are building tons of factories right now and typically the chip industry is, again I'm sure everybody in this room knows, tends to be cyclical. So the whiplash from this one might be pretty bad depending on exactly when it comes to an end if it does.
I think what Max is saying is that it's good for his job security because the more news there is the more he'll have to report on.
Not at all. No, I can't I'm very I'm very curious if there is a bust what that looks like exactly and what precipitates it. I think it'll be a lot of fun to cover.
Well, one of the big questions
Fun. I also think if I can say too, I think this is a particularly unique time because not only do we have these incredibly highly valued, ambitious frontier labs that are getting involved in these circular deals, but also we're in the middle of a memory shortage. There's this unrelenting demand for compute and also this is a midterm election year and so I think you're going to see a lot of politicizing of the data centers too as lawmakers try to appeal to their base because a lot of people are very unhappy about data centers.
Yeah. I mean they're not polling well at all.
No. So um you know when you think about where the collapse might happen um a popular thing for people to discuss is well maybe Nvidia which has been making such premiums on its hardware um it can't sustain it or it gets caught. So Lauren you spent a lot of time with Jensen Huang from Nvidia um what do you think about Jensen would enable him to sustain Nvidia's lead or do you think that some of these skeptics have a point?
So um you know when you think about where the collapse might happen um a popular thing for people to discuss is well maybe Nvidia which has been making such premiums on its hardware um it can't sustain it or it gets caught. So Lauren you spent a lot of time with Jensen Huang from Nvidia um what do you think about Jensen would enable him to sustain Nvidia's lead or do you think that some of these skeptics have a point?
Uh yes and yes I think some of the skeptics absolutely have a point and I think once you reach the zenith, Nvidia has, you know, people are always sort of looking to take you down a peg and compete but I think Nvidia and Jensen has been incredibly good in Nvidia's history at sort of pivoting the company at exactly the moment that they need to in order to make sure that they've sort of caught the next wave. Um, and we certainly saw that happen with not only like, you know, the GPU to begin with and parallel processing, but then again with sort of pivoting towards crypto which ultimately meant they were in good place for AI and now we see the company doing that by addressing the inference market a lot more closely too and you know Jensen coming out and making these big proclamations that actually they're the biggest CPU maker in the world which I know Intel and AMD must be thrilled about. So, you know, I think he's very smart and very strategic and that there's a good chance that they do maintain their dominance.
I mean, I think the question is like how much of the inference market they're going to get. I mean, it's whether it's 80% or 40% or somewhere in between or 90%. I mean, I think that's what everybody's fighting out at the moment. I don't think there's a question that they're going to have some big chunk of it. It's just how much, you know?
Yeah. Yeah, I'm going to go to Ana in a moment, but Lauren, I just want you to tell us a little bit of what it was like with Jensen on a cover shoot for Wired.
You're talking about when I brushed his hair.
Yeah. Okay. So, wait, you actually No, I really did. Yeah. Yeah. I'll be brushing I'll be lining up later to brush people's hair if anyone would like to. Yeah, so I did a cover story on Jensen for Wired a couple years ago and our Wired art department is world-class by the way. And so we had this big photo shoot set up and I asked if I could tag along to the photo shoot because I just wanted 15 more minutes with him to ask him some follow-up questions about Blackwell. And so they let me tag along down to the office in Santa Clara, set up this whole set for Jensen, who had like, you know, exactly two minutes to give us. And as the photographer was looking at Jensen under the bright lights, the photographer said, 'Oh, he's got some flyaways and that like gorgeous silver hair that he has.' And then he said, 'Does anyone have a brush?' Silence across the set. No one at NVIDIA apparently had a brush. I was like, 'Okay.' And as anyone who has long hair knows, you always have a brush. So I said, 'Well, I have a brush.' So then I went to my backpack and got the brush. And then I said, 'Okay, here's the brush.' And no one moved. And meanwhile, Jensen is like, 'What are we doing here, folks?' And I said, 'Oh, okay.' So, I walked up to Jensen and I took out my brush and I got really close to him and I said, 'Jensen, we're about to get a lot more close.' And he said, 'Oh, Christ.' And then I brushed his hair and I have to say, I did bring some evidence of this. I think it looks great, frankly.
Oh, wow. I think it turned out really well. So, yeah.
Now, the unfortunate conclusion to this story very quickly because I know we need to move on is that people were joking afterwards like, you know, the net worth of every individual strand of hair on that brush, right? Like you could sell this on eBay and which I was obviously not going to do as an ethical journalist. But then like several months later, my backpack was in my gym bag. It was in the back of my car and downtown San Francisco, I think you know where this is going. My car got broken into and the brush got stolen. So those thieves have no idea the value of what they got away with.
They are cloning Jensen as we speak.
Yes. I'll see you in the back afterwards if you on your hairbrushed.
Ana, can you talk to us a little bit about what we've kind of hinted at at this point that there's this, you know, Nvidia, the common perception on Nvidia is their GPUs were great for training, but you can actually do the inference or the act of using a model on a variety of different chips. And once these labs train their models and they're happy with their models, most of the computing is going to go to these inference chips. And therefore, even if Nvidia has a bet there, they're not going to be able to sustain their dominance. Is that what do you think about that? Is that a potential flaw in the armor for Nvidia?
I think it is a flaw and I think there's if anyone's going to sort of attack Nvidia's dominance they're going to do it on inference like you said and there is sort of a massive effort underway right now to make all inference chips under the sun work well and so every single company that buys Nvidia chips and is spending a lot of money on NVIDIA chips is trying really hard to make these other chips work whether it's in-house chips from Google or Amazon or even in OpenAI's case and potentially Anthropic's case, you know, do we make our own inference chip. So I'd have a hard time, you know, believing that none of those chips are going to pan out, but you know, they might not tackle the bulk of the inference workload even. But then again, even if they do 10% of your inference, maybe you're saving enough money that you think the effort is worth it and it gives you negotiating leverage with NVIDIA. If they know that you have an in-house chip team, you know, Jensen's going to be a little bit worried when negotiating with you when you're playing hardball with him. So, I think everyone's going to have to have an inference chip answer to Nvidia. But when I talk to data center companies about what they're seeing, you know, some data center companies don't really care what chip goes inside of their data center, but they try to get hints from the companies that they're working with. On the data center side, they're seeing a lot of NVIDIA and in the instances that I'm seeing where there are non-NVIDIA chips, it's because of some sort of financial backstop. So, I do wonder how long that will have to keep being the case because that will definitely hinder non-Nvidia chips if they need special financing arrangements to get inside the data center.
Am I wrong in thinking the AI world is sort of separating on two poles? There's the Nvidia OpenAI pole and the Anthropic Google Amazon pole, right? So, it's almost two separate ecosystems competing with each other.
I think OpenAI is investing a lot in non-Nvidia hardware.
Are they trying to move away from Nvidia?
Yeah. Yeah, they are. Um they're using chips from other companies. Um Cerebras is a good example and they have their own in-house chip. So I think maybe publicly they're doing some big announcements with Nvidia and I believe it's 5 gawatts they have to deploy in the next couple of years on Ver Rubin. That's a lot and that in itself might hinder them from doing more. But I think Sam Altman wants to diversify from Nvidia.
Because you know just like any other company it's so expensive and I don't you know they have their own in-house silicon effort. I don't think they think that they know their model better than anyone else like their AI model and so I think they think that they're the best to develop a chip that can run their model efficiently.
Max, you're back and forth to Taiwan very frequently. Um when we think about the position of Taiwan and Taiwan semiconductor in this world, you know the fact that it's in this like tenuous geopolitical place is something people tend to be like oh okay and then move on right away and I want to hear your perspective on whether the independence of Taiwan and the stability of TSMC is this like hidden black swan event that's just kind of in plain sight that people are not paying enough attention to.
I mean, it is uh for some reason the modern world decided to put all of our chip manufacturing or most of it next to Kim Jong-un and the rest of it is in China is in Taiwan. I don't really understand who decided that or why we decided that, but that's the nature of the beast. Chip companies, the design companies in the US do not plan for this. The contingency plan is something along the lines of like, well, we're all kind of screwed if China invades. Which okay, but like there's no real plan there. So, and when I say invades, I don't necessarily mean like a literal invasion. I think what's a lot more likely is some kind of soft power exchange like what happened in Hong Kong over time. I think realistically that's a more likely option. Although I'm not saying Xi Jinping won't just invade. Like he absolutely will and he said he would. Which could be catastrophic for TSMC and the modern world again. I do know that chip companies, as I said, there really isn't much planning and much consideration of it because I think people in the industry just think it's an event that's just so crazy, complicated, and out of this world that nobody's going to do anything about it.
Max, do you want to talk about what happens when you try to visit TSMC?
They send me to the gift shop.
you're in Taipei and that's where you go.
That's Yep. That's basically it.
They got a museum where they have like I don't know a bunch of chips that they've made over the years. One of the Cerebras chips is in there. They're very proud of it. The dinner plate size one. And I've never been to the museum, but that's where they always want to take me. I think Wired is one of the few organizations, news outlets that's actually been able to go inside of one of the factories. And they wrote an interesting story about it. But yeah, I do not get to go to the TSMC fabs unfortunately. They're not all that interesting inside. Like they're cool if you've never seen one, but they mostly look the same. It's like a bunch of white boxes with robots moving around. So it's not like people that know what they're looking for. It's helpful because you can count the number of EUV machines in there and figure out roughly how many chips they can make and do that kind of thing and figure out what kind of tools they're using. But the tours that most chip companies give are not particularly enlightening in general.
Yeah, they're cool, but they're just they're not useful in like a meaningful reporting sense or from an investor's perspective or something like that.
I had gotten very excited about TSMC. Did a ton of reporting, spoke with former employees there and then got the courage up and I called them and I said, 'I'm ready to come to Taiwan.' And they're like, 'You can come to the gift shop.' And I was like, 'All right, well, I guess I'm not coming.'
My understanding is that our freelancer who got in there spent about a year negotiating with the PR team to get in.
I've been working on it for three. So if she has any tips like I could definitely
call up Virginia Heffernan.
Lauren, you've been inside a fab in Arizona. What's it like?
Yeah. Well, like this was the Intel Fab 52, which is their most advanced fab. It's a 2 nanometer fab and it is on the level of what TSMC does in terms of 2nm, but of course at a fraction of the scale of what TSMC does. And to Max's point about, well, what is our redundancy or our resiliency plan in the event of an invasion of Taiwan? It seems like right now the US government is very interested in what Intel can do in its fabs, but it's pretty small. So yeah, I went to Chandler, Arizona. Saw Fab 52. You know, got the bunny suit on and like Max said, it's interesting to see everything is roboticized. It's like if you had this utopian fever dream of what the future of manufacturing looked like and everything was white and everyone was dressed in all white and everything's like moving around. They call them foops that actually carry the silicon around above head and then they just sort of automatically go down to get etched and carved and stamped and everything. It looks and feels like science fiction. But from a reporting perspective, oftentimes the companies will cover up the names of the vendors on the different machines and you may be able to get some insight. For example, there were ASML EUV machines in these giant bays the size of school buses, but then you might see some spaces where they're like, well, we have two more coming and we're waiting for them. And so you're like, okay, what does that mean? EUV is obviously a very coveted sort of technology and they're also very expensive and so you get some insights from that, but primarily it's just good for context for better understanding how this all works.
I was not allowed to go to this tour just for what it's worth.
Max, I think you're doing something wrong.
Max, you got to work on your emails.
just kidding. That means you're doing actually a great job.
Thanks, Lauren.
There was an interesting moment in the Darius interview with Jensen, which I guess will come up in every session today, where basically what Jensen is trying to get across is if you do not provide chips to China, they're going to develop their own models on their own chips. They could become the global standards and then put export controls on the US. Does anybody here think that that is a legitimate concern or do you think Jensen just wants to sell chips in China?
Okay, I mean it might be a legitimate concern in some sense, but Nvidia has run out of places to grow. They can't sell to any more countries. They can't sell to any more hyperscalers. They can't sell to any more even they're going after the enterprise market, which is messy and complicated and difficult to actually make a lot of money in. So, China is a big market that has basically no access to Nvidia chips right now. I mean, you know, if you're getting those kinds of gross margins, like, yeah, of course, that's an obvious place to be able to sell chips, the company's kind of obsessed with it at the moment. And it's why Jensen goes to Washington all the time. That's why he insisted on going to China. I mean, it seems pretty clear to me.
Max spends a little bit more time thinking about that than I do. Um, I do understand, you know, obviously he wants to sell to China. It's a huge market and they're taking a hit because they can't sell to China. So, you can't really answer the question without acknowledging that. But I do think that, you know, from the national security perspective, Chinese companies are developing AI chips no matter what happens. So, you know, whether we can or can't, I do think that they'll continue to develop alternatives to Nvidia.
I mean, their manufacturing process is a lot worse. Like one of the research firms just published an analysis of the sort of one of the new Chinese chips like homemade by SMIC and it's like it's good but they're running up to the limit of like they can't buy EUV machines, the big ones that Lauren was just talking about. So if you can't buy those things it's just fundamentally difficult to make transistors below a certain size and they can't do it effectively.
Yeah. And to Max's point, Nvidia only has so much more room to grow. I mean they're at the point where they have become a major investor in CoreWeave and then CoreWeave is using that money to buy Nvidia chips and so it's really and there was an earnings call earlier this year maybe late last year where Nvidia said that they had a domestic buyer of some of the chips that they weren't able to sell to China though they did not disclose the buyer and so they're at the point where they're basically trying to sell them wherever they can and so you'd have to imagine the motivation there is they want to sell chips to China.
No, those leftover H20s that they were
Yeah. China that somebody else wanted them because there's people want silicon right now so badly any kind will do.
I think they sold about was it $600 million worth of those?
Okay. So, a minute left. Let's do this an actual lightning round. So, yes or no? Do you think that are you all bitter or less pills? Which is basically like the reason why there's all this infrastructure is because the AI labs think the more GPUs and chips you string together the better the models will get. Do you think that it will continue to improve as these buildouts continue to explode in size or no?
Wait, ask the question once more.
Is the investment worth it in these data centers or are they ultimately not going to have much better models even though they have a lot more chips?
Are we going down the line here? You start with us. So you start with me. Um generally yes.
Uh not not sure. Sorry. Okay. Okay. I think it depends on the amortization what ends up happening there.
Okay. Ana, you announced your news on Twitter, right?
Why don't you tell everybody where you're going next?
Oh, um July 6th, I'm starting a new job at the Wall Street Journal.
All right, let's hear for Ana.
Congratulations.
And thank you, Lauren and Max, for joining us as well. Let's hear for them, the whole panel. Thank you guys.
Thank you very much. So, so often the story of AI's trajectory is told without the people using it. We hear about these concepts called token maxing that are talked about in the abstract and then one day you see a chart and you never actually speak to the people spending the tokens. So I think to fully understand how this technology is progressing and its potential to meet the way that people want it to go, we actually have to speak with the people who are building with the tokens. And that's why I'm thrilled to bring on Dallas Dolan, the TMT leader at PWC, who is actually implementing this AI, looking at the costs, and making sure that it's worth the money that they're spending on it, and will take us deep into the token maxing and the budgeting conversation. So folks, let's give it up for Dallas Dolan of PWC.
Great to see you, Dallas.
The tables have turned. You're interviewing me. This is good. Uh what do you think about Aaron Levie was here like 10 minutes ago and he talked about how token maxing was a BS media narrative effectively and it never really happened.
I don't know if I totally agree with it. We were actually talking backstage just before he came on and um I think there's absolutely like we'll call it above the plane sort of commentary that's out there that we're all seeing in both the mainstream media as well as like what plays really well on social media on X and other places and in podcasts. And then there's the reality within a lot of organizations, but it's happening enough where the behaviors, I'll call them, are slightly problematic from a cost and from an ROI point of view that it's real, right? You can't say that every circumstance is a problem necessarily, but it's coming through in a way that, you know, it's enough to think about and say, hey, are we doing this the right way, right? Broadly speaking from an enterprise strategy and also then from even a broader ecosystem point of view.
Yeah. So, there's this moment now where we're seeing actually a counter to token maxing. It's called token minimizing. And you're seeing companies like AT&T and Meta get really serious about cost. We also know that Uber of course spent their entire budget in less than half a year. Who do you think is going to win out at the end of the day, the token minimizers or the token maxers given the definition of token maxing you just gave us?
Yeah, I mean I think here's the good news. The good news is I don't think there's a winner and loser that's going to be defined by did you token max or not token max. I think the winner and loser is going to be defined by did you outcome max or not. It's in part going to be a function of how did you incentivize people which goes into that leaderboard and the things that got a lot of folks in trouble or certainly in the news. So there's that piece of it. It's how do you actually want to encourage people to do it without encouraging the wrong things. It's take it too far etc. So there's that piece and or spend too much money. And then the other part of it is going to be actually I think from a planning point of view within an organization and I deal with this as well. So I sit on the boards for our US and our global organization at PWC and we talk about this a lot. In fact we talk about this even contextually from a comparative point of view within different industries. So it's not just saying hey does one organization as a services company spend more or less than another but also how are we compared and spending against let's say like a tech company who's got a bunch of engineers and doing the coding there. And I think what we're looking for is what's like the baseline, right? So it's benchmarking with industry within the industry, outside the industry and against a given benchmark and saying, okay, are we actually getting an ROI? Are we way outside the bounds in terms of what we think the spend is or what we know the spend is, right? Some of the companies that you've named are certainly companies that we're aware of and work with too. So we have some decent intel on what's happening within the industry from that point of view. But it is going to be that like what outcomes did you get? And I will tell you, I've seen some things that people have built, for example, in the deal space that's just incredible. Like really, really amazing output. It's taking thousands of hours worth of work and creating a product within seven minutes. The cost is very high, but it's an amazing output and it does show very well from a consumer or from a customer point of view. It's also very exciting for the folks who work in the industry who don't have to stay up all night maybe working on something. That's great. There's other processes that people haven't even attacked yet. And I think the question is when do you actually go after each one of those and say how am I going to outcome maximize for each one of these processes. That's actually when you're going to see I think the benefits come through. There will be maximization of spend but it'll be maximized in a way that you're actually getting an outcome from it not just because you spent the most money.
So you're already doing ROI calculations.
So you're probably far ahead than most folks. Um how do you determine the ROI and what percentage of your projects would you say are actually generating a positive ROI?
Yeah. Well, I'll tell you this. I mean even from an external point of view, I think MIT did a recent publication talking about like what sort of activities are really, you know, replaceable just with generative AI in general, especially like in the vision and human interactive space. And I think they came up with an outcome of about 23%. As in you wouldn't use a human to do 23% of the work. I look at that and say in our business it's probably not as high as 23%, but it's going to be some percentage of every single thing that people do, right? So it's a different mathematical equation. It is absolutely going to be measured for our business. It's going to be measured in hours. The same thing that's going to be done in the engineering space for a lot of these companies as well. It's a measurement of hours. It's are you making the person more efficient? Are they creating more output for the amount of time they're spending on it, right? So is that lines of code they're producing or is it quality product? Which this is an interesting thing, right? Like I could build a larger slide deck or I can write way more lines of code, let's say, but am I actually getting a product at the end of the day that people are saying, 'Oh, yeah, that's something I'm willing to use.' And I had that conversation a bunch actually at Tech Week in New York a couple of weeks ago with a number of founders and also a number of the investors from Andre and others who were talking about the types of companies that they're investing in. I think it's really interesting, right? If you're an investor, you say, 'No, like what does your product do for the person who's using, especially if it's a coding product or something along those lines.' If you're not indicating that you're able to help them actually build a product, come out with something a product or service that's better that their customer is going to want to use, the fact that it produces more lines of code or a longer slide deck or a longer pitch deck or a longer memo is not better. I think we're going to come to a determination, Alex, really soon here where we start to say, 'Oh, wait a second. Like, what am I really trying to get to here? Is it more or is it actually a better product that I'm trying to get?'
Right. But that sort of goes to the way that the foundational labs and the cloud players are working with you. The things that I've heard is that these companies are making it so that when you plug into their systems, you're going to spend a lot of tokens and they don't really want you to be able to measure them. Um, are you finding that?
So the I think the big counter to the marketing and sales approach there is in the control plane that's being used to actually help companies make decisions on what model's being used or what interface is being used to do specific activities. And this is where this concept of central planning and I say that as someone who's traveling to China next week. This concept of centralized planning within an enterprise is going to be so important. It's you are not allowed to check the weather five times a day using Claude 5.7 or whatever it might be, right? Like it's unacceptable, right? Mythos is not a good use case for weather checking. Even if you're worried like I was last night about the tornadoes going through the Midwest and trying to get back here so I wasn't late for Alex. It's still not a good use, right? You can do that still with a weather app or if you really want to use something inexpensive like there's a lot of those options out there. I think what's going to happen, you know, the term control plane is sort of out there. It's relatively newish. It's got governance elements, it's got cost control elements, it's got access to data elements, it's got the human interaction elements like what can you put into it, not only what you can get out of it. That is going to be the surface of play so to say where people are going to spend a lot of the time and the energy at the AT&Ts and the metas and the services companies and a lot of these other spaces including in the engineering space but also in the sales and marketing and in the back office as well as they look at other activities that they will do better but they will do better with the right tool. Quick analogy, right? Like the fact of the matter is we are simply not best suited driving the Lamborghini to go pick up milk. Those two things just don't align, right? Unless you live in South Beach, in which case it's totally okay. But those are the crypto people. They're not here. So, but in this case, like that's just not what we'd want to encourage. And I think that that should eventually proliferate and get to everywhere, especially as the costs have gone up. And the last time we got together a few months ago was right at that click point of cost. And you did a great thing with the audience talking about, hey, how much are people willing to spend which is a super fun exercise which I hope maybe you'll do later.
We could do it now. Can we put let's put the house lights up for a second? Um I am curious. Sorry, we're going to ask you to vote. Um of the folks here,
would you be willing to spend double the amount that you're spending today for the current capabilities that you have with AI?
That's it. How many people are satisfied with the price that you're paying?
So, if they really Okay, sorry. I'm going to go with more questions. If let's say your Claude subscription doubled in price, how many people would cancel?
Is it Fable? Well, right now, no. No.
I think I think Okay, we could put the lights down again. I think basically what it proves is we saw about half the room's hands go up that they would pay double. And it does seem like there's a bunch of room for the labs to be able to raise prices and still do well. But Dallas, something interesting happened since we spoke last. So, we spoke at Google Cloud Next about this and someone asked about the margin, I think Dallas, you asked whether the margin of the business can be maintained at the current prices. And I thought, okay, well, forget about it because these labs have such an economically valuable tool for us that they'll raise prices. But we might be in the moment where they're going to get into a price war because OpenAI is rumored to be potentially dropping prices. And so what do you think about that?
I think we're absolutely right on the precipice of that. I think you actually saw in the audience here by comparison, we saw every hand go up when you said, 'Would you be willing to pay double when we were together 3 months ago in April?' And when Alex asked if people were willing to pay four and five times as much, there was still a quarter of the hands in the room that were up. And we're talking a room of about 200 people roughly, right? So there was a lot of people. was a good, you know, good tea sample, so to say. I contrast that to what we just saw right now, and I think it's a fairly, you know, even distribution, similar subset of people. And the reality is there's more skepticism of value that they're getting from it, especially when you start layering on the access and the capabilities associated with some of the models that are still per seat, as well as some of the open models, which you can get access to and, you know, for free. You can do a lot of really cool things. What would you do if the models were half price or tokens were half price?
What would I think would I do something differently? Is that the question? Yeah.
Let's say I'm like Open AI or Google and I come to you and say, 'Dallas, we're happy that you're using our technology. We want to make sure that you don't use Fable when it's back online. So, we're cutting your prices in by 50%.'
Would it change anything you're doing?
Absolutely. I think actually like I know this because I talked to my CEO and CIO yesterday and my CIO again this morning. And we are an enterprise environment, right? We have 350,000 people globally. They all have access to one tool or another. And so when you have 350,000 people and they're playing with tools, there's a high degree of price sensitivity just on sheer volume alone, right? Even if not all of them are doing super productive things. Just to be clear, those aren't all engineers, but they're all pretty smart people doing pretty interesting things and they have use cases that they're like, 'Hey, I want to play with this.' And we are encouraging them to do that. We think that's great, but we're also price sensitive, right? There's an elasticity there that says the higher the price goes up, the less we'd want them to use that model. We'd want them to use something cheaper. So, if somebody was coming to us with a cheaper model, 100% the direction of travel would go to that. And we've even built that into some of our control plane technology. So there's selectivity of model and there's also recommendations within it as well. So depending on who you are and what it is that we think you're going to do with it, it automatically configures to have a specific model and you have to break the initial setting in order to use something different. So we're already there, but I'm certain we would encourage more usage of cheaper things. There's no doubt.
Have you gotten any of those type of calls yet or you're waiting for them?
Um I I I won't go too far into it, but there's definitely conversations going right now. Yeah.
Very interesting.
When we spoke recently, you told me that you're seeing some limits with agents. And if we're going to see this continue, agents can't be limited. They have to be able to operate autonomously and spend all those tokens and be effective. So, what are the limits that you're seeing with agents today? And do you think that if we could extrapolate a little bit, it means that we're going to see some more speed bumps as the labs try to roll this technology out further?
I use the term limit—I think it's a function of both risk tolerance as well as cost tolerances, and then finally what is it you have expectations of these things doing on their own. So the limitations are actually in all three of those areas. From a risk tolerance point of view, I think people are saying, wait a second, I'm worried that the agent without some level of control or governance around it could go just about anywhere. What does that mean within my organization depending on what access I give to it—from client data, whether it's code itself, and what can it do to change code. If you ask it to do one thing in one area, will it simply think it needs to do that everywhere else? There's the ability to extrapolate on a single point and what control exists there. So that's the first limit.
The second limit is on the cost variance, which is: there are just things you're not going to want it to do because, back to the MIT study, there might be things that humans can do not only better but more cheaply, especially now if you have access to Claude and you're using that at a super high premium. And then the third bit is really in that decision-making from an organizational point of view.
I was actually at a funeral this morning for my grandmother who was born in San Francisco—literally born right up the street here. It's such a human thing, right? I was talking to a priest who was there—he's actually the priest who wrote the paper along with some of the team over at Anthropic—and we're having an interesting conversation on where does ethics play into the broader conversation of what's happening in this area. It's sort of workforce planning and then some. There's this additional component that I don't think we've even really gotten into, because here we are at the center of technology and we don't think about that part of it so much. But it is the benefit side, and the benefit has a risk we haven't even really gotten into—which is, do I want to scare all of my people that I'm watching what they're doing and waiting to replace what it is that they're doing? And the answer is absolutely not, because I met six of our interns on the plane back from Chicago last night and they couldn't be happier to be interns with us.
Absolutely. We're hiring as many people as we did last year. We're changing who we're hiring—we're actually hiring a lot more kids who are pursuing sciences and areas that go beyond engineering.
And where are you not hiring?
I think we're hiring a little less in pure accounting. It's also a function of where we're doing it. It's not that we're hiring less people overall—we're hiring less people in certain areas, but that's also a function of globalization and how we're delivering services. It's not really a function of saying we don't need these people because technology will do a lot of it. And that goes back to your original premise—which is, what are the limits on the agentics themselves. The limits are going to be a tolerance for error, a tolerance for cost, and a tolerance for what will be acceptable within the organization of having that thing do versus having a human do it. And there'll be a cost dynamic to that as well. But I think there's also how do I run my business and how am I a positive leader within a given community—it doesn't matter what type of company it is.
Dallas, I want to take a moment just to acknowledge your grandmother, and I'm sorry about her passing. She spent her life here in San Francisco. Can you tell us a little bit about her just briefly?
Sure. Daughter of an immigrant family, people who picked a lot of things in a lot of places around the world, including Argentina and Hawaii, funny enough. They thought picking stuff out of the ground isn't a good deal—let's move to San Francisco. Her parents worked in the canneries up in Fisherman's Wharf. She actually worked in the telecom industry. Her first job was working for Bell—so it must be the second T in the TMT, right? For Grandma Fernandez Cortezo. Really a cool existence and a great person. A great full-circle story, right? I think this whole story of technology is such a cool thing that all of this is happening in one place, in the Bay Area. It's really neat to be able to see that thread all the way through. There's actually a direct connection to the things that made this place a great place 150 years ago—or actually, still the things that make it a great place, because it's still better to work in a cannery or work for a company than it is to pick something out of the ground, as my ancestors concluded back in the 1800s. When you think about that, I think it goes back to some of the things of what do you want to do. I think you want to create technology that makes people's lives better, in much the same way that our grandmas make our lives better. Grandma makes everything better. The things we talk about every day are how do we build products—what are the entrepreneurs trying to build that make their customers' lives better, or make something easier to do, whether it's coding or customer support. We take that thread all the way through. That's actually where the ROI comes in. It's not going to be back to token maxing—it's not going to be how much you pay and how you pay it. It's a whole ecosystem shift in the way we think about the outputs themselves. Am I serving my clients well? Are they getting deal maximization out of what they're doing with any given company, frontier firm, hyperscaler, or neocloud.
Well, thank you for sharing with us.
Yeah, thanks for that. Interesting—San Francisco. One of the things I associate with San Francisco is it's a city where people are okay losing a little bit of control, whether that's building new technologies that do things a little differently than previous generations, or the fact that this city—or not everybody, but many people—love LSD and mushrooms.
You said you weren't going to mention that in our pre-call.
It is interesting—with agents, you do kind of lose control. It reminds me a little bit of skydiving in a way, where you jump out of a plane and you're like, whatever happens, but I hope there's a system ready to catch me. From your position, you're deploying agents in pretty high-stakes moments. Even if you have the best governance in place, you have to be okay to a degree with losing control. So, how do you become comfortable with that?
It's funny, right? How much can you really control in the engineering space where you have people doing the code for you, or in our space where individuals are preparing an audit or a tax return or a deal report? We're putting trust in these folks, many of whom you know at a superficial level. When you start layering technology into it—because my view is that it's an augmentation of those people to make them better, not necessarily ceding all control from them and putting it into something I know less of—that makes me feel a lot more comfortable. If you start getting into the space where 100% of everything we do—let's say booking travel—becomes unified, you just put in the query and the travel gets booked, that's where you get uncomfortable. I appreciated that two nights ago, knowing I had to get back to hang out with you, Alex, and to make it to the funeral this morning—like, I can ping my EA late at night and say, I need some help, I need to make sure with tornadoes coming through Chicago there's at least a plane on Thursday morning at 6 AM that could get me to the West Coast. If I'm pushing that into an agent and I'm hopeful that it works and I'm hopeful that it books a flight, I might see the outcome—but isn't it neat having the human on the other line? Like Liz says, 'Hey, I got you, bro. You're handled.'
She's augmented. She goes into our technology and can quickly query something and boom, pulls everything up and she books a flight within like 30 seconds. But having her there to make that a little bit better does make the whole thing feel better.
And candidly, by the way, as it relates to what my EA does or my chief of staff and others do—they're able to do multiple people's jobs. If you think about what it was before, one person to one person—no, it's one to twelve. We're doing a lot more with a lot less. So I look at this tech as just an extrapolation on that point. Yeah, there's some things you're just not going to stop doing, but that's totally okay. It's no different than driving—it's like the self-driving dynamic. We are going to get really comfortable with that. I see this move to agentic as very similar in a parallel run with that itself. It just so happens to be in the physical space.
Yeah, it definitely reminds me a lot of a Waymo. You sort of white-knuckle it in the beginning and then you start to go on your phone.
Exactly. That's what I've done at least.
Yeah, the whole thing. Anyway, don't take anything away from those previous comments about the mushrooms.
That's for later. That's happy hour.
Please join us on the roof at 5.
Dallas, thank you so much for being here with us. Always a pleasure to speak with you, and I do appreciate your support and PwC's support of this event. So, thank you. Thank you very much. Let's hear it for Dallas. Thank you.
All right. Amazing. Thank you, Dallas. All right, folks, we are going to have this next session—it's going to be an audience participation section. So we definitely want your questions for us, and then we're going to go to a coffee break. For many of you, he needs no introduction. Let me introduce Ranjan Roy. I first started reading Ranjan's market writing in 2021. He had written this newsletter called Margins, and I thought it was a terrific newsletter. I saved it, I spent my winter break reading it. I DM'd him. By that January, we had decided to do an emergency podcast about a crazy financial situation, and then Ranj and I kept talking more and more. By January 2023, I wrote to him and said, 'Hey, don't you want to just come and do this every week?' And lucky for me and lucky for us on Big Technology Podcast, Ranjan said yes. Getting a chance to speak with Ranjan every Friday is an absolute joy—it's definitely one of the highlights of my week. Today we're thrilled to be able to do our Friday show live here with your audience questions. We'll just run it tomorrow like a normal podcast. So I hope you're ready—we definitely need your participation. And please join me in welcoming Ranjan Roy.
I got them on. I'm going to take these off though. We'll get into the Snap Spectacles. This is a medium-risk maneuver because I have this microphone on. These are the Snapchat Spectacles—they're the original developer beta edition. They're not the new ones, but I had them in 2021.
Now, everyone knows how cool you are.
I'm as cool as Evan Spiegel.
Is he still cool, though?
Yeah. No, no, no—I didn't even write this down. Okay, well, I'll try to do it. All right: Snapchat comes out with new Spectacles, and we take your audience questions. That's coming up right after this on a Big Technology Podcast Friday edition, recorded on Thursday. Welcome to Big Technology Podcast Friday edition, where we break down the news in our traditional, coolheaded, and nuanced format. We're joined as always by Ranjan Roy, who is here with us, not in studio—live with the Big Technology AI Summit audience. Audience, let's hear you.
The way this is going to work is, Ranj and I will break down one story, and then we definitely welcome your questions, your prompts, your arguments with us. If you don't have anything, we have plenty to do, but we'd love to have your participation. You can line up at either of the mics there to ask us a question. Let's start with our top story this week. I promised myself when this summit was being planned that I would not be mean to Snapchat. However, Snapchat has left me with no choice. John, let's roll. Image C, please.
Listeners, if you are listening on the podcast, you'll see what we're looking at. The audience here is looking at an image of Evan Spiegel on CNBC wearing the latest Snap Specs. The headline is: 'Snap Stock Falls After AR Specs Debut.' Almost 20 years since the launch of the iPhone, people are ready to think about computing differently, Spiegel said in an interview with CNBC. The market reacted differently. Ranjan, let me just go to you quickly—do you think that Snapchat and Meta and all these other companies have been trying to build these AI device futures for years, and this is what we're looking at?
This is what we're looking at.
You kind of ruined my surprise. Can we go to D, please?
I brought them. I had to bring them after sending Alex a photo.
Is it time for us to finally accept that we're not going to have an AR device?
Interesting. So these again—I got these in 2021. It's actually a very cool technology. No, I'm serious. Augmented reality—if you ever used Magic Leap in the 2010s, like being able to paint throughout your room and walk around that painting, being able to play games where you're chasing zombies—my seven-year-old son actually is probably the only fan of Snap Spectacles in the world right now. He still loves using them. He also likes to watch YouTube on them, which is kind of amazing. But I think that form factor and the experience is amazing. Vision Pro has not quite captured it. I don't think what we just saw on Evan Spiegel's face is going to capture it. I think eventually maybe Apple or someone will, but I don't think we're there yet. But I think we will be. I still am betting on AR eyeglasses of some sort.
