Hey. What's up, friends and family? Back here on the Google live stream stage. My name is Jason Davenport. I work at Google Cloud. And I am joined today by two very special guests. I have Ben and David from the "Acquired" podcast. Super excited to have you guys on here.
Thanks for letting us crash your party.
What a party it is. I feel like I'm crashing the party with you two on stage, to be very candid. It's super fun to see the energy here. Let's start off something here, get just right into it with our audience. You've done three episodes on Google, right?
Yeah, it was the majority of our last year was studying the company.
I think 11 or 12 hours total saga. So that's a solid audiobook in there.
Yeah. So what's that, it's like 400 pages written?
Super cool. All right, so you've done so much so far. You obviously watched today's opening keynote with Thomas. How would you change what you've written so far?
Yeah, so the question is, do we need a part 4 here at some point? For anyone who's watching that hasn't listened, we did this big saga. Part 1 was the history of Search. Part 2, we called Alphabet, sort of era of web applications and developing the web as a platform. And then part 3 was this AI era and the craziness of Google seeming very behind when ChatGPT launched in 2022.
Well, first Google inventing everything.
Inventing the transformer.
The transformer. Everybody, everybody in AI worked at Google.
Just a small thing in 2018, right?
And then the crazy comeback in '23, '24, '25, with Gemini, with bringing together DeepMind and the core Google research team that predated DeepMind, and watching all the products evolve. Today is probably a reasonable chapter of part 4. I expect it'll be like 10 years before we make the episode, but I thought an underrated, very interesting part of today's announcement was TPUv8, the split to training and an inference chip, and just some of the specs on the generational change between v7 and v8 in just, I think, two years is the time frame between the last chip and this one.
When we made our NVIDIA series in-- what was that, '23, '24? Was it '25? '23, '24-- the dominant computing modality in AI was training. And inference was like, oh yeah, inference, too, but training was what everybody was focused on.
Yeah, inference is fine. Like yeah, I guess you could run that on the same chips or whatever, worry about that later. And the bets that the TPU team and that all of Google made a couple of years ago-- because a couple of years ago, when TPUv8 was being developed, it was still very much a training-first market. And now, I don't know if it's inference first, but it will be soon.
I think the compute loads for inference have now eclipsed the line for training, which makes sense. When you look at every single Google product-- let's just zoom in on Search-- every query now has an AI overview. Now, granted, that's a really small model, and it's really optimized, but it's kind of unbelievable that at the scale of full Google Search, you now are doing AI inference every time.
Yeah, it's fascinating. You think of frontier models, and there's three to five-ish companies in the world that are really probably pushing that. Everyone else is leveraging those frontier models or derivatives of those. So I do think it makes a lot of sense if you think about inference as that workload going forward. The other thing I think is really fascinating with some of the stuff today, if you think of some of the agent platform components, I see a lot of folks struggle from zero to one with agent building. And the lucky that get to one, you then run into these hardware problems where it's OK, well, I have one, but how do I give this to a million users, 10 users, even, for some cases? And solving that problem, I think, is probably one of the next things over the next 24 months, if I'm being honest.
What are your thoughts, though? How are you looking at this across the market, in terms of barriers to that kind of last mile of adoption with AI and agents?
So I-- there's this interesting question of, is AI as useful as the hype around it?
It's one way to frame the bubble question. Are people getting enough value versus all the conversation around it? And the fact that we're seeing inference eclipse training in workload volume--
--seems like a clear answer to that question.
And are customers getting value out of it? You're seeing revenue explosion, so, OK, check. Then the question becomes, well, what is holding AI back? And I suspect it is most people still don't think, oh, I should be using AI for this task. We have very narrowly defined, oh, I should ask Gemini. I should ask ChatGPT. I should ask Claude when I have a query that is more complicated than something I would just search the web for. But I think most people, for most tasks throughout the day, it still doesn't occur to you, even if you're like an AI maximalist who's really trying to lean into the tools, there's just not the muscle memory yet and the obvious tools to use for each task. And to your point, I think the biggest manifestation is around agentic. Most people, 99% plus of people still don't think, oh, I should create an agent to do that on my behalf.
And the UIs for this stuff are still getting figured out. I mean--
Do you need a UI with an agent?
I mean, the fact that it's getting better quickly, but until now, you kind of needed to have a terminal up to do a lot of this. Normal people are not going to have terminals up. There needs to be some kind of widely accepted, easily usable layer for people that the instant the word "terminal" or a terminal flashes on your screen, it's like, nope, not for me.
I mean, I did a lot of development, and I still have that feeling some of the time. It happens to everyone.
Oh, that's the first command that I run all the time. Come on. All right. Is AI like the internet? Is it like the manufacturing line, or is it like electricity in terms of the timeline of adoption we're thinking about? And maybe the scale, also.
The internet, but faster.
Yeah. We actually had a great conversation last night with Amin, who's on stage today announcing the new TPU. And he made this comment to us that, when we were talking about inference versus training workloads, and his point was, training is a large, expensive one-time task. Inference is a cheap over and over and over again ongoing task. There's an analog could make to the early web. Google in 2000, the main use of the infrastructure was to go and do a giant one-time web crawl, save that web crawl in the index and--
Ah, and then use the leftover computers to actually serve the web pages. And over time, by 2010, 2015, maybe even earlier than that, the vast majority of Google's infrastructure was serving pages for queries, and the crawl was this small thing running in the background. You see that exact same thing happening with training and inference, but on a much more compressed timescale of, call it, four years so far, rather than 15 years.
I think also, the internet was so much about creating new business models and creating new revenue for companies and enterprises, and that just takes longer to figure out. And AI, I think, will be that, too. But AI is also-- like we saw the demo of the YouTube TV customer service today. One of our partners is a company called Sierra that does this for companies. Today, you can use AI to massively efficientize your costs. And so that is going to drive adoption. That is driving adoption by enterprises so much faster than something like the internet.
Yeah, it's interesting. I think we're in an efficiency phase of AI right now. I don't know how much longer it's going to last. Candidly, I'm a little excited for when it's over and we start to really think about business transformation as a result of AI. Because I think if you think of business models and where these next rounds of companies are going to come from, it's going to be from that type of question. Hey, how am I building AI in a way that integrates into what the customer actually wants? Not like making the website two clicks easier for some customer to use.
The next Googles, the next YouTubes, the next-- yeah.
It's actually a good question. I would-- much like how in the internet, when web pages first went online, the newspapers put the exact newspaper up. But it was digital, and you could read it via HTML. And then over time, we realized, oh, the scrollable feed is the way to consume information on phones. And I think you're right that we aren't really seeing the killer native app of the agentic AI era. We don't even know what it looks like. We're trying to make all of our existing stuff more efficient and put it into this AI modality, but I don't think--
But it definitely will come. It feels like there are things enabled by AI that it'll come.
Like the YouTube Support demo. For those who are following along, we had a demo this morning of using AI with YouTube Support in a multilingual scenario. And like, how many times does that happen? It happens all the time.
