We've got a special guest who's going to come join us. This always happens. Another bread, everybody. Oh my god.
Somebody told me you started submitting code and it kind of freaked everybody out that daddy was home.
Models tend to do better if you threaten them with physical violence. Yes. Management is the easiest thing to do with AI. Absolutely.
It must be a weird experience to meet the bureaucracy in a company that you didn't hire. But on the other side, it's pretty amazing that some junior muck muck can basically look at you and say, 'Hey, go yourself.' No, but I'm serious, that's a sign of a healthy culture.
You're punching a clock, man. I hear the reports. You and I have talked about it. You're going to work every day.
Yeah, it's been some of the most fun I've had in my life, honestly. I retired like a month before COVID hit, in theory. I was like, 'This has been good. I want to do something else. I want to hang out in cafes, read physics books.' Then like a month later, I was like, 'That's not really happening.' So I just started to go to the office once we could. To be perfectly honest, there was a guy from OpenAI named Dan. I ran into him at a little party and he said, 'Look, what are you doing? This is the greatest transformative moment in computer science ever, completely. And you're a computer scientist. Forget that founder of Google, but you're a PhD student in computer science.' I haven't finished my PhD yet, but working on it. Keep working. Yeah, we'll get there. Technically on leave of absence. He told me this, and I'd already started going into the office a little bit. I was like, 'He's right.' It has been incredible. You guys all follow the AI technology, but being a computer scientist, it is the most exciting thing of my life, technologically. The exponential nature of this, the pace, dwarfs anything we've seen in our career. It's almost like everything we did over the last 30 or 40 years has led up to this moment, and it's all compounding on itself.
The pace. Maybe you could speak to that. You had a company, Google, that grew from 100 users and 10 employees to now over two billion people using, I think, six products or five products have over two billion. It's not even worth counting because it's the majority of the people on the planet. Describe the pace.
The excitement of the early web. I remember using Mosaic and later Netscape. How many of you remember Mosaic? Am I a weirdo? There was a 'What's New' page. You'd go through two or three new web pages. It was like, 'This week, these were the new websites.' Such and such elementary school, such and such a fish tank, Michael Jordan appreciation page. Those were the three new sites on the whole internet. The web developed very rapidly from there, and that was exciting. Then we had smartphones and whatnot. But the developments in AI are astonishing by comparison. The web spread but didn't technically change so much from month to month, year to year. These AI systems actually changed quite a lot. If you went away for a month and came back, you'd be like, 'Whoa, what happened?'
Somebody told me you started submitting code and it kind of freaked everybody out that daddy was home. Daddy needs a PR. What happened?
The code I submitted wasn't very exciting. I needed to add myself to get access to some things, a minor CL here or there. Nothing that's going to win any awards. But you need to do that to run basic experiments. I've tried to touch different parts of the system. First of all, it's fun. Secondly, I know what I'm talking about. It feels privileged to be able to go back to the company, not have any real executive responsibilities, but be able to go deep into every little pocket.
Are there parts of the AI stack that interest you more than others right now? Are there certain problems that are totally captivating you?
I started a couple years ago. Maybe a year ago, I was really close with what we call pre-training. Most of what people think of as AI training, for various historical reasons, is pre-training. That's the big supercomputer, throwing huge amounts of computers at it. I learned a lot being deeply involved, seeing us go from model to model, running little baby experiments for fun. More recently, post-training, especially as the thinking models have come around, has been another huge step up. We don't really know what the ceiling is.
Explain what's happening with prompt engineering to deep research to a civilian. How would you explain that step function? People are not hitting the down carrot and watching deep research in Gemini's mobile app. The number of threads, queries, follow-ups it's doing is 200, 300. Explain that jump and what you think the jump after that is.
The exciting thing about AI these days is it's not quite AGI yet, not superhuman intelligence, but it's pretty damn smart and can definitely surprise you. The superpower is when it can do things in a volume that I cannot. By default, when you use some of our AI systems, it'll suck down the top 10 search results and pull out what you need. I could do that myself, maybe a little more time. But if it sucks down the top thousand results, does follow-on searches for each, and reads them deeply, that's a week of work for me. I can't do that. People have not fully appreciated that who are not using deep research.
Before we had our F1 driver on stage, I'm a neophyte. I said, 'How many deaths occurred per decade?' I wanted to get to deaths per mile driven. At first, it was like, 'That's going to be really hard.' I said, 'I give you permission to make your best shot.' It came up with a system including practice miles, gave me deaths per mile estimated. I started cross-referencing and was like, 'Oh my god, this is like somebody's term paper for undergrad, done in minutes.'
Which model did you use? Gemini. The fabulous one. I treat it like I get sassy with it, and it works. Not just our models, but all models tend to do better if you threaten them with physical violence. People feel weird about that, so we don't talk about it.
I threatened it with not being fabulous, and it responded to that too.
Yeah, historically, you just say, 'I'm going to kidnap you if you don't.'
It went through and literally came up with a system. It said, 'I think we should include practice miles.' So let's say there's 100 practice miles for every mile on the track. It gave me the deaths per mile estimated. I started cross-referencing and was like, 'Oh my god, this is like somebody's term paper for undergrad, done in minutes.' It's amazing. All of us have had these experiences where you throw this AI, don't really expect it to work, and then you're like, 'Whoa, that actually worked.' As you have those moments and then go home to your life as a dad, have you gotten to the point where you're like, 'What will my children do? Are they learning the right way? Should I totally change everything they're doing right now?' Have you had any of those moments yet?
I don't really know how to think about it, to be perfectly honest. I don't have a magical way. I have a kid in high school and middle school. The AIs are already ahead. Obviously, there are some things AIs are particularly dumb at, they make certain mistakes a human would never make. But generally, if you talk about math or calculus, they're pretty damn good. They can win math contests and coding contests against some top humans. I look at my son going from sophomore to junior, what is he going to learn? I talk to him about what the AI will be in a few years. It's comparable. I don't know if you can plan your life around this. I didn't plan my life to be an entrepreneur. I just liked math and computer science. I got lucky it worked out. My kids should do what they like, hopefully it's challenging and they can overcome different kinds of problems.
What about college? Do you think college is going to continue to exist as it is today? It seems like college was already undergoing a revolution before this AI challenge. People are asking, 'Is it worth it? Should I be more vocational?' We're already entering a situation where questions are asked about colleges.
AI puts that at the forefront. As a parent, I think a lot about how much of education in America, middle class and upper class, is all about what college and how to get them there. Honestly, lately I'm like, 'I don't think they should go to college.' My son is a rising junior, and his entire focus is he wants to go to an SEC school because of the culture. Two years ago, I would have panicked and thought, 'Should I help him get into this school or that school?' Now I'm like, 'That's actually the best thing you could do.' Be socially well-adjusted, psychologically deal with different kinds of failures, enjoy a few years of exploration.
Sergey, can I ask you about hardware? Years ago, Google owned Boston Dynamics, maybe a little ahead of its time. But the way these systems are learning through visual and sensory information, learning how to adjust to the environment, is triggering profound learning curves in hardware. There are dozens of startups making robotic systems. What do you see in robotics and hardware? Is this a moment where things are really starting to work?
We acquired and later sold about five robotics companies, Boston being one. We built the hardware. More recently, we built out Everyday Robotics internally and later had to transition that. The robots are all cool, but the software wasn't quite there. Every time we've tried to make them truly useful, it hasn't worked. Presumably, one of these days that'll no longer be true.
Have you seen anything lately that does? And do you believe in the humanoid form factor, or do you think that's a little overkill?
I'm probably the one weirdo who's not a big fan of humanoids. Maybe I'm jaded because we acquired at least two humanoid robotics startups and later sold them. The reason people want humanoid robots is because the world is designed around this form factor, and you can train on YouTube videos of people doing things. I personally don't think that gives AI enough credit. AI can learn through simulation and real life pretty quickly how to handle different situations. I don't know that you need exactly the same number of arms, legs, and wheels as humans. I'm probably less bullish on that, but there are a lot of smart people making humanoid robots, so I wouldn't discount it.
What about the path of being a programmer? We're seeing that with finite data sets. Google has a 20-year-old code base now, so it could be quite impactful. What are you seeing in the company? Are the 10x developers going to become the norm, or is it going to be all done by computers and we just check it?
