I went and looked at a studio. Well, a little office that I was going to turn into a studio nearby, but it was not good at all. It had drop ceilings, so I could hear the guy in the office next to me. You would be able to hear him talking on episodes.
Third co-host. Third co-host. Is it Howard? No, it was like a lawyer. It seemed to be like talking through some horrible problem that I didn't want to listen to, but I could hear every word. Does he want millions of people listening to this conversation?
Right. All right. All right. Let's do a podcast. Let's do a podcast.
[Music] Who got the truth? Now, is it you? Is it you? Is it you? Sit me down. Say it straight. Another story on the way. Got the truth. Welcome to the fall 2025 season of Acquired, the podcast about great companies and the stories and playbooks behind them. I'm Ben Gilbert.
I'm David Rosenthal, and we are your hosts. Here's a dilemma. Imagine you have a profitable business. You make giant margins on every single unit you sell and the market you compete in is also giant. One of the largest in the world, you might say. But then on top of that, lucky for you, you also are a monopoly in that giant market with 90% share and a lot of lock in. And when you say monopoly, monopoly as defined by the US government. That is correct. But then imagine this. In your research lab, your brilliant scientists come up with an invention. This particular invention when combined with a whole bunch of your old inventions by all your other brilliant scientists turns out to create the product that is much better for most purposes than your current product. So you launched the new product based on this new invention. Right.
Right. I mean, especially because out of pure benevolence, your scientists had published research papers about how awesome the new invention is and lots of the inventions before also. So, now there's new startup competitors quickly commercializing that invention. So, of course, David, you change your whole product to be based on the new thing, right?
Uh, this sounds like a movie. Yes. But here is the problem. You haven't figured out how to make this new incredible product anywhere near as profitable as your old giant cash printing business. So maybe you shouldn't launch that new product.
David, this sounds like quite the uh dilemma to me. Of course listeners this is Google today and in perhaps the most classic textbook case of the innovators dilemma ever the entire AI revolution that we are in right now is predicated by the invention of the transformer out of the Google brain team in 2017. So think OpenAI and ChatGPT, Anthropic, NVIDIA hitting all-time highs, all the craziness right now depends on that one research paper published by Google in 2017. And consider this. Not only did Google have the densest concentration of AI talent in the world 10 years ago that led to this breakthrough, but today they have just about the best collection of assets that you could possibly ask for. They've got a top tier AI model with Gemini. They don't rely on some public cloud to host their model. They have their own in Google Cloud that now does $50 billion in revenue. That is real scale. They're a chip company with their tensor processing units or TPUs, which is the only real scale deployment of AI chips in the world besides Nvidia GPUs. Maybe AMD maybe, but these are definitely the top two. Somebody put it to me in research that if you don't have a foundational frontier model or you don't have an AI chip, you might just be a commodity in the AI market. And Google is the only company that has both. Google still has a crazy bench of talent. And despite ChatGPT becoming kind of the Kleenex of the era, Google does still own the textbox, the single one that is the front door to the internet for the vast majority of people anytime anyone has intent to do anything online. But the question remains, what should Google do strategically? Should they risk it all and lean into their birthright to win in artificial intelligence? Or will protecting their gobs of profits from search hamstring them as the AI wave passes them by? But perhaps first we must answer the question, how did Google get here? David Rosenthal.
So listeners, today we tell the story of Google, the AI company. Woo.
You like that, David? Was that good?
I love it. Did you hire like a Hollywood script writing consultant without telling me?
I wrote that 100% myself with no AI. Thank you very much. No AI.
Well, listeners, if you want to know every time an episode drops, vote on future episode topics or get access to corrections from past episodes, check out our email list. That's acquired.fm/e. Come talk about this episode with the entire acquired community in Slack after you listen. That's acquired.fm/slack. Speaking of the acquired community, we have an anniversary celebration coming up. We do 10 years of the show. We're going to do an open Zoom call with everyone to celebrate. Kind of like how we used to do our LP calls back in the day with LPs. And we are going to do that on October 20th, 2025 at 4:00 p.m. Pacific time. Check out the show notes for more details. If you want more acquired, check out our interview show, ACQ2. Our last interview was super fun. We uh sat down with Tobi Lütke, the founder and CEO of Shopify, about how AI has changed his life and where he thinks it will go from here. So, search ACQ2 in any podcast player. And before we dive in, we want to briefly thank our presenting partner JP Morgan Payments. Yes, just like how we say every company has a story, every company's story is powered by payments. And JP Morgan Payments is a part of so many of their journeys from seed to IPO and beyond. So, with that, this show is not investment advice. David and I may have investments in the companies we discuss and this show is for informational and entertainment purposes only. David, Google, the AI company.
So Ben, as you were alluding to in that fantastic intro, really, you're really up in the game here. If we rewind 10 years ago from today, before the Transformer paper comes out, all of the following people, as we've talked about before, were Google employees. Ilya Sutskever, founding chief scientist of OpenAI, who along with Jeff Hinton and Alex Krizhevsky had done the seminal AI work on AlexNet and just published that a few years before. All three of them were Google employees, as was Dario Amodei, the founder of Anthropic, Andrej Karpathy, chief scientist at Tesla until recently, Andrew Ng, Sebastian Thrun, Noam Shazeer, all the DeepMind folks. Demis Hassabis, Shane Legg, Mustafa Suleyman. Mustafa now, in addition to in the past having been a founder of DeepMind, runs AI at Microsoft. Basically, every single person of note in AI worked at Google with the one exception of Yann LeCun who worked at Facebook.
Yeah, it's pretty difficult to trace a big AI lab now back and not find Google in its origin story. Yeah, I mean the analogy here is it's almost as if at the dawn of the computer era itself, a single company like say IBM had hired every single person who knows how to code. So it'd be like, you know, if anybody else wants to write a computer program, oh sorry, you can't do that. Anybody who knows how to program works at IBM. This is how it was with AI and Google in the mid-2010s. But learning how to program a computer wasn't so hard that people out there couldn't learn how to do it. Learning how to be an AI researcher significantly more difficult, right? It was the stuff of very specific PhD programs with a very limited set of advisors and a lot of infighting in the field of where the direction of the field was going, what was legitimate versus what was crazy heretical religious stuff.
Yeah. So then yes, the question is how do we get to this point? Well, it goes back to the start of the company. I mean, Larry Page always thought of Google as an artificial intelligence company. And in fact, Larry Page's dad was a computer science professor and had done his PhD at the University of Michigan in machine learning and artificial intelligence, which was not a popular field in computer science back then.
Yeah. In fact, a lot of people thought specializing in AI was a waste of time because so many of the big theories from 30 years prior to that had been kind of disproven at that point, or at least people thought they were disproven. And so it was frankly contrarian for Larry's dad to spend his life and career and research work in AI. And that rubbed off on Larry. I mean, if you squint, PageRank, the PageRank algorithm that Google was founded upon is a statistical method. You could classify it as part of AI within computer science. And Larry, of course, was always dreaming much, much bigger here. I mean, there's the quote that we've said before on this show in the year 2000, 2 years after Google's founding when Larry says artificial intelligence would be the ultimate version of Google. If we had the ultimate search engine, it would understand everything on the web. It would understand exactly what you wanted and it would give you the right thing. That's obviously artificial intelligence. We're nowhere near doing that now. However, we can get incrementally closer and that is basically what we work on here. It's always been an AI company.
Yep. And that was in 2000. Well, one day in either late 2000 or early 2001, the timelines are a bit hazy here, a Google engineer named Greg Herak is talking over lunch with Ben Gomes, famous Google engineer who I think would go on to lead search, and a relatively new engineering hire named Noam Shazeer. Now, Greg was one of Google's first 10 employees, incredible engineer. And just like Larry Page's dad, he had a PhD in machine learning from the University of Michigan. And even when Greg went there, it was still a relatively rare contrarian subfield within computer science. So, the three of them are having lunch and Greg says off-handedly to the group that he has a theory from his time as a PhD student that compressing data is actually technically equivalent to understanding it. And the thought process is if you can take a given piece of information and make it smaller, store it away and then later reinstantiate it in its original form. The only way that you could possibly do that is if whatever force is acting on the data actually understands what it means because you're losing information going down to something smaller and then recreating the original thing. It's like you're a kid in school. You learn something in school. You read a long textbook. You store the information in your memory. Then you take a test to see if you really understood the material. And if you can recreate the concepts, then you really understand it.
Which kind of foreshadows big LLMs today are like compressing the entire world's knowledge into some number of terabytes that's just like this smash down little vector set. Little at least compared to all the information in the world. But it's kind of that idea, right? You can store all the world's information in an AI model in something that is like kind of incomprehensible and hard to understand. But then if you uncompress it, you can kind of bring knowledge back to its original form.
Yep. And these models demonstrate understanding, right?
Do they? That's the question. That's the question. They certainly mimic understanding.
So this conversation is happening. You know, this is 25 years ago. And Noam, the new hire, the, you know, young buck, he sort of stops in his tracks and he's like, "Wow, if that's true, that's really profound."
Is this in one of Google's micro kitchens?
This is in one of Google's micro kitchens. They're having lunch.
Where did you find this, by the way? A 25-year-old uh this is in the Plex.
This is like a small little passage in Steven Levy's great book that's been a source for all of our Google episodes, In the Plex. There's a small little throwaway passage in here about this because this book came out before ChatGPT and AI and all that. So Noam kind of latches on to Greg and keeps vibing over this idea and over the next couple months the two of them decide in the most googly fashion possible that they are just going to stop working on everything else and they're going to go work on this idea on language models and compressing data and can they generate machine understanding with data and if they can do that that would be good for Google.
I think this coincides with that period in 2001 when Larry Page fired all the managers in the engineering organization and so everybody was just doing whatever they wanted to do. Funny.
So there's this great quote from Greg in the book. A large number of people thought it was a really bad thing for Noam and I to spend our talents on, but Sanjay Ghemawat, Sanjay of course being Jeff Dean's famous prolific coding partner, thought it was cool. So Greg would posit the following argument to any doubters that they came across. Sanjay thinks it's a good idea and no one in the world is as smart as Sanjay. So why should Noam and I accept your view that it's a bad idea?
It's like if you beat the best team in football, are you the new best team in football no matter what?
Yeah. So all of this ends up taking Noam and Greg deep down the rabbit hole of probabilistic models for natural language. Meaning for any given sequence of words that appears on the internet, what is the probability for another specific sequence of words to follow? This should sound pretty familiar for anybody who knows about LLMs work today.
Oh, kind of like a next word predictor.
Yeah. Or next token predictor if you generalize it.
So, the first thing that they do with this work is they create the did you mean spelling correction in Google search.
Oh, that came out of this.
That came out of this. Noam created this.
So this is huge for Google because obviously it's a bad user experience when you mistype a query and then need to type another one. But it's a tax on Google's infrastructure because every time these mistyped queries are going well, Google's infrastructure goes and serves the results to that query that are useless and immediately overwritten with the new one, right? And it's a really tightly scoped problem where you can see like, oh wow, 80% of the time that someone types in god groomer, oh, they actually mean dog groomer and they retype it. And if it's really high confidence, then you actually just correct it without even asking them and then ask them if they want to opt out instead of opting in. It's a great feature and it's sort of a great first use case for this in a very narrowly scoped domain.
Totally. So they get this win and they keep working on it, Noam and Greg, and they end up creating a fairly large, I'm using large in quotes here, you know for the time, language model that they call affectionately PHIL, the Probabilistic Hierarchical Inferential Learner.
These AI researchers love creating their uh backronyms. They love their word puns.
Yeah. Yep. So fast forward to 2003 and Susan Wojcicki and Jeff Dean are getting ready to launch AdSense. They need a way to understand the content of these third party web pages, the publishers, in order to run the Google ad corpus against them. Well, PHIL is the tool that they use to do it.
Huh. I had no idea that language models were involved in this.
Yeah. So Jeff Dean borrows PHIL and famously uses it to code up his implementation of AdSense in a week because he's Jeff Dean. And boom, AdSense. This is billions of dollars of new revenue to Google overnight because it's the same corpus of ads that are AdWords that are search ads that they're now serving on third party pages. They just massively expanded the inventory for the ads that they already have in the system.
All right, this is a moment where we got to stop and just give some Jeff Dean facts. Jeff Dean is going to be the throughline of this episode.
Wait, how did Google pull that off? How did Jeff Dean just go home and over the weekend rewrite some entire giant distributed system and figure out all of Google's problems?
Back when Chuck Norris facts were big, Jeff Dean facts became a thing internally at Google. I just want to give you some of my favorites. The speed of light in a vacuum used to be about 35 mph. Then Jeff Dean spent a weekend optimizing physics.
So good. Jeff Dean's PIN is the last four digits of pi.
Only Googlers would come up with these.
Yes. To Jeff Dean, NP means no problemo.
Oh yeah, I've seen that one before. I think that one's my favorite.
Yes. Oh, man. So, so good. Also a wonderful human being who we spoke to in research and was very, very helpful. Thank you, Jeff.
Yes. So, language models definitely work, definitely going to drive a lot of value for Google, and they also fit pretty beautifully into Google's mission to organize the world's information and make it universally accessible and useful if you can understand the world's information and compress it and then recreate it.
Yeah, that fits the mission. I think I think that checks the box.
Absolutely. So PHIL gets so big that apparently by the mid 2000s PHIL is using 15% of Google's entire data center infrastructure and I assume a lot of that is AdSense ad serving but also did you mean and all the other stuff that they start using it for within Google.
So uh early natural language systems computationally expensive.
So okay now mid 2000s fast forward to 2007 which is a very very big year for the purposes of our story. Google had just recently launched the Google Translate product. This is the era of all the great great products coming out of Google that we've talked about. You know maps and Gmail and docs and all the wonderful things that Chrome and Android are going to come later. They had like a 10-year run where they basically launched everything you know of at Google except for search truly in a 10-year run. And then there were about 10 years after that from 2013 on where they basically didn't launch any new products that you've heard about until we get to Gemini, which is this fascinating thing. But this 2003 to 2013 era was just so rich with hit after hit after hit.
And so one of those products was Google Translate. You know, not the same level of user base or perhaps impact on the world as Gmail or maps or whatnot, but still a magical magical product. And the chief architect for Google Translate was another incredible machine learning PhD named Franz Och. So Franz had a background in natural language processing and machine learning and that was his PhD. He was German. He got his PhD in Germany at the time. DARPA, the Defense Advanced Research Projects Agency, division of the government, had one of their famous challenges going for machine translation. So Google and Franz of course enters this and Franz builds an even larger language model that blows away the competition in this year's version of the DARPA challenge. This is either 2006 or 2007. Gets an astronomically high BLEU score for the time. It's called the Bilingual Evaluation Understudy, is the sort of algorithmic benchmark for judging the quality of translations at the time, higher than anything else possible. So Jeff Dean hears about this and the work that Franz and the translate team have done and it's like this is great. This is amazing. Uh when are you guys going to ship this in production?
Oh, I heard this story. So Jeff and Noam talk about this on the Dwarkesh podcast.
Yes, that episode is so so good. And Franz is like, "No, no, no, no, Jeffy, you don't understand. This is research. This isn't for the product. We can't ship this model that we built. This is an n-gram language model." Grams are like number of words in a cluster. "And we've trained it on a corpus of two trillion words from the Google search index. This thing is so large it takes it 12 hours to translate a sentence." So the way the DARPA challenge worked in this case was you got a set of sentences on Monday and then you had to submit your machine translation of those set of sentences by Friday.
Plenty of time for the servers to run.
Yeah. They were like, "Okay, so we have whatever number of hours it is from Monday to Friday. Let's use as much compute as we can to translate these couple sentences."
Hey, learn the rules of the game and use them to your advantage.
Exactly. So Jeff Dean being the engineering equivalent of Chuck Norris, he's like, let me see your code. So Jeff goes and parachutes in and works with the translate team for a few months. And he rearchitects the algorithm to run on the words and the sentences in parallel instead of sequentially. Because when you're translating a set of sentences or a set of words in a sentence, you don't necessarily need to do it in order. You can break up the problem into different pieces, work on it independently. You can parallelize it and you won't get a perfect translation, but you know, imagine you just translate every single word. You can at least go translate those all at the same time in parallel, reassemble the sentence and like mostly understand what the initial meaning was.
Yeah. And as Jeff knows very well because he and Sanjay basically built it with Chozha, Google's infrastructure is extremely parallelizable, distributed. You can break up workloads into little chunks, send them all over the various data centers that Google has, reassemble the projects, return that to the user. They are the single best company in the world at parallelizing workloads across CPUs across multiple data centers.
CPUs. We're still talking CPUs here.
Yep. And Jeff's work with the team gets that average sentence translation time down from 12 hours to 100 milliseconds. And so then they ship it in Google Translate. And it's amazing.
This sounds like a Jeff Dean fact. Well, you know, it used to take 12 hours and then Jeff Dean took a few months with it. Now it's a 100 milliseconds.
Right. Right. Right. Right. Right. Right. So this is the first large, I'm using large in quotes here, language model used in production in a product at Google. They see how well this works like hm maybe we could use this for other things like predicting search queries as you type. That might be interesting, you know, and of course the crown jewel of Google's business. That also might be interesting application for this. The ad quality score for AdWords is literally the predicted click-through rate on a given set of ad copy. You can see how an LLM that is really good at ingesting information, understanding it, and predicting things based on that might be really useful for calculating ad quality for Google.
Yep. Which is the direct translation to Google's bottom line.
Okay. So, obviously all of that is great on the language model front. I said 2007 was a big year. Also in 2007 begins the sort of momentous intersection of several computer science professors on the Google campus. So in April of 2007, Larry Page hires Sebastian Thrun from Stanford to come to Google and work first part-time and then full-time on machine learning applications. Sebastian was the head of SAIL, the Stanford Artificial Intelligence Laboratory. Legendary AI laboratory that was big in the sort of first wave of AI back in the '60s, '70s when Larry's dad was active in the field then actually shut down for a while and then had been restarted and re-energized here in the early 2000s. And Sebastian was the leader, the head of SAIL. Funny story about Sebastian, the way that he actually comes to Google. Sebastian was kind enough to speak with us to prep for this episode. I didn't realize it was basically an acqui-hire. He and some I think it was grad students were in the process of starting a company had term sheets from Benchmark and Sequoia.
And Larry came over and said, "What if we just acquire your company before it's even started in the form of signing bonuses?"
Yes. Probably a very good decision on their part.
So SAIL, this group within the CS department at Stanford, not only had some of the most incredible, most accomplished professors and PhD AI researchers in the world, they also had this stream of Stanford undergrads that would come through and work there as researchers while they were working on their CS degrees or symbolic systems degrees or, you know, whatever it was that they were doing as Stanford undergrads. One of those people was Chris Cox, who's the chief product officer at Meta.
Yeah, that was kind of how he got his start in all of this and AI and obviously Facebook and Meta are going to come back into the story here in a little bit. Wow. You really can't make this up.
Another undergrad who passed through SAIL while Sebastian was there was a young freshman and sophomore who would later drop out of Stanford to start a company that went through Y Combinator's very first batch in summer 2005.
I'm on the edge of my seat. Who is this? Any guesses? Uh Dropbox, Reddit. I'm trying to think who else was in the first batch.
Oh, no. No. But way more on the nose for this episode. The company was a failed local mobile social network.
That's amazing. He was at SAIL at the same time.
He was at SAIL. Yep. As an undergrad researcher.
Wow. Wild, right? We told you that it's a very small set of people that are all doing all of this. Man, I miss those days. Sam presenting at the WWDC with Steve Jobs on stage with the double popped collar, right?
Yeah, the double popped collar. That was amazing. That was a vibe. That was a moment. Oh, man.
All right. So, April 2007, Sebastian comes over from SAIL into Google, Sebastian Thrun. And one of the first things he does over the next set of months is a project called Ground Truth for Google Maps.
Which is essentially Google Maps.
It is essentially Google Maps. Before Ground Truth, Google Maps existed as a product, but they had to get all the mapping data from a company called Tele Atlas. And I think there were two. They were sort of a duopoly. Navteq was the other one.
Yeah. Navteq and Tele Atlas.
But it was this like kind of crappy source of truth map data that everyone used and you really couldn't do any better than anyone else because you all just use the same data.
Yep. It was not that good and it cost a lot of money. Tele Atlas and Navteq were multi-billion dollar companies. I think maybe one or both of them were public at some point then got acquired but a lot of money lot of revenue.
Yep. And Sebastian's first thing was Street View, right? So he already had the experience of orchestrating this fleet of all these cars to drive around and take pictures.
So then coming into Google, Ground Truth is this sort of moonshot type project to recreate all the Tele Atlas data mostly from their own photographs of streets from Street View. And they incorporated some other data. There was like census data they used. I think it was 40 something data sources to bring it all together. But Ground Truth was this very ambitious effort to create new maps from whole cloth.
Yep. And just like all of the AI and AI enabled projects within Google that we're talking about here works very very well.
Well, especially when you hire a thousand people in India to help you uh sift through all the discrepancies in the data and actually hand-draw all the maps.
Yes, we are not yet in an era of a whole lot of AI automation. So on the back of this win with Ground Truth, Sebastian starts lobbying to Larry and Sergey. Hey, we should do this a lot. We should bring in AI professors, academics, I know all these people into Google part-time. They don't have to be full-time employees. Let them keep their posts in academia, but come here and work with us on projects for our products. They'll love it. They get to see their work used by millions and millions of people. We'll pay them. They'll make a lot of money. They'll get Google stock and they get to stay professors at their academic institutions. Win-win-win.
So, as you would expect, Larry and Sergey are like, "Yeah, yeah, yeah, that's a good idea. Let's do that. More of that." So, in December of 2007, Sebastian brings in a relatively little known machine learning professor from the University of Toronto named Jeff Hinton to the Google campus to come and give a tech talk, not yet hiring him, but come give a tech talk to, you know, all the folks at Google and talk about some of the new work, Jeff, that you and your PhD and postdoc students there at the University of Toronto are doing on blazing new paths with neural networks.
And Jeff Hinton for anybody who doesn't know the name now very much known as the godfather of neural networks and really the godfather of kind of the whole direction that AI went in, modern AI. He was kind of a fringe academic at this point in history. I mean neural networks were not a respected subtree of AI.
No, totally not. And part of the reason is there had been a lot of hype 30, 40 years before around neural networks that just didn't pan out. So it was effectively everyone thought disproven and certainly backwater.
Then do you remember from our Nvidia episodes my favorite piece of trivia about Jeff Hinton.
Oh yes. That his great-grandfather was George Boole.
Yep. He is the great-great grandson of George and Mary Boole who invented Boolean algebra and Boolean logic which is hilarious now that I know more about this because that's the basic building block of symbolic logic of defined deterministic computer science logic. And the hilarious thing about neural nets is it's not it's not symbolic AI. It's not I feed you these specific instructions and you follow a big if then tree. It is non-deterministic. It is the opposite of that field.
Which actually just underscores again how sort of heretical this branch of machine learning and computer science was.
Right. So Ben, as you were saying earlier, neural networks not a new idea and had all of this great promise in theory, but in practice just took too much computation to do multiple layers. You could really only have a single or maybe small single-digit number of layers in a computer neural network up until this time. But Jeff and his former postdoc, a guy named Yann LeCun, start evangelizing within the community, hey, if we can find a way to have multi-layered, deep layered neural networks, something we call deep learning, we could actually realize the promise here. It's not that the idea is bad. It's that the implementation which would take a ton of compute to actually do all the math to do all the multiplication required to propagate through layer after layer after layer of neural networks to sort of detect and understand and store patterns. If we could actually do that, a big multi-layered neural network would be very valuable and possibly could work.
Yes. Here we are now in 2007, mid-2000s. Moore's law has increased enough that you could actually start to try to test some of these theories.
Yep. So Jeff comes and he gives this talk at Google. It's on YouTube. You can go watch it. We'll link to it in the show notes.
This is incredible. This is an artifact of history sitting there on YouTube.
And people at Google, Sebastian, Jeff Dean, and all the other folks who are talking about, they get very, very, very excited because they've already been doing stuff like this with translate and the language models that they're working with. That's not using deep neural networks that Jeff's working on. So here's this whole new architectural approach that if they could get it to work would enable these models that they're building to work way better, recognize more sophisticated patterns, understand the data better. Very, very promising. Again, kind of all in theory at this point.
So Sebastian Thrun brings Jeff Hinton into the Google fold after this tech talk. I think first as a consultant over the next couple years and then this is amazing. Later, Jeff Hinton technically becomes an intern at Google.
Like that's how they get around the part-time, full-time policies here.
That's correct. Yep. He was a summer intern in somewhere around 2011, 2012.
And mind you, at this point, he's like 60 years old.
Yes. So in the next couple years after 2007 here, Sebastian's concept of bringing these computer science machine learning academics into Google as contractors or part-time or interns, basically letting them keep their academic posts and work on big projects for Google's products internally goes so well that by late 2009, Sebastian and Larry and Sergey decided, hey, we should just start a whole new division within Google and it becomes Google X, the moonshot factory. The first project within Google X Sebastian leads himself.
David, don't say it, don't say it.
I won't say the name of it. We will come back to it later. But for our purposes for now, the second project would be critically important not only for our story but to the whole world, everything in AI changing the entire world and that second project is called Google Brain. But before we tell the Google Brain story, now is a great time to thank our friends at JP Morgan Payments.
Yes. So today we are going to talk about one of the core components of JP Morgan Payments, their Treasury solutions. Now treasury is something that most listeners probably do not spend a lot of time thinking about, but it's fundamental to every company.
Yep. Treasury used to be just a back office function, but now great companies are using it as a strategic lever. With JP Morgan Payments Treasury Solutions, you can view and manage all your cash positions in real time and all of your financial activities across 120 currencies in 200 countries. And the other thing that they acknowledge really in their whole strategy is that every business has its own quirk. So, it's not a cookie cutter approach. They work with you to figure out what matters most for you and your business and then help you gain clarity, control, and confidence. So whether you need advanced automation or just want to cut down on manual processes and approvals, their real-time treasury solutions are designed to keep things running smoothly. Whether your treasury is in the millions or billions, or perhaps like the company we're talking about this episode, in the hundreds of billions of dollars.
And they have some great strategic offerings like Pay by Bank, which lets customers pay you directly from their bank account. It's simple, secure, tokenized, and you get faster access to funds and enhanced data to optimize revenue and reduce fees. This lets you send and receive real-time payments instantly just with a single API connection to JP Morgan. And because JP Morgan's platform is global, that one integration lets you access 45 countries and counting and lets you scale basically infinitely as you expand. As we've said before, JP Morgan Payments moves $10 trillion a day. So scale is not an issue for your business.
Not at all. If you're wondering how to actually manage all that global cash, JP Morgan again has you covered with their liquidity and account solutions that make sure you have the right amount of cash in the right currencies in the right places for what you need. So whether you're expanding into new markets or just want more control over your funds, JP Morgan Payments is the partner you want to optimize liquidity, streamline operations, and transform your treasury. To learn more about how JP Morgan can help you and your company, just go to jpmorgan.com/acquired and tell them that Ben and David sent you.
All right, David. So, Google Brain.
So, when Sebastian left Stanford full-time and joined Google full-time, of course, somebody else had to take over SAIL. And the person who did is another computer science professor, brilliant guy named Andrew Ng.
This is like all the hits.
All the hits. This is all the AI hits on this episode. So, what does Sebastian do? He recruits Andrew to come part-time, start spending a day a week on the Google campus. And this coincides right with the start of X and Sebastian formalizing this division. So, one day in 2010, 2011 time frame, Andrew's spending his day a week on the Google campus and he bumps into who else? Jeff Dean. And Jeff Dean is telling Andrew about what he and Franz have done with language models and what Jeff Hinton is doing in deep learning. Of course, Andrew knows all this. And Andrew is talking about what he and SAIL are doing at Stanford. And they decide, you know, the time might finally be right to try and take a real big swing on this within Google and build a massive really large deep learning model in the vein of what Jeff Hinton has been talking about on highly parallelizable Google infrastructure.
And when you say the time might be right, Google had tried twice before and neither project really worked. They tried this thing called Brain on Borg. Borg is sort of an internal system that they use to run all of their infrastructure. They tried the Cortex project and neither of these really worked. So there's a little bit of scar tissue in the sort of research group at Google of are large-scale neural networks actually going to work for us on Google infrastructure.
So the two of them, Andrew Ng and Jeff Dean, pull in Greg Corrado, who is a neuroscience PhD and amazing researcher who was already working at Google. And in 2011, the three of them launch the second official project within X, appropriately enough, called Google Brain. And the three of them get to work building a really, really big, deep neural network model. And if they're going to do this, they need a system to run it on. You know, Google is all about taking this sort of frontier research and then doing the architectural and engineering system to make it actually run.
So Jeff Dean is working on this system on the infrastructure and he decides to name the infrastructure DistBelief which of course is a pun both on the distributed nature of the system and also on of course the word disbelief because no one thought it was going to work. Most people in the field thought this was not going to work and most people in Google thought this was not going to work. And here's a little bit on why and it's a little technical but follow me for a second. All the research from that period of time pointed to the idea that you needed to be synchronous. So all the compute needed to be sort of really dense happening on a single machine with really high parallelism kind of like what GPUs do that you really would want it all sort of happening in one place so it's really easy to kind of go look up and see hey what are the computed values for everything else in the system before I take my next move. What Jeff Dean wrote with DistBelief was the opposite. It was distributed across a whole bunch of CPU cores and potentially all over a data center or maybe even in different data centers. So in theory, this is really bad because it means you would need to be constantly waiting around on any given machine for the other machines to sync their updated parameters before you could proceed. But instead, the system actually worked asynchronously without bothering to go and get the latest parameters from other cores. So you were sort of updating parameters on stale data. You would think that wouldn't work. The crazy thing is it did.
Yes. Okay. So you've got DistBelief. What do they do with it now?
They want to do some research. So they try out can we do cool neural network stuff? And what they do in a paper that they submitted in 2011 right at the end of the year is I'll give you the name of the paper first. Building high-level features using large-scale unsupervised learning. But everyone just calls it the cat paper.
The cat paper. You talk to anyone at Google, you talk to anyone in AI, they're like, "Oh yeah, the cat paper."
What they did was they trained a large nine layer neural network to recognize cats from unlabeled frames of YouTube videos using 16,000 CPU cores on a thousand different machines. And listeners, just to like underscore how seminal this is, we actually talked with Sundar in prep for the episode. And he cited seeing the cat paper come across his desk as one of the key moments that sticks in his brain in Google's story.
Yeah. A little later on, they would do a TGIF where they would present the results of the cat paper and you talk to people at Google, they're like, "That TGIF, oh my god, that's when it all changed."
Yeah. It proved that large neural networks could actually learn meaningful patterns without supervision and without labeled data. And not only that, it could run on a distributed system that Google built to actually make it work on their infrastructure. And that is a huge unlock of the whole thing. Google's got this big infrastructure asset. Can we take this theoretical computer science idea that the researchers have come up with and use DistBelief to actually run it on our system?
Yep, that is the amazing technical achievement here. That is almost secondary to the business impact of the cat paper. I think it's not that much of a leap to say that the cat paper led to probably hundreds of billions of dollars of revenue generated by Google and Facebook and ByteDance over the next decade.
Definitely pattern recognizers in data. So, YouTube had a big problem at this time, which was that people would upload these videos, and there's tons of videos being uploaded to YouTube, but people are really bad at describing what is in the videos that they're uploaded. And YouTube is trying to become more of a destination site, trying to get people to watch more videos, trying to build a feed, increase dwell time, etc., etc. And the problem is the recommender is trying to figure out what to feed and it's only just working off titles and descriptions that people were writing about their own videos, right? And whether you're searching for a video or they're trying to figure out what video to recommend next, they need to know what the video is about.
So the cat paper proves that you can use this technology, a deep neural network running on DistBelief to go inside of the videos in the YouTube library and understand what they were about and use that data to then figure out what videos to serve to people.
If you can answer the question, cat or not a cat, you can answer a whole lot more questions too.
Here's a quote from Jeff Dean about this. We built a system that enabled us to train pretty large neural nets through both model and data parallelism. We had a system for unsupervised learning on 10 million randomly selected YouTube frames. As you were saying, Ben, it would build up unsupervised representations based on trying to reconstruct the frame from the high-level representations. We got that working and training on 2,000 computers using 16,000 cores. After a little while, that model was actually able to build a representation at the highest neural net level where one neuron would get excited by images of cats. It had never been told what a cat was, but it had seen enough examples of them in the training data of head-on facial views of cats that that neuron would then turn on for cats and not much else.
It's so crazy. I mean, this is the craziest thing about unlabeled data, unsupervised learning, that a system can learn what a cat is without ever being explicitly told what a cat is and that there's a cat neuron.
Yeah. And so then there's an iPhone neuron and a San Francisco Giants neuron and all the things that YouTube recommends, not to mention porn filtering, explicit content filtering, not to mention copyright identification and enabling revenue share with copyright holders.
Yeah, this leads to everything in YouTube. Basically puts YouTube on the path to today becoming the single biggest property on the internet and the single biggest media company on the planet. This kicks off a 10-year period from 2012 when this happens until ChatGPT on November 30th, 2022 when AI is already shaping the human existence for all of us and driving hundreds of billions of dollars of revenue. It's just in the YouTube feed and then Facebook borrows it and they hire Yann LeCun and they start Facebook AI Research and then they bring it into Instagram and then TikTok and ByteDance take it and then it goes back to Facebook and YouTube with Reels and Shorts. This is the primary way that humans on the planet spend their leisure time for the next 10 years. This is my favorite David Rosenthalism. Everyone talks about 2022 onward as the AI era. And I love this point from you that actually for anyone that could make good use of a recommender system and a classifier system, basically any company with a social feed, the AI era started in 2012.
Yes, the AI era started in 2012 and part of it was the cat paper. The other part of it was what Jensen at NVIDIA always calls the big bang moment for AI, which was AlexNet.
Yes. So, we talked about Jeff Hinton back at the University of Toronto. He's got two grad students who he's working with in this era. Alex Krizhevsky and Ilya Sutskever, of course, future co-founder and chief scientist of OpenAI. And the three of them are working with Jeff's deep neural network ideas and algorithms to create an entry for the famous ImageNet competition in computer science.
This is Fei-Fei Li's thing from Stanford. It is an annual machine vision algorithm competition. And what it was was Fei-Fei had assembled a database of 14 million images that were hand-labeled. Famously, she used Mechanical Turk on Amazon, I think, to get them all hand-labeled.
Yes. And I think that's right.
And so then the competition was what team can write the algorithm that without looking at the labels, so just seeing the images could correctly identify the largest percentage, the best algorithms that would win the competitions year-over-year.
We're still getting more than a quarter of the images wrong. So like 75% success rate, great. Way worse than a human. Can't use it for much in a production setting when a quarter of the time you're wrong. So then the 2012 competition along comes AlexNet, its error rate was 15%. Still high but a 10% leap from the previous best being a 25% error rate all the way down to 15 in one year. A leap like that had never happened before.
It's 40% better than the next best.
Yes. On a relative basis.
And why is it so much better, David? What did they figure out that would create a $4 trillion company in the future?
So, what Jeff and Alex and Ilya did is they knew like we've been talking about all episode that deep neural networks had all this potential and Moore's law advanced enough that you could use CPUs to create a few layers. They had the aha moment of what if we rearchitected this stuff not to run on CPUs but to run on a whole different class of computer chips that were by their very nature highly highly highly parallelizable video game graphics cards made by the leading company in the space at the time, Nvidia.
Not obvious at the time and especially not obvious that this highly advanced cutting edge academic computer science research that was being done on supercomputers usually that was being done on supercomputers with incredible CPUs would use these toy video game cards that retail for $1,000.
Yeah. Less at that point in time. A couple hundred bucks.
So the team in Toronto, they go out to like the local Best Buy or something. They buy two Nvidia GeForce GTX 580s, which were Nvidia's top-of-the-line gaming cards at the time. The Toronto team rewrites their neural network algorithms in CUDA, Nvidia's programming language. They train it on these two off-the-shelf GTX 580s and this is how they achieve their deep neural network and do 40% better than any other entry in the ImageNet competition.
So when Jensen says that this was the big bang moment of artificial intelligence, A, he's right. This shows everybody that holy crap, if you can do this with two off-the-shelf GTX 580s, imagine what you could do with more of them or with specialized chips. And B, this event is what sets Nvidia on the path from a somewhat struggling PC gaming accessory maker to the leader of the AI wave and the most valuable company in the world today.
And this is how AI research tends to work is there's some breakthrough that gets you this big step change function and then there's actually a multi-year process of optimizing from there where you get these kind of diminishing returns curves on breakthroughs where the first half of the advancement happens all at once and then the second half takes many years after that to figure out. It's rare and amazing and it must be so cool when you have an idea, you do it, and then you realize, "Oh my god, I just found the next giant leap in the field." It's like I unlocked the next level to use the video game analogy.
Yes, I leveled up. So after AlexNet, the whole computer science world is a buzz. People are starting to stop doubting neural networks at this point.
So after AlexNet, the three of them from Toronto, Jeff Hinton, Alex Krizhevsky, and Ilya Sutskever do the natural thing, they start a company called DNN Research, deep neural network research. This company does not have any products. This company has AI researchers who just won a big competition. And predictably, as you might imagine, it gets acquired by Google almost immediately.
Oh, are you intentionally shortening this? That's what I thought the story was.
Oh, it is not immediately.
There's a whole crazy thing that happens where the first bid is actually from Baidu.
So, Baidu offers $12 million. Jeff Hinton doesn't really know how to value the company and doesn't know if that's fair. And so, he does what any academic would do to best determine the market value of the company. He says, "Thank you so much. I'm gonna run an auction now and I'm going to run it in a highly structured manner where every time anybody wants to bid the clock resets and there's another hour where anybody else can submit another bid."
No way. So, I didn't know this. This is crazy.
He gets in touch with everyone that he knows from the research community who is now working at a big company who he thinks, hey, this would be a good place for us to do our research. That includes Baidu, that includes Google, that includes Microsoft, and there's one other.
Of course, it's a two-year-old startup.
Oh, wait. So, it does not include Facebook.
It does not include Facebook. Think about the year. This is 2012. So, Facebook's not really in the AI game yet. They're still trying to build their own AI lab.
Yeah. Yeah. Because Yann LeCun and FAIR would start in 2013. Is it Instagram?
Nope. It is the most important part of the end of this episode.
Wait. Well, it can't be Tesla because Tesla is older than that.
Well, OpenAI wouldn't get founded for years. Wow. Okay, you really got me here.
What company slightly predated OpenAI doing effectively the same mission?
Oh, of course. Of course. Hiding in plain sight. DeepMind.
Wow. DeepMind, baby. They are the fourth bidder in a four-way auction for DNN Research. Now, of course, right after the bidding starts, DeepMind has to drop out. They're a startup. They don't actually have the cash to be able to buy.
Yeah. Didn't even cross my mind cuz my first question was like, where the hell would they get the money because they had no money.
But Jeff Hinton already knows and respects Demis.
Ah, even though he's just doing this at the time startup called DeepMind. That's amazing. Wait, how is DeepMind in the auction, but Facebook is not?
So, the timing of this is concurrent with the it was then called NIPS, now it's called NeurIPS conference. So, Jeff Hinton actually runs the auction from his hotel room at the Harrah's Casino in Lake Tahoe.
So, the bids all come in and we got to thank Cade Metz, the author of Genius Makers, great book on the whole history of AI that we're actually going to reference a lot in this episode. The bidding goes up and up and up. At some point, Microsoft drops out. They come back in. Told you DeepMind drops out. So, it's Baidu and Google really going at the end. And finally, at some point, the researchers look at each other and they say, "Where do we actually want to land? We want to land at Google." And so, they stop the bidding at $44 million and just say, "Google, this is more than enough money. We're going with you."
Wow. I knew it was about $40 million. I did not know that whole story. It's almost like Google itself and you know the Dutch auction IPO process, right? How fitting.
That's kind of a perfect DNA.
And the three of them were supposed to split it 33 each and Alex and Ilya go to Jeff and say, "I really think you should have a bigger percent. I think you should have 40% and we should each have 30." And that's how it ends up breaking down.
Well, that leads to the three of them joining Google Brain directly. And turbocharging everything going on there. Spoiler alert, a couple years later, Astro Teller, who would take over running Google X after Sebastian Thrun left, he would get quoted in the New York Times in a profile of Google X, that the gains to Google's core businesses in search and ads and YouTube from Google Brain have way more than funded all of the other bets that they have made within Google X and throughout the company over the years.
It's one of these things that if you make something a few% better that happens to do tens of billions of dollars or hundreds of billions of dollars in revenue, you find quite a bit of loose change in those couch cushions.
Yes, quite quite a bit of loose change. But that's not where the AI history ends within Google. There is another very important piece of the Google AI story that is an acquisition from outside of Google. The AI equivalent of Google's acquisition of YouTube. It's what we talked about a minute ago, DeepMind. But before we tell the DeepMind story, now is a great time to thank a new partner of ours, Sentry.
Yes, listeners, that is S-E-N-T-R-Y, like someone's standing guard.
Yes, Sentry helps developers debug everything from errors to latency and performance issues, pretty much any software problem, and fix them before users get mad. As their homepage puts it, they are considered quote unquote "not bad" by over four million software developers. And today we're talking about the way that Sentry works with another company in the acquired universe, Anthropic. Anthropic used to have some older monitoring systems in place, but as they scaled and became more complex, they adopted Sentry to find and fix issues faster. So when you're building AI models, like we're talking about all episode here, small issues can ripple out into big ones fast. Let's say you're running a huge compute job like training a model. If one node fails, it can have massive downstream impacts, costing huge amounts of time and money. Sentry helped Anthropic detect bad hardware early so they could reject it before causing a cascading problem and taking debugging down to hours instead of days for them. And one other fun update from Sentry, they now have an AI debugging agent called Seer. Seer uses all the context that Sentry has about your app usage to run root cause analysis as issues are detected. It uses errors, span data, logs, and tracing and your code to understand the root cause, fix it, and get you back to shipping. It even creates pull requests to merge code fixes in. And on top of that, they also recently launched agent and MCP server monitoring. AI tooling tends to offer um limited visibility into what's going on under the hood, shall we say. Sentry's new tools make it easy to understand exactly what's going on. This is everything from actual AI tool calls to performance across different models and interactions between AI and the downstream services. We're pumped to be working with Sentry. We're big fans of the company and of all the great folks we're working with there. They have an incredible customer list including not only Anthropic, but Cursor, Vercel, Linear, and more. And actually, if you're in San Francisco or the Bay Area, Sentry is hosting a small invite only event with Dave and I in San Francisco for product builders on October 23rd. You can register your interest at sentry.io/acquired. That's sentry.io/acquired. And just tell them that Ben and David sent you.
All right, David. DeepMind. I kind of like your framing. The YouTube of AI.
The YouTube of AI for Google.
They bought this thing for, we'll talk about the purchase price, but it's worth what $500 billion today. I mean, this is as good as Instagram or YouTube in terms of greatest acquisitions of all time.
100%. So, I remember when this deal happened, just like I remember when the Instagram deal happened cuz the number was big at the time.
It was big, but I remember it for a different reason. It was like when Facebook bought Instagram, like, "Oh my god, this is wow, what a tectonic shift in the landscape of tech." In January 2014, I remember reading on TechCrunch this random news, right? You're like, "Deep what?" That Google is spending a lot of money to buy something in London that I've never heard of that's working on artificial intelligence.
Right. This really illustrates how outside of mainstream tech AI was at the time.
Yeah. And then you dig in a little further and you're like, this company doesn't seem to have any products. And it also doesn't even really say anything on its website about what DeepMind is. It says it is a quote unquote cutting-edge artificial intelligence company.
Wait, did you look this up on the Wayback Machine?
To build general-purpose learning algorithms for simulations, e-commerce, and games. This is 2014. This does not compute, does not register. Simulations, e-commerce, and games. It's kind of a random spattering of.
Exactly. It turns out though, not only was that description of what DeepMind was fairly accurate, this company and this purchase of it by Google was the butterfly flapping its wings equivalent moment that directly leads to OpenAI, ChatGPT, Anthropic, and basically everything. Certainly Gemini that we know.
Yeah, Gemini directly in the world of AI today and probably xAI given Elon's involvement.
Yeah, of course xAI. In a weird way, it sort of leads to Tesla self-driving too.
Okay, so what is the
story here? DeepMind was founded in 2010 by a neuroscience PhD named Demis Hassabis who previously started a video game company.
And a postdoc named Shane Legg at University College London, and a third co-founder who was one of Demis' friends from growing up, Mustafa Suleyman. This was unlikely to say the least. This would go on to produce a knight and Nobel Prize winner.
Yes. So Demis, the CEO, was a childhood chess prodigy turned video game developer who, when he was aged 17 in 1994, had gotten accepted to the University of Cambridge, but he was too young and the university told him, 'Hey, take a gap year, come back.' He decided that he was going to go work at a video game studio called Bullfrog Productions for the year. And while he's there, he created the game Theme Park, if you remember that. It was like a theme park version of SimCity. This was a big game. This was very commercially successful. RollerCoaster Tycoon would be sort of a clone of this that would have many, many sequels over the years.
Oh, I played a ton of that.
Yeah, it sells 15 million copies in the mid-90s.
Then after this, he goes to Cambridge, studies computer science there. After Cambridge, he gets back into gaming, founds another game studio called Elixir that would ultimately fail. And then he decides, you know what, I'm going to go get my PhD in neuroscience. And that is how Demis ends up at University College London. There he meets Shane Legg, who's there as a postdoc. Shane is a self-described at the time member of the lunatic fringe in the AI community in that he believes — this is 2008, 9, 10 — he believes that AI is going to get more and more and more powerful every year and that it will become so powerful that it will become more intelligent than humans. And Shane is one of the people who actually popularizes the term artificial general intelligence, AGI.
Oh, interesting. Which of course lots of people talk about now and approximately zero people were afraid of that. I mean, you had like the Nick Bostrom type folks, but very few people were thinking about superintelligence or the singularity or anything like that.
For what it's worth, not Elon Musk. He's not included in that list because Demis would be the one who tells Elon about this.
So, Demis and Shane hit it off. They pull in Mustafa, Demis' childhood friend, who is himself extremely intelligent. He had gone to the University of Oxford and then dropped out, I think, at age 19 to do other startup-y type stuff. So, the three of them decided to start a company, DeepMind. The name, of course, being a reference to deep learning, Jeff Hinton's work and everything coming out of the University of Toronto. And the goal that the three of these guys have of actually creating an intelligent mind with deep learning. Like, Jeff and Yann and Alex aren't really thinking about this yet. As we said, this is lunatic fringe type stuff.
Yes. AlexNet, the cat paper, that whole world is about better classifying data. Can we better sort into patterns? It's a giant leap from there to say, 'Oh, we're going to create intelligence.'
Yes. I think probably some people, almost certainly at Google, were thinking, 'Oh, we can create narrow intelligence that'll be better than humans at certain tasks.' I mean, a calculator is better than humans at certain tasks, right? But I don't think too many people were thinking, 'Oh, this is going to be general intelligence, smarter than humans.' Right? So, they decide the tagline for the company is going to be: solve intelligence and use it to solve everything else.
Ooh, I like it. I like it.
Yeah. Yeah. I mean, they're good marketers too, these guys. So, there's just one problem to do what they want to do.
Money. Just saying. Money is the problem.
Right. Right. Right. Money is the problem for lots of reasons. But even more so than any other given startup in the 2010 era, it's not like they can just go spin up an AWS instance and build an app and deploy it to the App Store. They want to build really, really, really, really big deep learning neural networks that requires Google-scale levels of compute.
Well, it's interesting. Actually, they don't require that much funding yet. The AI of the time was: go grab a few GPUs. We're not training giant LLMs. That's the ambition eventually, but right now, what they just need to do is raise a few million bucks. But who's going to give you a few million bucks when there's no business plan? When you're just trying to solve intelligence, you need to find some lunatics.
It's a tough sell to VCs, except for the exact right, as you say, they need to find some lunatics.
Oh, I chose my words carefully, didn't you?
Yeah, we use the term 'lunatic' in its most endearing possible way here, given that they were all basically right. So, in June 2010, Demis and Shane managed to get invited to the Singularity Summit in San Francisco, California, because they're not raising money for this in London.
Yeah, definitely not. I think they tried for a couple months and learned that that was not going to be a viable path.
Yes. This summit, the Singularity Summit, organized by Ray Kurzweil, future Google employee, I think, chief futurist, noted futurist Eliezer Yudkowsky, and Peter Thiel.
Yes. So, Demis and Shane are excited about getting this invite. Like, this is probably our one chance to get funded. But we probably shouldn't just walk in guns blazing and say, 'Peter, can we pitch you?'
Yeah. So, they finagle their way into Demis getting to give a talk on stage at the summit.
They're like, 'This is great. This is going to be the hack. The talk is going to be our pitch to Peter and Founders Fund.' Peter has just started Founders Fund at this point. You know, obviously a member of the PayPal mafia, very wealthy. I think he had a big Roth IRA at this point is the right way to frame it. Big Roth IRA that he had invested in Facebook, first investor in Facebook. He is the perfect target. They architect the presentation at the summit to be a pitch directly to Peter, essentially a thinly veiled pitch. Shane has a quote in Parmy Olson's great book Supremacy that we used as a source for a lot of this DeepMind story. And Shane says, 'We needed someone crazy enough to fund an AGI company. Somebody who had the resources not to sweat a few million and liked super ambitious stuff.' They also had to be massively contrarian because every professor that he would go talk to would certainly tell him absolutely do not even think about funding this.
That Venn diagram sure sounds a lot like Peter Thiel.
So they show up at the conference. Demis is going to give the talk. Goes out on stage. He looks out into the audience. Peter is not there. Turns out Peter wasn't actually that involved in the conference. He's a busy guy. He's a co-founder or co-organizer, but is a busy guy.
Yes. Guy's like, shoot. Oh, we missed our chance. What are we gonna do?
And then fortune turns in their favor. They find out that Peter is hosting an afterparty that night at his house in San Francisco. They get into the party. Demis seeks out Peter, and Demis is very, very, very smart, as anybody who's ever listened to him talk would immediately know. He's like, rather than just pitching Peter head-on, I'm going to come about this obliquely. He starts talking to Peter about chess because he knows, as everybody does, that Peter Thiel loves chess. And Demis had been the second highest ranked player in the world as a teenager in the under-14 category.
Good strategy. Great strategy. The man knows his chess moves.
So Peter's like, 'Hmm, I like you. You seem smart. What do you do?' And Demis explains he's got this AGI startup. They were actually here. He gave a talk on stage as part of the conference. People are excited about this. And Peter says, 'Okay, all right. Come back to Founders Fund tomorrow and give me the pitch.' So they do. They make the pitch. It goes well. Founders Fund leads DeepMind's seed round of about $2 million.
My, how times have changed for AI company seed rounds these days.
Oh yes. Imagine leading DeepMind's seed round with a less than $2 million check. And through Peter and Founders Fund, they get introduced — 'Hey, Elon, you should meet this guy' — to another member of the PayPal mafia, Elon Musk.
Yes. So, it's teed up in a pretty low-key way. 'Hey, Elon, you should meet this guy. He's smart. He's thinking about artificial intelligence.' So, Elon says, 'Great. Come over to SpaceX. I'll give you the tour of the place.' So, Demis comes over for lunch and a tour of the factory. Of course, Demis thinks it's very cool, but really, he's trying to reorient the conversation over to artificial intelligence. And I'll read this great excerpt from an article in The Guardian. Musk told Hassabis his priority was getting to Mars as a backup planet in case something went wrong here. I don't think he'd thought much about AI at this point. Hassabis pointed out a flaw in his plan. 'I said, what if AI was the thing that went wrong here? Then being on Mars wouldn't help you, because if we got there, then it would obviously be easy for an AI to get there through our communication systems or whatever it was.' He hadn't thought about that. So he sat there for a minute without saying anything, just sort of thinking, hm, that's probably true. Shortly after, Musk too became an investor in DeepMind.
Yes. Yes. Yes. I think it's crazy that Demis is sort of the one that woke Elon up to this idea of we might not be safe from the AI on Mars either.
Right. Right. I hadn't considered that. So, this is the first time the bit flips for Elon of we really need to figure out a safe, secure AI for the good of the people. That sort of seed being planted in his head.
Yep. Which of course is what DeepMind's ambition is. We are here doing research for the good of humanity, like scientists, in a peer-reviewed way.
Yep. I think all that is true. Also, in the intervening months to year after this meeting between Demis and Elon and Elon investing in DeepMind, Elon also starts to get really, really excited and convinced about the capabilities of AI in the near term, and specifically the capabilities of AI for Tesla.
Yes. Like with everything else in Elon's world, once the bit flips and he becomes interested, he completely changes the way he views the world. Completely sheds all the old ways and actions that he was taking. And it's all about what do I most do to embrace this new worldview that I have? And other people have been working on for a while already by this point. AI driving cars.
Yep. That sounds like it would be a pretty good idea for Tesla.
It does. So Elon starts trying to recruit as many AI researchers as he possibly can and machine vision and machine learning experts into Tesla. And then AlexNet happens, and man, AlexNet's really, really, really good at identifying and classifying images and cat videos on YouTube and the YouTube recommender feed. Well, is that really that different from a live feed of video from a car that's being driven and understanding what's going on there? Can we process it in real time and look at differences between frames? Perhaps controlling the car, not all that different. So Elon's excitement, channeled initially through DeepMind and Demis about AI and AI for Tesla, starts ratcheting up big time.
Yep. Meanwhile, back in London, DeepMind is getting to work. They're hiring researchers. They're getting to work on models. They're making some vague noises about products to their investors. Maybe we could do something in shopping. Maybe something in gaming, like the description on the website at the time of acquisition said. But mostly what they really, really want to do is just build these models and work on intelligence. And then one day in late 2013, they get a call from Mark Zuckerberg. He wants to buy the company. Mark has woken up to everything that's going on at Google after AlexNet and what AI is doing for social media feed recommendations at YouTube, the possibility of what it can do at Facebook and for Instagram. He's gone out and recruited Yann LeCun, Jeff Hinton's old postdoc, who's together with Jeff one of the sort of godfathers of AI and deep learning and really popularized the idea of convolutional neural networks, the next hot thing in the field of AI at this point in time. And so with Yann, they have created FAIR, Facebook AI Research, which is a Google Brain rival within Facebook. And remember who the first investor in Facebook was, who's still on the board, Peter, and is also the lead investor in DeepMind. Where do you think Mark learned about DeepMind?
Do you know for sure that it was from Peter?
No, I don't know for sure, but like, how else could Mark have learned about this startup in London? I've got a great story of how Larry Page found out about it.
Oh, okay. Well, we'll get to that in one sec. So, Mark calls and offers to buy the company. And there are various rumors of how much Mark offered, but according to Parmy Olson in her book Supremacy, the reports are that it was up to $800 million. Company with no products and a long way from AGI.
That squares with what Cade Metz has in his book, that the founders would have made about twice as much money from taking Facebook's offer versus taking Google's offer.
Yep. So Demis, of course, takes this news to the investor group, which, by the way, is kind of against everything the company was founded on. The whole aim of the company and what he's promised the team is that DeepMind is going to stay independent, do research, publish in the scientific community. We're not going to be sort of captured and told what to do by the whims of a capitalist institution.
Yep. So definitely some deal point negotiating that has to happen with Mark and Facebook if this offer is going to come through. But Mark is so desperate at this point, he is open to these very large deal point negotiations, such as Yann LeCun gets to stay in New York. Yann LeCun gets to stay operating his lab at NYU. Yann LeCun is a professor. He's flexible on some things. Turns out Mark is not flexible on letting Demis keep control of DeepMind if he buys it. Demis sort of argued for we need to stay separate and carved out and we need this independent oversight board with his ability to intervene if the mission of DeepMind is no longer being followed. And Mark's like, 'No, you'll be a part of Facebook.'
Yeah. And you'll make a lot of money. So, as this negotiation is going on, of course, the investors in DeepMind get wind of this. Elon finds out about what's going on. He immediately calls up Demis and says, 'I will buy the company right now with Tesla stock.' This is late 2013, like early 2014. Tesla's market cap is about $20 billion. So Tesla stock from then to today is about a 70x run-up. Demis and Shane and Mustafa are like, 'Wow.' Okay, there's a lot going on right now. But to your point, they have the same issues with Elon and Tesla that they had with Mark. Elon wants them to come in and work on autonomous driving for Tesla. They don't want to work on autonomous driving, right? Or at least exclusively.
At least exclusively. Yep.
So then Demis gets a third call from Larry Page. Do you want my story of how Larry knows about the company?
I absolutely want your story of how Larry knows about the company. All right, so this is still early in DeepMind's life. We haven't progressed all the way to this acquisition point yet. Apparently, Elon Musk is on a private jet with Luke Nosek, who's another member of the PayPal mafia and an angel investor in DeepMind, and they're reading an email from Demis with an update about a breakthrough that they had, where DeepMind AI figured out a clever way to win at the Atari game Breakout.
Yes. And the strategy it figured out with no human training was that you could bounce the ball up around the edges of the bricks and then without needing to intervene, it could bounce around along the top and win the game faster without you needing to have a whole bunch of interactions with the paddle down at the bottom. They're watching this video of how clever it is.
And flying with them on the same private plane is Larry Page.
Of course, because Elon and Larry used to be very good friends.
Yes. And Larry is like, 'Wait, what are you watching? What company is this?' And that's how he finds out.
Elon must have been so angry about all this.
And the crazy thing is this kinship between Larry and Demis is, I think, the reason why the deal gets done at Google. Once the two of them get together, they are like peas in a pod. Larry has always viewed Google as an AI company.
Demis, of course, views DeepMind so much as an AI company that he doesn't even want to make any products until they can get to AGI. And Demis, in fact, we should share with listeners, Demis told us this when we were talking to him to prep for this episode, just felt like Larry got it. Larry was completely on board with the mission of everything that DeepMind was doing. And there's something else very convenient about Google. They already have Brain. So Larry doesn't need Demis and Shane and Mustafa and DeepMind to come work on products within Google. Right? Brain is already working on products within Google. Demis can really believe Larry when Larry says, 'Nah, stay in London. Keep working on intelligence. Do what you're doing. I don't need you to come work on products within Google.'
Brain is like actively going and engaging with the product groups, trying to figure out, hey, how can we deploy neural nets into your product to make it better? That's like their reason for being. So, they're happy to agree to this and it's working. Brain and neural nets are getting integrated into search, into ads, into Gmail, into everything. It is the perfect home for DeepMind. Home away from home, shall we say?
Yes. And there's a third reason why Google's the perfect fit for DeepMind. Infrastructure. Google has all the compute infrastructure you could ever want right there on tap.
Yes. At least with CPUs so far.
Yes. So, how's the deal actually happen? Well, after buying DNNresearch, Alan Eustace, who David you spoke with, right?
Was Google's head of engineering at the time, he makes up his mind that he wanted to hire all the best deep learning research talent that he possibly could, and he had a clear path to do so. A few months earlier, Larry Page held a strategy meeting on an island in the South Pacific. In Cade Metz's book, it's an undisclosed island.
Larry thought that deep learning was going to completely change the whole industry. And so he tells his team, this is a quote, 'Let's really go big.' Which effectively gave Alan a blank check to go secure all the best researchers that he possibly could. So, in 2013, he decides, I'm going to get on a plane in December before the holidays and go meet DeepMind. Crazy story about this. Jeff Hinton, who's at Google at the time, had a thing with his back where he couldn't sit down. He either has to stand or lay. And so a long flight across the ocean is not doable. But he needs to be there as a part of the diligence process. You have Jeff Hinton. You need to use him to figure out if you're going to buy a deep learning company. And so Alan Eustace decides he's going to charter a private jet. And he's going to build this crazy custom harness rig so that Jeff Hinton won't be sliding around when he's laying on the floor during takeoff and landing.
Wow. I was thinking the first part of this, I'm pretty sure Google has planes. They could just get into Google Plane. For whatever reason, this was a separate charter. But it's not solvable just with a private plane. You need also a harness, right?
And Alan is the guy who set the record for jumping out of the world's highest — was it a balloon? I actually don't know — the highest freefall jump that anyone has ever done, even higher than that Red Bull stunt a few years before. So, he's like very used to designing these custom rigs for airplanes. He's like, 'Oh, no problem. You just need a bed and some straps. I jumped out of the atmosphere in a scuba suit. I think we'll be fine.'
That is amazing. So, they fly to London. They do the diligence. They make the deal. Demis has true kinship with Larry and it's done. $550 million US. There's an independent oversight board that is set up to make sure that the mission and goals of DeepMind are actually being followed, and this is an asset that Google owns today that again I think is worth half a trillion dollars if it's independent. Do you know what other member of the PayPal mafia gets put on the ethics board after the acquisition?
Reid Hoffman has to be given the OpenAI tie later.
We are going to come back to Reid in just a little bit here.
Yes. So after the acquisition, it goes very well very quickly. Famously the data center cooling thing happens where DeepMind carved off some part of the team to go and be an emissary to Google and look for ways to use DeepMind. And one of them is around data center cooling. Very quickly, July of 2016, Google announces a 40% reduction in the energy required to cool data centers. I mean, Google's got a lot of data centers, a 40% energy reduction. I actually talked with Jim Gao, who's a friend of the show and actually led a big part of this project. And I mean, it was just the most obvious application of neural networks inside of Google right away.
Pays for itself. Yeah. Imagine that paid for the acquisition pretty quickly there.
Yes. David, should we talk about AlphaGo on this episode?
Yeah. Yeah. Yeah. I watched the whole documentary that Google produced about it. It's awesome.
This is actually something that you would enjoy watching. Even if you're not researching a podcast episode and you're just looking to pull something up and spend an hour or two, I highly recommend it. It's on YouTube. It's the story of how DeepMind, post-acquisition from Google, trained a model to beat the world Go champion at Go. And I mean, everyone in the whole Go community coming in thought there's no chance. This guy Lee Sedol is so good that there's no way that an AI could possibly beat him. It's a five-game thing. It just won the first three games straight. I mean, completely cleaned up and with inventive new creative moves that no human has played before. That's sort of the big crazy takeaway. There's a moment in one of the games, right, where it makes a move and people are like, is that a mistake? Like that must have just been an error.
Yeah. Yeah. And then a hundred moves later it plays out and it was like completely genius. And humans are now learning from DeepMind's strategy of playing the game and discovering new strategies. A fun thing for Acquired listeners who are like, why is it Go? Go is so complicated compared to chess. Chess has 20 moves that you can make at the beginning of the game in any given turn and then mid-game there's like 30 to 40 moves that you could make. Go on any given turn has about 200. And so if you think combinatorially, the number of possible configurations of the board is more than the number of atoms in the universe.
That's a great Demis quote by the way.
Yeah. And so he says, even if you took all the computers in the world and ran them for a million years as of 2017, that wouldn't be enough compute power to calculate all the possible variations. So it's cool because it's a problem that you can't brute force. You have to do something like neural networks. And there is this white space to be creative and explore. And so it served as this amazing breeding ground for watching a neural network be creative against a human.
Yeah. And of course it's totally in with Demis' background and the DNA of the company of playing games. You know, Demis was chess champion. And then after Go, then they play StarCraft, right?
Oh really? I actually didn't know that.
Yeah. That was the next game that they tackled was StarCraft, a real-time strategy game against an opponent. And that'll come back up in a sec with another opponent here in OpenAI.
Yes, David. But before we talk about the creation of the other opponent, should we thank another one of our friends here at Acquired?
All right, listeners, we are here today to tell you about a new friend of the show we are very excited about. WorkOS.
Yes. If you're building software that is used in enterprises, you've probably felt the pain of integrating things like SSO, SCIM or SKIM, permissions, audit logs, and all the other features that are required by big customers. And if you haven't felt this pain yet, just wait until you get your first big enterprise customer. And trust us, you will.
Yes, WorkOS turns these potential deal blockers into simple drop-in APIs. And while WorkOS had great product-market fit a few years ago with developers who just want to save on some headache, they really have become essential in the AI era.
Yeah, I was shocked when they sent over their latest customer list. Almost all the big AI companies use WorkOS today as the way that they've been able to rapidly scale revenue so fast. Companies like OpenAI, Anthropic, Cursor, Perplexity, Sierra, Replit, Vercel, hundreds of other AI startups all rely on WorkOS as their auth solution. So I called the founder to ask why, and he said it's basically two things. One, in the AI era, these companies scale so much faster that they need things like authorization, authentication, and SSO quickly to become enterprise-ready and keep up with customer demand even early in life, unlike older SaaS companies of yesteryear. And two, unlike that world where you could bring your own little SaaS product just for you and your little team, these AI products reach deep into your company's systems and data to become the most effective. So, IT departments are scrutinizing heavier than ever to make sure that new products are compliant before they can adopt them.
Yeah, it's this kind of like second-order effect of the AI era that the days of, oh, just swipe a credit card, bring your own SaaS solution for your product team. You actually need to be enterprise-ready a lot sooner than you did before.
Yeah, it's not just about picking up that big potential customer for the revenue itself either. It's about doing it so your competitors don't. Enterprise readiness has become so table stakes for companies no matter their stage. And WorkOS is basically the weapon of choice for the best software companies to shortcut this process and get back to focusing on what makes their beer taste better, building the product itself.
Amen. Amen. So if you're ready to get started with just a few lines of code for SAML, SCIM, OAuth, SSO, authorization, authentication, and everything else to please IT admins and their checklists, check out WorkOS. It's the modern software platform to make all this happen. That's workos.com and just tell them that Ben and David sent you. All right, David. So what are the second-order effects of Google buying DeepMind?
Well, there's one person who is really, really, really upset about this, and maybe two people if you include Mark Zuckerberg, but Mark tends to play his cards a little closer to the vest. Of course, Elon Musk is very upset about this acquisition. When Google buys DeepMind out from under him, Elon goes ballistic. As we said, Elon and Larry had always been very close. And now here's Google who Elon has already started to sour on a little bit as he's now trying to hire AI researchers. And you've got Alan Eustace flying around the world sucking up all of the AI researchers into Google, and Elon's invested in DeepMind, wanted to bring DeepMind into his own AI team at Tesla, and gone out from under him. So, this leads to one of the most fateful dinners in Silicon Valley's history, organized in the summer of 2015 at the Rosewood Hotel on Sand Hill Road. Of course, where else would you do a dinner in Silicon Valley but the Rosewood, by two of the most leading figures in the valley at the time, Elon Musk and Sam Altman. Sam, of course, being president of Y Combinator at the time. So what is the purpose of this dinner? They are there to make a pitch to all of the AI researchers that Google and to a certain extent Facebook have sucked up and basically created this duopoly status on. Again, Google's business model and Facebook's business model. Feed recommenders or these classifiers turn out to be unbelievably valuable. So they can, it's funny in hindsight saying this, pay tons of money to these people, tons of money, like millions of dollars, take them out of academia and put them into their dirty capitalist research labs inside the companies selling advertising.
Yes. How dirty could you be?
And the question and the pitch that Elon and Sam have for these researchers gathered at this dinner is: what would it take to get you out of Google, for you to leave? And the answer they go around the table from almost everybody is nothing. You can't. Why would we leave? We're getting paid way more money than we ever imagined. Many of us get to keep our academic positions and affiliations and we get to hang out here at Google with each other. Iron sharpens iron. These are some of the best minds in the world getting to do cutting-edge research with enormous amount of resources and hardware at their disposal. It's amazing. It's the best infrastructure in the world. We've got Jeff Dean here. There is nothing you could tell us that would cause us to leave Google. Except there's one person who is intrigued. And to quote from an amazing Wired article at the time by Cade Metz, who would later write Genius Makers, right?
Quote: 'The trouble was so many of the people most qualified to solve these problems were already working for Google. And no one at the dinner was quite sure that these thinkers could be lured into a new startup even if Musk and Altman were behind it. But one key player was at least open to the idea of jumping ship.' And then there's a quote from that key player: 'I felt like there were risks involved, but I also felt like it would be a very interesting thing to try.' It's the most Ilya quote of all time.
The most Ilya quote of all time because that person was Ilya Sutskever, of course, of AlexNet and DNNresearch and Google and about to become founding chief scientist of OpenAI. So the pitch that Elon and Sam are making to these researchers is: let's start a new nonprofit AI research lab where we can do all this work out in the open. You can publish free of the forces of Facebook and Google and independent of their control.
Yes, you don't have to work on products. You can only work on research. You can publish your work. It will be open. It will be for the good of humanity. All of these incredible advances, this intelligence that we believe is to come, will be for the good of everyone, not just for Google and Facebook. And for one of the researchers, it seemed too good to be true.
So, they basically weren't doing it because they didn't think anyone else would do it. It's sort of an activation energy problem where once Ilya said, 'Okay, I'm in.' And once he said, 'I'm in,' by the way, Google came back with a big counter, something like double the offer. And I think it was delivered from Jeff Dean personally, and Ilya said, 'Nope, I'm doing this.' That was massive for getting the rest of the top researchers to go with him. And it was nowhere near all of the top researchers who left Google to do this, but it was enough. It was a group of seven or so researchers who left Google and joined Elon and Sam and Greg Brockman from Stripe who came over to create OpenAI because that was the pitch. We're all going to do this in the open.
And that's totally what it was.
It totally is what it was. And the stated mission of OpenAI was to, quote, 'advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return,' which is fine as long as the thing that you need to fulfill your mission doesn't take tens of billions of dollars.
Yes. So here's how they would fund it. Originally there was a billion dollars pledged.
Yes. And that came from famously Elon Musk, Sam Altman, Reid Hoffman, Jessica Livingston, who I think most people don't realize was part of that initial tranche, and Peter Thiel.
Yep. Founders Fund, of course, would go on to put massive amounts of money into OpenAI itself later as well. The funny thing is it was later reported that a billion dollars was not actually collected. Only about 130 million of it was actually collected to fund this nonprofit. And for the first few years that was plenty for the type of research they were doing, the type of compute they needed. Most of that money was going to paying salaries to the researchers. Not as much as they could make at Google and Facebook, but still a million or $2 million for these folks, right?
And yeah. So that really worked until it really didn't. Yeah. So David, what were they doing in the early days?
Well, in the first days, it was all hands on deck recruiting and hiring researchers. And there was the initial crew that came over and then pretty quickly after that in early 2016, they get a big, big win when Dario Amodei leaves Google, comes over, joins Ilya and crew at OpenAI. Dream team, you know, assembling here.
And was he on Google Brain before this?
He was on Google Brain. Yep. And he along with Ilya would run large parts of OpenAI for the next couple years before of course leaving to start Anthropic.
But we're still a couple years away from Anthropic, Claude, ChatGPT, Gemini, everything today. For at least the first year or two. Basically, the plan at OpenAI is let's look at what's happening at DeepMind and show the research community that we can do as a new lab do the same incredible things that they're doing and maybe even do them better.
Is that why it looks so game-like and game-focused?
Yes. Yes. So, they started building models to play games. Famously, the big one that they do is Dota 2, Defense of the Ancients 2, the massively online battle arena video game. They're like, 'All right, well, DeepMind, you're playing StarCraft. Well, we'll go play Dota 2. That's even more complex, more real-time.' And similar to the emergent properties of Go, the game would devise unique strategies that you wouldn't see humans trying. So, it clearly wasn't humans coded their favorite strategies and rules in, it was emergent.
Yeah, they did other things. They had a product called Universe which was around training computers to play thousands of games from Atari games to open-world games like Grand Theft Auto. They had something where they were teaching a model how to do a Rubik's cube. And so it was a diverse set of projects that didn't seem to coalesce around one of these is going to be the big thing.
Yeah. It was research stuff. It was what DeepMind was doing.
Yeah. It was like a university research. It was like DeepMind. And if you think back to Elon being an investor in DeepMind, being really upset about Google acquiring it out from under him, makes sense. And I think Elon deserves a lot of credit for having his name and his time attached to OpenAI at the beginning. A lot of the big heavy-hitter recruiting was Elon throwing his weight behind this. I'm willing to take a chance.
Absolutely. Okay. So that's what's going on over at OpenAI, doing a lot of DeepMind-like stuff. Bunch of projects, not one single obvious big thing they're coalescing around. It's not ChatGPT time. Let's put it that way. Let's go back to Google because last we sort of checked in on them.
Yeah, they bought DeepMind, but they had their talent raided.
And I don't want you to get the wrong impression about where Google is sitting just because some people left to go to OpenAI. So back in 2013 when Alex Krizhevsky arrives at Google with Jeff Hinton and Ilya Sutskever, he was shocked to discover that all their existing machine learning models were running on CPUs. People had asked in the past for GPUs since machine learning workloads were well suited to run in parallel, but Google's infrastructure team had pushed back and said the added complexity and expanding and diversifying the fleet. Let's keep things simple. That doesn't seem important for us. We're a CPU shop here.
Yes. And so to quote from Genius Makers, in his first days at the company, he went out and bought a GPU machine, this is Alex, from a local electronics store, stuck it in the closet down the hall from his desk, plugged it into the network, and started training his neural networks on this lone piece of hardware just like he did in academia, except this time Google's paying for the electricity. Obviously, one GPU was not sufficient, especially as more Googlers wanted to start using it, too. And Jeff Dean and Alan Eustace had also come to the conclusion that DistBelief, while amazing, had to be rearchitected to run on GPUs and not CPUs. So spring of 2014 rolls around. Jeff Dean and John Giannandrea, who we haven't talked about this episode.
Yes, you might be wondering, wait, isn't that the Apple guy? Yes, he went on to be Apple's head of AI who at this point in time was at Google and oversaw Google Brain. 2014, they sit down to make a plan for how to actually formally put GPUs into the fleet of Google's data centers, which is a big deal. It's a big change, but they're seeing enough reactions to neural networks that they know to do this.
Yeah. After Alex, it's just a matter of time.
Yeah. So, they settle on a plan to order 40,000 GPUs from Nvidia.
Yeah, of course. Who else are you going to order them from?
For a cost of $130 million. That's a big enough price tag that the request gets elevated to Larry Page who personally approves it even though finance wanted to kill it because he goes, look, the future of Google is deep learning. As an aside, let's look at Nvidia at the time. This is a giant, giant order. Their total revenue was $4 billion. This is one order for 130 million. I mean, Nvidia is primarily a consumer graphics card company at this point.
Yes. And their market cap is $10 billion.
It's almost like Google gave Nvidia a secret that hey, not only does this work in research like the ImageNet competition, but neural networks are valuable enough to us as a business to make a hundred-plus million dollar investment in right now, no questions asked.
We got to ask Jensen about this at some point. This had to be a tell. This had to really give Nvidia the confidence, oh, we should way forward invest on this being a giant thing in the future.
