Why Top Founders Are Racing Into AI Infrastructure

We have a whole new technology. That's the most important technology ever. And you need a whole new infrastructure.

Normally when we talk about the infrastructure world, we're talking about the servers on the storage and the network. Here it goes all the way down to the mines of copper min. That's how widespread this thing is going to be.

It used to be when you built something, it was an engineering problem. And here it feels like it really is a resource limitation. So whether it's tokens or not, we're pouring a ton of money into systems and then those systems are producing a result. And right now we're bottlenecked on the systems ability to actually match the resources we're pouring into them.

The leading memory vendor said the demand they have today will take them 3 years of capacity to supply.

If this fund does what we think it will do, how do we see the world in 5 to 10 years?

America wins in the infrastructure and that would be awesome.

Ben Martin Ragu, welcome.

Thank you.

All right. Thank you.

I want to start with a Mark quote to introduce this new fund. This is the biggest technological revolution of my lifetime. This is clearly bigger than the internet. The comps on this are the microprocessor, the steam engine, and electricity, or maybe the wheel. Guys, the machine age fund. Please introduce it. Ben, start us off.

Yeah I would agree. I mean normally when we at least in the computing when we talk about the infrastructure world we talking about the servers and the storage in the network here it goes all the way down to the mines copper mines. That's how widespread this thing is going to be.

Um that's number one and number two I think what we have seen over the last 3 years is the steady increase of the capabilities of the models where the model is no longer the bottleneck and in fact using AI these models are getting better faster and faster and faster. Now the bottleneck is all what I call south of the model and so that's why we need to work on that. You know, the only thing I'd add very quickly is like we tend to follow founders and we've been watching over the last couple of years is the number of very strong teams going after complex hardware problems has increased. I don't know the actual numbers but I was trying to estimate it over the weekend. So I think we'd get you maybe 5% of the deals from top founders would come in would be hardwood before. Now I would say north of 20% or 30% right now. So like the founder community which tends to be much smarter than the VC community has identified this as a very active area for innovation and they're responding. I think 5% is probably generous.

Yeah, it's very low. Is very low. Yeah. 3%. Yeah.

And explain some of the macro conditions that have led to this this trans this change in terms of the surplus of founders pursuing these idea. Like what are they seeing that's that's enabled?

Well, I mean the obvious is like you know the the demand for AI is basically infinite and as a result of that every part of the supply chain is under under duress. I mean everything including like materials used to make things like like memory. Um it's also very interesting whether there's something unique about AI. Um which because the demand is infinite and growth is infinite. Um uh what you tend to worry about is the margin of companies which is how efficient it is. Like normally you worry about growth like can I just you know can I just get people to buy this stuff. You don't have to worry about that here. The question is is can you do this in a way that's profitable? And a lot of the um efficiencies are actually strictly a physical limitation of hardware. And so even the business model of the AI wave is really putting a lot of stress on the existing systems because they weren't built for AI. They weren't built for those workloads. And I think there's just this you know this global observation that we actually need to change the core components to get that efficiency to help drive the growth and to drive the value of the businesses.

Yeah. And how did we know that demand is actually outpacing supply here rather than this being you know another hype cycle? Well, I mean,

yeah, there are any number of cities today. Um, firstly, it is that some of the smartest judges of demand are cutting huge purchase orders. I mean, if you look at the hyperscalers, right?

Yeah.

Their capex spend has been exploding. Next year supposedly, it's going to reach a trillion dollars collectively across the big hyperscalers. This year, it's about 700 billion dollars, right?

And if you think about the hyperscalers position in the industry, they see demand from everywhere, right? They see obviously the frontier labs um wanting their compute, they see the AI native companies, they see the enterprise, they see the US geography, the international geography. So if anybody has visibility, it is down and they've been jacking up their capex like it's never been seen before, right? So that's a clear clear sign. And secondly, if you look at the companies that we see on a day-to-day basis, they are all ripping. All the application companies, the growth is insane. The frontier labs, the growth is insane. It's been documented. So I would say on the demand side, the signals have never been clearer that this is not a hype. It's to top it all, all of it is just and prices are going up.

Like we've never seen prices go like up on chips.

You keep your prices went down. They always go down. Yeah. It always goes down. If you look at the price curve, it it went like this and then went back this.

Yeah.

And we know only like 5 10% of the address for the market to stop today.

I mean, the the the supply, if you look, if you look at the supply across the board, it's basically all booked out to 2028.

I mean, it's so bad we've actually seen multi-day auctions for a few thousand GPUs.

Um, you know, the other side of that, of course, is demand. And as Ragu said, we've seen the fastest growing companies we've seen in the history of the industry.

But also the unit of work that AI can do, the value of that unit of work keeps increasing. But underneath the covers, the number of tokens that are consumed is going by orders of magnitude, right? If it's one token for I mean 100 tokens for chat or an agent, it's thousands of tokens, right? So you got expansion of both sides of demand. One is the unit of work is becoming more and more consumptive

of tokens and then secondly the number of people therefore that are going to be benefit it's not just the developers it's going to be all knowledge workers and then all of beyond that so that's that's what we see

you said the key components in supply are sold out to 2027 maybe in 2028

what does it mean for an entire industry to be sold out that that far like

I don't know this has ever happened before do you guys recall I mean remember in the Um in in in the internet days when we were doing massive buildout, the majority that was actually being put in the ground was speculative and was dark. Remember the dark fiber? And here basically every GPU that's being created is already pre-sold. So

yeah, we weren't quite there. I mean there was

there was a lack of bandwidth

like in the 989 time frame, but

there wasn't there there wasn't that much real demand for it because there just weren't that many people on the internet. So like it was a two-sided thing and the companies were all rushing there and needed more bandwidth theoretically, but there weren't the users on the other side to consume it necessarily. And then to really consume a lot of bandwidth, you have to do high bandwidth things like video, which weren't really viable for, you know, a number of reasons that that had nothing to do with how much bandwidth was in the data center. So it smelled similar, but it wasn't this. This is like we're flat out and people are reselling GPUs for four times what they bought them for and this kind of thing. Like it's just not. And then,

you know, we're we're also out of power and cooling. Uh and then on top of that, it's really hard to build because there's these incredible political headwinds going into it. So, it's it's really unprecedented in my career. uh that we've had anything anything like this. No,

I want to give you a quick anecdote. So, this I was talking to a CFO of a large company, large public company who had historically been very resistant about going into the cloud. So, they had a lot of servers and they were doing an inventory uh check and they realized that the memory in their servers had increased so much it could uh fund the entire migration to the cloud. So, I just feel like we're in a very unusual situation.

Yeah, that's right. We're out of many things. power, cooling, memory, GPUs, like you name it, we're out of it.

Yeah. So, the flagship uh conference for the industry is the one called hot chips is going on in Stanford and the leading memory vendor said the demand they have today it will take them 3 years of capacity to supply it. It's just today it's not even future demand.

So in terms of being about everything simultaneously, is it because people just underestimated how good the models would be, how how useful they would be, they just couldn't have foreseen the demand?

Well, I don't even think it's that. I mean, this stuff came out of nowhere, right? We we're only four years into this. So even if we had a perfect oracle, once it started working, I don't think

we could have built the capacity.

We could have built the capacity. There's no way. And we're talking about like chip cycles, which tend to be three to four years. We're talking about breaking ground and building data centers which is you know four to five years we're talking

and connecting breaking down and building them and having uh power source. So like you either have to build your own power or like usually both you've got to build your own power and have a power source which is not easy. Yeah. the ML industry that's used if it's growing at 20 30% it's a great growth rate right and it's being connected to an AI software industry that's like triple digits is the base you know

so you can see the disconnect right so it's it's just wide bank

and so why didn't this fund exist you know five years ago or seven years ago or why was it not a great category to to invest in in in the same way Brad

well I would say we're probably I'd like to think for just in time, but you know, we probably would have been uh well suited to have it at least a couple years ago.

I will say you could actually point on on on basically every epoch to uh an independent company that came up, right? Clearly the move from the client from uh in mainframe to the client server, we saw a bunch of companies come up. Um the move to the internet, this we got Cisco and Juniper. uh even in the mega data centers which by the way was largely driven by the incumbent cloud providers verticalizing you saw the uh arising of Arista so there has been the ability to invest in you know silicon and hardware but it's been relatively minor because the change has been relatively minor like one chip company one switch company where here everything is and so I I agree with Ben we're probably you know we probably could have started a little bit earlier um but the amount of change is so high now that it's just an obvious thing

And the other thing is the demand for intelligence is so vertical um with really no end in sight. I mean cuz every company that's adopted it is growing very fast in its usage and then most companies haven't adopted it to a high degree and then consumers are just getting started.

And so it's going to probably the demand for tokens is probably going to grow close to a,000% a year, which you cannot grow supply that fast. Like like we're not

like the amount of just work we're going to have to do across the board to get to the point where we can grow like infr at that kind of rate is is pretty vast. So I think there's a lot of investing opportunity on the way. And by the way, the other thing is like all the architectures

of the hardware systems were built for a whole different era of computing. And so more than just we need more capacity, we need capacity to build there there's lots of opportunities to build different kinds of infrastructure. Yeah, they're they're all reaching their the physics limits for what they were designed, right? Like what Ben was talking about copper and so on and so forth. And you could go across every one of these categories and you could find, okay, this is the limit of this type of technology. So now you got to get some technical breakthroughs to get to the next one.

Yeah. I want to dive deeper on the demand side for a second. As we've moved from chat bots to reasoning to agents to to multi- aents, each step has multiplied the number of tokens a single task takes up by orders of increasing orders.

Yeah, nobody likes to use AI more than AI.

So, um why does that keep happening instead of a leveling off? Do you just see that happening you know indefinitely just continuing to

uh well so there's a couple as that the first one is is for sure right now if you look at like the the way we're achieving scaling the way we're doing it is through a lot of inference so through a lot of token right if you think about what RL is you know it's it's a lot of inference if you think about chain of thought it's a lot of inference u think longunning agents of course it's a lot of inference and so that's just basically been one of the approaches that we've been using to um uh to scaling um I think if you want to step back and say kind of what is the macro trend here it used to be when you built something it was an engineering problem and you throw a bunch of engineers at it and that doesn't scale and that would have a natural law of engineering physics uh which is what the mythical manmouth came from and here it feels like it really is a resource limitation so whether it's tokens or not we're pouring a ton of money into systems and then those systems are producing a result and right now we're bottlenecked on those position those systems ability to actually uh match the resources we're pouring into them. And so I think like tokens right now is probably where we are on the scaling curve, but we don't have a natural regulator like engineering like we did before. So I think we should expect this to continue and we have to build a supply to support it.

Yeah. Like the simple way to think about it is

any problem that you have can be solved with enough

infrastructure

basically

GPUs and power and money. Uh and so until we run out of problems, we're not going to run out of demand. And that's the that's the uh challenge. I

mean AI's answer to getting better and better is to use more AI, right? inference is one basic building block that it keeps using over and over and over again and so that's why these tokens multiply at each step.

Yeah, even the autocatalytic effect so even the idea of using AI to create more AI like creating a GPU kernel of course is just using more AI um as part of the process. So again, one way that we think about it is in the past money would come in, you have an engineering problem, we know that it takes two years, normally fails, you know, it's a national governor and then you get the product on the other end. This there's there's there's nothing between the money going in and then the hardware, you know, creating intelligence. And so now we're just limited by our ability to create supply. It's a very very different

be more GPUs and our sol.

Yeah, that's right.

So that's the cycle. as long as you have the money, the GPUs, and the data, you know, for the foreseeable future, you'll be able to scale these things.

And it's it's fascinating because, you know, over the last decade, it feels like there so many, you know, people and the pervasive sentiment was there's too much money going to startups. We're overfunding these startups. There's too much money in in in in venture capital. Say more Ben about what that means because there there used to be this um sort of skepticism that the more money you put into into the industry that you know there would be bigger outcomes and now you know we were saying at the offsite that there's to some degree the the market is as big as we collectively contribute to it.

Yeah. So, this is this look, the one thing we all knew with in startup world is that if I have a two-year lead on you and you try and catch me by hiring a thousand engineers, you're going to wreck your company. Like, that never works. It's a mythical man month. Nine women can't have a baby in a month. That like that's it. Like, that never works. Okay, now that works.

But it's not hiring a hundred thousand engineers. It's taking $3 billion and like lighting up a magnificent cluster and then all of a sudden, you know, whatever Grock can come out of nowhere and like, oh, all of a sudden it's real or or a Kimmy or or what have you.

It's just like these leads um you can throw money at the problem and you can throw money at almost any problem and that works. And so that is just completely different than anything we've ever lived through. So we're, by the way, we're all psychologically adjusting to this. The Chad GBT app has a billion weekly activives. There's about 30 million uh developers um who are using, you know, relatively a big portion of compute demands. How do we think about compute compute demands needs now and in the future in light of what people are actually doing with AI?

That's the progression, right? So Chadic was a casual app and it's coding for professionals right now using coding you have now built amazing tools for knowledge workers so that's the next frontier and now there are over a billion knowledge workers in the line right and with that it's a long ways to go for that demand and by the way the work that they do all this workflow and automation and so on and then you get to the back office which is all the agents So progressively each of these things unlocks

uh I would say order of magnitude more demand. I mean just at the start of this

well and now you have Grockbot which is kind of uh you know what uh happened with coding is kind of happening with all use of computer viaot and so we're in a whole another wave of demand and most certainly there's going to be more to come. Uh so it does seem quite unlimited at the moment and we haven't even gotten into embodied AI or robots uh which are going to be another source of demand.

Martinez is expert but my understanding is that part uses computer use which is just like

human being sitting inside the computer typing away.

I literally used it over the weekend to to update my credit card with a bunch of services that I'd been like lazy to do and cancel a bunch of subscriptions. I mean, this is not coding or whatever. This is true computer use.

All of a sudden, you're creating like half a billion knowledge workers except they're all sitting inside of the computer doing what?

I I do I do think that that that Mark Mark Andre is right. It's like the right analog here is like the steam engine or electricity in the following way. Like we've we've introduced this new thing that you can turn to work and there are some very obvious applications now, but there's probably 30 40 years of throwing computer at problems. anything with a clear reward signal and we we're just starting like we've got language and code that's it and just starting computer use but like what else are we looking at we're looking at uh in terms of science materials biology I mean of course creativity is a massive use and so listen we're at the very very early part of a very long journey and we've removed this key bottleneck which is you know traditional software engineering now of course you know bottlenecks will move and there'll be kind of more complexity elsewhere but I think we're at a very early in a very long run of throwing computer problem. So let's expect you know this compute need to persist for decades.

But because we mentioned it u Marty talk about grabbot um because we um we're talking at the offsite about how you know what what struck you about it. Obviously we're involved in every possible way you could you could be involved but what um yeah what what did you find so interesting about it? So I think we've I think we've as an industry gone through kind of multiple realizations for how AI enters our lives, right? And and uh very early on we're like okay well you add AI to a product and it's like whatever it's like a search bar and then you kind of you know you you do chat with it and it chats back because that's kind of the traditional way to do it. Um uh and then Open Clock kind of showed up and that was it earlier in the year. And with Open I say, "Okay, well maybe like it just being like Google but better. Maybe that's not the full embodiment of it. How about we'll have it be a standalone thing but it'll be an extension of you and it'll share your keys and it'll know your passwords and it'll just kind of do stuff that you would do, right? So it's kind of an extension of you but it's more like a human, an extension of you." And then what I think Rockbot got really right is no, how about it's actually an employee. So now you have this thing that's an entity and it doesn't have like special access to your keys or whatever. It has its own computer and it has its own browser and because these are the smartest models in the world, it can do whatever an employee can do. And it's kind of interesting because now actually if I if I want something done my first thing I I think is like well can do it for me and and often the answer is yes even if it's something you wouldn't you know expect it. So, the obvious ones are like whatever. It'll like manage my calendar. It'll like book a meeting, but there's also non-obvious ones as well. Like, so, for example, I'll have it um uh read through my email and do triage. And and I did I don't tell it how to do that, but it will know to check with me before actually doing the triage. So, like these things are sophisticated enough that you can give a relatively high level task and it'll do kind of, you know, like sophisticated things as a result.

Ben, I know you're thinking a lot a lot about this and how this you know, works in the organization. you think a lot about culture of course. What are your thoughts here?

Well, I mean, I think if you just look at us, um,

you know, it's like having a new kind of employee and

there's going to be a lot of them and we have to just like, you know, with our we spent many, many, many years figuring out how to work with our kind of regular human employees and now we've got these other kinds of employees

and, you know, there is a learning curve with them. So um they can burn a lot of tokens and spend a lot of money and get nothing productive done. Um they can forget stuff. They can make stuff up. Um you know they can have good behavior. They can have bad behavior. They can they can create security problems. So like there there there's all those aspects to it, but like they can also be like super duper productive. And so I think figuring out how to integrate them in, have them work nicely with the people that they're working with, um, the actual humans, uh, is all something that we're learning how to do. I mean, I I don't want to sit up here and say I've cracked the code. We've got this more of a sloop and

the whole firm is just completely automated now, and I'm going to slowly get rid of all the humans because I can. Like, that's not at all where we are. were much more going like, okay, how do we make all our humans superhuman um without like wrecking the place um because the bots got out of control?

Yeah. And it's interesting because we've done we've tried a couple of different ways as how best to get agents into the system if you will and eventually it was Martin's insights just treat them as people and get it done. And that's what we're doing. that's turned out to be the most durable way of getting this thing going inside of an organization.

I want to go back to the supply side and go deeper into the the bottlenecks. You we were talking about how you know in terms of the data centers, the chip architecture, system software, facilities themselves that none of them were designed with with AI in mind. What would it look like for them to be designed with AI? Like what is sort of the mental model for thinking about what that could mean?

Yeah. So um I mean if you start with a statement that you just say hey original model um of infrastructure on any of these models has to change you can go se category by column and see how it bricks right and then you start uh unlocking the bottlenecks in each one of these things. So eventually you have to get to a system where if you look at what an inference engine does, right? It takes up a lot of memory, it generates new tokens along with the compute. And so you can just think about how do I optimize all of this? What does the memory need to be? What does the compute need to be? How do they need to talk to each other? How much power does each of them need? And if they need all of these power, how do you cool each of these? Right? And then how do you put the collections of these things together? That is the exercise that's underway in the industry right now with a lot of the founders. So they're breaking down the problem into its fundamental components and saying what is the exact nature of the compute that's getting done. Okay, it's how it's going to be matrix multiplications. How do I optimize my compute around that kind of a scenario and then they all need very progressively to generate these tokens? What is the best way of hierarchically arranging this memory right and then how does the power consume I mean then you got to connect it together what are the ways of connecting it on the same chip but across chips and across data centers how much power does each of these data transmission take so you have to progressively break it all down and rebuild it from these fundamental building blocks and that's what we see

Underway, and that's where we see the opportunity.

Well, to that end, rack power requirements are moving from roughly 5 to 10 kilowatts to 100 to 50 kilowatts. Compute density is climbing something like 70x. Cooling is moving from air to liquid as a requirement. What are the investment opportunities as a result of this?

The horse. Yeah.

Yeah. I think we're just doing something in space, too. The big guys that own the big cloud data, big data centers, they all are furiously experimenting with the robots, right? To do the work of assembling or putting servers into the data center, etc. And so you will see that increasing as a result of the evolution in AI.

Yeah, and saying more about the robotics arm in terms of what we'll be doing versus maybe American dynamism or how to think about that.

Yeah. Going back to the data centers, by 2028, new data centers are going to need something like 44 gigawatts of additional power against maybe 25 gigawatt of expected grid additions.

No, it's like, it's like we'll have 100 gigawatt.

Martine. What's a gigawatt?

What do you mean? What is it?

The equivalent is like 50,000 houses.

50,000, 50,000 homes. 50,000 homes. I grew up in Flagstaff, Arizona, which is a town of 40 to 60,000 people, depending on the universities. We have less than a gigawatt of power consumption. I mean, this is...

Yeah. I mean, this is...

No, but by the way, everybody talks about the gigawatt. There's very few gigawatt data centers that are actually up. I mean, we've got a long way to go, right? But then why can't utilities and hyperscalers just build faster?

By the way, it is so bad that right now if we have new companies going for GPUs, it's often in Mexico or Australia or another country just because it is so difficult in the United States. Yeah, we're creating huge both job and long-term economic opportunity in other countries by banning data centers here. I think, look, the right answer would be to set a standard where a data center contributes back to the community, like that power gets better, there's no noise, there's no water issue, and it's adding jobs. Like, that ought to be the standard.

Zooming out. Why do we think the, you know, we've been batting around the name for a little bit. Why do we think Machine Age is a compelling term for what we're doing here?