#334 The State of Data & AI with Tom Tunguz, VC at Theory Ventures

We thought that pre-training was dead and all the techniques about bringing in the total of human knowledge and compressing it into an LLM. We had solved that problem. But the launch of Gemini 3 basically shattered that illusion.

The nimblest companies, the companies that are relentlessly customer focused will be the ones who win because they need to anticipate and hear from their customers how those buyer preferences are changing.

How do you start to create bigger, more complex documents that describe an entire project in a way an agent can understand and work on while you are hosting a podcast, while you're attending a meeting, while you're having lunch? The status symbol now will be how many agents are working for me while I'm having tea.

Oh, thrilled to be back. Thanks for having me on, Richie.

Yes, I would say we are in a period of incredibly rapid investment in AI. And I mean just to kind of, so I think it's, we ran this analysis of comparing AI to other boom cycles or other bubbles and the most relevant one I think only for scale is railroads in America. So at the peak of railroads, we were spending, the US was spending about a little bit more than 6% of our GDP on railroad construction and it took about 40 years to get there. So the railroad started like 1810, 1820 and the peak decade was 1860 and we were spending the equivalent of what would be today of about five, five and a half trillion per year, right? I mean, it tells you like, wow. I mean, can you imagine how many people were chopping trees down? And so today AI is about 1.5, 1.6% of US GDP. Next year it'll probably be closer to two and a half to three. So it will be half of the scale of railroads.

Huge amount of infrastructure. And so anyway, so it's not, our spending is profligate, but it's not as banana as say it was 200 years ago. The other dynamic here is we thought there were scaling limits. We thought that pre-training was dead and all the techniques about bringing in the total of human knowledge and compressing it into an LLM. We had solved that problem. But the launch of Gemini 3 basically shattered that illusion. There was a tweet two days ago, I think when they launched Gemini 3, that the improvement from two and a half to three was the single largest improvement in performance of any model in Google's history.

Right. Which is, I mean, that'll make you sit up and re-evaluate. And what does that tell us? Well, you know, within the last six months, the team at Google have figured out how to completely break the perceived law that we thought or the scaling limits associated with pre-training. And that means that when the Blackwell chips ship and they're three to five times more powerful than the current Hopper chips, that we should see improved performance. So, you have two components contributing to performance. You have an ability to improve algorithms both on pre-training and post-training, and post-training was the theme of the summer, and then you have basically significant improvement on the chip side. And so yes, are we spending a lot of money on all this? No doubt. Are we seeing dramatic improvements in performance? Yes. And then the last thing is I think as a result of those performance improvements, the productivity gains that we'll see should be real and significant. And then will we overspend? No doubt. Look, we will build way more data centers than we... If nobody knows what that point is, we will overshoot.

Well, yeah, the theme of the summer was around post-training. So reinforcement learning environments. And the way I think about that, and this is not totally true, but the way I think about it is if you have a robotic vacuum that enters a room, it needs to do two different things. It needs to make a plan for how to clean around tables and the carpet, and then it needs to reward itself. So just the way that you and I might take a vacation after a long stint of work, or grade ourselves at our work, what our objective and key results are, a Roomba has to create OKRs for cleaning a room. So 10 points for every square foot or meter, and then 50 points for every chair and then 100 points for a full bin. Over the summer we have been focused on the automation of both of those. The first one is nearly fully automated. The second one is still research. And so that was a big push and the theme coming out of Q1 was, okay, all the pre-training, all the compression of human knowledge is done. My guess, and I don't have any knowledge here, but my guess is there's been some pretty significant use of synthetic data to drive the pre-training gains. And that's really exciting because if we can have AI generate data that is then used to improve the AI systems, we have this virtuous cycle that will last for some period of time. The naysayers will say, and I think with some reasonable caution, that at some point if you just keep generating synthetic data, you have what's called model collapse. So if you keep building fake data and fake data and fake data, all of a sudden your foundation is not very strong. But Gemini 3, and again I don't know the details, suggests that there are meaning, I mean huge, right, biggest gains ever to be had from intelligent applications of synthetic data.

Yeah. And if you can create, right now we're at a place where if you can create those loops and we can do it in coding where you can spin up a cloud agent and tell it keep iterating until you satisfy these conditions and satisfy these tests, then it works. And now we're trying to figure out what do those reinforcement learning loops look like for non-coding applications.

We've been deploying them inside of Theory. So I manage most of my email with an agent, manage most of my tasks, a customer relationship management system that we have is all through AI. We have agents developing algorithms to extract information from podcasts and lots of other applications. The challenge historically has been what's called tool calling. So read an email and then find a company URLs in there and then look them up in the CRM and then enrich them. There has been a significant amount of error at each step. When you have a multi-step process, the error explodes the longer the number of conditional probabilities. And what's been amazing, and I have been using Gemini 3 for the last three days, is the tool calling ability. I had looked at the benchmark. There's a benchmark that was created by Sierra called Tau. And the benchmarks would lead you to believe that the improvement in tool calling ability of Gemini 3 compared to Sonnet 4.5 are relatively modest, but in practice it's unbelievable the difference, right? And so we're getting there. We're getting very, very close.

Well, yeah. Summarization agents I think were the first ones to work very, very well. Take large volumes of information. We're investors in a company called Drop Zone that summarizes security information for people who analyze the incidents inside of security operation centers inside of enterprises. And there the agents will query a database and they'll go and read online about some particular vulnerability. They might read previous documentation and summarize it. That works. It's a brilliant application of it. It's sort of the same thing, right? Go and research within a particular domain. I think we'll get to a place in early 2026 where you will start to see some much more sophisticated applications. I mean, and we're starting, there's some evidence of it within the world of customer support. So, typical customer support call might have 20 to 25 agentic tool calls. That's not to say it's a single call with 25 trains pulled by a locomotive. It's 25 short base pairs combinations. I'm mixing metaphors here, but so we're starting to see some of these workflows with pretty significant tool calls. I hope if anything should give us hope, OpenAI released the updated version of Codex this week where they said they had broken their record of like two months ago of a seven-hour continuous job and now it's closer to eight and a half, nine hours, right? So if we can do it in coding, well, there's no reason that we can't do it with marketing or customer support or sales.

Yeah, that's right. Yeah, exactly right. Exactly right.

I'd say eight hours is the upper bound.

I think we consistently see agents operating for 15 to 30 minutes with success and that will be the sweet spot today.

Yeah.

Well, I think it's the opposite. I think it's how do you start to create bigger, more complex documents that describe an entire project in a way an agent can understand and work on while you are hosting a podcast, while you are attending a meeting, while you're having lunch. I think the status symbol now will be how many agents are working for me while I'm having tea.

How much passive work, right? There was a wave I think in COVID of like side hustles and now it's actually no, I want all these agents working for me. Can I have 10 of them simultaneously? Can I fill up a queue, a hopper, where all of the tasks are sufficiently well defined that the agent can go and just grind through it.

Oh, I think the, okay, so a very simple agent is upcoming meeting summarizer. That's a very simple one for a lot of people. You know, podcast listener was one of the first ones that we built, a newsletter summarizer. So I have an agent that goes through my inbox and looks at any email that has the word unsubscribe in it, archives all of those emails and then summarizes and tries to find anything that could be interesting and then sends me a compendium of all that. Those are the really simple agents.

Yeah, I think the main difference is that the agent should be self-improving, but that's not necessarily always the case. Like a coding agent does not necessarily improve. A coding agent is just constantly checking to see is there work for me to code and if so I code and if not I don't. And a cron job, well, it's kind of the same thing, right? Check to see if I have new emails or I should be checking to see if I have new emails and if I do I process it and if I don't. So maybe the only difference is that there's some form of non-deterministic programming within an agent. As I've developed all these different tools internally, my crontab file is enormously long because there are all these agents running at different intervals. Check every one minute. Check every three minutes. And in fact, a tip for anybody else building out there, don't use cron on Mac. You should use launchd. It's far more robust. Cron won't restart after you put your Mac to sleep.

No, no, no, no. It's just, I just like, what? I just built all these agents and they're so lazy. Every time I put the computer to sleep, they don't start up. And so anyways, it was Claude Code that taught me that cron is historically and has been for a really long time incredibly brittle.

Yeah, unemployment rates for college graduates are approaching 10%, which is quite high. I think the most important thing is there's a tremendous amount of demand for people who, irrespective of their age or seniority, can reinvent workflows with AI. Nobody knows. There's no one in the Valley who has a secret for the best way of working with a coding agent. There's no one who has discovered the new optimal way of designing a marketing campaign or the best way of structuring a customer support organization. It's all new. And in addition to it being all new, it seems to be all new every three months because the capabilities of the underlying models advance to such an extent that previous assumptions no longer hold. And consequently, if you're a person with ambition and curiosity, demonstrating a reinvention of a workflow in a domain you want to pursue is very likely to help you get a job because a hiring manager will look at that implementation and say, 'This may or may not be perfect. It is a meaningful step forward. But more importantly, I'm looking at someone who has both the curiosity and the ambition to iterate to what could be a meaningful differentiator for me internally and as a hiring manager I want to be promoted and then ultimately for the business as a differentiator.'

Yeah, that's right. We reached, I mean, with SaaS we had reached, you know, this is what the modern data stack looked like, then this is the Salesforce Marketo stack. We don't have any of that. It's all wide open.

Yeah. So you definitely see, I think you're right. We used to see engineering organizations structured as pyramids where you'd have one senior person and then two architects and a bunch of tech leads and then a lot of software engineers. And now we see them structured more like rocket ships where the wide part of the pyramid is smaller. I bet that is a temporary phenomenon. And I think it's a temporary phenomenon because the rate of code generation by those senior people will very quickly outstrip their ability to review the code. There's just no way. There's just no way. And we still need human in the loop to be able to review the code. And that will initially be some of the work of people coming out of school will be evaluating the code and effectively QA on top of the AI, quality assurance on top of the AI-generated software. And the other dynamic here is I think we're in this little nadir of people generating a tremendous amount of productivity while the rest of the market catches up. But as the rest of the market catches up, we'll go back to higher levels of burn, higher level of investment because the overall productivity to remain competitive will rise just the way the sea level rises. Everyone needs to respond.

I think it's true. Yeah, I think at some point, well, so within EPD, within engineering, product and design, an engineer can now code, design a product and then design the UX themselves. And the same is true for a designer. A designer can design the product, create a PRD and then ultimately vibe code it. So you have a skill set expansion that blurs the lines across the EPD departments. And so some people take, I mean, I've heard some perspectives where, and even Claire Vo will say this, AI is a death of product management.

Yeah. Why, if you're an engineering manager, why would you need a PM? PM is just orchestrating communication across multiple stakeholders, both internal and external. What does the customer want? How do we build it? When do we prioritize it? Can we validate it? Who are the initial design partners? Well, you could argue AI can do all of that. A product manager can look at an engineer's role and say, particularly for front-end work, well, an engineer is just implementing a spec. Well, what does the coding agent do? We need far fewer agents. I think we'll see short-term aberrations, but I don't think we'll see pretty, I don't, I think the AI engineer EPD teams of the future will look very similar to the ones that we have today. The ratios might change, the productivity will be hugely increased, but I don't know if any one role increases in scope or is obviated. I think they remain relatively similar.

Well, customer success is probably the one that has been the most acutely and immediately impacted with the advent of the forward deployed engineer.

Yeah, forward deployed engineer is exactly what it sounds like. It's a salesperson who is extreme, who's an engineer and they typically report to the VP of engineering that they are deployed or they are sent to the customer and the primary responsibility is taking an AI product and making sure it works really well at a customer. And so you need somebody who typically has an engineering degree to do that. Historically, customer success has been not so technical. And if we think about the boom in the SaaS ecosystem, if you're configuring HubSpot or if you're configuring Salesforce, you don't need an engineer to be able to do that. In fact, you probably want somebody who is more comfortable or looks more like the buyer. So the buyer has a certain level of comfort. Many of these AI systems are bought by very sophisticated engineering teams and as a result, you want engineers on both sides. And when the buyer is not a technical person, there's another need which is the configuration of the software to drive the intended outcome. Particularly because many of these pieces of software charge on a resolution basis, actually do the work. That's really important. And so the combination of those two things has basically reinvented customer success for many companies to be these forward deployed engineers, which are typically much more expensive and the contract sizes have to go up and they have. But so that's, I think, the team that has changed the most.

Yeah. Yeah. Very. I mean, I think the number of FDE job requisites at the beginning of the year up 10x.

Yeah. So I think, so Steve Blank wrote this book called The Four Steps to the Epiphany, which the design philosophy for companies around establishing product market fit. I remember reading it in 2008. It was a brilliant book. And then it kind of created this movement called the lean startup movement. The idea was remain as small as possible until you've achieved PMF and at that point you can scale. And then there was a counter movement, countercultural movement, which is called a fat startup. Raise as much money as possible, hire a lot of salespeople and then after you sold, figure the product out, which actually is the way Salesforce was built. If you read Marc Benioff's book, he had a tremendous amount of salespeople and was burning a huge amount of money very early on. And during that era, it was kind of taken for granted that once you had crossed the PMF threshold, once a startup was able to grow from founder sales to account executive sales, the founder could train a salesperson to sell the product and that they could demonstrate repeatability, that it was all about scaling. And that's because the underlying infrastructure really didn't change that much and buyer demands really didn't change that much. And today, as we've talked about, Richie, the capabilities of the models are changing so fast, the buyers don't really know what they want because nobody knows what's possible. The definition of PMF is no longer constant. It's this like you have to constantly reestablish PMF. So you might have a foundation model company moving up and competing with one of the clients, one of its customers. And so all of a sudden you might have had PMF as a business as an agentic company building on top of say Anthropic or OpenAI, but if OpenAI or Anthropic move into your market and they have the underlying model, well, you've lost PMF. Or there have been categories of software where initially the startup was developing like content generation prompts that were very, very good at generating prompts and then a new model iteration comes around and that model has intuited prompts and so there's no value in those additional prompts and you've lost product market fit. So I think that's a dynamic where we're seeing and it's happening so fast, faster than I've ever seen it.

Well, I think customers will be the winner because they have so much choice and the models are becoming that much better and the price for performance is significantly better. They're the nimblest companies, the companies that are relentlessly customer focused will be the ones who win because they need to anticipate and hear from their customers how those buyer preferences are changing.