Joining us now to introduce our next speakers, General Partner at Primary, Cassie. Young, this next session is going to be a very fun one for all of our enterprise enthusiasts in the room. I'm very excited to be welcoming Dev Ittycheria, who's the CEO of MongoDB. It's one of my favorite New York City success stories just because of the role the company has played in solidifying New York's presence on the enterprise SaaS map. Dave actually started his first technology company back in 1998, and in the time since, through his various leadership and board roles, he's actually played a hand in five different IPOs and three significant acquisitions, including when his own company, BladeLogic, he took that public in 2007 only to later sell it to BMC for close to $1 billion. A very fun fact for today is that Dave has spent time on both sides of the table, including as an MD at OpenView, where he led the company's investment in Datadog, where he is still on the board today. So Dave, we owe you many thanks for all of your contributions to New York City's tech market cap. Thank you for that. Dave is going to be interviewed today by Aaron Holmes. Aaron is a reporter with The Information. Before joining The Information, he also wrote for publications like Business Insider and the New York Daily News. His work has also been featured on media outlets like Good Morning America and CBS Los Angeles. Dave and Aaron, we are so appreciative of the time this morning. I'll turn it over to you guys.
All right, thank you all for being here, and thank you Dave.
Thank you. It's good to be here.
Yeah, nice to see everyone. Good to be in New York City. So unlike the last panel, I do not have enough self-control to stay away from AI for 20 minutes, so I want to start by jumping right into that. I think we're about two years deep into the AI hype cycle, and the signals are still mixed as ever in terms of where ROI is turning up. We're hearing about pretty much every Fortune 500 company using generative AI, but at the same time, I'm talking to CIOs every week who say the coding assistants are great, but it's not clear exactly where else ROI is going to show up. So I wanted to start by getting your thoughts on that. Where do you separate AI hype from reality?
Yeah, I think it's important. I'm unfortunately old enough to remember a few other platform shifts, and what's interesting is when you look at the platform shifts, adoption typically happens in S-curves. You get this peak of inflated expectations, then all of a sudden the hype surpasses reality, then you go through this trough, then there's another breakthrough, etc. So today, I would say because of the role that we play, really focusing on helping customers deploy inference workloads, I would say that there's a variety of inference workloads in production. There's lots of experimentation going on. I'll give you an example. I spoke to a very, very large financial services customer only about six weeks ago. I asked them how many AI workloads they have in production, and this is a firm that has about 45,000 developers, and they said 15. I said, okay, and how many of those are client facing? They said zero. The reason for that is they're terrified about the liability of giving bad information or bad recommendations for people who are making investment decisions. So for them, they're being very, very cautious. They're doing a lot of experimentation, but they're being very, very cautious in terms of what they roll out. But I do see a lot of interest. One of the things that we're seeing is there's a lot of interest in using AI as a means to an end rather than an end unto itself. We're seeing a lot of large customers who have all these legacy estates, all these applications. For another bank, they have like 5,000 COBOL apps. COBOL is going end of life. They don't have a coherent plan to get off COBOL in a very quick manner. So now suddenly with code generation tools and working with people like us, they're saying we could potentially short-circuit the process to migrate these legacy applications off these dated platforms to much more modern architectures that not only reduce tech debt but also enable them to be much more nimble. The CIO of a large insurance company said this is the first tangible use case I see for AI that really hits the bottom line. So we're starting to see pockets. Obviously as more breakthroughs come, as GPT-5 comes out maybe sometime this fall or by the end of the year, I think you're going to see increased value. But definitely the picture is very mixed.
Gotcha. Yeah, I mean the example that you mentioned of using generative AI to code your own tools in-house is interesting. And Klarna made some news this week because they came out and said that they are ditching Salesforce, they're ditching Workday because they essentially used generative AI to build their own solutions. But at the same time, I've heard concerns that CFOs don't feel like those kind of patched together generative AI built tools are ready yet. So do you buy what Klarna is saying, and do you see that happening in other software categories?
So I definitely believe code generation tools will reduce the cost of building software and also reduce the cost of switching from one piece of software to another. That being said, I'm a little skeptical. I've been around the block quite a bit, and I find that people tend to overestimate the impact of a technology in the short term but underestimate in the long term. If you're using Salesforce or Workday or some ERP system that's deeply embedded in your day-to-day workflow, just ripping it out seems very unrealistic. So if you're a light user of one of those platforms, then maybe it's more practical to say maybe I don't need to use that, or Slack, or some other tool, and I can just build my own homegrown solution. But I think it really depends on the intensity of usage, how deeply embedded it is in your day-to-day workflows, how much data is stored there. For someone to really think about getting off those platforms very quickly, I'm skeptical.
If you had to guess which categories of enterprise software might be threatened first by those kind of homebuilt generative AI solutions, what would you say?
I'll answer slightly differently. I think AI is going to go after work. I think it's going to go after a lot of white-collar work. I think that if I'm the leader of a systems integration company, I'd be deeply worried about what AI has in terms of implications for the work and the kind of projects my organization does. I think AI is going to go after things like law, tax, compliance, auditing, etc., where there's a lot of manual intensity as well as domain expertise, but it can be automated. Not that people no longer need accountants or developers or tax specialists, but you can do a lot of the mundane grunt work and automate that through AI. And I think that's where it'll have a truly profound impact on our potential lives, especially in the enterprise.
Gotcha. And you mentioned what your customers are doing, but I'm also curious what MongoDB is doing with generative AI and where specifically you are seeing the most return on investment right now internally.
Yeah, so our focus is really to be the data persistence layer for these new AI workloads. We've embedded our own vector search functionality, so MongoDB is the only place where you can unify your operational data, your metadata, your embeddings data, unstructured and structured data, all in one place. And it makes a developer's life incredibly easy to build these next-generation AI applications. And like I said, it's still early, but we're working with lots of customers. A lot of early-stage customers—if you're a founder thinking about building an app, please consider MongoDB, and we can really give you a lot of support and help. And we're also working with a lot of large enterprises like that bank I told you about. They're using MongoDB to experiment with a lot of AI workloads. And I think we're architecturally well-designed for AI because AI is all about querying rich and complex data structures. We're designed to do that. We're highly performant and scalable. We have massive share. We're the most popular modern database in the world today, so there are tons of developers out there who know MongoDB, and it's no longer risky using MongoDB like it was say 10 years ago.
Gotcha. Now you're also an investor, so curious to get your thoughts on where the most value is left to be created in generative AI. We're seeing foundation model companies like OpenAI is closing a $6 billion round right now, as outlets have been at $150 billion valuation supposedly. So that's the model layer. There's also the application layer, and then there's several infrastructure layers between. But I'm curious where you see the most opportunity now for founders and for investors who are making investments in AI.
Yeah, I would say actually I would slightly disagree with Devin in that I think there's three layers to invest in. He mentioned the model layer and the app layer. I think there's also the chip layer. I do think there's going to be a lot of innovation at the semiconductor area. I do think that because it's clear that the more compute you need, the bigger the breakthroughs you have, so people will start optimizing on driving more performance or better price performance at the compute layer. I just met with a really interesting company this morning called Etched. They're doing some really interesting work there. So I think there are three buckets. We are clearly focused on the app layer ourselves. I think what we've learned over the last two years is that if you're building a thin wrapper around an LLM, your long-term value is going to be questionable because with the next generation of LLMs, that'll probably be subsumed into the LLM. So my belief is the next generation workload will use the LLM as an enabling technology but marry that with novel data sets and deeply embedded workflows and a very compelling user experience. I think that's what the AI apps of the future are going to look like. It's hard for me to predict in what shape or form. Obviously we see a lot of work around AI apps focused on automating BDRs, AI apps focused on automating legal assistance, we see a lot of other examples like code generation, customer support, etc. But I think the other thing that will really differentiate AI apps is being able to embed real-time data. Because if you imagine you're talking to a chatbot and you want to figure out you placed an order with some e-commerce site and you haven't got any confirmation on when that order is going to be shipped, well that chatbot needs to know that you placed that order like an hour ago or two hours ago. It can't tell you the orders you placed three months ago, here's the history of them. That's not that useful. So embedding real-time information into your application will be increasingly critical in terms of really adding value to your business.
Gotcha. And I mean as someone with customers deploying AI and someone who's an AI practitioner, you have maybe more insight than most. But I'm curious what you make of the $150 billion valuation for OpenAI. Does that make sense to you?
How do I say this? Listen, if you have conviction that OpenAI is going to be the next seminal company, as investors typically like to see a 3x return, so that means you have to assume that you're going to see OpenAI worth potentially $450 billion or half a trillion dollars. And so that's a big hurdle. I do worry, much like Devin said, there seems to be a lot of competition in the space. A lot of people view LLMs as a means to an end, not an end unto itself, so they will open source and drive a lot of the value out of the LLMs into the end products. So I think it's actually good for us. It kind of hearkens back to the days of the internet when people were building out all this telecom infrastructure, this broadband networking, dark fiber, and then the bubble burst and a lot of people thought oh that was such a waste of money, but that's exactly when all our lives were transformed by the internet in terms of how we work, how we live, and how we socialize. So I feel like in some ways, while history does not completely repeat itself, it does rhyme. I feel like we're kind of going through a similar phenomenon, and I think we will all benefit from these investments, though I'm not sure the companies themselves. That's a decision for an investor to make in terms of how much money they're going to make on those investments.
