Box's Aaron Levie: On Reinventing Yourself in the AI Age and Enterprise Diffusion

I want to send an emergency alert to like everybody who's like a sophomore or junior in college and just be like follow these 20 accounts on Twitter and also join Twitter because like this will just help your career. You're either like a year ahead or a year behind simply based on your feed and my feed is like so so wired in. I'll still talk to 20-year-olds that are like yeah like you know I see some articles and I'm like what do you mean how do you see articles? Like I don't even know what that means. Like do you just like get lucky that somebody emailed you an article? You just have to be wired in. I enjoy it. It's a lot of fun. I play with everything.

Thanks for having me. I'm a big fan of the podcast. I religiously watch every episode and so great work. I am wondering though, I know this is a different podcast, but when do you talk about root canals in this? Oh my gosh.

We should just do this pod with a live root canal. Let's do it and see how it goes. Actually, that would be like hot ones, but you get a root canal and you have to talk about your strategy while you're in a dentist chair and Doug is just on the other end of it.

Has he seen every tweet?

Okay. Oh, yeah. He's very pleased with himself.

You know, two years ago, I think it would have not made that much sense as like what does that mean? But very clearly I think this is what's playing out in the market and it's all working out mostly because of open source. But what's pretty amazing is right now the whole concept of being a sort of LLM wrapper or model wrapper is actually working out because what I think people underappreciated was that in the real world in the enterprise what you need is some bridge from the model's capability to the actual workflow that the enterprise has and that bridge you basically like probably a trillion dollars has been bet on basically one of two outcomes. Either that bridge is very kind of limited or that bridge is actually very vast and needs to be able to take on a lot of depth within organizations. And that bet basically looks like are you only long the model itself and super intelligence or are you long this sort of application tier or you know maybe previously would have been just pure Neolab. But I think it's very clearly playing out that actually there's a lot of gap between the model and the workflow and as you bridge that gap over time you get to a point where you realize oh I should also do the model and then you have enough data you have enough sort of domain expertise where that becomes its own sort of flywheel. So I'm very bullish on this idea and what's cool is it's opening up multiple layers of opportunity for startups because you could either be the actual applied company itself i.e. the Neolab or you could be the infrastructure provider to the Neolab and you have multiple layers of going and attacking that space. But I think a huge update for the market and everybody's kind of view of it.

We already have it. It's a little bit secret but we pay very close attention right now. The main focus is let's make the agent really really good on any model but over time obviously you would peel off certain use cases either on a per customer basis or kind of across the whole data set.

Yeah. I think what I tend to see happen in the valley is and for very good reason and it's actually why these companies have been so successful is like everybody is so kind of research pilled which is again totally awesome big fan. It leads to all these breakthroughs. But the sort of sense that okay the model and the intelligence in the model is kind of the only form factor that matters. And then you go to the real world and you sort of see how intelligence actually gets rolled out in people's workflows and the model could be the most intelligent super intelligence in the world but that workflow still requires you to connect up to other data systems still requires these moments where there's a human in the loop interaction. There's delays in the workflow and so the agent has to sit idle. There's change management of the actual business process. There's legacy systems that haven't been modernized. So you kind of go through these five or 10 things that are way sort of much more operational, much more blocking and tackling than just the pure super intelligence of the model. And the last thing that I think a classic sort of research organization wants to go do is go attack every single one of those things. And this is sort of not a one-off in history. Like we've always had this relationship between infrastructure and application. Like obviously AWS or GCP or Azure have created trillions of dollars in value of market cap of infrastructure but guess what there's also trillions of dollars of value in software that only exist because of that infrastructure and if you were to go back 10 years ago and you were to look at what was happening in the data space and you looked at what GCP early versions of GCP was building or AWS is building I guarantee you would not have predicted Snowflake or Databricks existing you would have been like the infrastructure just already does that why would you pay another $10 billion of revenue to all these other products that are just making it so you can work with your data. The same thing is going to be true for intelligence which is the models will be insanely valuable but the application of bringing those models into real workflows in banking and life sciences and healthcare and government that's just going to be a lot of software. Now the sort of challenge for the next let's say two to five years is how much do the model providers need to move up that stack and also try and go and compete at that layer or do they either leave open intentionally or accidentally that entire space for the application sort of ecosystem and actually to some extent this is a big question strategically for them because on one hand you want to be closer to the customer on the other hand you also want to be able to have an ecosystem so people trust you so that's going to be a really interesting tension over the coming years.

Yes. You know, I don't want to speak for Sequoia, but you can sense some of the existential dread of a VC right now is like, should I just put another billion dollars into Anthropic or, you know, should I attempt to sort of see what at the applied layer is going to play out. So, nobody's, you know, that envious of your guys position having to figure that out. But obviously, it's even harder for the entrepreneur. But you know I am pretty long obviously the application layer like I'm also equally very biased like I have a very concentrated bet with very limited diversification on it working out that you still want to buy technology that understands your workflow and can get to the core enterprise data. But I don't see a different event happening than all of history. I mean, you know, we only have like 50 or 60 years of computer software history. But basically, when you go to that law firm and you go to that pharma company and you go to that bank, they need something that bridges the core technology to their workflow in their business process. And AI has not meaningfully changed the need or the shape of what that looks like. And the best sort of manifestation of this is sort of this chatbot versus kind of agentic workflow demarcation. So the chatbot can be totally universal and can be totally horizontal, but you know, all of a sudden you kind of look at that and you say, well, my workflow kind of needs something to kind of ping me at the right time in the process or it needs access to a certain kind of data that just the chatbot can't natively get access to. So somebody has to go into that organization like get it set up and somebody has to go and provide domain expertise to this model so it really understands our particular business process. And so then unless you really just underwrite the big labs at honestly like I'm not exaggerating, like 100,000 employees, if you don't underwrite that, then the diffusion kind of economy is going to be massive because every single one of those companies, whether that's a 50 person firm or certainly a multi-hundred-thousand person firm, is going to need an army of people to go in and help them with that transformation, that change management. And so this is, you know, like whether it's the FTE phenomenon or just again understanding that domain expertise that's going to be a very big deal. And then you kind of alluded to this point Fox kind of guarding the hen house and that might actually be singularly the biggest sort of reason this has to happen which is even under all sort of complete benevolence and nobody's actually doing something in any kind of sneaky way it just stands to reason that if I'm going to sort of give a task to an agentic system I just want that task to be cost optimized like with accuracy as holding constant. And so who can do that? It's the company that doesn't care among 10 different models which model is performing that task. So like definitionally you would want that to be the company that does not have a sort of a preference.

I don't know how long that lasts though because when these companies become public, I think they will be held to basically the same laws of capitalism that everybody else is. So the subsidization is working very well up to a certain threshold of spend and we might have exceeded that spend when you're at like tens of billions of dollars of capital and in fact if anything it might even be worse because eventually you have to go in and sort of pay for your training runs as well. So the subsidization of tokens is I think it just has to be a temporary phenomenon.

Entirely gross margin.

But then what are they subsidizing the like then it's actually then they're charging actually like decent rates.

So that part totally fair. You know there's an interesting dimension which is well like if API is the high margin thing that is sort of paying for the subsidization but you've moved all your customers over to the applied product and there's no API revenue. So like there is an equilibrium you have to strike with this.

Yep. Totally. Let's transition talking about Box. I'm going to come back to talking about tokens and Jevons paradox and China and all this stuff, but let's talk about Box for a second.

Give folks I mean I'm guessing a lot of people that listen to this podcast use Box, but give people like a brief kind of explanation of the history of Box and how you're reinventing yourself with AI.

We started the company as a way to be able to kind of securely store and share data in the cloud. And it was a very simple idea, but we kind of just cracked a nut or struck a nerve. And that you know for Doug out there we were able to sort of scale up quickly. We pivoted rapidly in the enterprise and the idea was enterprises would be moving from on-premises systems to the cloud and they would need a better way to be able to sort of store, share, collaborate, manage all of this unstructured data, their corporate documents, their financial documents, their marketing assets, their research materials in the cloud securely. So that was the kind of company and we had been sort of flirting with AI kind of products and sort of experiences really since like 2015. If you remember like the first kind of rise of the at least in modern times the AI winter that happened in like the 2015 to 2018 period which is like we think it's going to happen now and then it didn't. And that was a period where we were like okay these very early AI models were showing us signs that okay if you looked at an image and you could classify the image well that's pretty useful if you're in an enterprise because now maybe you take all of your image data and sort of label it or maybe you'd OCR something and you'd be able to kind of pull out the text in there. That's enormously helpful. The problem was is insanely expensive and you had to have a model for every single use case required a hyper trained model for each workflow that you wanted to do. So, we kind of shelved it. Few years later, started paying attention to the GPTs. We had some hackathons where people were like, oh, we could do like type ahead in one of our note-taking products. And that was sort of early versions. We did some early work in text detection and classification which helped with security use cases. Then obviously ChatGPT moment sort of hits and that was the big sort of head exploding moment. If for no other reason then they kind of figured out a form factor that opened up everybody's mind to oh these could be these interactive systems that you just ask a question get an answer back of sort of increasing complexity and length.

So we looked at that very quickly. We jumped all in. We sort of did the whole company pivot like everything was kind of exactly like academically what you should do. We had a team carved out. We put the best people on the team. We met every day looked at the updates and then slowly but surely sort of built out what today is kind of our AI stack and then basically the Box agent. And for us you can imagine the use case is very straightforward. We sit on hundreds of billions of files. Every single one of those files contains critical information for an enterprise. That could be their contracts, their research files, their marketing assets, their loan documents, like all of this critical information. The problem is they rarely know what's actually inside of it. So unless you literally look at the document and kind of search and find it, you just don't know what's inside of it. So now agents can go and basically be farmed out to go and answer questions about that data. They can pre-process it and extract metadata from those documents and turn it into structured data. You can use agents to automate steps and workflows. So, we've built a platform that basically lets you deploy agents against all of that unstructured data. And that's been the core focus.

We are. There's a couple modalities where that shows up. One is we have again an online collaborative product that an agent can just generate any amount of content in it. And then we've done most of the I think probably more exciting work with OpenAI and Anthropic on just how do you do like advanced document creation, PowerPoint creation. We've decided that their tech is at this point can always be frontier. So we have an agent that goes and interacts with those systems to produce a high quality PowerPoint etc.

Yeah. So the probably the easiest hero for again more of a traditional enterprise to think about is just as simple as you have a million contracts. Why don't you find out what's inside them? Or you have a million research documents, be able to go and pull out all of the critical structured data, put that into a database, and then be able to query, analyze, automate workflows around that. So that's kind of the thing that just knocks it out of the park every single time because it's been a long-standing problem that people have never been able to go and sort of apply human labor to because it's just too expensive to read every contract, every research document. You know, maybe you could do it if you had like a loan document process, but most other data just never gets read at that scale. And then I think the stuff that we're probably as much if not more excited by is really the equivalent of what we see with let's say coding agents or other complex agents which is you have long-running agents that are just executing your entire kind of workflow or process. And this would be in the form of you go to a bank and you're onboarding at a bank and they've basically automated every step that is possible to automate and then sort of jumps out to a person in the steps in the process for extra review or extra verification. But now instead of that sort of one or two week back and forth, it just happens in like an hour. Like that's the dream state of most of these enterprise workflows is what if we could onboard a client faster? What if we could discover kind of critical data inside of our research much more quickly? What if we can alert to a security event much more quickly? So to do that, you need these sort of background agents or workflows that are pre-established for those processes.

It is. It is.

I'm going to sort of separate the Box corporate hat and just now kind of maybe riff as a consumer of gosh I wish there was a better term but work slop as you know inside of an enterprise context.

So here's the difference I think on the acceptability. So there's probably more symbolism to this topic than just the slop element, but actually diffusion of AI in general sort of almost ties to this code like other than the top engineers that we hang out with that have deep taste in the code and the judgment is incredible and it is as much an art as a science. So take that group aside. For most of the world, code is a utility. It is just trying to accomplish something. You're just trying to automate something. You're just trying to put a sort of interface up there that somebody presses a button and moves to the next step. So for most of the world, the value creation of code has been to automate things and to be able to have it as a utility. So at the end of the day, like we'll probably still use the term slop for a while because there's taste in front-end design and there's taste in systems and you don't want to have vulnerabilities in your code. So, that's going to exist for a while. But, at the end of the day, if you can tell an agent like, please go and generate my entire backend system or my front-end system, like it's not only acceptable, it's preferable because it's just like that was the thing that was blocking us from moving forward. So, we need to go do that.

At least the way society functions and the way the world works and our brains work at the moment. Maybe this changes. You know, when you get a presentation from somebody, there's still this association which is like I'm trying to decide if I can trust that person to go and execute on that thing or deliver that result or understand that topic. And so if when you see work slop you're like I'm losing my ability to sort of know for a fact that like how much of the thought process was them versus how much was the AI? How much should I even care about that because I myself am doing the same thing. So like we have this weird collective issue that we have which is like I'm doing work slop for some of my brainstorms and decisions but when I get it from somebody else I'm like hm should I trust you? And I don't know I mean it just might be a thing that as a society we have to kind of keep cranking through over the next three to five years and end up at the other side. Like I hate to use these totally busted analogies but you know obviously you don't care when you see somebody's financial model you're like yeah that was generated clearly by a macro or that was not you did not personally go and compute all of that but you're showing it to me and we're talking about it. So, why can't the same exist for a strategy deck or whatnot? But I think right now we're going through this evolution of like what is the person's role? What is the content a proxy for? Is it supposed to be a proxy for how much that person knows? Is it a proxy for what we think that they can go and execute on? I think we're just in this very messy period where we have to kind of figure that out.

Yeah. The reaction to it, but I actually I didn't read it. But was it like very sloppy?

How many em dashes were in there?

Okay. And dashes there. I think they're okay. You can't do that. Yeah. But it was a nice counter example to me because normally I read something that's clearly written by AI and I just have this allergic reaction whereas with the Stan piece I didn't. And I'm not sure how much of that was just you know it's Stan therefore I trust in Stan versus

It does seem like the calculator is the closest analogy though.

Like some of these analogies are breaking down of like it's just a task because it's like well at some point like this thing is doing at least 10 tasks at once.

Great transition to harnesses.

This is back to the kind of Neolab applied layer. There's a bunch of things that we know about our file system, permission structures, our search engine that you know certainly and by all means we actually would love the labs to train on our understanding of this because it would only make external agents as good as possible. Like we always talk to labs like hey we'll give you as much data as you want about how the system works but let's just say that aside we have a lot of depth of understanding of what do people do in Box how do they search Box how do they decide when they look through 10 files which is the one to go pick what is their internal kind of calculus or heuristic on figuring out the most relevant document to look at. So we know all of that and we basically have built an agentic harness that attempts to kind of understand that set of domain understanding about our system. It obviously has access to our search system our file system. It has a bunch of mechanisms for just pulling out just the text of a document just pulling out chunks from the document doing embeddings on the document on the fly. So there's a set of tools that it can use. And effectively it's a harness for asking questions of a large data set. So in my Box account I have I don't even know the latest number but on the order of tens of millions of files. Just because it's like everything that has ever accumulated over 20 years. But I can now ask any question of all that data set using the Box agent and it goes around it does multiple searches in one. It reranks it. It then very quickly pulls out the most relevant information. And then it'll in some cases read the full document, does all the steps and then we compare that against like, well, what if we just gave Claude our API or gave OpenAI our API and we see meaningfully better results on accuracy and latency because again, we kind of know exactly how to tune it for our workflows. So that's effectively the harness that we built out.

So I have a couple like just funny personal ones that I just keep track of my own use cases. But we have I don't know hundreds of different tests that we do on every single model. We actually have two evals at the moment. One is we put out a thing called the complex work eval which is a set of domain specific work in life sciences financial services public sector tech etc. And it's kind of exactly what you'd think of as a document centric eval. So given these five documents and this set of problems like what would your answers be? And we test every single model against those with our agent. And then we have a hold back eval which is actually the first one is hold back also but the second one is just like then our Box instance and how Box employees use their data and then we eval every model again on that. So we're able to kind of roughly keep track of all the incremental progress like we see when things move by half a point in terms of model improvement. And then we roll out default models based on different cost and accuracy thresholds. And then we let customers also choose any model they want from effectively our model garden.

They more or less closely correlate with code with one exception which is actually in some of our use cases Gemini is disproportionately better than what you would see from coding. And it might be sort of just better tool use. Given the Gemini ecosystem and what they need to build for it solves a strong set of general knowledge work use cases as well. But I think mostly correlating to code. So, Claude 3.5 Sonnet was clearly state-of-the-art and the best model that we've seen. There's obviously rumors about other models and so we'll see how the race continues on this front but basically by and large like when you look at GPT-4o, MMLU has their Apex eval, these things will all generally follow the coding models and so I think it's we're just neck and neck on like Grok, Muse, the Claude class and GPT-5 whatever they're building next like these are just It's a total race right now.

So they by volume they use our default because it's just easy and it works extremely well and it's tuned for there's a few ways our agent manifests. So like the way that you'd most commonly experience it as an end user is you would just be searching and asking questions of your data. But by volume the volume of tokens tends to go through more of our workflow agents or data extraction. That's where actually you have customers doing eval basically saying okay I want to really make sure that at this cost profile I can get 98% accuracy on data extraction and that's a place where like we'll have an FTE that goes in and helps you understand your data environment tests against five different models and then you're just basically at the mercy of the eval.

So probably higher than people think, lower than what enterprises actually want and much much much much lower than what it'll be in five years. So like some mix of that would be the kind of message.

I have to probably attribute 30 plus percent to just kind of the sexiness of like I want to try open source.