Retool CEO David Hsu on Why AI Models Are No Longer the Bottleneck | Technovation 1106

And the reality is actually, what is blocking a company, an enterprise, from actually getting value from AI at this point is no longer how smart the model is. The model is smarter than any one of us put together.

Great to be here.

Yeah. So today what Retool is really focused on is how do you secure AI in your company, specifically how do you secure vibe coding in your company. And the core idea here is that with LLMs and with AI today everyone in your company can go and build applications, which is great. But the challenge is that when you don't have the guardrails, when you can't control who is building what, what data they're connecting to, where they're hosting the different applications, it becomes actually more of a headache than a boon to your business. And so, is there a way to go leverage the power and creativity of AI for all your employees for them to be able to create all the ideas they want, but do so in a centralized governed way that IT or some centralized team can actually go and manage? That's basically what Retool is.

Yeah. So, there's been quite a bit. Today Retool, we have around 10,000 or so customers, and they are companies ranging from Amazon to Taco Bell to Kagra to Boeing to DoorDash to Stripe to OpenAI, etc. So many, many companies today use Retool. It was born — so my co-founder and I were actually both engineers. It was born out of an idea that we were building a lot of business software, sort of business apps if you will, internal apps as they say in Silicon Valley. And after building so many of these apps, eventually we thought there's got to be a faster way of building all these different applications because if you look at enterprise applications, they all kind of have the same building blocks. They have authentication, authorization, audit logs, but on the front end they, you know, really it's just buttons, tables, dropdowns, forms, stuff like that. And so there's a lot of repetition, if you will, across this large class of software, enterprise apps. And so we thought, could there be a more effective or efficient way of building all these different applications instead of writing code from scratch all the time. So that's kind of where the idea of Retool first came from. And we discovered pretty quickly — we started actually by trying to sell to other startups. But quickly we discovered that startups don't have that many enterprise apps which is not too surprising. And instead one of our first customers was a company called Tao Electronics. I think it's around 140,000 person company so, you know, sort of a mid-sized Fortune 500 company, and they told us they had so many apps to go and build, and instead of building all these apps from scratch if they could use a platform like Retool they would really love that. And so that's where sort of Retool came from.

And what's changed over the last few years with the advent of LLMs, today now everyone can be a builder. It used to be the case actually with Retool that actually we mostly targeted engineers because we believed engineers are the ones building all the internal applications, all the enterprise apps, and this was true up until probably this year actually. This year we've seen in our customer base, especially amongst the enterprise customers, there's been a lot of interest in, given how powerful LLMs are, can we allow these non-technical, these non-software engineers to actually go and build apps? And of course, you know, everyone wants the answer to be yes, the answer should be yes, but I think in the enterprise there's really specific needs. And it's not just, you know, let's go write some code in ChatGPT, throw it up on the public internet and, you know, the app will work. You have to worry about what data to connect to, how do you connect to the data, could you connect to the data if it's on premise for example, who can use the application, how do I hook in a single sign-on so which groups in my company can actually go use this application, how do I make sure everything is audited and secure, how do I make sure it's hosted on my own VPC so, you know, you have to be on a VPN to even go and use this application. There's so many things you have to worry about besides just the code itself. But the really incredible thing is with LLMs now the code itself is actually so easy to write. So now anyone can be a builder.

Yeah, so the really surprising thing is if you look at most industry discourse, whether in the LLM industry or broadly in the technology industry, I think it's wrong. What most people are interested in talking about is how smart are the most recent models, and we're recording this in late July. Fable has come out, GP5.6, Soul has recently come out as well, and there's a lot of discussion. And CIOs that I meet with would love to learn about, you know, how smart are these models, how powerful are they. And the reality is actually, what is blocking a company, an enterprise, from actually getting value from AI at this point is no longer how smart the model is. The model is smarter than any one of us put together. It's way smarter actually if you think about the diversity of knowledge it has. You think about the fact that it's able to give a pretty intelligent reply to practically any question. But I would say it's pretty clearly smarter than any individual commit at this point at probably 99 if not 100% of tests. And so it's great that we're pushing the frontier with Fable and with 5.6. But to be honest, if you go back to 5.5, you know, is it that much less capable? It's not. And what's really bottlenecking or what's really preventing enterprises from getting the full value from AI or LLMs today is not in fact LLM itself anymore. It's what we would call the harness around the LLM, which is — harness is how does the LLM actually affect change or do work in your organization. Just the raw LLM by itself, you can imagine it just being sort of a chat window. And as we know, you know, chat is great. Again, you know, chat can be very smart. And in fact, I think at this point, chat could get smarter and I don't think it would make any impact on our business or your business at this point. But what's really powerful is the idea of the harness, which is it allows the AI to autonomously go do things in your organization.

One analogy I would give which I think is quite interesting is if you look at the difference between something like a ChatGPT versus a Claude Code example or even, you know, Claude Chat versus Claude Code, they're powered by the same — you can go use Fable in Claude Chat, you can go use Fable in Claude Code, and the innovation behind something like a Claude Code or a Cursor or a Codex is not a better model. It's actually a better way for the model to go and do work in your codebase because you can imagine, you know, before Claude Code came out what you could do is you could just chat to ChatGPT and you say hey write me this program, it writes you this program, you copy paste that program into your terminal window, you run the program, you see there's an error, let me copy paste the error back into Claude Code, into Chat, and the Chat says okay well you're right this is an error, let me fix it, okay here's the fix, here's this one line fix, you have to copy the one line over again into the text editor. So you're kind of going back and forth, right? You have effectively — the engineer has effectively become the API for the LLM because the LLM is using the engineer to affect change in the codebase basically. And today obviously no one does that anymore because we have a harness that allows the LLM to directly go make change in the codebase. And in fact the LLM's even able to not only change the code but even run the code. And so now you can actually see engineers are substantially more productive than having to copy paste back and forth. Right? What's really remarkable about this is that this change has only happened for engineers. If you look at how non-engineers at your company are using AI today, largely speaking, it's a lot of copy and paste. And so if you think about it that way, what your non-engineers, what the majority of your company is doing when they use AI is they are effectively acting as the API for AI to go affect change in your company, which if you think about it, it seems totally ridiculous. You know, it's almost like the human has become the API. The human is not a very good API, right? It takes a lot of work. You pay them a lot. It's not very efficient. They make mistakes even sometimes. What would be far more effective is if you could just somehow glue the LLM directly into doing actual business tasks. Now the problem is that LLMs do make mistakes and so you then need a governance layer. How do you supervise the LLM? But that I think is really the bottleneck today is if we can figure out that harness for non-knowledge work as an industry. I think what you saw happen in coding LLMs when I would say that LLMs have really totally reinvented what coding looks like in the span of 12 months. I think that kind of change will come for every single other job there is. But what's really missing is that harness layer. It's no longer the intelligence.

Hm. I would say that again from an intelligence perspective, I'm actually not sure there are too many limitations at this point. I recently saw a thread, a ChatGPT thread from this mathematician Terence Tao, and he was proving a new theorem or he was validating the proof of a new theorem with ChatGPT. If you look at the conversation, I'll send it to you later on for the show notes. You can see that the LLM is really, really, really smart. Terence Tao, who's one of the preeminent mathematicians of our time, is almost treating the LLM as an equal. And so it's, you know, you look at that, you're like, wow, you know, there's not much limitation there. Instead, I would say the limitation really is: are you able to effectively monitor what the LLM is doing? Are you effectively able to give it the context that it needs? Are you sometimes able to roll things back if necessary? Are you able to work with humans effectively? So some things will I think always require human approval because LLMs cannot be held responsible for certain things. And that's I think those are the features I would say of a harness where if you think about a coding harness, let's say like a Claude Code for example, what Claude Code does is not only does it give it the tools to go and do things but it also gives it the context because it gives it the whole codebase that it can read and create an index of, for example. The fact that it's code also makes it nice because you can actually write it and then put it into staging and so it doesn't, you know, directly affect anything yet until you promote it to production. And I suspect the promotion to production will be human-gated for some time to come, I think, and the reason is again because we just say who's responsible for doing some dangerous action. And so I think those — if you sort of think about the analogy between coding and the rest of knowledge work, and you think, you know, what has made the coding harness so effective, the context engineering, the gating, the human approvals — if you're able to bring that over to knowledge work, I think those are the bottlenecks at this point rather than the model itself.