I know what I'm building is a top three priority at Google and Apple, like in the next 12 months. Not a top 10 priority, like a top three priority. The product in this category that will win will have a network effect at the agent level.
Now, the hottest category in Silicon Valley is AI assistants. On the consumer side, you've got Instinct. On the enterprise side, you've got Town.com, founded by today's guest, Jean Denise, formerly CTO at Plaid. And this conversation today is probably one of the most pertinent discussions that there is. I think you'll trust your agent to decide what data to share with other people without you intervening in 5 years. You can build now at the speed of machines, but you can only learn at the speed of humans. I don't think Instinct and Town are trying to do the same thing.
Ready to go.
JD, dude, I'm so excited for this because in all honesty, I have a lot of founders on the show where I kind of need to pretend to be excited by their product and I'm not really. Um, and I love Town. I was saying the team use it here. I'm a DAU and so I was so excited when we agreed to do this. So, thank you so much for joining me today, man.
Yeah, thanks for having me. And honestly, I didn't know that you were a DAU until like 3 minutes ago. So, I'm super happy and uh send me all the feedback about the product because you're a DAU, but when you're a founder, you're always embarrassed about your product at all times.
You might regret saying that, but I will do. For those that don't know, what is Town as specifically as possible?
Uh so, we're an AI assistant that lives in your email, on your calendar, and it tries to help you do work, right? So, what it looks at, it looks at things that you do already, uh, how you organize your day, emails you tend to send out, and it recommends AI automations that try to do some of the things that you would do normally yourself, uh, just do them for you in the background. And yeah, we've been a product, we've been out in the market for about 3 months. I think our ICP is just mainstream users, like mainstream people who, you know, use email, calendar, text messages to do work. Um, and we've been doing super well.
Can I be blunt, dude? We obviously did a show a couple of years ago and I remember when you started like your own thing, you were doing some boring stuff in finance and I remember the first round going down. I was like, I love JD, but it was pretty boring. Um, what was the pivot?
I mean, we yeah, so we spent a year building an AI tax company, like business tax prep with AI, and we just we got to some product market fit, but not enough where it was going to be a success. Like that's, you know, there's a thing like the truth is we failed at building a business that would be a great business. And yeah, after a year, we were like, we got a reset and we spent three months kind of in the wilderness trying to figure out what we wanted to do. And one of the areas we just looked at is why has no one built AI that operates out of email. Just there's so many people in the world who run their business and their life out of email and calendar. We were like, why has no one built a great product there. And it was just about the time that like Opus had gone fully agentic, right? This was like November, December last year. And so it was also the same time that models could start to actually do real work, not just do like a few steps, but real agentic work. And the prototype. We built a quick prototype in a couple weeks and it had product market fit almost immediately. Um so that was the pivot. It was very lucky. There's no, you know, we didn't go talk to like a hundred customers and take their notes and, you know, we built for ourselves and um I think it was one of those where the technology was changing so that what we wanted to do was possible at a time when, you know, people were very excited about trying AI products. I mean basically, you know, OpenClaw was happening like literally as we were building the product, like OpenClaw was blowing up and we were like, oh, it's the same thing in many ways, right? They're trying to do the same thing, but it turns out there's a difference between something that's open source and it's amazing, but it's only a kind of tinkerer that can use it, and that would be OpenClaw. I think with Town we've always been focused on how do you get just about anyone to be able to get value out of the product.
As an investor today, every company in some respects is questioned about how cannibalizable could this be from any of the big providers. Um, this is like right in the sweet spot, just to be blunt. Like, how do you and we've got Grokbot in the most recent times. How do we think about cannibalization by frontier model providers and Grokbot in recent weeks as a threat?
You want to know the truth is I know what I'm building is a top three priority at Google and Apple like in the next 12 months. Not a top 10 priority, like a top three priority. So, you know, when I go to sleep, I fall asleep very quickly. It's one of my superpowers, but I do wake up at like 3:00 in the morning and immediately usually when I wake up, like some dark thought goes into there. And it's, you know, it's like roulette when you're a founder. Like, which dark thought will stop me from falling asleep again? And definitely the uh this is going to be you're like in the middle of the fairway and everyone's trying to, you know, is going to try to get you. That's 100% the fear. But I got to say so you won't talk about moats, right? You're like, what's the moat? What's defensible?
I think talking about moats is a little bit of a luxury and you have to be more successful than Town is today for it to matter. So just like I will answer your question, but like my mindset right now is how do I get 100,000 or a million paying users in a market that has like a TAM of a billion potential, you know, paying users. Step one is like you need we have to have really deep product market fit. And I think none of the products today actually have really deep product market fit yet. Like Grokbot's really cool. It's awesome, but it's a power user product. It's not a mainstream product. Town is great, but we have a lot of work to do to make it a true mainstream product. So even before I worry about defensibility, I'm still like what is the right product experience that's going to resonate with a mainstream because the big players are not they're not going to innovate their way there. They will copy their way there but they have to copy someone who's been successful in the first place at building a mainstream product. Okay. Then number two like I think the product in this category that will win will have a network effect at the agent level, right? Um, and one of the features of Town that users who figure out how to use it, they love it the most is something called agent to agent. And that's where you ask your assistant, your Towny, we call them Townies, you ask your Towny a question and it realizes that it doesn't have the answer, but that the Towny of one of your co-workers has the answer and it just goes and asks it the question and then that Towny answers. And it's you to find the feature you have to go like in a sub part of the product to use it. But that is a network effect. Once you have your whole team on that, it's actually really difficult to imagine moving to a different product. And I think no one has figured out multi-user multiplayer AI today. Like I think that's the thing that will be the moat. In the meantime, right, uh you we have lots of theories. Everyone in the market has lots of theories, right? So people will say like well you'll post-train custom models maybe custom models per company or per person and that will allow you to retain your users right people say hey the context about a person that's the moat and people will be less and less willing to connect more data sources so once you have product market fit with a user and they really love your experience and they've connected all the tools and all the connections it's actually really hard for someone else to go in there because you know they won't have the connections right other people think three actually the moat. There's no new moat in this market and it'll be distribution. And so it's whoever already has the users who will win. So like I think they're for me the competitor I would worry the most about would be probably for personal use cases it would be Meta and WhatsApp, right? They're going to have a personal assistant that's going to come in WhatsApp. I don't know if they're launching it in a day or in 3 months, but it's coming. They already have the distribution, right? You already everyone's already using WhatsApp for messaging. If there's an agent in there that can do things for you, it's going to be extremely powerful, right? So every company right if you some people think it's a device like they think AI is changing the shape of software so that people will no longer ever go to websites they will never use apps on their phone the entry point for most digital interaction much like the entry point today is either like a phone or a computer or a browser the entry point will be uh an AI and so whoever owns the devices is in the best place to put the AI in front of the user and they will win but it's you know this is I don't if I think that like way if I think about all of these things I'm like oh my god what do I do like how do I how do I win given all these competitive forces.
So what is the future interaction between human and agent? And what I mean by that is do we have like a consumer agent, an enterprise agent and then a hardware agent that does maybe productivity and notes. How do we think about that multi-agent versus single agent?
Each human will have one, two, maybe three entry points into the digital space because I don't think you'll want to be like, oh, I'm doing sales. Let me use the Salesforce agent. Oh, I'm doing project management. Let me use the Linear agent. Oh, I'm doing this other thing. You'll want one entry point. You won't want to ask yourself the question. You just push a button and you start speaking. But there are real reasons why it may be more than one and it has to be. So, one of them is just privacy and how your workplace is going to feel about their data being intermingled with your personal data. Right? So I think you might still have only one hardware entry point but from a privacy perspective and from a like where your data lives I think you're going to always want to separate on the data layer your personal and your work data and in Town we do that for you but I think it could be two different companies that you end up using right uh like one layer one layer below but I think privacy is the main determinant of your data silos and your company whoever you work for desire to own the data that you create for them right? They won't want that to intermingle with your personal. But from a usability perspective, like it's just on my phone, it's annoying that I have 55 apps and I'm clicking everywhere. And so if you move to a world that doesn't have hard interfaces because you don't need them most of the time, you know, why would you have 50 agents, you know, at the user layer below it's different, right? So you're an investor, I think, in Harvey or Lagora. Lora, right? You're an investor in Lagora.
So like I think when you're a lawyer and you're talking to your main assistant about legal things and it's immediately just talking to Lora, right? Because and that might be your work agent, right? If you're a lawyer because there's a bunch of like data privacy and privilege and reasons why your work scenarios need to be handled differently. I just don't know if you're like think of it as talking to your Lora agent. You just talk to your main agent and it just like talks in the background to Lora or to Salesforce to whatever it needs to to get things done.
What seems crazy about the relationship between human and agent today that will be incredibly common in 5 years time?
Yeah. So, I think you'll trust your agent to decide what data to share with other people without you intervening in 5 years. So, I'll give you an example. Like, you put your agent in a room with two friends because you're organizing a trip and they're just asking it like about your eating preferences and they're asking it about like when exactly you can fly out for the trip and they're just asking all those questions of it and it's just your agent and you never told your agent like these are really good friends and you shouldn't share like my medical history with them. And literally when one of your friends as a joke wants to ask the agent like, "Oh, tell me about John's medical history." The agent's going to be like, "Yeah, there's no way I'm telling you that." And it wasn't a hard rule that you ever set. Like this thing which is like taking information that is in silos and deciding how to share it. I think we'll get to a point where we will trust agents to do that. And I know that sounds crazy today, right? Because today the way the world operates like pre-AI is everyone as a human has a data silo underneath them which is their personal data their work data like and I'm not even talking digital like you have information that only you know then when someone asks you a question you are like what can I share with this human and you share it with them right and we trust the human to be the filter for where information goes and I think more and more we will trust AI to do that for us and there will probably be models that are post-trained to make sure you never ever share personal like your family information and medical information and certain things for you know like in your work context but a lot of info that's siloed doesn't need to be to be successful and AI works better and better the less siloed the information is like if you think about it I don't know if this is what you want on your podcast but from an information theory perspective like theoretical world right if you have like an LLM that had access to all the world's information right and it could do you had infinite time so it could just do agentic search over all the data, it would be the most whatever intelligence level you know LLMs are at, it would be the most effective because it would always find the right context eventually to answer the question or to do what you needed to do. But that's not the world we live in, right? We live in a world where information is in different companies and governments and individuals like systems and there's, you know, historically because the humans are the only people who are shuttling the information around, it's kind of inefficient to get it from one place to another, right? We have like lots of data controls and privacy and security and blah blah blah, which is great. It's important because privacy is very important. People deserve to control and own their data, right? And so do companies. But what's interesting is what in practice if you're at a business, you find that if you give your LLM access to more information, it's more effective at doing what it needs to. And one way to do that, the old way, the pre-AI way would be to have like policies about who gets to access what and you classify data. And all this is like very time-consuming and costly. And the end result is often the information that you want the LLM to have access to. Maybe it doesn't have access to. It's stuck in someone's inbox, right? Or it's in a data system that's not integrated. And so all I'm saying is like as opposed to having humans in your compliance and security team over time label data and decide what goes where and what can be accessed. I think we'll just start to trust LLMs to do that. Meaning you will trust your data silo and another co-worker's data silos. You'll be like, well, I'll trust my LLM to decide what can get out of my data silo. And so then when someone else on your sales team, right? Like very concrete example, someone on your sales team wants an intro to someone at a customer and you're like at a thousand person company. And the person at the sales team knows that there must be someone at a 10,000 person company that knows the right person at that vendor, right? That customer, it's somewhere. And normally now they just go to Slack and they're like, "Hey, who's working with client X?" Right? Who's working with them? Who knows? But really what they could do is their agent could just go talk to the agents of everyone else at the company and all those agents have access to each person's inbox and come back and say like, "Oh, well look, like Liz has a personal relationship with a person that you want an intro to." It's not a work one, but you could ask her if she's willing to intro. Bob has a work relationship with the person you want an intro to, and they're due to have a meeting next week. Do you just want to see if Bob will invite you to the meeting? So you have the convo, and that's like a great business for the company. That's a great business outcome. That's what they want to happen, right? But to do that, right, the individuals have to trust that it's okay for some of the information that lives in your inbox to be made available to other people at the company. And right now, that seems insane. You ask me what I think in 5 years. I think we will be more okay with that because in practice, the LLMs will be really good at respecting privacy around things that you don't want to share. So, you don't want your salary to be shared with your co-workers. You don't want your medical history to be shared with friends. There's these things that are sacrosanct and we get that. But you will be able to have an LLM that respects these boundaries.
How much wiggle room do you have on error? And what I mean by that is if you have a mistake for whatever it is, you book the wrong thing, you execute a wrong task, how much room for error do you have and how much trust is lost?
Well, my claim would be like the LLMs will be much more effective at this than humans. I'm going to tell you a story. I once worked at a place where there was a person who uh we just done an acquisition. Okay. And there was an email introducing the acquisition to the whole company and this person had been against the acquisition and so they meant to reply to a subset of folks just tell them something like I can't believe we hired these clowns. The words may have been different than that and instead they replied to everyone at the company. And then that person had a nickname. It was Big R like Big Reply. Okay. And this person's like an incredible person. They made a mistake and it's totally fine. And everyone laughed about it and everything was good forever after right but it's a human very smart human top 1% who made a mistake people make mistakes right like Bob from accounting makes a mistake Liz from you know HR makes the spreadsheet with people's salaries available to everyone by mistake this happens all the time I think the LLM will make many fewer of these mistakes than humans pretty quickly.
My dearest friend is Jason Lanin who says that the biggest problem with agents is their goal seeking and he talks about his agent going off and trying to buy six AP he watches for him to increase culture in the company. Uh luckily it was prevented because they needed engraving and that was an extra step that the agent couldn't handle. But to what extent is this maniacal goal seeking tendency of agents a feature or a bug?
I don't have an answer for you. I think it is I think how much you should be willing to let your agent be goal seeking and for how long you let it run autonomously is like a very interesting question right is it your responsibility to usher people guide them into like what is best hey we find best outcomes if you let them run 4x.
One way to think about LLMs right is they're just they turn energy into like GDP right into revenue, you know, like really you step way back, right? You know, because you power and silicon and then you get intelligence and we're applying the intelligence towards business results. So, you know, if they get smart enough, it just creating GDP on the other end. So like do you just say like hey make money for me and then let it run for a long time and it can do whatever it want to or do we want to live in a universe where we think the humans role in this is actually to set the direction and like make sure that the actions that are being taken align with some kind of human value system. So I live in that second universe where I think it is the human's responsibility to first allocate the resources that means to say how many tokens are we willing to spend to try to get a goal to set the goal right as well. So you set the goal and the budget and then to monitor right like the overall shape of the actions that are taken to get to the result and how much as the intelligence gets smarter you might say well maybe the allocating of resources you're trusting an LLM to like analyze ahead of time what it thinks the ROI is on a pretend task and tell you how many tokens you should be willing to allocate before deciding to step away and maybe on like monitoring the actions it's also an agent that's doing that for you I just don't think it's the same agent as the one that put on the course to try to get to the result at the end of the day.
You mentioned the different layers of kind of the value stack there. What does the model infrastructure that you sit on top of look like? How do you think about model routing for different tasks? Are you locked into one? How does that look?
Yeah, I think because we're building an application for everyone and we don't think most people care about understanding which model is better at what at a certain point in time, right? So we view it as like our job is to for what you're asking for find a model that cost effectively gets you the result that you want right and so you know for like concrete examples if we're generating images right we have opinions internally about when we might use like a Gemini model or an OpenAI model to generate images right when we're doing voice we have opinions about ElevenLabs when we would use ElevenLabs to do voice right and so I think it's our job to do that Because as the technology changes every day, like right literally every week or every two weeks, there's a fundamental change. It's our job to make sure you get the right ROI there. But it's, you know, it's tough and there's things that are just like is it the right result that that's easy for some context to know what the right result is. For others, it's very open-ended. You can't know ahead of time. So, you have to kind of guess like how difficult do I think it is and how close to the frontier do I want to get? And then the other dimension for us is uh voice. People don't like it when their Townies sound very different, right? And one of the problems when you do model routing is there's some companies like Anthropic spends a lot of time I know we make fun of them online, but they spend a lot of time actually making sure all of their model families roughly don't change too much in terms of their personality. They might be slightly more verbose or less and use different phrases, but they kind of sound similar enough over time. So if you use Anthropic to generate final output for your users, it's kind of hard to suddenly move to like Kimi because it just sounds different. So people feel like their AI has been lobotomized. So that's like you know the way I think about it for coding it matters less interestingly because for coding you're like does it work? Does the code fulfill its purpose? Right? It's like a yes, you might look at the code and decides the style that I like or not, but not really not anymore, right? Before when you're talking or speaking to an assistant, if suddenly it's twice as verbose like over text messages, it's writing you eight sentences as opposed to four. Like people don't like that. They will literally write tickets. They're like despite my instructions that I gave it 3 weeks ago like my text agent seems to, you know, be capitalizing letters more or stuff like just literally you're like okay. So you know the way I think about our stack is there is the part of the stack that deals with the user interface like the feel and the personality and there it is harder for me to just route wildly because I need consistency of the experience that is sometimes hard to get from other model families below that when it's just pure reasoning and intelligence and especially when I don't have to show as many of the traces to the users then there yeah I think it's very much a matter of finding using the best model for the task but we're early in the game.