We have all the resources in the world. The number of SaaS apps that we have replaced as a result of all this kind of army of resources and talent that we have — we're a software company through and through — is exactly zero.
Okay folks, welcome to Agents of Scale. I'm here with Dharmesh Shah, who is the co-founder and CTO of HubSpot. Dharmesh needs no introduction. Dharmesh, welcome to Agents of Scale.
Thanks for having me. Great to see you again.
Yeah. So, Dharmesh, I must admit, for the CTO of a fairly large public company, I end up talking to you more often about culture. Somehow you got deemed the culture guy at HubSpot. And you know, one of the things we end up talking about a lot on the show is how different cultures are navigating AI right now. And you have a particularly interesting phrase I've heard you say, which is, the job isn't to compete with AI, it's to compete with AI. Can you talk about what you mean by that and like...
Yeah. So, you know, I had this experiment I ran with an online poll and the question was kind of posed randomly, it's like, how do you compete with AI? And as it turns out, that's an ambiguous question and it could be interpreted in one of two ways. One is how do you compete against AI is one possible interpretation and the other one is how do you compete using AI? So that kind of width carries those kind of dual meanings and I would posit that those that are thinking of competing against AI, that's not the optimal way to think about it. I think the right way to think about it is how do I, as a person, compete better, grow better using AI. I think that's a more productive, just generally a better approach. I'm a techno-optimist and I'm optimistic about AI, but I generally believe that, and lots of technologies have come before, and this one obviously is very different and much more transformative, but I think it just elevates what humans are capable of and that's one of the reasons I'm excited for it.
So yeah, I tend to think that that trait is fairly common amongst founders who are optimistic about the future and what's possible here. Can you share how have you all built a culture inside of HubSpot that takes that approach versus maybe takes a less optimistic approach? You know, I think when you get to a certain scale, you have an organization that's filled of all types of people and maybe not everyone comes in with that default positive framing of that with AI.
No. And that's true. It's been a journey in terms of getting people comfortable with the idea of AI. We were kind of in the game early. And I think one thing that helps just on the kind of cultural side of it is you sort of have to tie it back to the things that we believe at HubSpot in terms of culture and one of those things is around that we are here to create lots of customer value. That's the kind of the overall mission is to help millions grow better. But if you kind of double click on that, the only real way for us to deliver on that is for us to continually get better, is to continually grow, to continually learn. And it's easy to say those words in any organization, right? It's like, okay, fine. But you sort of have to make the investment. And so what we've done at HubSpot is to say, okay, we think AI is going to be a really big deal both for us internally and for kind of imbuing into our products and our platform. But we need to kind of carve out the mental space. We need to make sure it's okay for people to experiment. It's like, oh, I went off down this rabbit hole. I tried out X on AI or something like that with AI and it didn't quite work. All of those things are not just tolerated but encouraged. It's like, okay, if we're not trying enough things, playing with the new tools and kind of getting excited about them, I don't think we're doing our jobs. And so then we also kind of carve out time. So we had a week earlier this year where we spent, and you know, we had outside speakers, I spoke, and just to kind of talk about what's going on, how I'm using it, you know, and just kind of encourage folks to take some time away from their day-to-day and actually invest in learning and playing and experimenting and trying things. Yeah.
So are there rituals or practices that you all have instituted that have helped with that experimentation, that play time?
Not yet. And I regret not doing something sooner. It's just been a busier year than usual this year. But it's a good idea. I'm a big believer in kind of taking good practices and turning them into habits and making them rituals and making them something that's kind of part of the fabric of the company. We're not quite there yet. I think you folks have done a better job of doing that, just kind of baking in some of the rituals.
Yeah. Well, I think for us, we were sort of able to just fall back to the founding moments where Zapier started at a hackathon. We have a long history of running hackathons internally and so AI showed up and we're like, well, we'll do what we always do, another hackathon. And I think that has been a huge accelerator of adoption at least for us internally, especially in the non-technical orgs where we open these hackathons up to your marketing teams, your sales teams, your finance teams and whatnot and give them access to tools that maybe are a little friendlier to them than maybe these codegen tools are. And as a result, they're able to start to imagine, hey, here's how this process might change for me or this day-to-day task might look a little bit different. And that has been a huge vector for us. I'm curious for you, if it's not a ritual, what have been the practices that have worked the best inside of HubSpot at encouraging this? Has it been these like your own demo, show-and-tell, these internal talks or are there other practices or other cultural elements?
Yeah, that works really well in terms of the kind of internal demos and we've had hackathons and things like that to do internal projects. But what we found sort of the most, you know, we are at heart like problem solvers both internally and then externally for our customers. And so we'll make a list of, you know, what are the things that we think, and when I say we, I mean the collective we, that are just inefficient or we could do better with AI that like maybe works, right? And we have, just like many organizations have, these internal champions that are often a little bit ahead of the game. You know, they either started sooner or just have a more experimental mindset and they just jump in or someone comes like, you know, this is sort of painful that we have to do this, this, this and this and isn't it possible now? And so one thing I do encourage people, and I mean this literally, they don't take me literally but they should, is that let's say you try something with AI, it's like, oh, I tried this thing or whatever. I'm going to try and automate this or do that, whatever it happens to be, and it doesn't work. It's like, that's okay. What you should do is, if it was important enough to try, then put a date on your calendar for six months from now and try it again because this stuff moves so quickly that often I think a common mistake is that, oh, I did that, I tried that, I tried ChatGPT for X or Cursor for Y, and it just didn't do it. And then they said, oh, well, AI doesn't work for that. It's like, well, it doesn't work now for that. But with an exponential curve, you need to sort of kind of retry it and it might just work.
One of, I heard a story that I think is worth sharing. I don't know if you have much to say about it, but I think it represents how you approach the role inside of HubSpot. I'd heard from somebody who shared with me that one of the reasons Dharmesh is so effective inside of HubSpot is you can act as elbow grease to get the org doing more stuff. And they'd shared that they were, I forget what tool it was, but they were trying to use this or the that or the other and maybe things were getting stuck in procurement and Dharmesh was like, here's my API key, like you just go nuts and have some fun on this thing. And I think that's another great example of just someone in your stature being able to say, hey, I'm going to help encourage this innovation and just find ways to continue to cut through the orange tape in this regard.
Yeah. And part of it is just, you know, and we do this, Yamini, our CEO, has like a weekly internal communications that she does and she's a daily user of AI, right? So she's sort of leading from the front in terms of just how she uses it in her day-to-day life and encouraging people to kind of adopt it. And I think it helps when people see their leaders actually not just saying the words that seem to be popular, but actually digging in, doing real work and making it part of their daily lives. So yeah.
Yeah. I think that's, it's hard to, I mean, it's hard for anyone to want to follow someone who they themselves aren't willing to do the thing. And so like you say AI is important, you probably got to be one that's saying, hey, I'm dogfooding this stuff as well too.
Yeah. And it's one of the things I found helpful for me personally because of my excitement around it in terms of galvanizing the organization is that I'm usually kind of customer zero or user zero for a lot of these things partly through my startup activity so I get early access to them. And so by the time they'll kind of read about it and things like that, like, oh yeah, that's the thing that Dharmesh was talking about a week ago or a month ago or something like that. And yeah, that helps for just, yeah.
So another thing I want to talk about is demystifying AI. HubSpot sells predominantly to entrepreneurs, marketers and small and medium-sized businesses. So these are folks that are usually not coming at AI from a strong engineering mindset, but they are builders. They like to make things. What have you learned about helping these folks actually deploy AI inside their companies?
Yeah, I think the first thing is that there's a bar every organization has in terms of what's the highest amount of risk they're willing to take or how readily are they willing to put AI in their organization. And the thing that's worked well for us is to kind of tell folks it's okay to have that bar. You can define it, it's like, okay, I'm not going to put something that's going to be front-facing for customers until we've gotten either this CSAT or tried it internally or whatever your criteria are. That's completely okay. What I think is not okay, and we try to talk our customers off of this, is to have some arbitrary line. It's like, oh, it's just too risky. It's like, okay, well, it's risky, why? What is it and what would it have to do in order to demonstrate that it's ready? And it doesn't have to be production-oriented, even if it's an internal project, even if it's for a small team, whatever your constraints are, you sort of have to define those constraints. Otherwise, this is just human nature, the tendency is that it'll never be ready because you keep moving the bar, right? It's like, okay, well, fine, if you have something that says we need to see 500 tickets handled in an automatic way with a CSAT above X before we do a wider rollout, that's completely fine. And so alongside with that advice, and this goes to the, sometimes fear is actually a good motivator, is that if you do wait until the point where it's completely risk-mitigated — not completely, nothing's ever completely risk-mitigated — but you sort of do fall behind. Things are moving really, really fast and you don't necessarily have to keep up with every model update and things like that, but you can't really sit on the sidelines. And we saw this before with the internet, right? It wasn't, and it took years, right? But within a few years, like, okay, if you were a business and you didn't have a website, that was sort of unthinkable. And we're sort of at that stage. You should be using AI in some way and you should sort of be expanding that circle of what AI can bring to your organization. You don't have to take a leap into the deep end at time t equals zero, but you should start somewhere and then kind of increase your confidence level over time.
On that start somewhere, are there really good starting spots that you've run into with your customers? It's like, hey, you're maybe thinking too big about this, like if you actually start in these more practical places, you can start to stack wins.
Yeah. I mean the obvious ones. So at one end of the spectrum is around, and this is where generative AI kind of got started, is around creation, content creation, writing, writing a blog post, creating an image, writing an email, all those kinds of things are relatively low risk, right? So let's say you give it a chance at writing a blog post or at least the first draft of an email or whatever it happens to be, there's still a human in the loop, it's you, right? And you control whether you use the output or not, fine, right? That's an easy, low-pain kind of thing to try out. And over time as you move down the spectrum, you sort of bring in other people. It's like, oh, I'm going to try to automate this process that three people touched or whatever, but I'm going to take one of the steps and I'm going to automate it and see. It's like, oh, it's going to do some sort of lead routing algorithm or whatever, but then someone still verifies that that was the actual right thing to do. And then you get to like the, oh, I'm going to put an agent in front of customers to do frontline support kinds of things, right? And it's like all of those are kind of along a spectrum, but it's easy to sort of do the single-player mode, it's just me and the machine. Let's see what it can do. And I get to be the sole arbiter of whether it was good enough or not and nobody even needs to know. And then you sort of move it. It's like, okay, now I've gotten some level of comfort for these kinds of things. Now maybe it's not just creation. Now maybe it's synthesis, right? I'm going to be able to take a body of information and distill it down into either a different media format or I want to create this 300-word post of a 50-page PDF that's really important to my industry, whatever it happens to be. So there's these kind of levels of the game that you can move through and it's usually creation is the easiest one to get started. Then synthesis is likely the next. There's automation and the other one that's really fun, and it's just short of the customer one, but is simulation, which is very interesting. That says, okay, well, if I can describe to AI what my business is, who my customers are, all these things with some degree of fidelity, spend some time on it as if you were being interviewed by someone, but then you can pose the what-ifs, right? Similar to what we might do in Excel 20 years ago. It's like, what if I launched a product in this bucket that sort of intersected with our platform in these ways? What if we introduced a new pricing tier? What might happen? And AI is actually shockingly good at coming up with a response to that, right? And it's not going to be perfect, but it's a really good idea generator. And once you kind of pull on this thread far enough, you can say here are the types of customers and you can create effectively virtual audiences or virtual panels. It's like you had a customer advisory board of 12 people that had these profiles and then you put this idea in front of them and it's like the AI can sort of simulate that. It's like, oh yeah, one of my customers is this. It's a partner at a 50-person law firm, whatever it is, and you sort of simulate what the real world might look like.
That's fascinating. Is there a particular problem that you've used simulation for recently and how did that work?
Yeah, I use it a lot for speaking things, right? It's like, okay, well, I'm going to speak to this kind of audience and maybe I have some sort of sense for the kind of questions or topics I'm trying to address. It's like, how will this be received? Is it controversial? And I'll try to kind of, and this is I think my most common use for AI for me personally outside of agent coding, which we can talk about, is just as a sparring partner. So, not as a writer, but as an editor, as a critic to say, okay, Dharmesh, you said this right here, but it's logically inconsistent with this thing you said earlier in the thing or whatever. Or, you said this, here are the gaps in your reasoning, whatever. It doesn't quite make sense, or this was not clear enough, or this is controversial, or for this particular audience, it doesn't feel like that's going to land as well as you might think it's going to. And I like to treat it very conversationally. I don't like to kind of attribute human qualities, but it's very good at that.
Yeah. Are there particular tools you're using for simulation or is this like, hey, I'm just going back and forth with ChatGPT asking it to...
It's a combination of GPT, but I have my own LLM because I like to do a lot of these things outside of the classic bot and ChatGPT interfaces and I just use the API, but it's those core models.
Yeah, we've seen, in fact I've started to experiment with this too where we'll build out GPTs for different people internally that sort of have their personality tests, tendencies, principles that they like to obey by. And then to your point, you can create like, oh, here's the council internally and you can ask each how they might respond to a different situation.
Yeah. What gets interesting with these things, and this is going off track a little bit but I think it's fun and interesting, is that the more you kind of give these models in terms of the grounding information they have access to, and we've seen this with all forms of retrieval augmented generation. One of the things I found very useful, so I have lots and lots of emails over 30-odd years of being a professional and I have all of them and I've done multiple passes at creating vector embeddings for all of them. And so I can then say, oh, well, based on my conversations with Brian, my co-founder, how did we arrive on this particular thing or whatever, what were the sides that we considered? And it's really, really good at kind of taking that prior, because you sort of forget and it's got the historical record because most of the conversations that have happened internally, especially in the early days, were all in the form of email. Even if Brian and I were sitting in the same room, we'd still send each other emails. That's just how we operated. So yeah.
That's fascinating. Have you tried asking it things like, what have we changed our mind on over the last 10 years or questions like that?
I haven't asked that. That's actually a good one. What things I have asked is around if I were to try to summarize a decision, if I'm trying to onboard new people or we're kind of revisiting a decision or something like that. One thing I'm a big believer in is that I don't like second-guessing decisions. But after a certain amount of time has elapsed, I think some decisions are actually worth revisiting because the context may have changed dramatically since the last time we made it and we don't remember, sometimes the people weren't there for that initial debate. To go back through that history, just educate ourselves and educate new people, it's like, here's how we kind of came to that, and be able to write a meaningful kind of article that captures it.
Yeah, this is one of the use cases I'm most excited about for call recording. So I've fallen in love with this tool, Granola. Not just because it takes nice notes, that part is great, but the context of being able to search and ask questions over a large corpus of conversations is incredible. We've done internally a lot of work trying to write and describe our culture. And I tend to think we generally do a good job at this. And so one of the things I tried with Granola is I said, hey, can you tell me what the unspoken culture of Zapier is based on all of the meetings we're in? And it writes up, here's what I think it is. And as I read through that, I was shocked. I was like, wow, this is a better answer than all these thoughtful things that I had thought I'd written over the years. And so it's just fascinating to see how these tools know us better than we know ourselves in a lot of ways.
Yeah. One, I'll call it a pro tip. I'm not sure if it qualifies to be a pro tip, but so if you think back, Wade, back when you were originally building Zapier, chances are since it's a web app, you were sort of logging events. It's like, oh, the user did this or this or whatever, and you haven't built the dashboards or whatever. You may or may not have been using an analytical tool for just kind of doing the classic product analytics type stuff, but you just like, I'm just going to throw this and you would just dump the data because the reality is the cost of dumping the data was super simple. It was a single line of code that you could just write to whatever data store you're writing to. But then you had that data that you could then later say, oh, well, now I've got this and I can do something with it, right? And I have had very, and I talk about the email example. Another good example is screenshots. If folks are not already using something that will actually do a permanent storage, like you do the clipboard thing, whatever, I'm going to take a screenshot of this and paste it into an email, whatever. You should be using a tool, there's hundreds of them out there, that will actually save it to a local folder. Not that you're going to go back, but there will be a day that future self will thank you for having that kind of historical record of all the screenshots that you took because you'll be able to do video analysis. You'll be able to find, it's like, I know I looked at this particular version of this app or whatever, internally, externally, doesn't matter. And you'll have questions that you could not answer before that you will be able to answer someday.
Yeah. I had heard an interview with Tobi from Shopify where he had shared he'd been logging his day at a deep level. He was like logging in. And I was just like, I mean, it was one of those things where I had FOMO all of a sudden where I was like, oh my, that was not even a thing that had dawned on me that I should maybe consider doing, but in this day and age, how valuable to have that data set. It's incredible.
So maybe shifting gears again, kind of back to the more human side of this. Walk me through some of the techniques you've used building AI to make it feel more approachable to this non-technical audience that HubSpot has built up.
Yeah. A couple of things is we have to sort of internalize. So yes, AI is very, very new and it is transformative and there's all these things and you can sort of geek out about it as we do, but looking at it from an SMB's perspective, mere mortals that are out there kind of living their lives and growing their businesses, you sort of have to accept and kind of say this is software, that's what it is. It happens to be smarter software. It can do more, but it's not magical in that way, right? Yes, it's kind of lifted it. And one of the things I think a mistake the industry is making right now, and we're getting better about it, is that because of ChatGPT and its popularity and because we all had that magical moment the first time we typed in an English prompt and got a meaningful cohesive response back, it was life-changing, right? We all had that light bulb moment. But just because that magical moment happened doesn't necessarily mean that's the best possible way for a human to interact with software, right? So one of the things we're doing right now is like, yes, there are certain use cases where a conversational UI makes perfectly good sense. There are other use cases where you can use AI that the user doesn't even have to know that there's AI being used. It's like, oh, well, this thing works pretty well. I don't know how the heck it does it and they're going to assume AI. But I think over time, like any good technology, it should sort of fade into the background, right? So now we have the internet, it's like, oh, all the computers are connected, we have REST APIs, we have all this stuff, right? We don't really talk about that anymore because it's sort of built in the fabric of the way machines work and software works and we work. I think AI, we should sort of be nudging humans along that in a gradual way and we should not say, oh, everything's different now, everything you ever thought you knew about software is gone. It's like, no, actually a lot of that still makes sense. You still have the same problems that you used to have. We have better ways of solving some of them and sort of help people bridge a path to the current.
I totally cosign with that. I feel like a lot of the products I'm using right now, there still is a fair amount of configuration involved and that configuration feels like an unnecessary thing in this day and age. And an agent, and every time I have that conversation, I'm like, they don't care. Why am I having to have this conversation? It should just fade totally into the background and you're just saying, hey, I want to automate this. I want to build this. I want to do this thing. And it knows how to go figure these things out.
Yeah. It helps a lot actually, and we can all do it when we do it. It's like if you use your own products, even if you are an expert in AI, there are some things that are just cumbersome, using a particular modality or something like that. It's like, okay, well, yeah, I know AI could do this in a textual kind of way, but if I had this UI, that would be a much more efficient, better way to do this thing. Yeah.
Are there sleeper use cases right now that you feel like are just not more broadly talked about?
Yes. I don't think we're yet looking at what I think of as dynamic or fluid UIs yet, right? So I think that's the bridge between the two. I think the thing we like about conversational UI is that there's this nice easy thing where I'm not converting the thoughts in my head of what I'm trying to do. I can just sort of tell the computer in English or whatever my natural language happens to be and the computer goes and does it. The other end of the spectrum is this very static UI that some designer went through and we did a bunch of customer testing, whatever, and we have this. There is an in-between that says, okay, well, something that's more natural than the static UI, more personalized than the static UI, and maybe it's some sort of a hybrid dynamic interface, right? And that's the thing that I'm starting to kind of play with now. So I'll give you a concrete example because it's a fun one. So one of the things I'm working on right now personally is this AI agent for image generation for businesses, right? So right now we know image generation. There's a bunch of really good image generation models and most of them are kind of text-to-image, right? You type in whatever image you're looking for and it comes out with an image. It's like, okay, well, that's useful. But for my personal use cases, whether I'm doing it for a blog post or need a visual for a slide deck or something like that, it's like, okay, well, here's my actual use case. My actual use case is I have a particular style that I'd like to use for my social media posts or whatever and I have already five, 10, 50 images that are sort of along those lines. What I want to be able to do, just like if I actually had access to a designer, which I've had in the past, I would just say I'm going to give it these five images. Okay, great. They would come back with, I give a rough idea of what I'm looking for. It'll come back with something, right? It's like, okay, well, it generated an infographic of this or whatever. It's got some text and the models are really good now. And so here's the leap I had last night from a UI perspective. It's like, okay, so it did the thing with a single prompt. It came up with an infographic that describes a timeline of HubSpot's evolution or something like that if I'm talking about our history. And then to be able to do a revision on that to say, okay, well, I can say, okay, well, tweak this and tweak that. And we've seen that now with Nano Banana and others. The leap now is, okay, well, we can do an interpretation of the image that it generated because I didn't give it the text. So it's like, oh, here are the milestones it put. Change the text for what happened in 2006 from calling it an IPO to whatever, right? And what it does though, it presents it in a, here's all the elements of text that were autogenerated. Just overwrite the ones that you don't like. Not have to describe it and say change this text to that text. Actually give me a visual, like a modal dialogue that lets me say, okay, well, here are the texts that I detected in this image that I...
Generated, and let me override and kind of tweak it. That's interesting.
Yeah. So you could, you could put kind of more of like a classic editor in front of you where you can just come in and be like, I'm going to physically change the text on this like I would in any other tool.