If the quality is great, no matter how bad you are to everything else, you win. Right? If the quality is bad, then no matter how good you are to everything else, you lose.
Jason, excited to have you. One of the things I saw in your book that I knew we had to come on and talk about, multipliers that kind of undercur and AI search. One of the things you talked about in the book is how affiliate marketing was a powerful channel for you and WP Engine. You helped grow it to almost I think 300, 400 million ARR which is really impressive. But you spoke to the fact that you kind of took a specific lens on the affiliate channel that helps you dominate, made other competitors ignore that. What was that specific?
Yeah, it's a great story and it's in the introduction of the book because the book is about things like this. But I'll tell you the whole story. You don't have to buy the book for the story. I'll give it to you right now all the details. So the first observation is that affiliates when you plot them in terms of how much they can send you, they it's this crazy power curve. It's like maybe it's an ultra power curve, right? It's like there's a few big ones and then there's this massive tail that doesn't matter. And in fact a lot of them in the tail, what they do is they send you one forever. Like they'll never send you another one because what's really happened is they're a freelancer or something like that and they have a client and the client already wanted you and they realize if they sign up as an affiliate they can just get paid a little bit but they're not really, you know, so that's what a lot of affiliate quote unquote affiliates are in an affiliate program. Okay. So anyway, so that's why it's this super exaggerated thing. So once we realized that which of course this took time but I'm shortcutting this right. So once we understood that just a few affiliates in our case it was something like seven or 10 but obviously that number will vary are like all of what the affiliate program can be right the rest of it doesn't matter. And so therefore it's valuable to quote unquote overspend or over or take too much time quote unquote to woo them or to have them. And then the other thing we realized is since there's so few. It's a zero-sum game. It's not like social media is not a zero-sum game. I could have a great Twitter feed and then my competitor has a great Twitter feed and that's fine. Right. Like one doesn't really take from another. You could try to argue about attention but it doesn't really, right? But an affiliate since there's so few that matter. It is a zero sum game. If I get one that's one fewer that you can get and since there's only seven, right? So what we realized is okay so if we get even half of these then it's very hard for any competitor to have meaningful growth through this particular channel. So, if we can overspend to dominate this channel, then you could say if you just look on a dollar for dollar basis, you might say, 'Well, this isn't the greatest ROI thing.' But when you consider that that means a competitor can't use this channel and will have to go elsewhere. Okay? You'd say, 'Oh, okay.' And the way I would describe this internally was this way. I would say what would if I just said I have a magic wand and I can just make it so no competitor can use affiliates. What would you pay me? That doesn't even mean that you get it. I'm just saying nobody else can have it. What would you pay to like shut that down for everybody? And it's like well you would pay more than a dollar and you wouldn't pay $10 million. So that proves there's a number. You can decide what it is. Then okay you don't have to decide exactly but what you can do is say okay so what's the ROI on affiliates you would like to have because now you're computing like what can I pay for a new sign up or whatever the metric is plus I can pay this extra amount if the affiliate is exclusive to me. Okay so you negotiate an exclusive deal but you pay them more because that means they can't sell it right so you're going to have to pay them more so you might realize now these are made up numbers but you might realize is I would have paid $100 per conversion, but I'm actually going to pay $200 per conversion. But this is the logic because $100 of that is actual ROI. And the other half is this thing of like, but if they do it exclusively, I shut someone else out and I'm willing to pay another $100. But this thought experiment of what would you pay is how you can arrive at the actual figures that you can actually pay. Okay? So, you know, you're being logical and objective about the ROI dollars that you compute in the normal way, plus this like exclusivity dollars that you're sure you're still thinking of whatever you think it's worth. You may not care about this, but if you did care, so it's still a number you came up with and now you can justify this in the budget. So, in that way, we were able to pay for exclusivity or otherwise understand how this affiliate system works. And the way I knew this worked, and again this is in the book, but I'll tell you the way I knew this really was working besides the fact we were getting, you know, signups, right? But the way the rest of this was working is I was at a conference and in WordPress, which is the space that WP Engine is in, people are very friendly. Even competitors will talk to each other at the conference. Like it's a very friendly community. So, I was in a circle and there was another I think it was like a director of marketing or somebody who ran marketing for a competitor and he was telling the circle yeah affiliates don't work. We tried it and you know they just they don't give you anything. It's all crap. And so only big companies like WP Engine who waste money would do that. But what they don't understand is that's because they were at affiliate number 12 or number 15. And so of course I know that that doesn't work very well. I know that. So this theory of like it would make the channel difficult or impossible for anyone else to be effective in worked. Now his perspective was that means we're dumb. Incorrect. So whenever you see a big company doing things like this, it could be that they're dumb. That totally is true too because as we all know things like Google ads, the Google ads are sold to the dumbest bidder. Like whoever's willing to spend the most for that keyword whether or not that's a good ROI is who wins it right so that's true sometimes it's a waste sometimes but sometimes you want to take a step back and say are they dumb or is this happening to me the thing I just described is that happening to me are they overpaying quote unquote for Google ads to shut me out of the top spots. Now the answer is I don't know I'm just saying like maybe take a more circumspect view of what could be going on okay So this specific channel in particular played out that way and you're willing to kind of take break even. I don't even know maybe take a loss on it because it was a zero sum channel. We weren't going to take a loss but we didn't have to have the best ROI ever. Right. That was kind of our attitude. But again the way we computed it is this is the ROI we would like. And so this is normally how we would try to price it. Like what we'd be willing to pay for a conversion, right? But we're willing to pay more for this other thing. And we can compute how much more because we can answer this question as a thought experiment. Would we pay, you know, what would we pay per month to do that? And we can just imagine, okay, and if we get 50 conversions a month and divide that by that, that's how much I'm willing to pay. Right? So, so it was still an objective number with a clear decision of how we got there.
Okay. How one of the things that's very interesting smart thinking here is you're not just putting it in context of your own kind of business economics but the impact on another competitor. It sounds like you're directly trying to hold their growth down effectively by this as a byproduct. Obviously some people just kind of ignore their competitors. In this matrix you're very clearly thinking about the context of your competitor against yourself. Is that fair to say?
Yeah. I always say, you know, it's true that you shouldn't focus on your competitors. You know, you don't want to drive by looking out the side window. You want to look ahead and you want to play your own game, your own strategy. If you copy them, half the stuff they do is stupid. Half the things they do are wrong even for them. So, copying them is a bad idea. Plus, they have a different strategy. They have different goals. They may be going after different markets. So, if you copy them, that could be completely wrong for you. So, you don't drive by looking at the competitors, right? But you also can't ignore them because your customers, for example, are comparing you to them before they buy. So you can't just not know about them because your customers are going to look and you need a good answer to them. So to me, the general advice I give on competitors is you have to be aware of them otherwise you're ignorant. But you don't want to make all these decisions based on them because then you're not playing your game, your strategy. The time you want to do something like copy a feature of a competitor is when your own customers are telling you, 'Dude, I love you, but that thing they have.' I mean, and this other guy's telling me we should leave you for that. And I mean, and once you start hearing that, you should do the feature, but because your customers want it, not because your competitor just made them aware that they want it, maybe, you know, but that's why because your customers told you, not because you're copying them just wholesale because you don't know what else to do. Okay, so that's the right amount to pay attention. In this case, this is kind of special. I didn't do this all the time. This was again this was a case where it was a small channel really like it looks big but it's not and once you understand that this is huge asymmetry in there then you realize it's actually a small zero sum game well once it's a small zero sum game the rules might change versus something where there's long tails and all kinds of other stuff or the dynamics could be different however now I'm not an SEO expert but my understanding is that the top spot is valuable worth a lot is like worth two or 3x the next spot which is worth two or 3x the next spot And by the time you're on page three, you might as well not exist. So, while it may or you will tell me whether it's like affiliates, maybe it is, maybe it's not, but I bet it's similar in that there's this thing. And so, getting the top spot might be more useful than the direct ROI or more valuable than the direct ROI. So, you still need a logical way of arriving at what am I going to invest in this and how much effort am I going to put and is it even possible or you know because if it's impossible then it's a dumb idea, right? So you still have to be thoughtful, but does this change your calculus in what you're going to invest in and how?
I mean, it's a good question. Yeah. Yeah. So I mean, search we're going this AI search way. I ask about affiliate because I think you know this, but some people are not as deeply aware. It's highly influencing AI search. So, I think about the halo effect of and I mean I don't even know if this was an affiliate you're referencing, but I remember seeing WP Engine all across WP Beginner when I was coming up in my career. And I'm there's undoubtedly just brand effects, I'm guessing, being on these top affiliates that do not have direct conversion that probably help the math on those.
Yeah. And might be direct with algorithms that see that. It might be indirect just that you know of course the general idea of branding is that if people see it a lot it just seems familiar and if it seems familiar it's more likely to convert and of course you can't really measure that. I guess maybe some people can at scale and if they're really good, but let's say most of us, you know, you kind of give up on a direct measurement of that and just say, look, it exists. And once again, you could use my same technique and I could ask, well, what would you pay to have more branding knowing that you can't measure it? Again, more than $1, not $10 million, right? And so once again, one of the ways I thought about budgeting in marketing in general is that you have two buckets in marketing. One is the direct results. So this is stuff like SEO, AdWords, affiliates where you know you spend money, you make money, let's just say immediately enough that you can try to track the ROI, right? There's always, you know, and there's multi-attribution. There's all kinds of issues, but let's just say it's variable in the sense that the better it's going as a whole, the more money you can spend in it because it's making money. Yeah. So it will be variable. And then you have another bucket which is fixed and that's stuff like brand or maybe things like events or other things that are sort of air cover or it's marketing but there's no way you're going to tie it in either because it's actually impossible or it's just impossible for you. We're not worth it to try or something like this, right? So branding is a good example and you just have a bucket of money and you're like that's what we're willing to spend for that because we know it matters. We don't know how much so we're willing to spend only a certain amount. The end. So with that in mind you could then decide oh there is a brand value in being in all these places or something like that especially with high traffic this and that. And so we can in a sense spend some of that brand budget even if it's into a variable area like affiliates. Right. I can still spend some of my budget by overpaying the affiliate from a direct perspective. And that way because I believe I'm getting some brand value just as you said. Look, I saw you on here. Okay. Well, that's brand value. You can put dollars in a bucket of what that is and then spend it either, you know, on something like events, which is often like that or even into a variable bucket. No, as long as it's providing that brand awareness. Some people say that AdWords do that, right? They do. You know, like at least anecdotally, if you spend on AdWords, you get more organic clicks and if you stop the AdWords, you get fewer organic clicks. Now I don't know whether that's true or not but there's old studies there's a lot of studies right why I mean one possibility is branding.
Yeah, makes complete sense. So one thing I one lever I wanted to pull on there was the exclusivity piece. I'm curious so we run affiliate services for clients. You maybe this is more rare but you basically went to that affiliate and said we would pay you double or whatever number if you literally stop mentioning other people.
Well, in competitors of ours, they may sell other things. Right. But it's like, oh god, within this category, you can only mention us. Yeah. Well, it's easier for them because they don't have to manage so many people. And it's also more specific in their content. Hey, I just love this host. I just love this plugin. I just love this software. Not hey I'm gonna give you three and we're going to talk about it and actually that might cause a crisis of decision whereas if you just had one you might even convert more. Um so but you could look at it the other way. You could say if it's just one then that won't be the best solution for all of my customers. No no one thing is the best for everyone like in any product category. So I want to have three or five so I can say this one's best if you're like this. This one's best if you're like that. That makes me more honest. And it's true. I'm matching up people better. So I don't want to do just one. So it's not like everyone's going to do this. And they may be right. I mean there's easy arguments in both directions. So you know I mean in your book you and we can touch on it separately but you talk about kind of Carol I believe the ideal customer. I don't know if you define WP Engine as kind of enterprise solution. Would you in that No, you wouldn't. No. How would you define it? I guess well we have 200,000 customers and most of them are not even in the top 1 million domains by traffic. So we actually are in the top 10. We host we're in the top 10 of hosting the most domains that are in the top million by traffic. So we're big in that and yet most of our customers aren't there. So that means it's not enterprise. Now we have tons of enterprise customers. But see, when you say sell enterprise, to me that means you're running McDonald's.com, right? You're running google.com, right? And so we sell to tons of enterprises, but we're running some marketing campaign for McDonald's. We're running some marketing campaign for Google. We're running, you know, a dev tool site for some tech company, you know, like it's not their.com. Yeah. Yeah. All right. We go down a rabbit hole on that, but I guess I say it out to say increasingly in the AI search context, we're trying to influence these best X roundups of like best WordPress hosting software. Yeah. And I don't know if you feel like it'd be enough to simply obviously you don't want to be defined as enterprise solution but say there was an enterprise solution where their care role was enterprise is it sounds like we it may behoove us to try to negotiate exclusivity for enterprise as a segment and allow small business to be there startups maybe. Yeah. I mean I think the affiliates again I can see both sides for an affiliate. So I think it just depends on what are you doing and what's your persona like? What do you you know there are affiliates who like to say you know I have these three brands that I love this one for hosting this one for plugins this one for SEO that's what I do I'm an expert in it and that's what and I'm super deep in it and I love it and here's all the stuff about I want to try to infect you with my love for these things yes and there's affiliates who say what I love is matching the person with the right tool for that person and so I'm going to show you 10 things now both of those to me sound genuine honest and valuable So they both sound good to me, you know. So it depends on the person. Yeah. Yeah. How about just curious your if there's like a tactical negotiation thing of this because this is my first time I've even like considered the exclusivity piece. Would you go to an affiliate and say I guess you could just open the conversation, but like would you be willing to do exclusivity if we doubled for a do you lead with like a five-year term? I don't how well you could but like you and the affiliate may not like to be locked in that much. So you could do three months. Okay. So just for the next three months in your newsletters and your YouTube, you just do this or one month. I mean you could do whatever you want. And yes, you should go to them, but you should already have a relationship with them. Again, if there's only seven, why don't you know them all by name and be talking to them every quarter or month, right? Yes. What are you doing? That's what you should be doing, right? You know, it's a relationship. So yes, you should be talking to them about all kinds of stuff. Okay. And they want to do some special thing for Black Friday and they want to Yeah. You should be talking to them all the time about stuff. What should we do together? Yeah. And of course that will be illuminated into how to have that negotiation. I suppose again if there was a thousand affiliates then some of that will have to be many to many and emails like you still want to like contact but it's going to have to be a program. When there's five, talk to them. What are you doing? You know who they are. Pick up the phone. I'm curious how you thought about the threshold. I'm guessing I mean I know you were very big of just acquiring that website at a certain juncture. Um never. Never. Okay. You could Yeah, you could call that vertical integration or something. I don't think that's the worst idea. Um but just didn't enter calculus. It's different business model. It's definitely not a business model anymore because the point of doing that would be that you're the exclusive one now, right? So now it becomes your marketing site. Yes. That's another content marketing site. So, if you have a content marketing program that or strategy that says our strategy is we're going to have 10 content sites that are going to look independent, but they're actually all us. Maybe not the best strategy. No, it could be like that's what for example, that's what Blue Host does. Blue Host is a hosting company that has 40 some odd brands and they all host in the same location physically. They're all the same thing. Yes. literally the same product, but there's 40 brands. So, so they can have 40 websites. They can have an affiliate with 10 options, but it's all Bluehost. Okay. Yes. And then what Blue Host does is stuff like this. They'll make a review of hosting of a website that's a review. They're all Blue Host though brands. And then they run AdWords to promote the review page. Yeah. And then also, you know, in AdWords, you can only only one of your ads to one of your URLs can be listed at a time. Like in one search, you can only come up once. Your company can't come up more than once. But they have 40 brands. They can come up 10 times, right? So, they can dominate AdWords by coming up 10 times. They can have review sites that look like a review, but really it's still them even though it looks like choice. Yeah. So here's an example a real example in my space where yes you have many many kinds of sites and you win that way by covering all this stuff. So no it's not a bad strategy. It just again many of these strategies it's like the strategy is not intrinsically good or bad. There are different strategies. There's a multibrand strategy like that. There's a single brand of course where you go deep or whatever and you're the expert and so on that makes sense. There's lots of them that make sense. Yeah, I think when strategies don't make sense, they're when the strategy is inconsistent within itself. It's internally inconsistent. For example, it's an enterprise tool for $29 a month and you go that can't be because enterprises need a bunch of stuff. You couldn't afford that for that just doesn't make sense. So that like you could say pricing or positioning strategy quote unquote because we just that's a very small shard of a strategy but that strategy doesn't make any sense because just intrinsically it makes no sense, right? It's not because of a competitor or a market just it doesn't make any sense, right? Okay. So I think that invalidates the strategy, right? Because like what do we do and we can't fulfill the promise of this positioning because this is all the revenue we have and the customer doesn't understand and like this is a bad strategy. Yeah. Yeah. But of course there's a multibrand strategy that does make sense. Maybe many of them and of course there's a single brand strategy many of them that make sense.
Got it. Right. Yeah. Yeah. I mean it's definitely a rabbit hole to go down and I'm sure very layered in making those decisions but I think that's enough good questions and answers about affiliate for now but super top of mind for everyone in AI search another kind of search focus thing you have a chapter in the book about experiments and how they may be challenged sometimes and how people do not get the results that seeming outcome prophesizes at the end or maybe at the end you I'll let you detail it further but it's interesting to our audience because there's tool providers there's other people just at larger companies with a lot of page views that are running AB tests for search benefit themselves and many of these I'm sure do not see a very similar outcome where they look back in two years and they're like why didn't my organic search go up 500% at the end of the rainbow.
Yeah. What you're referring to is so you run an AB test, you see no result. Fine. Another AB test, no result. This is all normal. Oh, look, 15% left. Hooray. And so you take the winning thing and you repeat ad nauseum, right? And then the idea is okay. Oh, you got 15%. Oh, you got another 10%. Oh, you got 20%. And like so theoretically after a year of this with a couple of these wins, you should be at like 50% more, 100% more because they stacked up. And you look back after a year and you don't have 2x the conversion. Yeah. Often you have nothing. It's nothing. It's exactly the same as when you started. And it's like, well, what the hell? I had all these wins. And there's several things that explain why this happens. Mostly it's false positives because if you don't do the math right, and sadly a lot of these AB testing tools don't do the math right. So, they tell you this is the winner, and really it's not statistically significant. It says it is, but really it's not. And as you say in the book I give like the actual answers to what is statistically significant? Why do the tools get tell you the wrong thing? I of course and of course I give real world examples of a favorite tool that you think are right I'm not really sure and they change their algorithms anyway so I don't know but they'll say like yeah this is a winner but it's a winner at like 80 90% confidence if we're using statistics language but that means one out of five or one out of 10 times you get a false positive right but like getting a real positive about one every 5 7 n tests is about what people normally See, but that's also the false positive rate. So, how many of those are real? Maybe none or maybe one, you know. So, that's the first reason is like actually the false positives are real bad and it's hard to get something that actually is better that it's actually quite difficult to come up with something where that's the case. The next thing that happens is people are just throwing spaghetti at the wall like they don't know why. They're just like, 'Uh, say click here. Say click now. Say click. Say next. Say sign up. Say, you know, right?' They don't know. They're just like you know, this color, that color, you know, right? And so then when something happens, they're like, 'Ah, something happened.' But they haven't learned anything. Like, so what does that tell us about our customers or our positioning or the mindset of the person at this moment? Nothing. Because I didn't have a theory. I just like did stuff. And so maybe it's even a false positive, too. So I've really learned nothing at all. The point of an experiment is to learn something, is it not? You know, but I've learned nothing. So the right way to do it is you form a theory about what's going on with the visitor's mind at this moment and therefore what kinds of changes would have the effect that you want. So if it's a landing page, what is it that you think they want on this landing page? Are they wanting to learn more? Are they wanting to buy now? Are they wanting to compare you to something? Like what's happening? And it may depend on where they came from. From this source they may be wanting to compare you to a competitor from this source or keywords or something they may be like ready to buy you know so but anyway you have to make a theory about this right then when you have a theory about the pricing page are they ready to buy when they see the pricing page you know let's say you think yes and when once they reach the pricing page they're ready to buy okay well then on your pricing page you would have this theory like what okay if that theory is true what should we do to increase conversions at this moment well I might reduce the stuff on the website around the pricing page so that they're not tempted to leave and go off to other parts of the page. I want to force them down the buying funnel. I want to make that buy button bigger. I want to answer objections. Maybe I want to, you know, I don't know. But like you would have all these ideas of what to do. And then you can enact that. Now you have an AB test, current pricing page, and this stuff. Now, what happens? Now you're going to learn something. Suppose conversions go up. Now, does that prove everything in your theory is true? Well, it gives evidence. And then what do you do next? You say, 'Well, if that's true, could we go further? Could we do more?' And maybe you can get even more of a result because you know what you're learning and what you're doing. On the other hand, let's suppose nothing happens. It doesn't change the thing. You still learn something. Your theory wasn't right. Maybe they're not ready to buy yet. Well, that's really useful information. Why not? Oh, because they still have objections. Oh, because they still want to compare competitors. You can have other theories. Let's suppose your next theory is because they're worried that if they the trial period is not long enough. They're worried that they don't know how to compare to other things. They don't know which of these things to buy. You can change what it looks like to address that proactively if that's your theory and so forth. So you see whatever happens you now know what to do next. Either get a new theory or try a different thing. And now you're sort of doing like this actually learning and doing better. And there can still be false positives. I could, you know, you get a positive result and say, 'Yay, my theory was right.' But then you say, 'Okay, so I'm going to go even further.' And if it is a false positive, conversions will come back down and it'll fail again.
Are you saying people should still remind me how what incurs false positive besides pulling out at like 90% confidence interval or is that it basically
That and the fact that we keep running experiments as we should? Yes. And so we're sort of accruing the probability that we're going to get a false positive at some point. Okay. Right. Both of those things, right? If I just did one, then at a 90% confidence, then there's only one out of 10 chance that it's false positive. But if I run 20 tests, it's almost guaranteed I'm going to get a false positive. And that's what I do. Of course, that's what you do, right? That's your job. It's okay. It's just that's what's going to happen, right? It's going to generate false positives. Yeah. So, in our world, I mean, it's a lot of engagement rate type testing, dwell time, that kind of thing. Yeah. So, I think the challenge with SEO though is there's a whole lot of factors that I think maybe negate AB testing because you can't really AB test the same content because that content is somewhere in you know rank somewhere. So is it that content and totally different content and you're testing what and there's so many other factors that go into each of those. So what are you really comparing only that one variable and then of course your search position doesn't change in a day. So are you waiting two months? Okay. So iteration time is super long and then in that two months the algorithm changes. And your other competitors, you might say content competitors, they're changing. So the test is not stable over that time. What are you measuring? I'm not sure I know what you're measuring at that point. So I don't really know what AB testing means in terms of that sort of situation because of those confounding factors. Yeah, I think to my understanding definitely comment on super high high traffic pages consumer e-commerce and they'll basically like say it's an e-commerce site take similar product pages maybe there's like 30 of them with the same design and can compare it to a control group and if there's 30 and 30 of course that's still not the same there's 30 different things but yeah 60 different things so I guess what I would say is number one either say that's not a good use of your time. Okay, maybe you should do other things with your time because this is not scientific. Yes. Okay, that's one idea. The next idea is okay, but only look for massive changes. In other words, if one converts 2x more than another, I can kind of believe you that this is way better. Yes, because it's such a big change. It's like this isn't just the roll of the dice. Like that's crazy. Whereas if it's a subtle change, I would say you can't measure subtle changes. There's too much noise. The noise amount is the same magnitude, right, as the real thing that you think you're measuring. So, I don't believe you. Whereas, if the noise is this amount, but you have a change this amount, I believe you that you surpass the noise and it would be unusual for the noise to explain that. Of course, statisticians have very precise ways of saying what I just said. Yeah. So, what I said is actually a precise statistical thing. Okay. And so when you use the formulas, right, you're doing what I just said. But that's why I would say the upshot is big effects are probably real because you've surpassed the noise. So maybe that but it's hard to get a big impact. And by the way, if you get one, did you even need the test or was it obvious anyway, right? So like we're right back to that like but it's still useful me mentally to say I'm going for big effects not small because if you think I'm going for a small incremental effect, then you think of things like I'll just change this color. I'll just change this word. But if you think I need to big impact then you start thinking how would we redesign this whole page and again I recommend having a theory about things at this point they you know they're not going to read they're blowing through we haven't earned the right to someone read a whole paragraph blah blah blah so it needs to be like picture five words get going super compelling now that's a theory because that's a whole different page layout look positioning words right yeah Now that feels a lot better in terms of something that might 2x convert rather than I just changed a word which is unlikely to do that. Even that notion of it got to be big, that does change your mentality of what is it that I'm going to try, right? How do you think about So with search, we're going to make a lot of these changes iteratively sometimes. Some of it seemingly is done for downside protection. Like would that change your mindset at all? Like let's say we're updating a category page on Walmart and we're going to roll that out, right, to 30 different pages. Hopefully, we think it's going to have a positive effect, but maybe more so we it's almost more exciting to do an AB test and find that this did not hemorrhage conversions. I understand what you're saying. Yeah. The often the way this happens is designer says, 'But I want this design.' Yeah. And business person says, 'Well, if it decreases conversions, you can't have your design.' Yeah. But if conversions are just the same, it's a win. Okay, fine. Because you I think it's better like it's whatever fine you know but you can't harm the numbers but okay maybe not harm is a sufficient a brand bucket in there yeah so sure right like we can't lower conversions because you like the design but yes you could of course do that kind of thing one thing that is
Also in the book that I think is relevant here is the notion of the funnel and the quality of the traffic coming to the top of the funnel, which is what SEO does. It brings in traffic to the top of your marketing funnel to some page, whether it's a homepage, landing page, whatever. And then whatever happens next, that's a funnel. SEO is at the top of the funnel of a certain funnel, right? And I have a whole bunch of stuff in the book about the top of the funnel. But again, I'll tell you it all now, so you don't have to buy the book. This isn't that great. It's okay because as you know, I have so much in there.
Yeah. Yeah. There's like something for everybody.
We've covered like three-quarters of one of 12 chapters so far, right? There's tons of stuff.
It's a good fast read, too. Yeah.
So, I don't mind just giving you everything in there. So here's the top of the funnel thing. My contention is that the quality of traffic, quality, not quantity, quality of traffic at the top of the funnel is more impactful than anything else in the funnel, even combined.
Here's how I'll prove it. Let's suppose I send only perfect customers to your website. Like they're exactly your target market. They got budget. They're ready to buy now. They don't like your competitors. And every feature you have, they're like, 'That's what I always wanted.' And your UX is like, 'Oh my gosh, this is so convenient.' The price, they already have a budget bigger than your price, so they're ready to go. Perfect customers, and that's all I send to your website. How will that website convert? And the answer is, well, like even if you're kind of crappy at saying what it is and kind of bad at everything, and let's say you have a sales team and it goes to the sales team and they're just like mediocre, you'll still sell a lot because they actually want it.
It's going to go pretty well. Even if you're terrible at the entire funnel, it'll go pretty well for you.
Now, suppose I send nothing but garbage to the top. It's just fraud clicks and, you know, whatever. People that are not interested. Then it doesn't matter how good you are at everything. Oh, I'm so good at the rest of the funnel. I AB test my all my sales people are the best ever. It doesn't matter.
Because if it's garbage, none of it matters.
So, how do you summarize what I just said?
If the quality is great, no matter how bad you are at everything else, you win, right? If the quality is bad, then no matter how good you are at everything else, you lose. Right? So, the main thing is the quality. That's the main determination of whether this succeeds or fails. So that's my little proof, quote unquote. Of course, reality is more complicated than just that. But that's my proof that it might be your most significant lever, your multiplier if we can use the book title, right? Okay. So that's my sort of thesis on that. So it's actually worse than that.
Because think about this. You have a current website with stuff and then what do the demand side like SEO people, what are we told? Bring in more traffic, right? Okay. Hopefully, you know, so more keywords, more content, better keywords, better placement, whatever. Do we do the thing? Then we bring in more traffic. Traffic goes up 30%. Now, not all of that traffic is high quality, right? But what do we say about that? No problem. That's what the rest of our funnel does. It'll hit a landing page and then what we do is we AB test on things like landing pages and home pages, right? And through the AB test, what we'll do is we'll just figure out what works best for this traffic. And that's how we'll improve the conversion rate anyway.
So, we're all good, right? And here's why this is a huge fallacy. Because when we go get more traffic in SEO, let's say all that new traffic is garbage. Let's just say, then what is going through the AB test? Garbage. So when we improve the conversion rate, we are converting garbage. We're convincing more garbage to continue.
Not the customers that are going to buy. The customers that are not going to buy is what we're optimizing for.
Oops. And then even if they buy or they get to sales, are we wasting everyone's time or they buy and then cancel because it's garbage, right? So if we put garbage on the top, the AB testing that comes next down the funnel makes sales go down because the way to increase clicks is to convince the garbage that you brought in.
That's how you convert, right? So it's, I know it's a little bit of a mind bend, but like when you do that, when we bring in garbage incrementally, we lower sales and the AB testing folks are making it worse. We're gunking up the whole marketing system. So, it's not just that garbage in is not valuable, right? We're actually gunking up the rest of our marketing and sales system and wasting their time and actually unoptimizing our funnel. So, this is my whole pitch for oh my god, the quality at the top of the funnel that we in SEO are bringing in. The quality is hyper important and we're not fully understanding that. We're not acting like it because the way we should act is it's not volume, it's quality. I would rather have a tenth of the volume but very high quality because that means we're going to win regardless and that means those AB tests are going to in fact convert more real customers and so on. The whole system works well at that point. That's how we're going to grow this company is quality. Now you could say but how do you know quality? I would agree and I have some ideas, right? But the point being like but first before we even get there do you have to first say oh my god that's our actual job because that's actually what the deal is. And so if you don't agree with that like then that's what you're doing. But if you agree with that, see it's another multiplier there at the top, right? It multiplies to the crap and it also multiplies to the good, right? But once you agree, you can go, okay, well then shoot, how do I do that? How do I know what the quality is? How do I go for higher quality? But at least you're asking the correct questions now. So, we're already making progress, right? So, a couple of ideas are one is of course targeting. The more targeted the more likely that, right? But also, what is it that it's hitting and what is that whether it's content marketing or whatever it is? Is that speaking to the widest audience or is that speaking to our perfect customer? So, you know, when someone lands on there, are you just trying to appeal to the masses just to get another click because that means the quality is low.
Or in fact sometimes making the content confusing causes them to click because they're like what?
Right. But that's low quality as opposed to this is so sharp and it's so good at if you're our target customer you're going to love this and everyone else is going to go what and then bounce off. That's correct. That's what our SEO and then content marketing should be doing because that's this immediate filter to high quality traffic. Now we're good to go.
So that's one thing. The specificity of our target audience rather than appealing to the masses is one. I'll just give you one more. The book has more but I'll give you one more which is the further you can measure down the funnel to say what is the conversion to define conversion the better. So in other words the worst case is the conversion is just did I click? Did I just get to the next step? That's the worst because it results in everything I just said. But suppose the best case would be how many of them bought and then stayed for a year.
Now I know you can't, right? I'm just if we're just theoretical because now you're right if we increase that conversion that's it that's the answer and now I am selecting I'm not selecting for garbage just definitionally. So of course that's too much but you see what I mean. The further down we go it could be that they got to the pricing page. It could be that they got to a sales lead that was qualified. It could be, you know, so the further down we can measure the conversion, the more indeed you are not falling into the trap I said and you have fixed this machine a bit because you're saying you're actually measuring quality because that's what we're doing. We're saying a sales-qualified lead is quality. We're saying a sale is quality. We're bringing that notion of quality up to the front of the funnel. So we really are getting quality, right? So that's what we're effectively doing. So there could be other ways but that's the idea. I'll give you yet another way is you may have a pretty good system. Now, if you have brand new traffic from, let's say, new keywords, new whatever from SEO, if you put them through those same funnels and those same tests, we might be poisoning our machine like I was saying.
So, can we keep those segmented so that the AB tests for those are separate from these so that we don't poison what's already working? So just in case it's bad quality, which we won't find out for a while because it has to go through the funnel, has to go through and then we could find out later. Oh no, this is good. Put it in with the good. Pile it up with the good. Or we could realize, oh no, this is bad. But we know that. So now we can make different choices and we didn't poison our normal machine with that. So our normal machine can still optimize and thankfully we segmented and didn't poison. So simply segmenting traffic like that, setting aside the new stuff allows you to find out if that's garbage or not and then decide what to do next. Right? So that's another technique for dealing with this idea or for reacting to this insight that the quality at the top of the funnel is maybe the most important thing. And certainly for SEO can completely change your job description. Now you have to get buy-in from everyone else. I get it. I get it. But if it's the truth, then if you can show that, maybe you get the book and show them the book. Oh yes, we're going to sell books. But okay, you don't have to do that. But making that case maybe that's insightful and smart. So if you can make that case, yeah, I mean then you're doing a lot more than just finding the next keyword.
So yeah, we've had, I mean our industry's gone through a disruption obviously and clicks are less, I think it's more okay to be showing flat traffic recently as long as leads and sales are going up. So it's not a bad time to I think have that conversation you're talking about.
Yeah. I mean I think like if you just said hey if actual traffic was flat but we got more sales leads would you want that? Of course everyone says yes. That's obvious. So what I'm getting at is something quite different.
Yeah. Right. It's like sure, but redefining my job description as quality leads both measuring, doing everything I just said about the actions we're going to take, how we're going to measure it even downstream. So that traffic could go down, but we're still happy because reasons like that's a whole different thing. And like what metrics am I going to use and how will we know and what are the AB testing people have to do and what other systems need flags need like tagging or whatever so we can know.
Totally different so yeah in the abstract sure but that doesn't mean that's what we're doing, right? And so if you bring that in though you're being strategic in the sense that you're making the whole company, the whole department, the whole go-to-market department smarter and better because again what we're doing is even making the AB testing smarter. We're making better SQLs so sales is more efficient. Like we're making everything better. That is a huge impact to the business. Not just, hey, I'm really good at getting the next, you know, getting from slot three to slot two, which of course is also the job. I know, you know, but now you're having this impact on the business. I mean, all of our jobs are at risk with AI. That kind of thinking, bringing that kind of thing in is the kind of thing where like that's not a person you want to fire. A person that like bringing that kind of insight to the table even beyond their narrow role is not the kind of person you want to get rid of next week, right? Like so I can't promise anything but like I don't know it seems like a reasonable idea, right? Like there beyond just hey do you want more sales leads? Yes. That's not insight.
Yeah. One thing I've been thinking about recently is just the concept that I don't know your position on it, but with clicks dropping sometimes dramatically, especially in B2B SaaS in particular, there's less kind of wiggle room to even participate in experiments in a statistically significant way. I'm not optimistic about AB testing with SEO.
Yeah. Now, what happens next? Sure. Like the AB testing of the next conversion as long as we're doing conversions better. Sure. And of course with N being big enough and looking for big effects, but just AB testing in SEO again like you were saying, oh there's this example and it's Walmart, so N is enormous and also this and also that. I'm like, okay, yeah, sure, that sounds pretty good. But for most companies that aren't Walmart and etc., etc., like maybe that's just not a good technique and that's not where you should be spending your efforts in SEO. How do we get into that? It's changing all the time. I got to be up on that. That's where I could be spending my time as opposed to my testing tools that probably are random, right? You know, it's like or I could be doing some tests, but I have a totally different mindset now. Big effects only, big changes, theory based. Okay, maybe now I'm doing something.
Yes. Okay, makes sense. So, I mean you got a lot of multipliers in here. You've already kind of hit indirectly things that you think would be impacting search and content marketing. Was there any other ones in particular you felt like would resonate with our audience?
I think there's actually two chapters in there about finding actually identifying your perfect customer. Like how do I know who that is and what do I write down exactly? Because I don't believe in writing down personas. Like Sally is 34 and has a cat and she also likes design tools and so she's a customer. You know, that's not helpful. So how do you identify what they are? How do you describe what they are? What dimensions are actually useful? So there's two chapters of how to do that. Oh, I should mention too there are free AI skills. It's at skills.asmartbear.com. That's my website is a smart bear. And free as in like totally open. You don't even have to put in your email or any of that stuff. Like it's just there. The GitHub repo is there. Like just do whatever you want. Okay. And not the AB testing, but stuff like how do I find my perfect customer is there like you don't have to buy the book or anything, you know, you could just run through those skills and it'll, it doesn't tell you answers. It's an interrogator. It's a facilitator that walks you through my framework to do it. So anyway, you can just do that. But the reason I'm saying all that is I find that most marketing departments or product departments and high-tech sometimes it's product that owns this ideal customer, whatever, don't have a really crisp idea, actionable, crisp notion of who the perfect customer is. And if you don't have that, then it's hard to say what to do. It's hard to say what keywords and what, yeah, you know, what am I targeting and what should I put on the and what kind of content marketing should be behind that or you know, what is the hook on the landing page where the right customer would click and other people wouldn't like if you haven't told me who the perfect customer is in great detail so I can action that then I don't know how to make quality because you haven't told me what quality would be, you know. So that's an exercise that would be wise to do.
Again useful for the whole marketing department and also product because you need to be making features for that customer and not features for everybody.
Yeah. Yeah. Right. So it's really the whole company but okay. So again another strategic thing to do, strategic I'm using that word because that word might mean nothing or means too many things. I mean it to mean the macro decisions that all other decisions and things flow up into so that it's like we're aligned on this and as we all go to our own jobs and do we are aligned on this and so we're all moving in the same direction without having to talk all the time because we've made these macro decisions. To me that's strategy. Yes. Like how are we going to win? Who are we selling to? What's the product? You know, like just and then everyone has to be within that as they go off and do all their jobs, right? Yes. So, if you don't have a crisp definition, actionable definition of your target customer, then you're missing that piece of your strategy and so people don't know how to make all these decisions and so they run in different directions. And sort of like how you know the old thing of like if every water molecule is going in a different direction, then the water just sits there in the cup.
But if all the water is moving in the same direction, then it's a water jet and you can cut through metal.
So like strategy is we need to be going the same general direction, right? Not just all scattered and like not doing work, right? So if you're missing that customer definition, you're not all aimed in the same direction. You're missing that part of the strategy. You could even though it doesn't matter who it is. It could be an SEO. You could say, I need this. We all need this. I'm going to lead us through this. You could use the skills. You could use that whatever. You could use whatever to some other framework that you like, you know, whatever somehow get there. Right? So I think that is another thing that I find even in whether it's small or large companies the tendency is to say well it's for any small business it's like okay so it's hyper-focused on 300 million businesses really are you sure it's good for concrete manufacturers and also restaurants and also high-tech startups from YC and also are you sure you know and so that's one reason you know people are loathed to even make the decision which is a bad reason to not make a decision. They don't want to cut anyone out. And the reason that's faulty is what you're really doing is saying, well, there's a bullseye of our target market, like a whole bullseye with rings, but then there's a center that's our hyper-perfect customer. If we can define that center, then we know what we're aiming for. We can write everything for that. Now, here's the truth. There's people who aren't exactly that. They're adjacent. They're at this ring. They're at this ring. This ring, and they're going to pick us, too, because we're close enough. Close enough. Or also the our trade-offs are more or less the trade-offs they want. And so they'll pick us. But because we knew where the center was because we were so hyper-specific, we knew what to do. We knew what to say. We know our positioning. We know the keywords to do. We know. And even if you're in an outer ring, you understand what this product is. You understand what the trade-offs are. You can make that decision. And if you don't know what the trade-offs are, you don't know the product. If it's wishy-washy, it's hard to choose that product because you're like, I don't know what I'm getting into. So that specificity always wins you people near the center. So it's already a win.
