Back
Craig Saldanha
Chief Product Officer, YELP INC

Product Roadmap: Building a Platform for the Next Decade with Craig Saldanha

🎥 Mar 13, 2024 📺 Product Thinking by Melissa Perri ⏱ 48m 👁 262 views
In this episode of Product Thinking, Craig Saldanha, Chief Product Officer at Yelp, joins Melissa Perri to explore user research, ...
Watch on YouTube

About Craig Saldanha

Craig Saldanha, Chief Product Officer at Yelp, has discussed the company's approach to artificial intelligence and user trust in several recent appearances. In December 2024, he described two categories of search intent on the platform: objective queries, such as business hours, which he said large language models can answer effectively, and subjective queries, such as whether a restaurant is romantic, which he said require opinion rather than fact. Saldanha stated that for subjective searches, Yelp uses AI to find the best answer but then connects users to a specific human review rather than providing an AI-generated response, adding that when done well it can "feel like finding a needle in a haystack." He also outlined three reasons he believes consumers trust Yelp: the breadth and depth of its content accumulated over 20 years, the authenticity of reviews from actual human beings, and the ability for users to make meaningful connections with other users or service professionals. Saldanha cautioned that for user-generated content platforms, it is important to understand where trust comes from, saying AI can elevate trust if used correctly but can "very quickly erode that consumer trust" if used incorrectly. In a September 2024 podcast appearance, Saldanha described his mental model for Yelp as focusing on consumer trust, making the experience engaging, and elevating services by removing friction. He stated that the company optimizes for connections between local businesses and consumershol, and that its planning process involves both top-down goals from leadership and bottom-up goals from teams based on consumer and business feedback. Saldanha also noted that Yelp has a product ops team that reconciles dependencies across teamshol. In September 2023, he received the Tepper Recent Masters Alumni Achievement Award from Carnegie Mellon University's Tepper School of Business, where he credited his career success to the support of fellow Tepper alumni, including mentors who helped him with presentations and friends who moved into his home to care for his newborn while he moved abroad for a career opportunity.

Source: AI-verified profile updated from Craig Saldanha's recent appearances. Browse all interviews →

Transcript (56 segments)
H
Host0:00
Hello and welcome to another episode of the Product Thinking podcast. Today we're joined by Craig Saldanha, who's the Chief Product Officer at Yelp, responsible for bringing in huge investments in the implementation of AI and LLMs to their services. Craig spent almost nine years working his way up the ranks at Amazon, from an internship all the way through to being Director of Product and Engineering for Prime Video International. I'm excited to talk to him about how he leads product management teams at scale. But before we talk to Craig, it's time for Dear Melissa. So in this segment, you can ask me all of your burning product management questions, and this week we have a caller. So let's go to the phones.
C
Caller0:37
Hi Melissa, so I'm a biomedical technician and I want to pursue product management in the health tech industry. So do you have any advice on how to achieve that position, the steps I need to achieve, and also can you recommend a book or a person that I need to follow, some companies that have product management in the health tech industry? Thank you very much.
H
Host1:00
So breaking into product management is complicated, I'm not going to dismiss that, but there are ways to do it. There are two different ways to think about entering product management and what kind of companies you might want to target. There are actually a lot of software providers out there for biomedical things. I've worked with two of them, MetaData and Citel, that do biomedical data and they build software for people like you. So you might want to look at what kind of software you use and see if there are any openings in those companies for you to come in as a junior product manager and learn. Or you might want to start in a different role where your expertise and your clinical knowledge could really help, and then try to transition your way into product management. So that's a start. You can go to a pure software company and look for where your experience as a biomedical technician is really going to help. There are a lot of companies out there who really need that domain expertise, so that could be a good starting point. The other way to look at it is to go to a place that has software as a component. Maybe they don't sell software, but they use software in the way that they run their business, and express that you want to learn product management. You have the expertise already in your domain, you want to learn product management. I'm not exactly sure where you work right now, but let's say that you work for a lab or a pharmaceutical company. In pharmaceutical companies, they do have a huge engineering department, they have a bunch of product managers. I work with a couple of them, and they have thousands of product managers. That might be an opportunity for you too, to switch inside of your company and say, 'Hey, I'm really interested in it.' If you want to do a switch internally, what you really need to look at first is making that known. Make it known to your boss, go find the people who are doing product management, find the leaders, tell them that you're really interested in these things. A lot of companies have a way to develop people and help them transition into product management. That might be a good opportunity for you to get your foot in the door there too. So I would look at those two options. If you want to go pure software, try to leverage your expertise to get hired by somebody who builds software for people like you. Or internally, look at where the software department is and see if you can make a move internally. Express that you really want to learn this and then do that. You're going to have to learn other things though, besides just your domain experience. So I would look at trying to understand what product management is, what some of the tools of the trade are, and make sure that you really want to do this first, because a lot of people get into this role and then they hate it. It involves a lot of people dynamics and involves working with engineers and tech. So I would really get up to speed on some tech concepts. We do have a course at Product Institute called Tech Fundamentals for Product Managers that might be a good place if you want to learn more about technology. We also do have a Basics for Product Foundations there too that you could take. If you want to read a book, Inspired is a great place to start with Marty Cagan. I'll plug my own book, Escaping the Build Trap, for you to read as well. There are lots of product management books out there for you to start to get up to speed on what that role actually entails. I think if you walk in the door knowing some of those things, knowing what it entails, that's going to make it a little bit easier for you to transition. So I hope that helps and good luck with your search. For those of you listening, if you have a question for me, please go to dearmelissa.com, drop me a line. We do prioritize these voicemails, so I love to hear all your pretty voices. Let me know what your question is and I will answer them on an upcoming episode. Now it's time to talk to Craig. Welcome Craig, it's great to have you here.
C
Craig Saldanha4:41
Thank you for having me. It's great to be here.
H
Host4:44
So it is the 20th anniversary of Yelp being around, and I think everybody out there knows Yelp. That's where we get our reviews, that's where we figure out where to go and what restaurants to eat at, what service places to visit, all that stuff. Can you tell us a little bit about what Yelp looked like when you walked into the organization?
C
Craig Saldanha5:02
Yeah, so first let me tell you just a little bit about my history with Yelp, which is a good segue to why I joined and the org that I inherited. I first started using Yelp, I think, in 2008. I remember because I was dating someone who is now my wife, who was a foodie and she really prided herself on finding the hot and new restaurants. At the time, I was a very nerdy engineer who viewed food as a vehicle to get through the day. I spent a lot of time with Chipotle, just getting food. So when we were dating, Yelp was my lifeline. I would go to Yelp to essentially impress her and pick all of these nice restaurants. Even more than that, when we went to a restaurant, there were often times that I would look at the menu and have no idea what these fancy dishes were. She would make fun of me because I'd pull up Yelp and look at the pictures and decide what I wanted to order. The opportunity to shape a product that I had loved for so long and used for so long, such a key part of my life, was what essentially brought me here. I was already a fan. I inherited a product and a team that was in very good shape. The focus was really looking ahead: how do we future-proof Yelp? How do we make Yelp appealing for the next decade?
H
Host6:43
That's awesome. I love that. I mean, I use Yelp for the same reasons, and I like that it has such a unique part of your relationship there too. When you talk about future-proofing Yelp, can you tell us a little bit about what the situation looks like for Yelp? There's Google reviews out there, there are other places for reviews. I know Yelp was one of the first ones on the market. Tell us a little bit about what it looks like from your perspective and how you think about differentiation.
C
Craig Saldanha7:08
Yeah, I think what makes Yelp special is our consumers. We have such passionate folks who write the most in-depth reviews about their experiences, about the dishes they've had, about the ambiance. I think what separates us from our peers is the depth and the breadth of the reviews that they have. I think it's why customers trust us when they want to pick a local business to connect with. That's really the focus. Going back to your question on what we're doing to future-proof Yelp, what we're doing to make it appealing, it starts with the customer. That's the essence.
H
Host8:04
I had a friend in New York City who was one of the Yelp top users, and we got invited to all these restaurants and launch parties and all these different things all the time. I know she was very passionate about writing her Yelp reviews. So I know you've got a collection of people who are really the backbone. Do you structure how to engage those people all the time? Tell me a little bit about the customer obsession.
C
Craig Saldanha8:27
Our Yelp Elite Squad is just awesome. They champion us in the community, but they also provide guidance for everyone else. The reviews that they write are so in-depth and so helpful. It does form the backbone for a lot of what folks are looking for. But the nice part about Yelp is that every review is equal, every review is important, every review is valuable. Our Elite Squad is always a source of feedback. We have a lot of feedback sessions with them. But for me personally, I end every week with looking at customer feedback. Folks who write on Twitter, folks who post on our page, folks who engage with CS. We have so many different channels, and that really forms the basis for what we prioritize, how we think about our roadmap, and how we think about evolving our product overall.
H
Host9:42
So when you're looking at the product strategy as a Chief Product Officer coming into a new organization, it's got to be slightly overwhelming. I think you had an interesting time too because you were fully remote onboarded, right?
C
Craig Saldanha9:55
Oh my God, yeah.
H
Host9:57
That's got to be different, and Yelp is not a small company by any means. When you came in, how did you get up to speed with what was going on and what problems you needed to tackle? What's your way to get set up there?
C
Craig Saldanha10:08
That's such a great question. Prior to Yelp, I spent a decade at the same company, so I knew everyone and I used to walk the floor every day. I was really worried about this, and it actually turned out to be a super cool and super interesting onboarding experience. There were a couple of things that were very helpful, some that I was forced to do and others that were just a byproduct of being remote. The first was I got a chance to shadow our CEO and founder, Jeremy. I basically shadowed him for a couple of weeks, attended all of the meetings he was in, and even beyond product, got a sense for the whole company and how it worked and what actually made it tick. I don't think that would have been an option had we been in the office, just walking into meetings with him. This was really helpful, just to be a fly on the wall. I thought that was super interesting. But what was even cooler for me was because we were a fully remote organization, I knew that there would be folks coming in after me who would need the kind of context that I was getting as I was ramping up. So I decided that I had to codify what the product culture was and what the product ethos was. I ended up writing five tenets that I think are very core to how we think about our product. What was interesting was I remember taking them to Jeremy the first time and I said, 'Hey, from all of our conversations, I've written down these five tenets.' He read them and said, 'Maybe one and two make a lot of sense. Three I would modify, and four and five I actually don't agree with.' That set up three months of debates that me and Jeremy and other members of the exec team had. There were these really in-depth debates that we had based on these five tenets that I had written. After three months, we rolled them out to the whole organization, and that's the basis of what we use for our decision-making. It ended up being a really positive experience. I got a chance to really tap into what made the organization tick and then put it down in actual words and writing, which is very cool.
H
Host12:43
What are your five tenets?
C
Craig Saldanha12:48
I have them on my desk. Let me just pull them up and read them out. Tenet number one is 'Delight a two-sided marketplace,' which essentially means we acknowledge that we have two different constituencies that have different goals and needs. While we always look for a win-win, there are some instances where there's going to be conflict and we have to use extremely high judgment to resolve those. Two is 'Decision-making.' We talk about one-way doors and two-way doors. We want to move very quickly on two-way doors and experiment quickly, but when there's a one-way door decision, like a pricing change which would be hard to roll back, we want to be very slow and thoughtful and deliberate. Tenet three is 'We want to be consistent, scalable, and self-service.' That speaks to building an MVP is great, but we want to build it at scale and we want to be thinking about that from the time you write down the requirements. Four is 'We want to be diverse and inclusive in our product thinking.' We have a very diverse and inclusive set of consumers and a very diverse and inclusive team, and we want to be very thoughtful and deliberate about making sure that every feature that we roll out has the broad appeal that we want. The last one is 'Think big and learn fast,' which essentially means we want to take big swings, but we want to be honest and upfront about the risks and hypotheses. We don't want to be scared of failing. When we fail, we want to learn fast and shine a light on it so that other people don't make the same mistakes. They sound somewhat generic, but if you spend time with Yelp, you'll see that they're at the very core of everything.
H
Host14:50
I love that you baked in the diversity tenet in there too. I see a lot of people not explicitly call that out, especially in consumer-based products. How has that helped shape the thinking for your product managers on how they build products for scale or for many types of different people?
C
Craig Saldanha15:11
First, I think that implicitly a lot of product managers want to build diverse and inclusive products. What we wanted to do was by making this explicit, we thought that we were empowering them to make the right trade-offs. Very often, I saw that people would come up with really good ideas that had broad appeal, but they might favor narrower ideas that would drive some metric that they were being graded on. I found this especially with more junior PMs. By codifying this, they can now point to a tenet, and even if someone like me or someone else says, 'Hey, I think they should follow on the roadmap,' they can remind us that it's built into what we're doing. We've said this explicitly, and it's a good checkpoint for us to say, 'Let's go back and move it to the top of the stack.'
H
Host16:18
That's such a good point. I feel like there's almost a war sometimes between our OKRs and our goals that we want to hit on our product strategy, and then coming back to our tenets or the principles that we actually believe in. So often in companies, the tenets will get sacrificed for the OKRs because people will see them as higher priority. How as a leader do you make sure, or how do you work with the executives or your product managers to balance what you want to hit from the OKR perspective but then drive it back to that mission and goals? Where's the leeway? I think a lot of people have a question about how do I know these things need to get done? I've heard this from product managers: 'I know these things are going to be good for us, but I'm trying to justify it back to our OKRs and it might not align perfectly, but it's one of those fundamental things where I think it's going to make a difference.' How do you balance those things?
C
Craig Saldanha17:13
I want to tell you a quick story from my past life that speaks to your question. I used to work at Amazon and I had the privilege of working with Mr. Bezos a few times. One time, we had launched this new sports feature where you could instantly bring up highlights on your TV. Normally, if you're watching a broadcast, you wait for the broadcast to show highlights, but we had built this incredible AI model that was able to tell whether something was worthy of being a highlight clip and then put it away. We went in to take a meeting with Mr. Bezos, and everyone was excited to share how many people watched the broadcast and how well it was going. He looked at the room and said, 'You know how I know that you guys are doing a good job?' Obviously a rhetorical question. He said, 'Because of this feature. The reason I know that you're doing a good job is because this feature is purely to delight the consumer. No one is going to watch more, no one is going to like it, but we're not going to make any more revenue. But he built this because he knew that consumers would absolutely love it, and it'll take us a long time to figure out whether it moves any metric at all. But that's important. If you build things that people love, you're definitely going to be okay in the long term.' That shaped a lot of my product thinking. To go back to your question, being a fully remote organization, we spend a lot of time upfront in every product meeting, every time there's a new product or a feature. We spend time upfront with the tenets and other mental models to make sure that we're not losing sight of the bigger picture. If we're aligned on that, then the OKRs and the short-term goals are almost secondary. We feel really good about our roadmap overall. The minutia of 'Does this make a difference if we pull this board to Q1? It may drive an OKR versus if it shifts to the end of the year' — that kind of stuff is where we let our OKRs guide us. But we have a good feeling that our overall roadmap is true to our culture and true to our consumers.
H
Host19:59
Do you do anything to balance that where you're looking at the portfolio roadmap or you're looking at your percentage of time people are spending? How do you also ensure that it doesn't swing in a super different direction where you're not hitting any goals and you're just delighting a bunch of customers but not bringing in revenue? I feel like this is one of those things where product people just fundamentally have such a hard time balancing. They're like, 'How do I make the case for this, but then how do I oversee it and allow people the space to go do that, but not get carried away in a way where we sacrifice the whole company for that?'
C
Craig Saldanha20:37
Great question. I think it's all about balance. We think about our OKRs or goals as what we want to achieve, and our tenets as how we want to achieve them. The way our planning process works is there's a top-down set of goals that we start with, and then there's a bottom-up set of goals that the teams come up with. The top-down set of goals are generally from me, and they're based on what I want to see Yelp deliver in the next three to five years, working back to what that means for next year. The bottom-up goals are really what our consumers are asking for, what our local businesses are asking for, what our focus groups are telling us. Over the course of a couple of months, we essentially marry both of those and come up with the right balance. A lot of it is the ROI, both in terms of the impact it has on our top line and our bottom line, but also the impact that it has on consumer delight. That's how we balance the short-term roadmap versus the long-term.
H
Host21:56
So when you set the goals, do you go back to the teams in those planning sessions and then tell them, 'Hey, tell me what we can do to meet these goals?'
C
Craig Saldanha22:03
Yes. We give the team an initial set of goals as a guide, and that helps order how they're thinking about their work. But independently, they have an opportunity to come up with things that they think consumers need, independent of where the company needs to be. This is what consumers have been asking for, either directly through the various forums that I talked about, or the data is telling us that engagement is great up until this point in the flow and then there's a drop-off that we have to fix. So they have a backlog of items that they know they want to get to. Then we kind of marry that with, as we get through all of these, how much of our goal do we achieve already? Do we achieve all of it? Are there new initiatives that we have to focus on? And then where do we prioritize those on the roadmap?
H
Host23:10
Cool. So one other part that I see with organizations when they're doing this top-down, bottom-up type planning is they have a really hard time aligning all these things also across products or across teams. What do you do to make sure that the people below you, directors of product on different types of teams or VPs of product that might be reporting to you, how do you make sure they're all collaborating too towards the same goal?
C
Craig Saldanha23:32
That's a great question. I'll say that some of the podcasts that you've done on product ops are very interesting to me for this exact reason. Working across teams can sometimes be very hard. The short answer is every team does their best to work out all of their dependencies, and then we recognize that they're probably only going to get 60% of them. So we do have a small product ops team that then owns the overall Yelp-level roadmap, and they really get us from that 60% to 100%. They'll say, 'Well, these three teams say that they have a dependency, but last time team D also had a dependency, how do we reconcile that?' They find areas that are unfunded. We typically do an initial roadmap planning, then we give product ops an opportunity to go in and see where some of the gaps are, reconcile all of that, and then do a final round of planning.
H
Host24:49
Are they like, do they have a product background? The product ops people you have on your team, what type of background do they have? Are they looking at data around what you're covering, what you're not covering? Are they putting into the tools? What do they do to build that final roadmap?
C
Craig Saldanha25:00
First, I have to say they are amazing. They wear many different hats. They have a combination of both product and business backgrounds. They work across product, engineering, sales, marketing, data science, and business operations. They're the unifying force. A very small but mighty team. They're the unifying force of Yelp. I'll just say that I think every year the team gets better at understanding where their dependencies are, but we do have this safety net.
H
Host25:49
When they're looking into dependencies too, what types of work do they do to surface that up and reconcile it?
C
Craig Saldanha25:55
Once they sign a dependency, the vast majority of the time, by simply flagging it to a PM or the engineer or the engineering leader, teams can essentially work amongst themselves. We definitely follow a distributed decision-making process where we focus on context and judgment and really let people autonomously operate in a way that they see fit. In the rare occasions where accommodating a dependency would significantly push out a team's focus, for example, our commerce team does payments, accounting, tax, a whole bunch of things. Rearranging their roadmap is a high-judgment endeavor that bubbles up to me and the CTO, with the team working it out.
H
Host26:54
It sounds like your product ops people are making critical decisions that impact the business, and they have to know enough to be dangerous there.
C
Craig Saldanha27:01
Beyond planning, we also have them be liaisons between teams on our most critical projects, because that's where things can fall apart. When you have product, engineering, marketing, and sales all trying to march towards the same goal but at very different paces and at very different stages of maturity, we have these folks kind of be the unifying force and essentially report out on how projects are going. They also function as program managers for some of these smaller projects.
H
Host27:42
And are those projects that don't quite align perfectly to a specific area of the product, or what kind of projects are we talking about?
C
Craig Saldanha27:50
These are things where, as an example, let's take our multi-location ads product. The teams involved there include the product team that's actually building attribution products and ad products for much bigger companies that need a lot of white-label treatment. You've got marketing teams that are helping market these, and then you've got sales teams that are pitching these. For a new product in the space, you've got to start planning for when sales is able to pitch, you've got to train the reps to be able to go in with the right knowledge, you've got to develop marketing materials. If product slips by a month, now sales has to reconfigure and update all these folks. The product ops folks will play this role. They'll attend all of the standups, they'll understand what's going on in product and engineering, they develop a really good sense of when folks say things are going to slip by a week versus two weeks, and then they're able to translate that for the leaders who are marshalling these huge resources. We have a big sales team. They're basically able to play that role.
H
Host29:16
Okay, so they're streamlining the communications between the different areas and keeping them up to date on the progress from the product roadmap and making sure that's transparent to people. Great, yeah, that's really neat. I love hearing about how other people are doing product operations. I think it's always really fascinating. So we talked about you came in, you shadowed the CEO when you started at Yelp, and then you built these product tenets. What else did you do to think about where Yelp was going? I know when you came in, we talked a little bit about it, but maybe you can tell us about how far they came and what else needed to be done. Yelp had moved in 2019 from a sales-led model to a product-led model. When you walked in and you were trying to assess how far did we get or what's changing, what did it kind of look like for you? Because in my experience, that can take years to actually manifest and keep continuing. What do you think was critical to make that change stick, and what do you continue to do as a CPO today to make sure that you don't revert back to a sales-led model?
C
Craig Saldanha30:23
I'll start by just sharing my mental model for what I thought Yelp needed to be, and then we'll talk about some of the challenges in making that happen. My mental model was actually fairly simple, and looking back, it hasn't changed in two years. There were three things that I wanted the organization to focus on, and I thought we had a very good start. The first was consumer trust. I feel like there are two ways that we earn consumer trust: one is by the breadth and depth of our content that we talked about, so protect our content and make it easier for folks to contribute. The second was helping people effectively define their intent. Folks come to Yelp for all kinds of reasons, from finding a restaurant to finding a plumber. I wanted us to focus on our search and discovery experience, really helping people from the simple things that they told us they wanted to do, get them very quickly to actually making a connection. So that was the first piece: trust. The second was making Yelp enjoyable and engaging. That just came from being a fan for so long. We've done a lot in that space. We rolled out reactions, we just rolled out a home feed yesterday which is very visual and has video now, and just makes the whole experience more enjoyable. We have a personalized log, taking out friction and allowing folks to enjoy what Yelp is offering. The third was to focus on elevating services. I've got an old house and I do a lot of work around the house, and I hire a lot of pros. There's so much friction in the hiring process. One of my focuses was how do we come in and remove friction from the process, but also give people peace of mind. We've launched Yelp Guaranteed, which actually gives you money if you use our service and then find a job not done right. We've introduced a new way to get hold of trades as well. Just three very simple things that the team has focused a lot on, and consumers really want us to focus on that. That was the mental model.
H
Host33:09
I love that. I didn't know there was a guaranteed for Yelp thing. I also have to hire a bunch of contractors because I have crazy house renovations and stuff that I'm doing too, so I didn't even know that existed. But that's awesome.
C
Craig Saldanha33:22
Yeah, if you use our request-a-quote feature that our local businesses offer, then we guarantee your work up to $2,500, which is great.
H
Host33:38
That's awesome, and that's such a differentiator from Google as well, because they don't do something like that. I focus very little on our competition and more on our consumer, so I don't know whether they've started offering that. But I actually use this feature a lot to give me confidence to hire new and undiscovered businesses. We recently bought an electric car and I had to install the charger. I looked up Yelp and found a five-star or four-and-a-half-star business with 400 reviews that serviced my area. I used the request-a-quote feature, and our software recommended I get a quote from this new business. It turned out to be a phenomenal experience. I basically met this pro, he was just starting on his own, his business was only live for like a month. It's such a great story. We really hit it off, and it turned out he was from the city where I was born. In the past, I would have been apprehensive to hire someone that had no reviews and was just starting out, but knowing that this was like a $1,500 install and that I was backed by the guarantee, that actually pushed me to give this guy a shot. He was telling me that he's basically gotten a lot of business from it, which is cool. This is a win for everyone.
Well, yeah, that's what I was going to say. I don't think it only helps consumers. It's definitely what consumers want, but on the other side, it sounds like it's really helping small businesses that are just getting started out there too, to let people take a chance on them. This feels like one of those problems where you solved both sides of that marketplace very nicely.
C
Craig Saldanha35:40
It's still early, but it looks to be that way so far.
H
Host35:43
So when you're thinking about building two-sided marketplaces like this, we have the businesses on one side and the consumers here. How do you think about that balance? We talked a lot about making the consumer happy. How do you also think about what the businesses need, since they're usually the ones who are paying, whether they get advertised or the revenue? How do you balance the two?
C
Craig Saldanha36:11
It's hard. I think the best advice I can give PMs is when you're building a two-sided marketplace, have a clear mental model for how you're going to resolve the conflict. I've worked in three two-sided marketplaces. At Amazon, I launched an ebook subscription service where the two sides were authors and viewers. I worked in Prime Video where the two sides were consumers who were watching and movie houses that were making movies. Now at Yelp, we have consumers and local businesses. I think as long as we have a clear framework, you'll be okay. At Yelp, we always look for the win-win. The metric that we optimize on is connections between local businesses and consumers. At Amazon, we were very focused on consumers, so we focused on giving them the best experience while being fair to the other side of the marketplace. I think you can build a marketplace as long as you have a clear framework for how you're going to resolve that conflict.
H
Host37:32
How did you land on connections between the consumers and marketplaces as that one metric that rules them all, or the one that you really try to go after as the primary metric? How do you get to that?
C
Craig Saldanha37:55
That's been Yelp's mission: to make quality connections seamlessly between consumers and local businesses. The way it plays into this two-sided marketplace is local businesses come to Yelp because there are a lot of high-intent consumers looking to hire businesses, and consumers come to Yelp because there are a lot of local businesses that are reviewed, and the reviews are deep and broad. We landed on connections because that's core to making our flywheel go. I think that's the other big insight that I got as a PM: you need to understand your flywheel before you design your product. If you focus on the wrong thing, you basically design a product which might be successful in moving the metric that you think. We could have easily focused on revenue or something else, but we focus on connections, whether it's advertisers or non-advertisers, whether it's a phone call or an email or a request for a quote or going to their website. We want to know that we made a meaningful connection between a consumer and a local business, because we know that's what drives the flywheel.
H
Host39:21
The flywheel concept, I feel like, is incredibly important in product management but not super well understood. Can you walk us through how you would, if you were coming into an organization like you came into Yelp, how do you figure out or how should a product manager figure out what their flywheel is? How do they model it so that they can wrap their head around it?
C
Craig Saldanha39:46
Product managers are always looking for product-market fit. That's the core thing that we're taught to look for. In a two-sided marketplace, a lot of the time, product-market fit comes from growth. You need to be big enough for one side of the marketplace to attract the other side. I think of the flywheel as what is the mechanism that's going to deliver sustainable growth over time. That's how we think about product initiatives to drive the flywheel. We think of a flywheel as self-sustaining growth, and we think of the product levers that will help drive that growth. That's how we come up with the two-sided marketplace roadmap.
H
Host40:36
Okay, so for Yelp, you've got your growth metric is trying to make those connections. The more businesses and consumers on the platform, the more you can do that, those connections is what people keep coming back to. How do you think about the product levers connecting that?
C
Craig Saldanha40:52
We think about both sides of the marketplace. For a consumer, we think about the needs that they have to satisfy to make a connection. If you boil it down to the very basics, most consumers, when they're thinking about connecting with a local business, have three basic parts. The first is quality, whether it's the quality of the food or the quality of the service. They think about quality first. The second is price. Everybody wants to know how much they're paying. The third is timing. If it's a restaurant, am I going to get a reservation? Am I going to have to wait to eat when I want to eat? If it's a pro, is this an emergency job that they have to get done now, or can I wait a week or a month? We think about how do we give them all of the information with the least amount of friction to address their questions on quality, price, and time. On the pro side or local businesses side, what they've told us they're looking for in quality leads is high intent. Is this person really interested in connecting with a local business? And service area: are they in my service area? Do I provide the type of service they want? Our product is really focused on taking both sides of that marketplace, taking the consumers' needs, taking the businesses' needs, and essentially matching them. That's what we do.
H
Host42:32
The part that I love about you explaining this is that it really all goes back down to deeply understanding your customers on both sides of this marketplace, what they want, what their problems are, which takes a lot of user research and empathy and really modeling that out. How do you instill in your organization as a Chief Product Officer the concept of user research and customer research? What kind of research do you do? Who's going out and talking to people? How do you make sure that you're keeping in touch with your users?
C
Craig Saldanha43:06
The customer obsession piece is something that we look at from the interview stage. I think most PMs are customer obsessed, with a real hunger to understand what customers want at their core and then find creative ways to remember that. I feel that's very core to our organization. We talk about it in every meeting, we talk about it when we're making trade-off decisions, it's in our tenets. We really focus on that. We have a really good research team. For products where we're not quite sure how consumers will react, we do a lot of testing. We have sessions set up where they show prototypes and they give us feedback. Beyond that, I think we're very blessed that we have a very large, engaged, and vocal consumer base. You talked about Yelp Elites. They're always giving us feedback about what they like and what they don't like, and we take that very seriously. We have listening sessions where PMs will listen to calls of local businesses calling into CS to understand what their pain points are. Sometimes the pain points are that we just built a product that's too complicated for them to understand. Some of the folks we're building for are sophisticated and have been in business for 20 years, and some are just really good at being plumbers and don't know how to navigate our product. We have PMs listening in to those calls all the time. Even me, I will end every single week with the voice of the customer. I have teams that go through all the feedback that comes in, and the top 20 or top 50 they'll send to me for review, with screenshots or Twitter or a detailed transcript. I spend time looking at what is bubbling up to the top. I think we have a pretty robust mechanism in place, but you can always do better. Consumers never come to a product and say this is the best possible product. There's always ways to improve.
H
Host45:55
There is this big debate going on right now about companies letting go of user researchers and seeing that user research is not important from a user researcher perspective because they're trying to have all the product managers replace it. This is what I'm hearing from user researchers. I don't agree with this, just to put that out there. You mentioned you do have a research team. How do you see user researchers and product managers working together on these things?
C
Craig Saldanha46:22
My personal opinion is that user research is very powerful, and regardless of who is doing it, we should never let go of that. Testing and stuff is great and gives you the quantitative answers, but sometimes the qualitative answers are what you're looking for. As a PM, you have to feel the pain of a customer in order to understand how to simplify a product. We find that it's most powerful when it's directed to solve a specific problem. When we're looking at why a drop-off happens at a certain point, and we give our researchers the tools to go and come back with answers without really biasing them, we find that very powerful. We find it less effective when it's a broad question like, 'Do we think users are going to respond well to LLMs?' That's not a useful prompt, and the research that comes back then is not helpful to help product managers redesign their thinking. I always think of it as a redesigning step, as opposed to an ideation step.
H
Host47:41
That's really insightful. I like the way that you think about that. Thank you so much, Craig, for being with us here and for sharing your story at Yelp and how you think about product management. I think a lot of people are going to listen to this and have so much to take away from it. If people want to learn more about you or product management at Yelp, where can they go?
C
Craig Saldanha48:03
First, I'd say sign me on Yelp. I always have a bunch of recommendations. And then LinkedIn as well.
H
Host48:10
Great. We will put all of those links so that you can find Craig in our show notes at productthinkingpodcast.com. Thank you so much for listening to this episode of the Product Thinking podcast. We'll be back next Wednesday with another wonderful guest, and I'll be answering all your questions if you send them to dearmelissa.com. So make sure that you go do that, and we'll see you next Wednesday.