Season 2 Ep. 16 Amit Prakash of ThoughtSpot on empowering companies to make data-driven decisions

Yeah, it's been an interesting journey. In a little bit unique, I should say. In some ways, the journey started with my father. He was a professor in engineering college and he came here to do his PhD at Purdue, and he got really, really sick, and then he had to go back. As I was growing up, there was this dream that someday I'm going to go to one of the top universities and do a PhD. That was pretty much with me since I was young. I had to do it, and I was planning to follow the footsteps of my father and be a professor in engineering college and do some research. But by the end of my PhD, I had taken a very practical problem, and I liked theory, and in the end I had produced a solution that was very theoretical—I was proving lockstar inbounds and things like that. I realized that what I produced was very elegant theory but not very practical. That got me to want to work in the industry for a little bit before jumping back into academia. Once I went to industry and started working on interesting problems, that sort of attracted me so much that I decided to just stay. I continued to work with some of my PhD advisor's students and do some academic work, but it was just too much fun in industry to work on hard problems. At some point I was like, okay, I like this, but what's the next thing that I want to do? The natural answer was to go start a company and invent something new, and that's how my startup journey began.

Yeah, yeah.

So I was at Google from 2007 to 2012. I was working on the ads team, and that was building the click-through rate models for AdSense. We were training really, really large machine learning models. It was more of a systems problem than really a machine learning problem. We would tweak around with the algorithm and the parameters a little bit, but primarily it was how do you pump billions of training data points—actually trillions of training data points—within a week or so through distributed gradient descent code. That's what I was doing over there. That was the very beginning of the Google Brain team forming, and I remember talking to Andrew Ng and Jeff Dean at the time when the team was getting formed.

So I was talking to my co-founder, who had co-founded Nutanix and spent some time at Aster, and he was talking about when we start a company, I think the best thing to do is to first narrow down a market, because that's the most important thing. Let's go pick a really large market with important problems, and then let's go pick a problem in that space that we think we can solve better than anybody else, and then let's go work on that problem. Analytics seemed like a really great market—a lot of enterprises care about it a lot, it's been one of the fastest growing segments. We looked at some of the surveys from CIOs, and we realized that for the last three years consistently, CIOs have been saying that analytics is the number one area of disruption. So that's how we kind of zeroed in on it.

Yes, so in some ways it's really simple stuff, but getting it right is probably the most important thing. What happens in industry mostly is that if you draw two Venn diagrams of people who really know business—people who are making business decisions and they see where the competition is coming, where the market is—and people who know data, they are completely two different sets of people, and the intersection is usually very narrow and thin in the middle. As a result, even though there's been massive investment in data infrastructure and collecting data and making it queryable, people are not able to get the benefit from it. Imagine, one of our customers is Canadian Tire, one of the biggest retailers in Canada. During COVID time, the demand was changing rapidly for them—one week it's toilet paper, next week it's exercise bikes, next week it's men's shaving equipment. They were supposed to respond to these things quickly. It was not just that the demand was changing, but the supply chain was also changing. In traditional systems, to deal with that, you usually aggregate the data to scale where you can interact with it, and then you put in tools like Tableau and Click where an expert can go and interrogate data. But that creates some distance between the people who need to make the decision and ask the question. Typically it might take a week, in some cases a month, before the right person can get the answers. So the primary mission for us has been to empower the people who understand the business, who understand the significance of data, to be able to interrogate data and get answers to their questions.

We were extremely lucky to have, very early on, assemble some of the really talented people that you could find in the Valley. We quickly added another five people who were really amazing engineers and had a lot of experience doing different things. We had someone who was one of the architects of the BI systems at Arco, we had someone who was one of the leads for Google's backend query engine for what's called Mustang internally, we had one of the leads who was building the orchestration systems for Google data centers.

So early on, we spent about six months writing very little code and just imagining what the solution is going to be. It wasn't obvious how we were going to solve this problem of non-experts being able to get to data as fast as they could. We traded through many different ideas. In the beginning, we thought about using natural language processing to translate questions into SQL. But we said there's no way that we can do this and make it into a successful product, because given the state of technology, even the best research teams with hundreds of engineers can only get to maybe about 80% accuracy. If people are asking business questions and making important decisions, there's not even a 0.1% chance that you could take of giving the wrong answer, so it has to be very deterministic. So we scrapped that whole thing and went in different directions with different UX ideas to make it simple. Then one of the people who joined us said, 'You know what, I'm writing code. The simple autocomplete in Eclipse is extremely helpful for me to write code. So if we move the state of the art significantly towards a non-technical user being able to ask questions, that will be an amazing product.' That became the inspiration for what we did next. We built a Google-like interface to be able to ask questions, but behind the scenes it was a very deterministic system. You could think of it almost as a DSL factory—something that can take the entities that the customer would care about, everybody in that company would care about, and build a language on the fly that allows them to ask questions in natural language. That became the foundation of our very first product. We went to many potential customers and showed it to them in just design mockups, and it resonated very well. Then it took a long time to build, about three to four months, and then we did our first user study. It was really amazing to see the kind of response from the business users who had never been able to interrogate questions. The response we got was, 'Our jaws are on the floor.' That was a light bulb moment—okay, we are going in the right direction. We have a lot more to build, but let's continue building.

Yeah, so I was talking to one of the largest car manufacturers in the world. As you can imagine, they have a fairly sophisticated operation with parts coming from all over. It just gets hard for them to figure out what's the optimal way of procuring something unless they can quickly ask a question and then ask follow-up questions. They were comparing their life before ThoughtSpot and after ThoughtSpot, and they were saying that they're able to get these decisions much faster.

So there are three things, the three pillars on which ThoughtSpot stands, and each one is as important as the other. They all come together. The three pillars are: building large-scale systems that can handle a lot of data; the machine learning and AI part that makes it intelligent and smart; and the UX part that makes it intuitive for users who are not necessarily the most technical. On the systems side, we build a really fast, really efficient indexing system that can index pretty much everything the user might care about. This could mean anything from names of products to names of customers to street addresses and product categories. It builds a map of these entities—which column they live in, how do you join this thing to this thing. There's a lot of computer science innovation that went into the systems piece because of two reasons. One, we were shooting for a Google-like experience, meaning every keystroke returns under 100-200 milliseconds so it feels natural. But the other thing is that different users have different access to data. For example, a store manager might be able to see all the stores in California, while the supply chain manager can see all the data around supply chain for the United States but nothing else. As someone hits a keystroke, you need to go in the back end and look for everything that they can access. This sort of overlapping sets indexing becomes a really hard problem. You're trying to run essentially a compiler on top of it. The next piece in the systems is this DSL factory that builds a language and a compiler and auto-completion system for it on the fly. It's a very constrained language—it doesn't contain a comma or a bracket or two spaces in the end—people are just typing those things. The third piece is more around ranking, and this is where some of the machine learning comes in. We're trying to guide a user to be able to ask meaningful, relevant questions in a language that they have no idea it's actually a language. They're just thinking in terms of their business entities. Being able to surface within a few keystrokes exactly the right entities, tokens, and questions is the most important part of this UX. The choices are very limited so that the user is not bombarded with choices. They progressively build their question without knowing that they're actually doing this progressively. That's the search piece. The other part that I haven't talked about yet is the automated insight piece. After we had mastered the search piece, the question was: there are a lot of things where the user could later wish they had asked a question but they didn't think of asking that question. How can we help the user there? We build models that say if the user asked this question, what are the next likely questions that the user might want to ask? We generate a thousand hypotheses and then evaluate them to see which ones are interesting and which ones we should surface. That's the core stack. There's a lot of other things that go into running a large-scale data system efficiently and making it available all the time, but as far as the differentiating part of the stack, these are the things that go into the system.

Yes, so there are multiple sources of information that we have to combine together to be able to do this ranking. The first one is really where having a surgeon in the product really helps us. Just like at Google, if lots of people ask questions about dogs and then their very next question is about puppies, then you know that dogs and puppies are related and you want to guide users from dogs to puppies. We're doing the same thing: if lots of people are asking questions about revenue and their next question is about profit, then we know revenue and profit are related. But for new customers, they don't have that source of data to benefit from. The next one is purely statistical: if you're seeing a large anomaly in the data, then obviously that's going to be interesting to the user. We're looking for either large anomalies or some interesting correlation or some sort of interesting trend that stands out as an anomaly—like your entire business is growing at 5% but your fidget spinners seem to be growing at 200%. The third source is user feedback. We have a feedback system where users can say they don't like a particular insight, and we use that to infer what was not interesting. Think of it almost like Facebook posts: you can say 'I don't like this thing' but you can also say 'I don't like this thing because I just in general don't like posts from this particular user' or 'I don't like posts on this particular topic' or 'I don't like this specific post but everything else is fine.' So we have that kind of feedback system that flows into this ranking, where somebody could say 'I don't care about costs, I just care about revenue, so don't show me anomalies about cost' or 'I do care about revenue but you're showing me revenue anomalies that are not interesting.' Taking all that information together, we do the ranking.

It's one of the problems that I'm really passionate about, and I've spent quite a bit of time finding a solution for this that can be turned into a product. We also have an alpha version of this thing out there where people are trying it. But I think it's going to take a lot more than something like GPT and BERT to be able to solve this specific problem. For lack of a better word—and I know some people frown on it—I think the solution to this problem really lies in how we can represent knowledge efficiently and how we can capture knowledge efficiently. GPT is capturing a lot of real-world knowledge, but I'll give you an example. I was talking to an airline company, and they were trying out our alpha product and being able to ask questions in natural language. In their domain, 'A0' means average departure delay for a flight segment, and 'D0' means average arrival delay. When they ask 'What's the A0 for DFW?', that means what's the average arrival delay for any flight segment where the arrival airport was Dallas Fort Worth. When they say 'What's the D0 for DFW?', that means what's the average flight segment delay where the departure airport was DFW. Only the people in that company know this correlation—that when I say A0, this filter changes to arrival, or when I say D0, this filter changes to departure airport. They're busy people, they're not going to generate a million training examples for you to be able to learn this context. This is the determining factor that's affecting this thing. Things get arbitrarily complex. I was talking to one of the travel agency companies, and one of their prime questions was 'How many of my customers are in New York right now?' New York happens to have 17 different meanings in their data set—whether it's the arrival airport, departure port, hotel, their office headquarters, the travel agent's headquarters, city, state, all the combinations. In this specific case, what they really want is that their departure date should be today. So the problem is about capturing the context and representing it and presenting it. That's one aspect of it that I've spent many cycles trying to understand what would be the right UX to capture it, what's the right data structure to capture it, what's the right way of inferencing. But we're not close. The other thing that's very interesting is that even though natural language is fantastic for short questions, when people go deep into analytics, typically the questions grow very, very fast. At some point, very quickly, the questions have no equivalent in natural language. For example, 'Show me all the products that were returned and discount the revenue by for each partner by the margin that we are providing them'—that kind of thing is very common. If you look at Google queries, the average query length is like three words. If you look at BI queries, the average query length is like 20-30 words. At that point, natural language is too noisy and ambiguous to even communicate precisely the meaning of the question. If you took their question and repeated it back to them a week later, they wouldn't know what exactly the interpretation for this question should be. So I think building a 99.9% accurate translation layer from natural language to this DSL will definitely move the state of the art.

Most people have either learned or it comes naturally to them that if they're looking for Starbucks near them, they just say 'Starbucks near me.' They don't say 'Give me a list of Starbucks within five miles from here.' The same thing happens in our search engine. If you want to know revenue for the last three weeks, you just type 'revenue last three weeks.' You think of the important entities in your question and just type those entities, and you'll get the answer. You can say 'revenue for North America last three weeks' and that will give you the answer.

That's a very interesting idea, and maybe in some time we will. But I also right now at least struggle to see in that unsupervised way where do you get that information efficiently from. For example, when I say 'What's the longest movie ever?', does that mean the duration of the movie or the length of the name of the movie? If I have an unsupervised data set where people are just talking about movies, will I be able to capture that inference to correctly translate that question into 'duration of the movie'?

In some ways, it was somewhat counterintuitive in the early days, but it just happened to be exactly the right thing that the industry needed, and we are seeing the results of that. When we were getting started in 2012 and we wanted to build this product where you ask a question and get back a response almost like Google, part of that deal was that the response should come back in the same latency as Google—half a second or a second. We were trying to solve the problem for the entire enterprise, which means that some enterprises have tens of billions of records that they want to interrogate routinely, and you can't pre-aggregate that because that constrains what kind of questions you can ask. So we built our own database layer where you load all the data and we hold that in memory across possibly hundreds of nodes. When you ask the question, we translate that question from this DSL all the way to very efficient C++, do a compilation just in time, and then run it over these in-memory data structures and come back. That served us well. But with a lot of enterprises moving to the cloud and solutions like Snowflake coming in, we saw that these databases were pretty fast and they kind of served the need we were trying to fill. For the enterprise, it's much better to just have one data warehouse where they put all the data, as opposed to having multiple copies. So we partnered with Snowflake. The reason this partnership worked out really well was that we were the only analytics tool that was built on the assumption that the database is going to be super fast, super scalable, and can handle large queries over large volumes of data. We were architected that way. When we started sending queries to Snowflake instead of our own internal database, it just fit right in and created much better integration than what anybody else could create, because most of the other tools in the industry were designed for a world where databases were much slower or dealt with much smaller volumes of data. In some ways, Snowflake removes the constraints around data, and we have a lot of customers who love this combination.

That's a really interesting question. There are probably three different directions that we're going to pursue, each one in some sense is kind of an AI-complete problem, so we'll keep making progress, but until general artificial intelligence is solved, they'll be somewhere in that spectrum. We already talked about this problem of being able to ask questions in natural language and being able to answer them. One way to approach that is to realize that most questions that people ask are just a variation of a question that was asked before. If we take the history of all the questions that were asked by the experts before and figure out the right way to ask that question in natural language, then every question in its neighborhood can be answered. That's kind of our putting product—how can we build a good solution from the technologies that exist today? The second direction is around SpotIQ, which is our automated insight engine. There, I think it's a very interesting problem. If you look at all the advancements in AI in the last few years, I don't know if many people are training very deep neural networks and getting benefit from it by trying to figure out what's the best decision for their business next. Part of it is because a lot of context is still in people's heads, and the kinds of problems that people use predictive techniques for are still somewhat low-dimensional problems, like trying to forecast your revenue. We're trying to figure out how we can leverage some of the advancements that have happened in AI in this particular domain to help a marketing manager or finance controller spot patterns in their data and benefit from that. A lot of the problem again lies in how do you capture the context that's in somebody's head to be able to give them something meaningful. For example, if I tell someone in retail that your revenue from California is an outlier, they're going to say 'Tell me something I don't know—California is the most populous, one of the richest states where I have the most number of stores.' So how can I capture that context in automated and efficient ways so that when I present insights, they're more useful and meaningful and don't just look like random anomalies in data? The third direction is again about capturing business context. As great as we think our tool is, for anyone to be able to benefit from it, there are a lot of initial steps involved in bringing that use case up. For example, somebody wants to analyze their sales pipeline. They have to define what a lead is, what an opportunity is, what the stages are, what the conversion rates are—that's when it gets ready. We're looking to figure out ways of making that push-button, so that for people doing similar tasks hundreds of times across different industries, we can capture that knowledge and automate that whole process. As soon as you connect our product to your data, it knows most of the business context and can be ready for you to interrogate it, as opposed to an expert providing that knowledge.

Yeah, that's a really interesting direction to think about. We recently acquired a company called SeekWell. The reason for doing that is that people don't just ask questions for the sake of asking questions; they ask questions to take some action. You may decide to send out emails to a bunch of your customers about a particular topic, or you may decide to drop a promotion, or in a procurement system, you may push certain items to be ordered with quantity. Right now, we're providing convenience: you may have asked the question 'What are the products on which inventory is running low and what's the difference between the forecasted demand and the current inventory?' and then once you build that data set, you push that to the procurement system. But we want to close the loop so that the action taken is recorded and we can learn from it.

We were very, very fortunate in our relationship with our investors, and in particular Lightspeed. They trusted us from the very beginning and have been amazing in supporting us. Even before we had a concrete idea that we wanted to pursue, because Lightspeed was also an investor in Nutanix, which was my co-founder's previous company, they were willing to back us. But we had to hit the right milestones and show success.

Two things. One, in general, things have gotten so much better for entrepreneurs than in 2012. There's a lot of desire to invest in brilliant tech entrepreneurs, and there are so many problems that are now accessible because you don't need a really large investment to build a solution. Once you jump in, the infrastructure is so much better than before in terms of supporting would-be entrepreneurs. The other thing I feel is that there are two ways to start something. One is to have an idea and just take out six months of your life investing in it with a couple of co-founders, building something, and then going and showing it to someone. The other is to have an idea and start talking about the version and validating it with people, and once you have some validation, then go and build it. For engineers like us, it's even more important to get the problem right. Spending more time early on to get the problem right, even before you've written the first line of code, is one recommendation that I would have. Once you have a concrete problem statement that you believe in, you have conviction on it, and you have some validation, it's a lot easier to have those conversations with investors.

Yeah, yeah. So I think getting the market and the problem right is probably the most important thing. If you can get those things not necessarily right but in the right zone, everything else from there—whether it's raising money, whether it's recruiting people—all of those things become relatively easier. It's still entrepreneurship, and you're still shooting for something really ambitious, so there's going to be a lot of struggle, but it's definitely one way to derisk your venture.

Yeah, yeah. It's not something that I've always been able to do, but in the last couple of years I've made it a point to do that, and it's been really rewarding. I guess partly I have the pandemic to thank for that because all the commute is out, so you got to use that time for something. When I was at Google, I felt like I was at the forefront of where machine learning was at the time, and then I got busy with ThoughtSpot, and so much happened in this field that I felt like I was falling behind. So I started using that time to learn, whether it's about AI or politics or whatever. That extra hour that we saved from commute has been really fantastic for me to learn, and it gives me a lot of joy, and it's useful as well.

No, so I think in general, the day-to-day fires always take priority. You have your customer meetings, you have your strategy meetings. In general, those things always take priority. But I do make it a point to set aside some time. If it gets preempted, it gets preempted, but it doesn't get preempted that often, so that works well. The other thing I was going to say is having someone like Smither, who's leading the engineering for us, and I'm not in the day-to-day management of engineering, helps a lot as well. Basically, hiring amazing leaders allows you to focus on other things.

Thank you so much. I really enjoyed it. It was a fun conversation.