How Glean CEO Arvind Jain Solved the Enterprise Search Problem – and What It Means for AI at Work

Majority of the work that we do today is not going to be done by us anymore in five years from now, and that applies to me, that applies to you. We both do very different things, but still, I think we are knowledge workers, and I think a lot of our work is actually going to be done by these amazing AI assistants that are actually, in many ways, more powerful than us. They have access to all of our company data, our knowledge. They have all the context from all the past conversations and meetings. They don't forget anything, and they can really sort of... and on top of that, they have the reasoning capabilities that allow them to be super helpful to you in any task that you do. So that's sort of our core belief: that majority of our work is actually going to be done by these AI companions or assistants, and we want Glean to be that assistant in the workplace.

Yeah, first of all, thank you for having me. Glean, think of it as the Google or ChatGPT but inside your enterprise. It's a place where your employees go and ask questions, and Glean answers all of those questions using your company knowledge, regardless of where that knowledge is. It brings it all back to you. So that's what Glean does. Glean is also an AI platform, so if you want to actually build AI applications inside your company, you can use the Glean RAG platform to build those applications quickly.

Well, I mean, search is hard because it's actually magic in some sense. You can come and ask any question that you have, and you expect the system to actually give you back the right answer. So expectations are always high, and it's a difficult problem, especially in the enterprises because there's so much information inside the enterprises spread across so many different systems. It's both hard to actually even get hold of that information, but then even harder to actually make sense of what information is actually good, what has become out of date. So there are lots and lots of challenges around building that system. And in the past, I would say there were no good attempts made because the problem was so hard. It requires so much R&D, so much investment. It was not really startup-friendly in many ways. And you also couldn't even, in the pre-SaaS world, you couldn't even build a product because just getting connected with all of your enterprise data meant that you had to spend a year sitting with an enterprise trying to actually bring the data into your search system and then actually solve the real problem, which is making that information searchable.

Absolutely. So as you said, search starts first with the data and the knowledge that you need to actually make searchable. So the first part of the Glean tech stack is these deep integrations that we've built with most common enterprise systems. So think of systems like Salesforce or Confluence, Jira, Google Drive, SharePoint, ServiceNow. Your enterprise data typically lives in all of these different systems, and you need to bring it all together in one place. So that's the first part of our technology stack: these integrations. But then if you think about enterprise data, and this is one of the most unique things about enterprise search versus if you think about Google search on the web, is most of your enterprise information is actually private in nature. When you author a document in Google Drive, this document may actually be private to you, or you may share it with a few other people. And so you can't build a search engine where you just dump all the company knowledge and make it accessible to everyone. You have to actually understand permissions of each content. So when you go and search, the system should understand who you are and only retrieve information that you actually have access to. So that's our governance layer: understanding governance across all of these hundreds of different systems, which is quite complicated. And then the third part, and this is where really most products have failed in the past, is search is not about just putting a whole bunch of documents in an index and then when somebody comes and asks a question, take those words or take that question and just match it up semantically or with keywords with the right content. You got to actually also understand who's the person who's asking a question. I can come in and ask for an onboarding guide because I'm a new employee, but then which onboarding guide should be actually given to me? It depends on whether I'm in the marketing team or I'm in the engineering team. So understanding people and understanding knowledge and relationships between them, that's a big part of actually making search or a question answering service work inside an enterprise. So we do that. We actually build a deep knowledge graph where we look at all the employees, understand what roles do they play in the company, look at all the documents, and then we sort of try to understand what documents are meant for what departments, what documents actually are popular, what are the relationships between a particular individual and a particular document. And that is what we use as a core foundation that sort of then governs like when somebody comes and asks a question, what are the most relevant pieces of knowledge for them. So we had to do all of that work. And actually, interestingly, you mentioned what came before AI even became relevant for us. The AI was actually part of the core search technology from day one. We were actually working with LLMs in 2019, or I mean at least language models. These were BERT-based language models that you could use. And so when Glean... LLMs, that's right, is that the name for them now? But yeah, in the search engineering community, we just called them language models at the time. And so the language models were actually part of the core search experience from day one because it really allowed us to understand content at a semantic level. So that was sort of already baked in our core search experience on day one. When we were actually trying to look at a question from a user, we were never sort of limited by the actual exact keywords that users used. We were able to actually understand the meaning behind the question and actually match it up with the right documents. But still, that's sort of all the work that you have to do before you can actually even do anything with LLMs.

Yeah, so that's of course the core of the product: all the effort that we put in to actually build a really good ranking system for our search. I'll give you some examples of the kind of things that go into determining what documents are the best ones to rank for a given question. So of course, if you imagine that there's a document that people in the company are constantly looking at, so that gives you obviously a signal that there must be something about it, it's actually important, people like to spend time on it. If there's a document that was actually written in the last one or two weeks and there's some engagement around it, this again is information that people care about, it's not actually become obsolete yet. Then if you think about a particular document that we see is not popular when you look at a company level, but you look at one individual team inside the company, we see there's heavy usage for that document inside that particular team. So that sort of tells us more about that, hey, this document may actually be relevant for this particular set of people. Or the last example: imagine that somebody had a question and they're not bothering to search, they go and Slack and ask a question, and then somebody else posts a link to a document as a response to that, and the person who asked the question gave a thumbs up to it. Just imagine this interaction, like what it means. It actually means that that particular document was actually a really good answer for that question that the user had asked. And so if you keep that association in mind, it's going to help you later when somebody else came and asked a similar question. So those are some of the signals you have to sort of constantly look for. All these signals, they have to collect them differently in the enterprise setting than on the web. Google only has to look at all the activity that's happening right on Google itself because that's the gateway to any sort of knowledge quest. But if you look at in the enterprise, not all the things are happening through the search paradigm. So you have to sort of go and look at all the activity around all the knowledge in different systems, like your communication systems, your document systems, and just try to learn from that human behavior because ultimately that's how you learn. You learn from what people are doing inside the company. The more you can actually collect that information, the better your ranking systems are going to be.

Yeah, well, so first of all, I think since you were talking about a five-year-old, let's first talk about what RAG is. A ten-year-old? Ten-year-old? So yeah, no, let's keep it five. I need five. Let's start with five. Yeah, so I mean, if you think about all these amazing models, you know, GPT and Gemini and Claude, they're all... these models are all trained on the world's public knowledge and data. And so if you were to actually go in ChatGPT and ask a question like, hey, how many days do I get off with my PTO policy, it has no idea. It can't answer that question because that's my company's private knowledge. The answer is somewhere there, and the model is not trained on it. So how do you bring your private enterprise data to these models so that you can actually have AI create that magic for you? That's what a RAG-based AI application architecture allows you to do. So the way it works is that you come and ask a question, and you have a search engine or a retrieval engine, whatever you want to call it. And given the question, this retrieval engine finds potentially relevant documents that could actually answer your question. And then you're going to take those documents or those sort of content fragments and make the model work on it. You'll tell the model, like GPT, that hey, I have this question and I have this company knowledge that I think is relevant in terms of answering that question. Now you answer that question using this knowledge. So this is how most AI applications today are being built in the enterprise. The only way to actually connect your private enterprise data to the power of these language models is basically a search engine that's sort of sitting in the middle. So we, given that at Glean also, we of course built a search engine over all of your enterprise content in the last five years, it actually allowed us to become one of the best RAG systems that allows you to now not only, of course, we deliver our own end-user application, which is a Glean assistant using this RAG-based application architecture, but we're also allowing companies to actually build more and more applications using RAG. Now, I think in terms of while this is the architecture that is sort of emerging as the canonical architecture for building AI applications, I think it is still full of challenges. It's actually really hard to build great AI applications using RAG because one of the things, how models themselves are sort of, while very powerful, they're also still an emerging technology. Models hallucinate, they make things up. And what you're doing now is you're actually adding one more complex layer of technology in this application architecture. So think of it like you're chaining two things which are both not perfect. So oftentimes you will see a RAG-based AI application not perform well because you asked a question and the failure actually happens at the RAG stage, at the retrieval stage, where you didn't even actually were able to find the right pieces of knowledge, or maybe you found stale information that then you're actually giving to the LLM to work on, and then of course it's going to give you bad results. So while it's the only way to sort of bring knowledge together, it creates these interesting challenges for you as well.

So I think of search and RAG as being, in some sense, they're one and the same thing. The real core technology is taking all of your enterprise knowledge and putting them into a search system where now you can actually ask questions and the system is able to actually give you relevant pieces of information back. So that's sort of the core technology. Now you can actually use this technology either as a standalone product, so that's what the Glean search product is, for example, where people come in, they ask questions, and we can give them the relevant documents that potentially are useful to them based on that question. Or you could actually use this as an API layer in your overall AI application, where the search system, the search module, now is only one component of your overall AI application architecture. And so I think in that sense, it's sort of similar. But the industry, on the other hand, I think what we've seen, most of these RAG-based applications in the enterprise today, they actually use a much more simpler version of retrieval system in their RAG application, typically a vector search-based system which doesn't really have full enterprise context. And so that's the key difference. So for us, our approach always has been that really think about how to build a standalone search system, something that is as good that you can actually put them in front of the users as a standalone product. That's really the real test for how good the search is, and that then actually when you put it behind the scenes in a RAG-based application, it's actually going to create better AI experiences.

Yeah, yeah, that's correct. I mean, I think it's really hard to build these systems yourself and build them in a matter of weeks. I think you can build a great AI demo in one day, maybe in two hours now. But I think to actually build where it's robust, it's stable, it actually adds value within your enterprise, it's a hard problem.

So Glean is actually a product that is used quite heavily by people. There are many different types of things, we're often surprised by what people are using Glean for. But I would give you a few examples. So for engineering teams, they find Glean super useful in terms of troubleshooting. Whenever you run into any kind of a roadblock or an issue, like sometimes an error, your programs are not working properly, and so Glean serves as a really good troubleshooting tool for them. It's a place where you go and debug because more often than not, you're not the first one who's going to experience an issue. Somebody else has experienced those issues before. So just getting the context from all the other people and how they solved that problem before sort of helps you solve that issue for yourself. So that's a big use case for engineering. For some roles, like for support, their life day in day out is about resolving, answering people's questions. And I think the tool like Glean actually fundamentally changes how they work now because by default now they don't think about trying to go and look for answers in different knowledge bases and whatnot. Instead, the first reaction that they have now is that there's a question that's coming in from a customer, and then Glean on the side is actually already answering those questions for them. So their model of working changes from trying to find things to actually trying to validate what AI is telling them is the right answer and then just share that back with users. Some teams have actually really changed their behavior. Salespeople, for example, they use Glean as a way to prep for meetings. So before a customer call is coming up, they will just ask Glean, they can ask Glean, help me prep for this meeting, and Glean is going to bring that 360 view of all of the data from that customer, like what happened in the last meeting, what opportunities are open with them, and things like that. So it sort of really helps them prep for a meeting, then actually run a meeting well, because customers always have lots of questions, and so the salespeople feel more confident running that meeting because if somebody throws a curveball at them, they can just ask Glean right in the meeting, get the answer, and quickly give those responses back. So in fact, in our company, we don't allow salespeople to actually bring in sales engineers in the call, they have to answer those questions themselves. So that's one change in behavior that we drive in the first few calls. Yeah, those are some of the things, but overall the use cases are unbounded. The one that I think is the one that is universal across everybody inside the company is finding other people who can help you. That is one of the things that Glean makes it really easy for people. We help you connect with the right subject matter experts based on what questions you have. So that's one thing that we see everybody in the company make use of a lot.

Yes, so our key metric is how many questions people ask on a daily basis and actually get successful, like we were successful in answering those questions correctly for them.

Yeah.

Well, we have... so we have this technical metric, I don't know how much sense it's going to make, but we tend to actually keep that number at 80%. So I think it's a proxy for that 80% of the sessions that users had with us, they were actually successful in getting what they needed.