Arvind Jain (Glean): Solving enterprise search with AI

Thank you so much for having me here.

Yeah. So we started Glean in early 2019 and this was exactly the problem that we started out to solve, which is how do we make it easy for people to find information inside their company. The reason it's hard is first I think if you look at enterprises, there's lots and lots of information and data that they have internally. Often times it's spread across hundreds or thousands of systems internally, and a lot of that information is produced over a long period of time. Like a business that's been in existence for the last 30 years, you're constantly producing information, you never delete anything, so what you're left with is a fragmented mess of information in some sense. Some high quality, good up-to-date information, and there's a lot that has now become obsolete and spread across so many different places. To build a search system on top of it, it's ironic that the amount of information in business is actually not as much as the information on the internet. But on the internet, you can actually easily connect. Web pages are all accessible, all you need is a browser. You can hit a URL, you get to a web page. But internally in enterprises, your systems are sort of siloed. You don't have an easy way to actually connect into each one of them and enumerate and list all the content so that your search system that you're going to build can actually understand what knowledge is even there. So that's the first problem that you have to solve. As you know, we went through this SaaS transformation, the number of applications and systems that businesses ended up with went from tens to hundreds to thousands. And so that was one of the key problems: how do I even get connected to all the enterprise systems so I can know what information is there and I can actually search on it. That was just one problem. The other problem was that on the internet there are a lot of great signals. You know how popular a website is, so you can actually rate it higher in Google. We would see all feedback because Google is a gateway to the world. So when you ask a question, users will always tell us what sites they prefer, and so we have a lot of input to work on to figure out how to prioritize, how to rank certain websites over some other one. But in the enterprise, you don't have that signal. There aren't enough people, and you also don't have a notion of a gateway where you can actually track and observe what content people are really using inside the company. So you had to sort of work around that a little bit. These problems were never really looked at. All the search products in the enterprise were basically just the same product that we were actually trying to search on the internet for. So you would not go and try to understand the enterprise. You wouldn't go and see what content is actually high quality, written by subject matter experts. Those were the things that you needed to disambiguate what content matters for what question that you have at hand. I think the problem was not solved because people didn't bother to solve it. When we started, we went about it like any engineering project and we were able to go and figure out how to solve these engineering problems and make progress. So ultimately we did build a great product, but of course as you know, Glean has evolved far beyond that. That's the core foundations we built on, but we do a few more things. I would love to tell you more if you're interested to know.

Yeah, great question. The core of Glean is first understand the business that we are serving right now. The people who are going to be using Glean inside a company, we need to first of all understand who they are. We need to understand the business at a deeper level. Take an enterprise company. We want to understand what products do they sell, who are their customers. And the goal with AI is first understand how work happens inside your company. And the way you understand that is by observing people's activity inside the enterprise. So what Glean does is constantly looking at every person whenever they go and read a document or whenever they go and answer a question that some other employee had internally, when they actually take a task that's assigned to them. We actually observe what are the steps they take to actually go and complete that task. So a lot of those things are actually all observations that we make inside the enterprise. We're learning how people are working, and that sort of sets the foundation for building this ever evolving, automatically updating system that we build which learns as things dynamically change. Just a simple example with all the AI technologies that we have today. Take a product feature or a component. It works in a certain way today, but now your engineers have gone and changed the system and it works somehow differently tomorrow. Right? And if you look at documentation that talks about how it used to be done before, there's documentation or maybe there's no documentation which talks about how it is done today. But if you actually keep track of all the discussions, all the conversations, all the meetings, you can actually keep building that context and update their understanding of this business and how things work there. So that's the main framework about Glean is that it's auto updating. It actually observes activities and then keeps itself updated with what's the latest inside this business.

Let's take one particular topic and you have potentially a lot of information inside the business. There probably some documents that have been written on that topic. There are people who work on that particular subject matter. There are tasks that are being tracked on that topic maybe in some task tracking system like Jira. There are conversations that have been happening. Now you have to look at all of this data, all of this information on this particular subject matter. And you have to decide what information is the highest quality, what information is up to date and fresh. And for that you rely on signals. The signals are all there. For example, when you look at documents inside a business, you can see which documents have been authored recently versus which ones are old. When you look at questions and answers that people have given maybe a year back versus answers that are being given now. When you look at two documents which are written roughly at the same time but one of them is written by somebody who is an authority on that topic, you have to sort of think just like a human. AI allows you to think like a human. Imagine if you're looking for information on a given topic, you will start searching and maybe you'll search 10 different times, you'll come across a few different pieces of materials, and you have that judgment. You'll be able to figure out that hey, look, this looks obsolete, this looks old. So now AI actually has that same kind of abilities. That's how you have to think about it. And we built our system exactly how a human would react to all of this information. So when you come into Glean and you ask a question, we have thousands of resources that all looked semantically like relevant to what we have. Then we'll start to apply these signals to disambiguate and pick the things that we think are most authoritative. When you're asking a technical question, we'll actually go inside the code and actually see if the latest code reflects what the documentation says. All of this is only going to work if you are able to truly connect Glean with all the enterprise systems because if you have partial context then you can never give the right answers. When we deployed at a customer, we tell them that hey look, you have to connect Glean to everything inside your company. It has to know, it has to have the full picture for it to actually do the right job and provide the right information back to people.

Yeah, when Glean started, we had a singular focus. We wanted to build a Google for your work life. And the interesting thing is that we were fortunate, we were lucky to build a great search product. We actually needed transformers. So we were the first ones, probably the first ones to bring the transformer technology into the enterprise. We would take all of your enterprise corpus and we would pre-train a language model that would sort of now start to semantically and conceptually understand all of your information and data. And we were the first ones to actually then go and deploy embeddings and vector-based search. These terms didn't exist when we were building these, but it was like, I would not say that it was because we had the vision, we could look 10 years ahead. I don't think anybody can really do that, but for us this made sense to use this technology. Originally, transformers were built to actually make Google search better. So when we're building a great search product, it made all the sense for us to use it. Now if you think about our product, people come and ask questions and we're surfacing that information back to them. Over the years, the AI models got more capable. So we could actually go one step further in our product and say that hey, you came and asked a question and we have all this great information that we think is up-to-date, fresh, high quality, but I think we can also read it with an AI model and instead of just giving you the links to the information, we can actually give you an answer that AI generates for you. So that sort of was the next evolution of our product, which was very natural. It was exactly the next thing that people expected us to do. And so we launched a product that looked and felt like ChatGPT roughly at the same time when ChatGPT came out, but was meant for the enterprise. It was connected to all of your company's context. Inside a company, if you have any questions, we were using AI and a combination of search to provide the right answers back. As the capabilities increased, at some point the models were smart enough to not just extract and summarize an answer from a document or set of documents. They could actually also write on their own. You could actually create artifacts. And so we started to sort of transition more into that. Glean became more and more not just a search or a question answering product, but actually became in some sort of an assistant in some way. You can go in and you can ask it to do some work for you. It will create documents. So whatever journey you're seeing that AI is taking in the consumer world, we were basically in parallel taking that journey in the enterprise world.

Now in 2024, we launched something called AI apps. And the idea was that Glean is connected to all enterprise systems and you can create these conversational AI experiences which are incredibly powerful. You can get answers to questions but you can actually do some work with AI within this framework. But our customers started to tell us that here I've already done all the hard work of connecting Glean to all these different systems. But I have a very specific need. I have a very specific use case that I'm trying to solve for. For example, I have internal tickets that come to us. There are a lot of employees filing IT tickets and I want those tickets to be automatically resolved by AI whenever AI can do it. And that's all I want to do. I don't want you to resolve those tickets. I want you to actually work only with my approved knowledge base that the IT team has created to go and resolve those tickets. So people wanted to sort of scope Glean into these specific use cases. So we created applications and those were the precursor to what everybody in the industry started to call them agents. So we sort of were again like one of the first ones to bring agents into the enterprise. And now today, AI has evolved significantly. The capabilities of models are so much greater. Even in the last 2 months, everything has changed in terms of what models can do for us now. Now we're ready to sort of help any AI application that a business wants to build. We can actually act as that MCP endpoint, as a substrate that brings the enterprise context, that brings actions capability into your enterprise systems through one secure platform. So as you go and build agents, applications, workflows, you can either do all of that in our own product because we have our own agent builder, you can build workflows, you can build more complicated code-based agents in our platform. But in an enterprise, different people will have different preferences. They will build agents in different systems, they will create their own custom applications. And we want to make sure that we actually provide a central horizontal enterprise context layer to all of those other systems. You could actually go for example and build an agent in Copilot Studio or in Bedrock. Glean basically provides that connectivity tissue and layer within those agent frameworks. So that's sort of how we evolved.

Like a few months from now, we will be actually talking about something else, not agents anymore. So one of the most important things that I think is going to happen this year is that AI so far is a tool of a few. Not everybody in an enterprise actually benefits from all the power of AI because people are busy, they're not really figuring out how to sort of start using AI on a daily basis in their workflows. We have our habits and we don't change them that quickly. So if you have to go to AI, then very few people go to it. We see this power law. But that's going to change this year. We'll see that AI is going to become very proactive. You don't have to do anything as an employee. I think that's the right model. You don't have to constantly think about hey, what can I do with AI? Instead, the new AI that's going to already know everything about you and your work life, the work you need to get done this week, and you don't have to come and ask us to actually help with those. We will proactively actually tell you that hey, here are some of the things that you need to do which we can do on your behalf. And that is when AI will truly get democratized. And you'll see massive adoption of AI across the enterprise.

Yeah. Companies are actually doing a pretty good job in general. Everybody has understood that AI is a powerful technology and they have to invest in it. And so we don't see enterprises who are very skeptical of it. Yes, there are doubts like am I getting value yet or not, but everybody still understands that it's important. So I think some of the best successes that we have seen where enterprises have been able to really make a significant dent on their bottom line or top line is one, having clear ownership. You have to have people, you have to have leaders who are not busy with their main job and AI becomes a side initiative. You have to have people whose only job is to actually bring AI transformation inside the enterprise. So creating a role like a Chief AI Officer is actually critical in our opinion. We've seen some companies have done it and some companies haven't. In the latter category, it's a good mindset where the CEO comes and tells that everybody needs to be AI first. Everybody needs to think that they are an AI officer and that's true, but the problem is that leaders are already busy. They're fully subscribed and if you just add one more thing to their plate, it becomes hard. So that's sort of one thing is just having a center of excellence, an AI function actually makes a big difference. Second thing that I would say which is important is making sure that you are thinking about AI more broadly. There are two parts to it. One is you first want to make sure that your own products, your own services are fully AI in some sense. You want to actually deliver great capabilities to your customers to evolve your company. You have to make everybody AI first. So education has to be a priority. You have to think about yes, top line and bottom line is great, but you should also be happy as long as people are learning, as long as they are sort of changing their habits and trying to leverage AI more and more in the day-to-day work. Bringing a good set of tools to your employee base is also very important in terms of making progress. And then maybe I'll just give you one tactical good example of this approach. I've heard it from a few people and I actually suggested it to many. We have our ways of doing things and AI is a new thing and sometimes hard to change habits. But then if you have a young workforce, if you have people who just joined and this is their first job, they don't really know how to do anything the traditional way. If you couple them with experienced people. So for example, have every leader of yours get them an AI chief of staff which is let's say a new graduate. They will have that instinct of thinking outside the box and using AI fundamentally at any new piece of work. And so you can get a lot of AI momentum in the enterprise going by bringing folks who actually don't know the traditional way. The people who don't know the traditional way are actually going to figure out the new way. We see this happen in our company. A lot of AI transformation is driven by the younger folks.

Yeah, so I think just tracking adoption metrics is good from a point of view of is education happening inside the enterprise. So we should do that and most AI tools that you buy will hopefully give you those insights. But I think we should also not get confused about that correlating with business value. More usage is not necessarily going to correlate with business value. Usage metrics are good from the point of view of is education happening like we said. So from a point of view of how do you measure the success of AI, you have to first go and think about do you even have metrics to track productivity, to track efficiency regardless of AI. Like take for example customer support. What are your key metrics that describe how well you're doing? So things like case resolution time or time to first response to customer or how many tickets one agent can solve per day. Those are like some of your key business metrics. And so you don't have to actually create new ones to think about is AI helping me succeed or not. Just look at you deploy a great AI solution. If it is actually making an impact, then you should see that your people should now be able to do twice as many cases per day. How do you sort of really measure success with AI is to have those good hygiene from the get-go in terms of these performance metrics across your different functions and then just see the movement that AI is driving to those metrics. Those are the most believable. Those are the ones that your CFO is going to admit. They're not going to actually admit a new AI specific metric that you created to show value of AI.

Yeah, so some example scenario. Take Monday morning and the first thing you do is you have to commute to work. How about you get a briefing for the day and you're driving so it's not that you're reading something. It's kind of like a podcast. You sit in the car and your AI, your personal coworker which is on your mobile device recognizes that and offers you to make this commute more productive for you. So that hey, I can get you ready for the day. These are the five people that you're going to be meeting for this particular meeting. You need to have read this doc, but you haven't. So I can actually give you a summary. I can get you ready with the key points and help you with whatever context you need. And so there's that very interactive voice-based interface that you have with your personal coworker, which is on your device, which has all the context about your day, and it helps you get ready. And it actually also knows that it's going to take you 38 minutes to reach your office. It can actually figure out how to pack information within that amount of time. And so that's how you show up at work fully confident, feeling like you're in control for what's happening to happen next throughout your day.

Like imagine again, AI is looking at your calendar. It's looking at your tasks which if you're an engineer, your tasks have been tracked in Jira or a tool like that. It's also looking at all the communication that you're having inside the company. It figures out who are the people who are waiting for you on different things. And as those things happen, it sort of gives you again this list of things that it's figured out that you need to get done this week and maybe knows how to do 30% of those tasks and it keeps it ready for you. It says, 'Hey, by the way, I figured it out. I've done it for you.' So that it's effortless. You don't have to think about how to use AI. It came to you and completed some of those tasks for you. So those are the basic things from when AI starts to understand you better can do for you. I'll just give you one more example though for executives. I'm actually always frustrated about am I spending time on the right things or is my organization actually focused on the right set of projects. For example, we did a strategy shift and we have a few new focus areas and we want to defocus from some of them. Has that shift in strategy has it shifted down the organization? Did people who are working on projects that are no longer relevant to the strategy, did we actually stop working on them? Like these are things where I feel helpless in the sense that I can't figure out in a large organization with so many people, it's very hard to keep track of what everybody's doing and is everybody moving in the same direction. The new future, the super intelligent enterprise, is where very complex problems like those are also where you can actually feel in control. So for example, AI is powerful enough to actually go and catalog all the different projects that all the different people are working on. For each one of them, it can actually see what their work is and does it align with our new quarterly goals or our strategy, and it can just give me a list of projects that it thinks are potentially something we should reinvest. Similarly, as a CEO, a company is no more than its people. We want to know who are our best people in the company. We want to make sure that we are creating the right opportunities for them. And of course there are mechanisms to learn that through the hierarchy, but often times some great awesome people, they're not the loudest in the room and they're sitting somewhere and not fully getting leveraged. I want to actually identify who are those amazing people so that we can invest more in them. These kind of things where today you will think that this is going to take too long to actually go figure these things out and I don't know if I'm going to get an unbiased green signal. In the future, AI will be able to really help you with all of that. That's what is our vision for a super intelligent enterprise.

Like first of all, we believe that AI is a tool in our toolkit. I don't think the concept of hey does AI replace humans, and I don't like to think of it that way. It's more that AI enhances us, enables us, and actually sometimes it enables us to do twice as much work or 10 times as much more work than we could do before. So in that sense, yes it does replace some human work, but I think the success of a company even in the future is going to be proportional to the talent that you have inside your company. I don't believe in a one person $1 million company or a very lean company. Yes, you want to have super high productivity, but your scale is still going to be proportional to the number of people that you have. So now in terms of how we recruit, how we hire, right now the fundamental skills have not changed. AI doesn't change anything in terms of who we want to bring into our company. We're still trying to bring the same people: first of all who are passionate about what we build as products, who are excited about the technology that we are creating. We're of course looking for all the standard skills, people who work hard, people who are motivated, who want to succeed in their career, who are great team players. It's the same set of requirements, but I think AI is giving you a way to evaluate things more precisely. For example, right now we've changed the interview process for all the roles inside our company. We are looking for are you using AI in your job? And that's a key part of our assessment. Now take an engineer. We used to give them an assignment that could be done in 2 hours before and then would test how good they are at coding. That assignment has now changed to we give them a task which would have taken one month but with AI now you can actually do it in 2 hours. We don't tell the candidate to use AI to do this, but we want to see their instinct: will they actually use it and not feel daunted and actually complete that task. And what that is allowing us to test for is that is this a person who has a learning mindset, who's keeping track of all the technology trends, and are they ready to sort of change their habits and are willing to learn new skills. So that's sort of what we do and I think that's the right approach to build that modern AI first organization.