Interview Sherman Ye by The Next Database Platform

All right, first, thank you, Tim, to provide this opportunity. So, I started doing the graph database when I was at Facebook. I think we started the project back in late 2011. At that time, in Facebook, of course, we have a social graph, so it's very natural to have a graph database to serve relation queries for the products. So we started thinking to do something because there was no distributed graph database at that time. So we started thinking to index the relations shifts between people, like friendship, all those things, so we can provide easy access for the product. So we started building the project prototype. The prototype at the time, the project called the Dragon, it can serve like a friend-of-friends, that's a two-hop query, and we can even serve multiple hop queries like friend-of-friends photo, something like that. So after about a year, the system goes online and starts serving the products, and we get pretty positive feedback. After that, more and more products connected to the Dragon system. In 2015, I left Facebook and joined Ant Financial. I was lucky at the time because Ant Financial was looking to use the graph database in their portfolio, like risk management. So because of my background and experience with the graph database back at Facebook, they allowed me to lead the team to spearhead the graph database deployment at Ant Financial. First, we looked around to try to find some open source project or third-party commercial products. Unfortunately, at that time, no product could satisfy the amount of data and the high QPS of the financial. So after several months, in May 2015, we decided to build the system by ourselves. So we started gathering the team and building the system. Internally, we called it GBase, short for Graph Exploration and Analytic Database.

Yeah, it's all in C++. Yes, because of my background. I mean, I'm not a super expert guy, so I've been working on C process for many years.

Very good question. Actually, I think the main difference came from the different use scenario. In the previous two companies, in Facebook and Ant Financial, the system will be used basically for internal use, and all the users are internal product teams. So in that case, we not only provide the technology, we also maintain the system, right? We can provide a lot of consulting to the product team. But when we start doing our own stuff, like NebulaGraph, as a product to provide to external users, we cannot always do these things. We cannot maintain all the system for all the users, right? Of course, and we cannot provide all kinds of consulting work for free to all the other users. So that requires us to design the system in a way to be easily maintained, and the learning curve has to be smooth. Not like the internal system where the learning curve is not the first thing we consider. Because if it's hard to learn, that's fine because we can help the product team to build whatever and how to write the query as we want. But for other products, we cannot always do that. So two things we consider have the highest priority: one is that they're easy to use.

Very good question. The graph actually is a very special data structure. The scale of the graph we tested or experienced is at least at the trillion level, like a trillion edges.

Very good question. The graph actually is a very special data structure. If we only look at the structure of the data, the graph, even when we talk about the trillion edges, the amount of data is not huge. We can use a couple of machines because today storage is huge. We can have several terabyte storage easily on one machine. So if we only look at the skeleton or the structure of the graph, it's something. But unfortunately, only the structure of the graph doesn't mean a lot or doesn't help the business logic a lot. So we have to use what we call the property graph. Property graph means you have properties on nodes or edges. So the amount of property decides how big the data is. Some graphs we run into have hundreds of properties on each edge or each node, and that makes the amount of data huge. Like the largest data we run into is close to one terabyte, hundreds of terabytes on one property graph. So it's like a hundred terabytes. Given that we usually store one to two terabytes data on each machine, the size of the cluster runs close to like 100 machines.

First, I'll say, at least in this domain, in the graph database, so far there's no one dominant in this field. So there's a lot of rooms, but you also have a lot of players, and no player at least at this time dominates the field. So that's basically the background. When we come in, the advantage we have is the experience in a large amount of data. When I was at Facebook, we processed or experienced the largest social network in the world. But when I came back to China and joined Ant Financial, I was surprised because the amount of data is even larger, a different type. It's the transaction data. So actually, the number of edges is way more than Facebook. It's pretty much the largest transaction network in China or pretty much in the world. So we were fortunate to have this kind of experience. That's one advantage we have. The second advantage is the architecture of the system. Basically, we from the ground up would build a distributed system. It's not like, for example, Neo4j. I mean, I'll always pay a lot of respect to them; they are the first player in this field. They started the graph database more than a decade ago. But because they started early, their technology kind of limits the technology at that time. So basically, they're very good on a single machine, so they do not scale very well. But when we build our system, at the very beginning and the very first day, we started thinking about distributed system.

So basically, I'll say the magic here is how you organize the data. The nature of a graph is basically you have a node, a bunch of edges going out from this node, and then you have neighbors and neighbors going out. So it's a connected node. In a traditional database, the data was organized in tables, which means they put the same type of data together. But in the graph database, the data is organized in such a way that the node and its relations, or the edges starting from this node, we put those data together. So that means when we try to traverse out, it's much faster. So it's kind of like local access when we go out. But it's a distributed system, but when we go out one step, we always access one node, but we can go out.

It's very important, basically from two aspects. First, because the graph database is relatively new. Even Neo4j started the graph database more than a decade ago, but at that time, it's still very limited use of the graph database. It's not very popular. Graph database only started getting popular in the last couple of years. So if you start hearing people say, "Oh, graph database, I want to use graph database," but still, today, more people know about the graph database or have heard about it than people really know what's going on in the graph database and how to use it. I mean, basically, most people don't know how to apply the graph database into their business. So for us, dealing with this new technology, it's a very high cost to educate people, to tell or teach people how to use graph database in their business and scenario. The best way we think is to give users the technology and let them handle this new technology, let them prove it by themselves. So the best way to provide that is to make the software open source, so people start using it and can easily get access to the technology and apply it into their use case. So that's one way we try to use open source to reduce the cost of educating the users by letting them hands-on with the new technology to try and adopt it.

Yes, the first sector we're going after is the internet customer. The reason we choose the internet customer is because first, internet customers are more willing to access the new technology than traditional financial institutes or other customers. Second, usually, internet companies or internet customers have better technologies; they have a more technical team. For example, their development team is, I would say, a little bit stronger than a traditional financial institute R&D team. So in that way, we decided to choose the internet sector as our first customer sector.

All right, thank you.