Zero to $5B+ Sales: Anshul Sadana’s Arista Journey

Trading groups and that was the era of high frequency trading where a one microsecond advantage could give you enough of a lead over your competition where you can make a million dollars of profit every day. You know what was the most requested feature from our customers in 2008? It was a sticker to put on top of our switch to hide our logo so that none of their competitors could find out how great our product was and that they were using it. We won Microsoft Azure in 2012. They had notified us for 40 gig, we had been selected, we were able to beat Cisco and others to get to that point. Bought for multiple years, it was risky. If it didn't work out, we would have been in serious trouble. The reason it's so important is in fact Meta had published a paper about a year ago which showed that in their clusters, the GPUs are sitting idle about 33% of the time waiting for network I/O to complete.

Thank you. Thanks for having me over here and it's great being with you.

Well, thank you. It's been an amazing journey. I started my undergrad in electrical engineering, then I came to the US to do my masters in the era of the internet, and Cisco was the company making the internet, so it was a great place, maybe like OpenAI today. The systems world was complex, and not many people know this, but the code that runs on these switches or routers is a good 30 to 50 million lines of code. So joining one of the teams there, this was a Cat 4K team, this was an acquisition Cisco had done of Andy Bechtolsheim's Granite Systems, and just after the acquisition I happened to join that team, so I got to work with some fabulous people.

Well, Andy had founded Granite Systems, and Granite got acquired by Cisco and was powering the internet at the time. That is correct. So I joined this team, these are fabulous people, really smart, and I got to work on projects where we're working on the world's first triple speed product. So the same port could support 10 Meg, 100 Meg, and 1 Gig, doing all of that for desktop connectivity. That was the era of getting gigabit to the desktop, power over Ethernet, wireless was just starting to happen, so it was a great time. And as I started doing that, we got into some trouble. Our products ran into issues, and my management team came to me and said, 'Hey, can you help us look at these customer problems? You can work on this other product later.' And in life, that gave me immense exposure to how customers use and think of technology. These customers running into issues, they wanted to return the product, we had to root cause it, we had to give them workarounds, you get into negotiations with the sales teams and the customers on discounts for upgrades because the customer has issues. I had to work with the finance team on warranty and reserves and calculations, I had to work with the manufacturing team on tracking fixes in the field and repairing and replacing products. So that exposure was pretty wide. I had a pretty good understanding of how the company functions end to end. It looks boring from the outside, just to support a customer, but there's so much complexity around it.

That is around 2007. We were a small company, we were about 12 plus engineers, and we were just deciding what product to launch and which markets to tackle. When we got started, we launched our products in 2007 at Supercomputing in Reno, and some of the customers or prospective customers came to us at the trade show, just gave us their visiting cards saying, 'Just call me, I'm ready to buy your product.' We said, 'But I haven't even told you what the product does.' They said, 'We know, but we're ready to buy the product.' And it turned out we had the lowest latency switches on the planet. These customers were from financial trading groups, and that was the era of high frequency trading where a one microsecond advantage could give you enough of a lead over your competition where you can make a million dollars of profit every day. Our products were three to four microseconds faster than the best competitor, so a huge advantage. Now there was one other requirement though: the product had to be high quality because you cannot cause an outage in a financial trading network. And we had built a rock solid foundation with our operating system, and as a result of that we were very successful in the market. But that also gave us a customer base that we could call for references, and they would vouch for the quality of the product.

It was very hard to know upfront which markets we should be chasing. Clearly we were in the switching space, these are high-speed switches, these were 10 gig switches when the world was still at 1 gig. So HPC, high performance computing, was a natural fit, so we had some customers in that space that were interested. But we also had these financial trading customers. We were keeping an eye on the web companies during those days, they were not called cloud, they were called Web 2.0, so we were keeping a watch on them as well. And the products we were building for financial trading were essentially a stepping stone to the cloud, because it was building on top of the features the financial trading needed and going to the next level.

Major recession. One of our early customers was Lehman Brothers, and as you know they went bankrupt and they shut down. So in that market, you're chasing every potential customer that you can get to generate revenue. And high frequency trading was very, very important for us to get to the first $100 million of business, and equally important in getting us a customer base that we could count on as a reference, because these other large companies including the cloud would want that before they can use your product. So it was an amazing stepping stone. In hindsight it worked out, but directionally I think we were always aligned to what the cloud needed.

Yeah, there's a lot of luck in terms of finding that product market fit, because as you said, we were not expecting high frequency trading or HFT to just show up. Sometimes luck matters. At the same time, when you're given an opportunity, you have to grab it and execute on it. If you try to be everything to everyone, you will just not be the best in the world in the product category that you're trying to aim for. So what we ended up doing is focusing on financial trading for the first few years. Once we realized this can work out, we just simply executed and executed and executed, did every feature that they needed, and became essentially the talk in the town. You know what was the most requested feature from our customers in 2008? It was a sticker to put on top of our switch to hide our logo so that none of their competitors could find out how great our product was and that they were using it. That is the number one feature from your financial industry customer.

So while we were working with financial trading, we were also working with the cloud companies. At that time, the cloud was in its very early stages, and they needed products that could scale to significant network sizes that had not been imagined by the bigger companies at that point. A little bit of non-blocking at $100 per port, and that was unheard of. The larger companies, number one, didn't have such a product, and number two, were not interested. They said, 'Hey Google, you're the only company that wants this. It's a distraction for us, and we're not even sure whether we make enough margin. $100 a port, they won't make enough margin.' So Google went on to build their own switches once the industry said no. But by the time Arista was born and growing up, we still had that RFP in mind. We knew there's a need for large scalable networks. Not Google, but Microsoft happened to be one of our largest cloud customers early on.

We know about terabytes, we know about petabytes, and then comes exabyte. So during that era, when Microsoft came to say, 'Can you build us a cluster for so many storage nodes?' These are over 10,000 storage nodes, non-blocking at 10 gig. Everything just clicked. We had the product direction, we had the software that was very reliable to build a cloud network, and we had an architecture which got transformed into what is now known as the leaf and spine to truly keep on scaling it like a distributed cluster. So this was many years later when Microsoft had a requirement, you felt like, 'Hey, this is my architecture.' Literally, the product direction was that there's a need for non-blocking high-speed networks in the world.

What was happening in the ecosystem was there was a lot of talk about active-active, because the early days of networking were active-standby. One of the protocols used in many networks was Spanning Tree. And if you are from the network industry, you know how many outages that has cost, because it's active-standby and the standby doesn't really take over when you have a failure because you trigger a bug and the network just crashes and you have an outage. And whoever managed the Spanning Tree needs a PhD degree in networking to manage it. That's correct, it was very complicated, so it's very fragile.

Being made in the routing protocols in the IETF as well, we looked at all these choices and said it has to be distributed. The only way you can do it distributed is just like the internet. All these routers run BGP. If one router goes down, the internet doesn't go down, just route around it. We did the same thing with our architecture. We built a spine which is distributed. It can be four different switches, it can be eight-way, it can be 16-way, it can be 64 of them. If one switch fails, only 1/64 of your bandwidth is lost and you can route around it. So with that spine, we had many leaf switches or edge switches or top of rack switches. That pretty much defined the architecture of the future or the contemporary cloud networking, the leaf-spine architecture.

Of course, the cloud was a foundation for us at Arista to go after other markets including enterprise, media and entertainment, and so on. The same product set is used by everyone now. The cloud is obviously lower margin because of the scale, but you get economies of scale and leverage within the company. The same R&D effort, the same manufacturing effort can be leveraged for a much greater volume, and over time that results in reasonable profitability. At Arista, the cloud customers have been phenomenal for us because they form roughly 40% plus of our business. We call them the cloud titans, but we also have enterprises and other segments.

Absolutely. So first of all, these cloud designs have allowed our customers to scale significantly and more importantly also provide a lot of uptime. You know, you never get an email from a cloud company saying, 'Dear customer, the cloud is down from 2 a.m. to 4 a.m.' They don't do that. Does that mean they don't need any maintenance or fixes or patches? Not true at all. They have a very complex amount of infrastructure, but because it's a distributed system, they can schedule the maintenance such that you route around it and never get impacted. That ability gave our customers a large scalable infrastructure that evolved and kept on growing and growing. Some of our largest customers have millions and millions of ports active running in the infrastructure. Now you come to the world of AI. So we had the enterprise networks, they were highly oversubscribed, generally active-standby. That was 15 years ago. Now they showcase at OCP, Amazon and Google do their own things as well. So the world has evolved to very large scale in the front end of the cloud. From the back end are these AI clusters. The AI clusters drive a lot more bandwidth than a standard CPU or a storage node. So as an example, for the current cloud networks today, most customers are happy with 100 Gig, and for some use cases they have 400 Gig. But for AI, there's a mad rush to get to 800 Gig as quickly as possible, and then 1.6 terabit per server or per GPU as quickly as possible, because there's that much bandwidth that is needed by these AI applications.

Starting to adopt 800 Gig, 800 Gbps per GPU. And the reason it's so important is in fact Meta had published a paper about a year ago which showed that in their clusters, the GPUs are sitting idle about 33% of the time waiting for network I/O to complete. And that's wasting a lot of power, that's wasting a lot of capital cost as well. GPUs are expensive as everyone knows. So as a result of that, you want the network to go as fast as possible because all these GPUs are working in lockstep. They do a calculation, they exchange results, they do a calculation, they exchange results, they do a calculation, and so on until you're done with your entire training data training model. But that can take a long time. So everyone wants it instantly.

So there are certainly new topologies that the cloud companies are using or deploying for AI networks, but they are largely derived from the same leaf-spine architecture because they're resilient. Even if one fails, you're not dependent on a single point. Each GPU today takes somewhere between 1.5 to 2.5 kW, and some future GPUs are projected to be at 3 kW per GPU. Each motherboard has eight GPUs interconnected together, and then you have multiple GPU server blades connected together in a rack, then you have multiple of these racks connected together. So today, in fact Meta recently announced they built a cluster with 24,000 GPUs, H100s, and the other companies claiming to be very soon building 32,000 GPU clusters or 64,000 GPU clusters. That's where the world is now. If you can make that more efficient, it is a huge saving to the customer and essentially even for global warming because we need to reduce power consumption where we can.

It's a very interesting time. Many years ago when Arista was just started, there was this company called Myricom and they had a technology called Myrinet. It worked at smaller scale. I worked with them from my VMware days. They were chugging around all the time, never found the killer use case until now. Correct. Now because Nvidia has an end-to-end bundle stack, what they've done really well with InfiniBand is making it work out of the box for these customers. And when you have a small scale, that works okay. When you have a larger scale, you get a lot of single points of failure, whether it's a subnet manager or other complexities if there's outages or a link flap and so on. So as a result of that, there is a need to find a more scalable solution. Guess what? It already exists. It's called IP and Ethernet, and it works at scale. AI is very sensitive to loss. Whereas in TCP/IP networks, if you get a little bit of loss, you can retransmit. You cannot do that in AI networks because the GPUs are waiting. And while you can retry and retransmit, you're going to bring down the performance of the entire cluster because all these GPUs are waiting for this one last guy to finish its job. So as a result of that, there's a search for lossless networks. What's happening in the Ethernet world? Number one, Ethernet is already quite good, works really well, and customers who are already deploying it have been very happy with the results. There's lots of benchmarking data available in the industry and so on. In addition to that, there's a new consortium formed called UEC or Ultra Ethernet Consortium. These massive economies of scale and efficiencies with an Ethernet network that you don't get with InfiniBand, but you also get the reliability that the customer is searching for. And at scale, you can't really use InfiniBand anyway, so everyone has to go that route. Whether it's a year or two, it's only a matter of time.

Well, Ethernet has been around for over 50 years. In fact, 2023 was the 50th birthday of Ethernet. And Dave Metcalf, who's the founder of Ethernet, was there at the Computer Science Museum in Mountain View and he talked about it at that celebration. Many, many technologies have come and gone, but Ethernet has survived. Ethernet versus the rest, it's a good word.

Well, a lot of the focus in AI right now is on the application space, because that's where there's tremendous value. But the infrastructure is equally important. Arista works on merchant silicon, but merchant silicon has come a long way and delivers great performance and reliability. It's a lot of tuning that happens on buffer tuning of packet memory and so on to make RDMA work or to make ECMP hashing work. And I think along with our Broadcom partnership, it worked out really, really well.

Absolutely. There are so many things that come to mind, but I'll share a few of them that are key in my view. Number one, when you are in startup mode, clearly you need to have a good business plan, you need to go after a large market. It's very rare to have a successful startup in a market that is too small. And that entrepreneurial spirit of 'I will take whatever it takes to be successful' is extremely important. We were talking about this the other day. In the early days at Arista, I had to be the first IT engineer. I set up the phone lines for the company. I did research on how do you buy an 800 number and a 500 phone number or a direct dial block. We had to set up all of our labs, we had to set up our badge readers and our badge access facilities infrastructure, all the way to the customer facing stuff, whether it's the website, the support portal, how will customers interact with you, and so on. But that's one side. On the back end, documentation, things like that. On the other side, when you're working with customers, you have to go above and beyond. For example, in 2010, 2011, we got selected to be the network of choice for a company that was doing all of the broadcast production for the Winter Olympics, which was being held in Canada. They were streaming all the content directly to a big data center in Las Vegas. Over there, the product, the packets get cached, and after that it's okay because they can mix with ads and whatnot. But if you drop anything before it hits the cache, that frame is lost forever. This is IP production, very new. They had a failure and they used their last spare switch to replace that failed device, and called us and said, 'Look, we have no spares left. You cannot have any more issues.' And we're saying, 'Well, we'll try our very best, but what can we do?' And Murphy's Law sometimes kicks in. Did it kick in? Well, I think we were a little bit better prepared for that. So around 4 p.m., I get this phone call on a Sunday or Saturday, I forget, over the weekend that, 'Hey, we have the situation. I have no spares left on site.' So I called one of my support engineers, I drive quickly to San Jose Airport, the engineer meets me and gives me a spare switch. I hand carry that all the way to Las Vegas. We were very popular with that customer when we brought that replacement switch for them. It was a very successful event, there were no outages. Murphy's Law did not kick in. We had a successful event and we became very popular. You do whatever it takes to even be ahead of Murphy's Law. That's sort of the important part. Correct, absolutely. So I would say that was another direction. The third one we talked about is you have to stay focused on your goal, on your domain. And the hardest part as an entrepreneur, you're involved in product management, product strategy, is to say no. We had so many early customers where we said no, we're not going to do it right now. Can we come back and meet you in a few years when we're ready? That is extremely important to maintain that discipline.

Well, based on my previous comment, sometimes you have to say no. But it seems like a distraction. So what was happening in the industry was OpenFlow was being proposed as a way to control which flows get programmed and then you can forward packets that way. But to some extent, it's like going backwards. You're building a static network with a small controller rather than a distributed network using BGP, which is what our leaf-spine designs were already doing. And the OpenFlow based networks would be more fragile, they don't scale, and the controller becomes a single point of failure. Now there were a few companies that were already doing SDN and were doing it really well. Those happen to be the cloud companies. And of all the companies that have talked about it openly that I can share, Google is one of them. They have a lot of SDN controllers, their controllers are distributed, they don't have a single point of failure. They have a controller and it gets them the outcome they're looking for. However, for the rest of the market, including the enterprises and so on, this was never going to work in our mind. So as a result, we decided to stay a little bit behind on OpenFlow and SDN controllers in the early days. However, as the market evolved, we did see the need for automation rather than the protocol stack being in a controller. That was the basis for CloudVision. We built an automation plane for the entire network. This was the first time where you can manage hundreds or even thousands of switches all from one place. And one of the most interesting features we had added at that point wasn't just that you can configure, but you can also roll back. Customers would call us at 5 or 6 a.m. and say, 'Hey, last night I was making a change to my network and I modified the configuration. I did this and then I did this, but now it doesn't work. Can you help me?' We'd say, 'Wait, wait, wait. Can you walk me back on what did you change?' They'd say, 'I'm not 100% sure. I think I did this, but I'm not really sure.' So we'd give them a minute. We would ask them what exactly did you change, and they would say, 'I'm not really sure.' So then it clicked in our minds: why not build a system that can restore all configuration, bring back the entire system to the state it used to be? That was our sort of focus on SDN, basically an automation controller for the network. Stay focused on the value to the customer rather than what's the buzzword out there.

Absolutely. Well, as an entrepreneur, you have to start with a small team. As the business needs grow, you hire really good people and scale with them. And I think that part, maintaining the culture of the team, was extremely important and is extremely important for any successful entrepreneur out there as well.

Absolutely. Hiring is the most important part for an entrepreneur because you want to bring people on board who are smart, who can execute well, and also are a good cultural match for the team. Because without that, you'll start going in different directions. When you're a small company, you can't afford to have people who are not aligned. So we spent a lot of time hiring, interviewing, making sure these are the best in the industry and were able to carry on the mission that we were on. Setting the bar very high early on pays off over time.

Absolutely. There are two aspects here. One, I just finished my MBA before I joined Arista, and that gave me immense confidence that I could do anything new. I could take on more as well, whether it was running the IT for the company, building our first phone system, getting all the toll-free numbers for support, building the website, getting the pre-sales team going, product management, product priorities, margin analysis for cost structure of the product, and so on. Everything just started to come back together. But as you do that, there's always more. Actually, let's stick this part out now. At the same time, work at a startup or any company as you're growing is quite intense. And we had a global team. We had engineers here in Silicon Valley, on the East Coast, we had employees covering sales and support in Europe and Asia. The work was intense. And right during that window, we had two kids. Our daughter was born first, and that changed my schedule. What I started to do was start getting at 5 a.m. I would finish my phone calls for Asia first, then I would go on to Europe, then I would go on to New York, finish that, go to work by 9 a.m., then cover all of the needs for US customers, and by 3 or 4 p.m. again you're on to Asia calls. But I used to end by 6 or 7 p.m., and that way you're ready to be back with family, especially for dinner. That was extremely important to me and to my wife. She was a tiny startup at that point and taking the risk while she took care of the family and was also working full-time. But it has been tremendous because of her I could take on more risk and more work as well. So it was very important to find a way to still go home to do dinner and then balance a reasonable work-life balance over the years, even though as being a global company you're a global executive, but you still manage to make this happen.

Yes, yes.

Well, it's still way early on. Right now my focus is to take a break with the kids. We'll be traveling this summer, taking time off, recharging. And then when I'm back in the fall, I'll start meeting entrepreneurs and VCs and others yet again to explore more options. But not there yet.

Well, we had several challenges. The first one was the lawsuit with Cisco. That was very hard to predict because you're not sure what the outcome will be. You don't control everything. In that sense, we did come through. We had to design around a few patterns. We won. Many of the other claims that they had thrown at us that accused us of copying or copyright violation, we proved they were all wrong. So we came through. But that was a challenging period because we had to convince customers that yes, this lawsuit is going on, but believe me, you can still buy my product. And a lot of trust and credibility was on the line. But I think we came through. The second one was also an unpredictable event, which was COVID. We were doing great, our products are great. But during COVID, because of the boom in work from home and the use of internet technologies and social media and so on, during that period I had to spend a lot of energy looking out for alternate components, talking to suppliers, talking to vendors, researching how does the ecosystem work all the way up to TSMC and everything in the middle. And we went ahead and leaned in at Arista. We invested over $5 billion in buying components ahead of time. And as a result of that, we could ensure supply to our customers. We could look at our suppliers and get our fair allocation, and sometimes have more than what we had originally planned for. As a result of that, throughout COVID, at Arista, not only did we not have any supply issues, we actually grew. Just at the start of COVID, we used to have $500 million of inventory on hand. And then during COVID, we went ahead and increased that 10x and bought for multiple years. It was risky. If it didn't work out, we would have been in serious trouble. But it really worked out.

In the early days of Arista, a big challenge. Anything you can share on that one?

Well, in the very early days when we got started, in the networking industry the world was known as Cisco and the dwarfs. I was going to say that. And in fact, some companies talked to us and said, 'Hey, welcome to being the eighth dwarf.' And that was not a lot of hope for us to survive and succeed.

Well, we were actually quite confident that if we built great products, we could have a better outcome. In no scenario could we imagine at that time that we would reach the level we have achieved today. But as a young entrepreneur, and I talk to a lot of VCs and others as well, a lot of people tell me sometimes when people just show up saying, 'Hey, I have a business plan, this is just like the Google idea, this will be as successful as Google,' that's a bad plan because that's not how real life works. You start out with a goal, you achieve that, then you go on to the next goal, you achieve that, then you go on to the next goal. So at Arista, our first goal was to achieve $100 million in revenue. Our next goal was to become a $1 billion company. It was an amazing journey to get to that point. It was scary. We lost quite often in the early days. At the same time, we knew which customers could potentially work out and invested all our energy in that, whether it's financial tradings, some of the largest banks of the United States, whether it's some of the cloud companies. And as a result of that, we are where we are today.

Absolutely. We had to differentiate on software. Our products are not just hardware but a lot more on software. So we had to go educate our customers on why EOS or our operating system is more reliable than the alternates. In addition, you're right, we had to counter 'free.' Our competition was ready to give away switches for free. How do you beat free? We had designed switches that were so efficient that even if our competition gave switches for free, the cost of powering those switches for 5 years versus buying Arista switches and powering them for 5 years was still lower cost and more efficient than going to competition. That's really how the cloud also pivoted towards us. They knew it was more reliable and more efficient.

Absolutely. I'm pretty sure there are 200, but I'll share a few that truly stand out. First one is when we won Microsoft Azure in 2012. They notified us for 40 gig, we had been selected. We were able to beat Cisco and others to get to that point. It was a massive opportunity for us, and we were going to power all of Azure at that point. Now we didn't have the ability to build so many switches, but we did a mad dash to get the capacity and the scale up, and we delivered and we came through. But getting that award was essentially 3-4 years of work finally coming to fruition. So you got the award first and then figured out how to fulfill that contract. The cloud is very hard to forecast. Even when they come through, they say, 'That's great, we want your product, but now can you deliver?' The second one was when we went public. We had worked so hard to become a $100 million company and then a billion dollar company, then we IPO'd and became a $3 billion company instantly. And of course the company has grown since then. Very recently we celebrated our 10-year IPO anniversary in fact last week. So that was a proud moment. Packer, they're all standing there, yes, that's right. I would say the third one wasn't as much of a public event, but it was still very, very memorable, very, very enjoyable for me. So what was happening during our lawsuit with Cisco is that they were accusing us publicly through blogs and so on of stealing IP, copying, and so on. And we had to go to trial in San Jose. And as you may have it, just based on how our legal system works, the jury that had been hand selected included a grocery store manager, a gardener, a UPS truck driver, and many similar folks. And we had to convince them that a command like 'show ip bgp neighbor' is not copyrightable, it comes from the IETF. But they're not technical people. So going through the paces and convincing them that this is what we did, this is how we build the company, this is what we listened to the customers, the customers said this is an industry standard CLI, just go ahead and use it, and so on. And we were able to finally get to a point where the trial is over, the hearing is over, the jury is now making their decision. I was on a flight from Denmark back to the Bay Area, and during my flight I get this text message: 'We won the copyright trial.' Wow. I was in tears at 30,000 feet. I knew we had come through, and every employee had truly come through because we were able to prove that we were genuine people, we had done the right thing, and we should be the ones winning, not losing. And the jury absolutely supported us. We proved to gardeners that BGP related technology is not copyrightable. 30,000 feet above the ground in tears. That's a fantastic story.