Cloudflare & the Edge — John Engates, Field CTO

Hey guys, how are you?

I just put on my best ugly coat. I didn't have a sweater. I'm from Texas and it's not cold enough here for sweaters most of the time, so I had this sport coat from years ago and threw it over this Cloudflare t-shirt, so it should make for a festive sort of decoration for the podcast.

Well, I've always been fascinated with technology, computers, gadgets, whatever it is, toys that make noise and do things. And you know, sort of that led me to computers in the early days. And I think, I guess it was like middle school when I got my TI-99 and then later the PC. And so you know, tinkering around with computers ever since I was a kid. And in college, you know, kind of off the back of interest in BBSs and technology like that, I actually found out about the internet. The internet was sort of still relatively unknown in the '80s and '90s to most people, and maybe tech people knew about it but most other people didn't.

I ran one sort of part-time because we only had one phone line in the house, so you had to basically run the BBS like at night, just at night when nobody needed the phone. And you could call a friend and say, hey, dial into my BBS, but that's about it. I never ran a full-time BBS. I ran a Renegade BBS in central California in the '90s that was popular. I had to get like a second phone line and those were the good old days back before the internet. It was all very local, you know, and we would all the different sysops would get together and have barbecues and stuff. It was very local, but people still sometimes dialed long distance for that kind of stuff. And I remember the whole crazy era of hacking calling cards for people to use to get free long distance. That people used to do that kind of to basically cheat a little bit.

But you know, one of the things that pulled me into the internet was my — again back to my nerdy roots — I was into HP calculators in that day. HP had all those programmable calculators and I was in engineering and physics classes and needed to learn how to program that thing. And there was a BBS that was basically available via FTP. It was HP's BBS and if you wanted to dial into it, it was in Oregon, so again long distance, but FTP. And so I kind of saw that FTP link and had to go figure out what that was, and that's what led me down that rabbit hole of getting an account at the university on the internet and learning about Unix and starting to figure out how it all worked. And so that was sort of my entry into this industry, really.

Yeah, it's still a valuable part of our world, right? People do use those things during certain times of the year. And anyway, learned about the internet, about to graduate from college. I got a degree in accounting, by the way, so not really a technology degree, but I was still fascinated with it. Right out of school, got a job working for a company that did big IBM mainframe type mid-range systems, you know, AS/400s, RS/6000 kind of gear, but it wasn't my passion, it was just a job. And one of my buddies from college called me up one day and says, do you want to come over and work on this ISP thing that I'm building? Because we really couldn't get internet access at that point. It wasn't readily available, at least again back to some national ISPs, and you know, you could dial long distance. But we started our own ISP in the mid-'90s and, you know, that was what sort of drew me in even further and, you know, built that up, had dialup accounts, ISDN accounts, how did it get? I don't know, we had maybe 10,000 customers, something like that, 12,000 customers. Pretty big, yeah. It was local, it wasn't, you know, regional or national or anything, but it was local. And, you know, back in the early days they were paying like 50 bucks a month. It was pretty expensive then. It went down to 20 and then it went down to 15 and then, you know, the ISP market changed radically in just the course of about five or six years because people started to realize, like the telcos and the cable companies started to see the potential for it. And that really is when I jumped and went to Rackspace because I could see the writing on the wall that the ISP business was not going to be a big thing for a local company. So one of my customers at that ISP at the time became one of the founders and investors in Rackspace, and that's what led me to join Rackspace in 2000.

They hired me as VP of Operations, so that just meant I basically did a little bit of everything. I managed the customer support people, all the customer support. I managed the teams that built servers and deployed them into the data center. I managed basically all the things that were not financial or marketing, sales kind of thing. So I was sort of the tech side of the house. There were some people that were smarter than me that actually managed the network itself, like the network engineering folks. They were on a different team under our VP of Engineering, but we had a very close relationship and worked together on helping build out customers. So all the big customers that showed up really were my kind of responsibility to get them up and running and built, and the teams that I led basically did all that work.

You're going to laugh. In the early days we didn't really monitor things. We let customers call us and tell us something was broken, and we were basically supporting them in their efforts. We didn't really take full responsibility for their uptime. We took responsibility for the network and whatever, but there were customers that were starting to push us and say, hey, we need you to tell us when something's broken. And I downloaded — you're going to laugh — but there was a program for Windows that was called What'sUp Gold. What'sUp Gold, What'sUp Local, little piece of software that you needed to run on the desktop and it would ping every hour. And then eventually we wrote our own tools for that stuff. But I was usually the one that just sort of figured out some quick and dirty hack to figure out how to get across some challenge and then have to be adopted by our team to go build out a tool or a platform. But there was all kinds of monitoring tools — ping and port monitoring, uptime, availability, pinging lots of servers on a regular basis. It's harder than it sounds, you know, really, you have to have some pretty scalable hardware to do it. So we built a lot of that ourselves.

Well, we would obviously try to keep equipment in production as long as we reasonably could because it had to pay for itself, you know. So you had to sort of get customers up and running, get them deployed on some hardware. A lot of the servers at Rackspace were built in-house. We built our own from components. We were not buying off-the-shelf ready-to-go hardware; we were building a lot of that ourselves. And so we were buying the latest motherboards and chassis and hardware and building it. And so it was sort of this — you could kind of move across the data center, one of Rackspace's data centers, and see sort of different eras of hardware based on when a customer joined or when they got deployed. And we were dedicating hardware, dedicating an entire server to a customer. And typically the good part for us was that customers didn't want to necessarily move to the new hardware because it was a huge migration challenge. It wasn't like today where you've got a VM and you can just pick it up and move it. This was like, okay, we've got to migrate from this version of Linux to this version and everything had to be compiled and everything had to be upgraded. It was a lot of work, right? And so people were more than happy to stay on a particular piece of hardware as long as it stayed up and it performed well.

Okay, so easiest was when we started to actually buy hardware sort of off the shelf and buy these big Dell servers, you know, just sort of the typical 2U rack mount. I remember seeing the first one of those show up and it was modular and it was all easy to troubleshoot and upgrade and make changes to. There was a piece of hardware in the early days of the hosting business — and I'm trying to remember what the heck the name was — it was like a 1U pizza box server that was kind of an out-of-the-box ready-to-go web server. And it was so hard to maintain. I'm going to have to look up what that machine was called. It was like a proprietary version of Linux on top of some proprietary hardware, and the customers loved it because it had this really cool control panel that made it easy for them, but it made it really hard for us. And so that was one that sticks out in my mind. And then the other hard thing is just as customers scaled, you had more and more complexity associated with one given customer. So the hardest thing as they got big was like, okay, they've expanded beyond the bounds of one rack or one row of racks, and then all of a sudden you've got to play Tetris in your data center and move stuff around, right? Lift them up and move them to another part of the data center to allocate more space. That was always a challenge when we got these really big customers and you never knew when a customer was going to blow up and grow.

I always remember one that was Hot or Not — do you remember that site, Hot or Not? I remember that one. So shout out to James Hong, the founder, he's one of my Twitter followers or friends, and him and his friend brought this site to Rackspace and it just blew up in a kind of a crazy viral way. But there were others. Before we got on the show here we were talking about favorite movies and I said Anchorman, you know Will Ferrell, he was one of the guys behind Funny or Die, right? The website Funny or Die, that was a big viral sensation on the Rackspace platform. YouTube was another one that showed up at our doorstep in the early days. YouTube started at Rackspace and basically they needed infrastructure for the brains of YouTube. The truth is even one host was not enough for YouTube, so they had infrastructure at data centers all over the world. But we were sort of the brains, we were where the databases were and the actual main website was, but they were serving content all over the place.

We had some of that, that was not unknown territory. There was a time in one of our data centers when the air conditioners were not functioning properly and machines were starting to overheat. And I remember our VP of data center engineering was going to basically go smash out windows. Our data center had windows, by the way — most data centers do not have windows. This was early, early days, it was in a high-rise office building and they did have windows all the way around. And the air conditioners failed but the machines were still on and he was panicking, like, okay, we don't have that much time because these things are going to start to melt. And so he was threatening to go smash out windows. It wouldn't have helped honestly because you needed a lot more cooling than that. What else happened? You know, we did build redundancy — you remember the days of having a failover redundant pair of firewalls or load balancers or even servers? You had these sort of architectures that were supposed to be more resilient but they rarely functioned as promised, right? They usually ended up with a failover that caused more problems than it solved. And so we did try to build that for customers, but a lot of times we were just scrappy kind of folks that just did things with a lot of humans. We were running into the data center, we had people positioned with these crash carts that we could go out into the data center and sort of plug in a console and troubleshoot it right there. Sometimes a machine wouldn't boot. Sometimes on a Windows machine it would do a blue screen, you had to figure out what the heck was going on with that. And sometimes it was just human intervention, right? It was like, do what you can to get things back up and running. The bigger problems that were more outside of the scope of an individual, we had to figure out larger scale solutions to those things. And we did have our share of issues over the years, but fun stories.

We were certainly paying attention to Amazon. It was not like we didn't understand the threat or the opportunity. You know, you can look at it in two ways. At that time they were coming at it from — I think they launched a queuing service, they launched S3, they launched EC2, they launched a few things sort of in order in the early days. When we first saw that, it was like, this is pretty interesting stuff, this is worth exploring, worth thinking about. And yet at the same time we were not seeing it as an immediate threat because the big enterprise customers were still demanding large-scale hardware, redundancy, things that were sort of more bulletproof. I call it like the gold-plated hardware. They wanted a SAN with redundancy, they wanted a NAS with redundancy, they wanted large-scale enterprise-type equipment. They wanted to see your data center, they wanted to tour it, go inside and walk around the facility and make sure it's compliant with all these standards and whatever. And so at the early stages of AWS, none of that stuff existed, right? None. And people were not 100% sure, certainly not enterprises, how they were supposed to use that stuff. So you could see it in the context of maybe a smaller startup that was engineering their own product and could control everything, but you couldn't see it as these larger companies. So we started tinkering with it. I had a customer early days of Rackspace that came to me and said, do you have anything like S3? And I said, well, no, not exactly. But we put some people on working on that and we ended up building a prototype for a distributed storage platform that eventually led to us launching the OpenStack project. So we were embracing the idea of cloud. We bought a company that was called Slicehost. Slicehost was a by-the-month virtual server rental kind of company — not by the hour, not by the minute — but they were sort of taking servers that were physical and slicing them up and carving off a piece of it and renting it by the month. That company got folded into Rackspace and we built out the Rackspace Cloud.

I think that we did do a pretty good job. There were a lot of people that used that cloud product at that time. The challenge was it's just the scale of AWS was so big. They were spending in a quarter to build out data centers — they were spending our revenue for a year. I mean, they were just a different scale of beast at that time, and it's certainly only gotten bigger and bigger. But the velocity they had was just hard to match. And so we sort of always went back to the strengths of the company in terms of our support. We used to have this thing called Fanatical Support, which was basically do anything the customer needed, do everything that we could to make sure that they were successful. I remember talking to one of our third-shift engineers overnight, he said, I've logged into people's AWS servers and fixed things for them and they're not even on our data center floor, they're in AWS, but they were integrated with our cloud and our server infrastructure, and so we were supporting some of that. And I think that's really where we always leaned, is how do we make the customers' lives a little easier, how do we support them? And to this day that's what Rackspace is doing, is helping people move to cloud, get to cloud. I've not been involved for many years so I don't want to speak like I know what's going on inside the company, but it's a great story of sticking to your roots and doing what you do best and not trying to be something that you aren't. They did a good job at continuing down that path of supporting customers and serving the needs of these companies that were trying to embrace new technology at the leading edge.

Yeah, and we did. Some of the products in that cloud context allowed you to push a template to the cloud and deploy a database and a web server and a MySQL or Postgres at the time — LAMP stack. And we did build developer-centric services. The OpenStack platform itself really spoke to people that wanted to build their own clouds for a developer audience. And the developer mindset, it kind of was intertwined at the time with the startup — startups and developers kind of went together. The most forward-leaning developers were sort of in that startup space and you really had to lead with documentation. That was kind of the way you marketed to a developer community, is by having really good docs and having really good a way to sort of engage them. And so putting a little bit of code on your website — I mean, you remember these days — that was sort of the way you enticed a developer to check out your platform: just make it super easy, make an entry point. That made it dead simple for them to get started. And we did try to build in the cloud at Rackspace the simplest way to spin up a server or to take advantage of that infrastructure. We wanted to make it super simple, so that was part of it as well.

So I've been listening to a lot of Reid Hoffman lately. He was on a recent episode of Diary of a CEO, which is a podcast I watch, but I've watched all of his Blitzscaling videos on YouTube. I mean, I think what we could have done is again try to build at that scale, go faster, you know, but again you can't do it if it's not in your DNA. I mean, we didn't really have an army of software engineers at Rackspace at the time, you know, we were mostly data center engineers, network engineers, system administrators. We were kind of a different animal. And so what I think we should have done earlier is just embrace the idea that people were going to be multicloud and put our arms around supporting and helping make that more practical sooner, right? And a lot of people struggled. Even the best software engineers are not always good in terms of keeping things running or making things work at sort of a large-scale, distributed-systems kind of scale. And we had some expertise there. Building these big, complicated deployments — the other thing we used to do quite well, every Christmas, this time of year, we would build out massive amounts of server infrastructure for whatever Xbox game was about to launch that Christmas, right? Every game developer needed tons of infrastructure, and then in the pre-cloud era this was a challenge, right? How did you get all that infrastructure for Christmas Day and then how do you taper it off after Christmas? And we had lots of experience and expertise at building out those systems in a physical world, and it wasn't that much of a jump into the virtual. In fact, it made our lives a lot easier. And I think we should have just leveraged that and sort of went out and told that story of embracing all the other clouds instead of sort of fighting with them, but really going and making them easier to adopt sooner. Rackspace did eventually do that, but it was not, I don't think, the right timing necessarily.

That's amazing. I've heard that before, it's a great story. Yeah, and companies still struggle to do multicloud stuff. I still see it, it's very, very hard to do. Everybody has that problem. I mean, I did a panel the other day with some CISOs, CIOs at a conference, and I had raise your hand if you're in more than two clouds. And most of them, almost all of them, were in multiple clouds, and then connecting into SaaS platforms and API integrations and lots of complexity, right? Everybody is dealing with lots of moving parts. And I feel like it's only getting more complicated, right? We're now living in this distributed world where we work from home, all of us are sitting in a home office, we're traveling all over the world and trying to continue to do business. The internet is now the backbone of modern commerce and modern business. That's really what — if you want to know what led me to Cloudflare, it's basically the idea that the world has changed, right? We are not in centralized data centers, we are not in headquarters buildings. The network is no longer private and controllable, it's sort of this wild west, uncontrolled world. And so how do you wrap some sanity and some control and some visibility around that? And that's what I think I see in Cloudflare's platform that's really, really powerful.

Yeah, it is a great story, great company, great people behind it. I got to know Cloudflare when I was at Rackspace in the early days. I met the founders out in Silicon Valley, you know, one of those developer conferences or investor conferences or whatever I was doing. And I met them in the very early days. I actually had a guy that I hired at Rackspace ended up joining Cloudflare in the very early days as well. He was one of their smartest engineers and knew how to make DNS work and all the things that you needed to scale at that time. And I paid attention to the company for many years. I spoke at one of their early conferences. I met Tim Berners-Lee, the founder of the web — I met him at a Cloudflare conference, so it was kind of cool to have dinner with him. But I paid attention to the company for many years and saw what they were doing. And I didn't go straight from Rackspace to Cloudflare. So in the interim, I joined a company called NTT, which is this big Japanese telco, and I did some large-scale deployments of SD-WAN and some things that were really transforming enterprise. And there is where I saw even more potential for what Cloudflare was building. Because they started, like you said, helping scale websites, helping make websites perform better, helping defend them from attacks — that was sort of the origin of the platform. They later added SSL, made SSL easy to adopt, sort of made it a default thing on the Cloudflare network so that people get security on their website, make it super simple. And then over the years just lots and lots of more products and services that are layered in.

But after the pandemic or even during the pandemic, I just saw the potential for what their platform was becoming gradually. Not just protecting websites but protecting end users as well. So it's a large proxy and it goes both ways, right? You put traffic in and proxy it through Cloudflare for websites — that means helping scale them. On the other end of the platform, the reverse proxy is sort of from an end-user perspective — how do you defend them from all the threats and all the things that might land on their desktop via the internet? And zero trust and that sort of architecture of replacing that legacy VPN infrastructure and using our Cloudflare network as the entry point to secure that traffic. And when you have sort of both ends of it, you really have some power because now you can optimize traffic across that network. So I'm sitting here in my home office going through Cloudflare's zero-trust platform, through our edge, through one of the local points of presence that's nearby, and connecting into — what are we using, StreamYard? I don't know if they're a customer or not, but lots of people are customers of Cloudflare on the SaaS platform side of things. And so therefore you have this really nice path between the origin and the user, and we can optimize that. We have this mission statement at Cloudflare to help build a better internet, and it doesn't mean build another internet, it just means make the internet better for the kinds of use cases that people want to use it for. And today business relies so heavily on it. In my early days nobody cared if the internet was down for a few minutes, but now if it has a blip, people care a lot, right? And the threat landscape is so sophisticated that people need to start to figure out how to defend against all the kinds of attacks that are going on. And I think that's really where we shine.

The tubes, the inner tubes. If you are curious about the Cloudflare architecture, one of the things that I tell people when I meet them for the first time and how it helps me sort of explain what Cloudflare is, because it's hard to put it in one sentence — but if you go to our speed test website, speed.cloudflare.com, anybody can do this, and it has a little map about halfway down the page. And the map will show you how you are connecting into the Cloudflare network and it'll tell you kind of where you are on the planet and where the nearest point of presence is. And it's not always physical geography, but sometimes it's based on routing, BGP and how you're routing with your upstream ISP. But the advantage is once we get you to one of those edge locations, we can control routing across that network. We can improve routing between you and whatever services that we happen to be interconnected with. And we're highly interconnected — with ISP peering arrangements, with network operators, cloud operators, data center infrastructure operators — all those people are peered with Cloudflare. And then on top of that we have our own fiber backbone that interconnects many of the locations around the world. And so I have found there is potential for us to improve the performance. I was doing a demo at a bank one time and we were on the bank's Wi-Fi, and they were shocked that the entire conference I was doing like a Zoom call or a Teams call or whatever it was, and I was sitting there on their Wi-Fi the whole time we were going through the Cloudflare network. And they said, this is better than we get. And then I showed them the routing.