head in the clouds (with Cloudera Founder Mike Olson)

Let me dive in and actually let me back up a whole bunch of decades. My very first job in the computer industry was working for a company in Kansas City, Missouri, that sold time-sharing computer services. This was '77-'78, I was in high school. At the time, companies wanted computers to do some data processing or maybe payroll processing; they had specific jobs they wanted to run. Computers were super expensive and also really finicky to run. So most businesses, rather than going out buying their own computers, installing them, setting them up, running them, keeping them happy, would contract with a firm like United Computing Services to buy time on computers that those folks owned, and they'd connect over a modem or a dedicated line. Maybe for 15 or 20 minutes a day, if they were super technical, maybe for as much as an hour or two a day, they would use the computer at United Computing to run their jobs. In fact, they'd be able to share it; a lot of people could use it at the same time.

One of our customers at United Computing was an engineering firm. Someone wants to design a building, you've got to know how sturdy, how strong all of the supporting members need to be, how far apart they need to be. A whole bunch of mathematics. You could do that with a calculator and a piece of paper, but you'd probably screw it up, you'd probably fat finger it. Using computer programs was a much more predictable way to do that. So they would write their own computer programs and then run them on the UCS machines. Every business that was a customer had some specific application it needed to run but didn't want the hassle of buying and running a general purpose computer. So they would just buy time on the ones that we had.

My answer is really it lasted forever. It ebbed and flowed over time. My contention is that the move to the cloud is kind of a move back to the philosophy of those time-share days. Buying and running computers is hard and finicky; securing them is difficult and the bad guys are clever. Why not find somebody who can do that for you? In the '80s and '90s, companies like Sun Microsystems, IBM, and Microsoft began to offer computers and computer programs that were way less complicated, and most companies were able to afford and operate these computers on their own. So they built their own data centers and stopped using the timeshare services and started using their own. That was decades of the industry. Through the '90s and into the early 2000s, most companies had their own data centers and ran their own computer programs. In 2006, Amazon, which had up to then been just an e-commerce company, a bookstore and then a general purpose e-commerce company, announced two novel services: the Simple Storage Service, S3, storage online, and their Elastic Compute, EC2. This was 2006. These were the very first AWS offerings. Amazon Web Services is now hundreds of different services, but S3 and EC2 remain. Those were very interesting novel services at the time. Most of us had wired telephones in our house; you could plug a cable into the wall and get dial tone. The analogy, at least in my view, from Amazon was being able to just connect to this internet service and get data tone. I could get a computer whenever I wanted it; I could get as much storage as I needed. I didn't have to plan ahead and buy disks and make room for them; I could just buy that stuff. Amazon pioneered the cloud, of course, and they were followed by a whole bunch of other very great vendors who offer their services as well. The growth of those services turned into what we now know as the cloud.

Fair summary. The reason that EC2 and S3, the reason that compute and storage made sense to offer was it saved me, just an ordinary guy living in my house with my kids in Berkeley, the trouble of needing to go buy a whole bunch of disks or buy computers that I would use for specific things. But it was still kind of hard to use. I had to go configure the compute instances in the Amazon cloud, be sure I got the right one, spin them up, spin them down. You didn't want to leave them running because the meter just turned over; it was sort of like leaving your oven on when you went away for vacation. There was a lot of complexity in it. As Amazon rolled out more services, there were more things I would need to configure. They built, and the other cloud providers likewise have built, a sort of ease of use and integration. The administration overhead has been taken away; they do those things now so that I don't need to worry about it. No doubt we're going to talk later on about security and privacy. Securing your data, making sure you don't have any back doors into your system is both imperative and surprisingly difficult. If I can rely on a cloud vendor who's got a lot of money to spend on those services and is going to be really expert -- probably better than me -- doing it on my own, I'm likely to get it wrong and get penetrated, whereas the cloud vendors are able to offer much more secure services.

I absolutely don't want to minimize the claims I just made about security and configuration and operations. All of that stuff is hard work and the cloud vendors are really good at it. So offloading that work to those folks makes huge sense to me. But flexibility is a really big deal. Imagine I've got an idea for my business. I think if I can build some new software, I can measure how customers interact with my online services, and I can direct much better offers at them, give them more of what they want. But that's just my idea; I don't really know if that's the case. If I got to walk down the hall and talk to the CIO or the CFO and ask for 15 computers that I can put into our data center in order to run this experiment, it's going to be a tough conversation. Even if they say yes, they're going to buy them, and then if my experiment fails, they're just going to get mothballed. Even if they agree to that, I'm going to have to call up Dell or Supermicro, order the machines, it's going to take a month or two for them to show up because of supply chain problems. When they get delivered to my loading dock, it's going to be a bunch of boxes; I'm going to have to have a whole bunch of IT staff go rack and stack and cable those things up. We're gonna have to be sure we got enough electricity flowing into the room, it's gonna generate a bunch more heat, do I have enough AC? If I can just go touch the Microsoft Azure button and get the 15 servers I need and use them for a week and a half and spin them down with no fuss and no obligation to get rid of the boxes at the end, I'm going to do those experiments a lot more. That flexibility is absolutely huge.

If I'm running computer services, I still need to think about security: how do I protect my passwords, when I hire and fire someone, do I give them and then revoke their access to my systems, and so on. But baseline security – is the very latest version of the kernel and the operating system and all of the key utilities installed, and is somebody monitoring the network looking for people that are trying to get in and noticing when that happens and shutting down those accesses – that is super complicated. It's something I'm likely to mess up because I'm not a specialist. I'd much rather staff that out to somebody who does nothing but that every single day. All of the cloud vendors offer remarkably good security services.

Their problem is that they are very big targets because of the volume of data that they have. I would argue that it's not multiplicative, it's not even additive. If you've got to secure your systems, you fundamentally need to secure them, but adding one or two new customers doesn't make it fundamentally harder to secure. Your first 100, your first thousand, maybe your first million customers are hard, but after that, the incremental cost of making sure you stay secure as you deploy new hardware doesn't get worse. But it becomes an attractive target, as you say. Why do you rob banks? Because that's where the money is. Why do you attack a cloud vendor? Because if you get in, that's where the data is.

I'm gonna answer the question in a couple of different ways. First, there is just some gritty technical work you need to do if you're going to be sure that your systems are secured. A lot of liability you take on too. Hackers are trying to break into systems all the time; it's a very clever constituency. As vendors release new code and that gets deployed in the cloud, that opens opportunities for basically new vulnerabilities. There's a big attack surface if you're running a lot of software at massive scale. The cloud vendors are able to spend a lot of money to solve that problem: hire the very best people, send them to the conferences that they're going to learn the most at, educate them continually, and work very closely with the software vendors to secure that stuff. It's not a small problem, it's not a solved problem; there are vulnerabilities in all the software. But my point of view is that Amazon, Microsoft, and Google do a much better job of that than the local credit union is going to be able to do. But then there's this other philosophical question. You and I, all of us, we've got an enormous amount of personal data that we just sort of glibly store in the cloud, and most of us don't even know what that means: which cloud? I drove down here from Berkeley to Palo Alto, and I have a Fast Track transponder in my car. The reason I like that transponder, it lets me blow right through the toll booths on the bridges. But it turns out there are transponder readers buried in the road. When you see the signs on the side of the highway that tell you how long the driving time is from Burlingame to Oakland, the reason they know that is they're watching the cars go by with the transponders in them all the time. Nobody -- the convenience of blowing through the bridges and the convenience of knowing how long the drive is going to be has totally inured us. We didn't even ask the question: should that data be collected? Should it be kept? Where and how well? Who has it? Is it the government? I don't know the answer to that question. I don't know what the storage philosophy is there. And the government would contract out to a cloud provider perhaps to store that data anyway. Even if they're the ones who are collecting it, almost certainly the state of California is not running that infrastructure; it's working with some third-party vendor that provides that service. Then you've got to ask: are their security, where's their data stored?

Almost certainly that was traffic cameras photographing people that were running red lights. Those cameras are now pretty widely deployed. In Berkeley, all of the traffic and all of the parking enforcement little cars have cameras on them and are able to roll along reading every single license plate they go by. That data gets uploaded to a third party who actually keeps it for a year. Look, it's useful for the police because if you're a scofflaw with a whole bunch of tickets and you're parked somewhere, they'd like to boot your car. That's a perfectly legitimate enforcement job. But if I'm legally parked, why should my license plate be saved? 'You have nothing to hide,' as they like to say. But if you had all that data, you could start to notice who frequented what areas by what license plates were where. You could notice perhaps I am commonly near the same coffee shop regularly. Perhaps you could start to know who I'm associating with. If you were a stalker, that'd be great data to break into because you'd be able to target whoever you were stalking. There are a lot of ethical questions around that data. We need to ask those questions as a society. I don't think we do a good enough job at it yet. I do think that for the big three cloud providers -- Amazon, Microsoft, and Google, actually I'll add Apple -- at least their policies on data retention and access are pretty well spelled out. I think this topic is worth exploring a little more for those four in particular. But let me pause because I've kind of rambled on for a while. Any questions on this stuff?