I think the incentive structure we tried to gamify the Champions program actually didn't work. We realized that the incentives were in them starting to get some kind of badge of honor of being a security person. That badge of honor was not a gamified approach; they actually wanted certifications and security training.
Hi, you're listening to The Secure Developer. It's part of the DevSecOps community, a platform for developers, operators, and security people to share their views and practices on DevSecOps, Dev and Sec collaboration, cloud security, and more. Check out devseccon.com to join the community and find other great resources. This podcast is sponsored by Snyk. Snyk is a dev-first security company helping companies fix vulnerabilities in their open source components and containers without slowing down development. To learn more, visit snyk.io.
On today's episode, Guy Podjarny, founder of Snyk, talks to Rinki Sethi, Vice President and Chief Information Security Officer at Twitter. Rinki is responsible for leading efforts to protect Twitter's information and technology assets and advising the company's continued product innovations in the security space. Prior to Twitter, Rinki was most recently the VP and CISO at Rubrik. She has been at the forefront of developing cutting-edge online security infrastructure at several Fortune 500 companies such as IBM, Palo Alto Networks, Intuit, eBay, Walmart, and PG&E. We hope you enjoy their conversation, and don't forget to leave us a review on iTunes if you enjoyed today's episode.
Rinki, excited to have you on. Thanks for coming on to the show.
It's an honor to be here.
So Rinki, before we dig in, can you tell us a little bit about what it is that you do today, but also a bit about the journey that got you into security and into the role you're in today?
Today I'm the Chief Information Security Officer for Twitter, so I run all of information security at Twitter. I've been here for just about six and a half months now, and I joined at a really interesting time at Twitter, when Twitter had gone through its breach, but it was right before the election. It was a really interesting time to join as the head of security here at Twitter, which I can dive into more later. When I was in high school, I remember before I knew what cybersecurity was, before I knew what hacking even was, I remember that instant messenger—AOL Instant Messenger, for those of you that used it, dating yourself—yeah, I am dating myself. But that was the new thing and how you communicated with your friends. I was a late adopter of it compared to my friends. I started chatting and I thought it was the coolest thing ever, until I overheard my dad telling my mom about something I had typed to a friend. I was thinking, how the hell did he know about that? So I started digging around in my machine and realized what we now call a keylogger was something he was aware of. It was a parental spy tool that he had installed on my computer, recording all my chats and reading them, probably to make sure I wasn't talking to boys or something like that. I found out about it, told my sister, and we started messing with him. I deleted it at first, but then I'd have to go and check again, so I wrote a little program that would alert me every time he installed it. We would actually mess with him, start writing stuff that wasn't real, and then hear him talking about it later. So I think my mind was always a little twisted in that direction towards that security hacker type thing. Although I left that alone, I went and pursued my computer science engineering degree at UC Davis. When I graduated, it was a real bad time in the economy, and I was willing to take any job. I just wanted to have a job when I graduated. Fortunately enough, I was recruited by PG&E, and that was my first role in becoming a cybersecurity analyst—back then they called it an information protection analyst. I started my career there and have been in cybersecurity for about two decades now. I worked at Walmart.com, at Intuit, at Palo Alto Networks, at eBay, at IBM for a really short stint, Rubrik, and now here at Twitter. So I've been around the block in terms of industries, types of companies, and cultures as it relates to cybersecurity.
Yeah, absolutely. And I love the motivation to get into the world of security through teenage insubordination. As a parent, I think about the appeal—I've resisted it so far, but it is tempting. I know how to do it and track the kids' activities, but I've preserved their privacy to date. I hope I keep that up. Let's dig a little bit into those companies. I think one of the super interesting lenses you have on the world is the different perspectives you have on product security. You've worked at companies big and small, you've touched security as a whole and product security specifically. You've seen the industry evolve, and you've even been over to the dark side a bit, being at a security provider like Palo Alto. How do you think about product security and how you approach security? What are your key takeaways about how it's different across these different profiles of companies?
It's so different from company to company, industry to industry, and depending on what the company itself is going through at the time. They might be going through a transformation. When I joined Walmart.com, product security was easy. It was a really small development team—Walmart.com was a wholly owned subsidiary by Walmart—so it was easy to influence, easy to train developers, easy to push security into the product. Product was different back then; it was a website, e-commerce was a new thing, and there was a different type of culture at Walmart.com. Then going to eBay was very complicated. When you thought about the eBay platform back then, developers were given a lot of access. There was this culture that made it such a cool, innovative company at the time that let developers do what they want. It was very difficult to bring security in. It was a larger company too, and how do you scale security and train developers? That's when I introduced at eBay the idea of security champions and really training and developing core competency into the development teams, because it was the only way to secure product and really scale product security. Then I went over to Intuit, and I actually led product security there. It was a really interesting time because Intuit was broken up into business units: TurboTax was a business unit, Mint was a business unit, and they had a small business group. Each one had completely different development practices. One was in desktop packaged software where you bought TurboTax from a CD, the other was starting to be completely in the cloud. So you saw all these variations, and product security was nuanced. The product security organization was centralized, but the development teams were separate, so product security was one central entity supporting all these different groups.
That's exactly right. Such an interesting model. Again, it was in a position where you can't scale that. How are you going to have a central team when everything's different? Maybe you can hire one person for the big business units, but what beyond that? The other interesting thing that threw a monkey wrench into product security and how we thought about it was walking away from desktop packaged software and moving to the cloud, but not just private cloud. They were moving TurboTax data—they were the first company to move very private, sensitive data into public cloud AWS. So they were going through a massive transformation, re-architecting applications, having to rethink product security as a company. Again, that security champions model started coming back. Very different from company to company. And you talked about me being on the security vendor side at Palo Alto Networks and at Rubrik. Thinking about how we did product security there, it was again a little bit simpler, but then you're also thinking this is critical infrastructure for many companies where you play as a security vendor. The scrutiny and importance of security was at a whole different level, and you have to role model what security is for the rest of the world at a company like that. Very different from company to company. And of course, Palo Alto Networks had a hardware business, so your product is not just software. Different nuances, different challenges along the way, different ways of organizing, different cultures, different success stories in how you organize. Really interesting.
Yeah, it's fascinating. There's a world of questions that come out of this. For starters, when you refer to security champions and you're describing things very much in relation or proximity of the security organization to the development organization, is that a correct way to read what you're saying? You talk about how at Walmart it was easy because it was small enough and close enough, while at Intuit it was hard because you were distant and it was very diverse. Is that the fundamental thing, and you address many of those with security champions? The conceptual thing is that security champions is the implementation, but what you're trying to do is achieve proximity between the security practice and the development practice. Is that a decent representation?
I like the way you articulated that. I think the proximity is such an important thing, and that's what you're trying to get. The closer you are, the more effective security is going to be for the company. Proximity is one way of talking about it. The other way is, you know, at Twitter we talk about embedded security, same type of thing. Champions takes different forms depending on what company or what team you're talking to, but it can also be that you're actually building security into engineering. I think it could be an actual full integration, and you're getting as close to that as you can in building that champions program so that you can scale.
For security champions, what do you feel are the core tenets when you say security champions program? What does that mean to you?
I've failed at this actually more than once to know what works and what doesn't. What doesn't work is that you say, 'I'm going to go identify security champions,' you tell them they're responsible for security, they're not trained, you don't have a long-term strategy for how they stay trained, there's no accountability as to what you're expecting from them, or that they're the sole representation for that business or development team. It might even create a bottleneck in ways you don't want it to. Those are things that don't work. When I think about security champions and successful models, it's either where out of engineering or development teams you actually identify several folks that are responsible for security. You build a scorecard that they are responsible for that development team, showing the things we're doing to ensure we're securing our product in the right way or our feature in the right way. There's some kind of pullback into the central security organization, and we're also providing them with the latest and greatest training. If the security team is responsible for building tooling and things like that, those security champions are helping you go and drive adoption. Security champions have to be within the engineering organizations; they can't be sitting outside of it. They're really driving security, and you're holding them accountable with some kind of accountability model and pullback to the security team.
So the security champions in this model have official time allocated to this from their management. How do you kind of tee them up for success around being that representative for security within their daily jobs?
It has to be tied back to goals. There has to be a percentage of time dedicated to it. If it's just extracurricular work, it's never going to work. That was actually one of the failures I had at Intuit. We didn't have the buy-in from the top; I was working the program ground up, and it just wasn't the right way to do it. As soon as we went to the top and said, 'Okay, GM of this business or engineering head of this business, we want you to help us make this a part of the goals for the security champions,' as soon as that flip happened, you saw a different accountability model, a different engagement back with the security organization to help craft what the model would even look like. So you have to have this carved out as a part of an individual's goals.
Yeah, no, it makes a lot of sense. You talk about pullback to the security organization. How do you see the delineation between what's the responsibility of the security team versus what's the responsibility of these security champions?
This is an interesting question. Security is so nuanced; there's not one size fits all. When you talk about finance teams and other teams, it's pretty much similar structures—you have an accounting team and a controller team, and you know what to expect. Security is so nuanced company to company. But I think the ideal model is security builds the standards, security should be building and owning the services, and that team drives adoption. In many cases where security doesn't have the capabilities to build services, then you might be asking engineering teams to do that or building a central team within engineering to build the right services for security. It's ensuring that there's a real back-and-forth feedback channel between the two teams, where if a tool's not working, there's feedback back so you can improve that service or tool from a security perspective, so that the teams are driving adoption and truly reducing security risk. I also think where it works well is where security is plugged into development and understands that their customer is engineering in this case, and they're really feeling the pain of their customer as well. So what we talked about—the proximity—is so important.
Yeah, I mean, it's hard to define those, but I like the general guidelines you're describing. You talked about how it's a failing if you don't train them or equip them to do it. What are some suggestions of how you can help beyond ensuring their management is aligned? From a security team perspective, what type of training or enablement do you need to give these security champions to make them successful?
If you're launching a security champions program, this was another failure I had that was really short-term thinking in terms of what training content I was going to have available and what I thought they needed. I was wrong on both fronts. I didn't test it out properly with the engineering team. I think having a pilot group that you actually test out initially is key. Say, 'Okay, I'm launching this security champions program,' and engineering teams are not shy of giving feedback, so they will help you shape the program in the right way. As soon as I engaged the engineering teams—this was iteration number two after failing the first time—what happened the first time was I developed content for what I thought was six months of content for training, with different modules: ensuring you had the right unit tests, training them on security standards, or even more specific training to different languages. When I developed that, the engineers were able to consume it in less than two months, and they were hungry for what's next and how to apply this. So what I realized is you need to have almost 12 to 18 months of training content. It needs to be very applicable to their role, but it also has to be a combination of practices and standards that you're training them on that are specific to your company, not just generic off-the-shelf training. Then there are some more future-looking things as well. Developers were craving things like SANS trainings and certifications. They almost saw that as an incentive to continue because they were starting to get security under their belt in addition to their core engineering role. The incentive structure we tried to gamify the Champions program actually didn't work. We realized that the incentives were in them starting to get some kind of badge of honor of being a security person. That badge of honor was not a gamified approach; they actually wanted certifications and security training.
I think that makes a lot of sense. In a recent roundtable I was on, we got to an interesting perspective around the delineation between developer security training and the training of these security champions, not just from an investment but from a motivation standpoint. It sounds very similar to what you're talking about. For a lot of developers, maybe they conflate the two together, but in practice, there are developers who just want to get back to their day job, so for them you might want minimalist security and you might need gamification to lure them in. But the security champions are security-curious, hungry for more, so for them you actually want to give them high engagement, high depth, but they're still engineers, so you have to keep that in mind. I think it's a similar perspective to what you're describing.
That's exactly right. I think I was more gearing towards what the champions are looking for, but in broader engineering teams, I get into debates about the carrot and the stick approach. Whether you call it the carrot or the stick, I think recognizing developers that are doing the right thing—whether you gamify that or not—but then you have to hold those that are not accountable as well. There needs to be a combined approach. Whether you want to call that carrot or stick, or accountability and recognition, I think that's a really important concept more broadly.
Yeah, it's amazing how important naming is. Accountability and recognition are both good terms, while a stick sounds bad, and even a carrot sometimes sounds condescending. But recognition and accountability are really good words, so I'm definitely going to adopt those. Before we talked about the journey, we actually stopped just shy of Twitter. When you're at Twitter—and I recognize you're only there for about six months—what are you looking to apply from this sort of champions program? Is there already a champions program running?
We actually have a pretty large champions program at Twitter already. When I joined, I think they're really good at training and ensuring that if there's something new, like a new standard, people are aware of it. It's the largest champions program I've seen so far that I've worked with. But I think there's one thing that's still missing that we have a bit of a bridge to build: that proximity to engineering and really integrating security into everything they do. Right now, we're making it really hard for engineers and developers to do security. There are a lot of manual processes. If they want to introduce a new product or feature, they go through ten security reviews—one for privacy, one for security, and so many others. I think the more we can automate, the closer we get to them to understand that this is not going to help us innovate quickly. There's a lot more work we need to do, whether it's champions or building. Even from a central security perspective, we're starting to think about automation, the right tooling and services. One of the products we're looking at is Snyk because of that—how can we get closer to the engineers so that it's not that they don't need to worry about security, but that security is enabling them to move faster rather than slowing them down. So I think there's still more work we need to do on that front.
