Security in the Age of AI: Vanta CEO on Compliance and Risk

Thanks so much for having me.

Yeah, I think there's a ton of excitement about these technologies. I mean, some of the way I think about it is they're letting builders create these magical experiences that people have thought of before but have never been able to do. So a specific example in Vanta's context is security questionnaires are just a part of selling software to enterprises. Basically you get this long spreadsheet of questions you have to answer. Everyone has always wanted to build a product that just answers the questions correctly the first time. That wasn't possible prior to some of these foundation models, but now it is. And so just an example of kind of the magical product experiences that can be built now, but couldn't even three, five years ago.

Yeah, there definitely are risks. The one stat I found interesting is over half of the Fortune 500 companies in their most recent annual financial reports cited AI as a risk factor. And so, right, it's kind of prevalent. I think I think about it in two ways. One, just general technology deployment. AI is a new technology. We're still figuring out its parameters. Anytime there's something new, there's a ton of excitement, but there's also just a ton of risk as we figure it out. So, there's something just general there. You know, there's a story of mobile 10 years ago, maybe cloud 15 years ago. It's AI now. And then there's a specific piece of these tools can create incredibly accurate text and video that looks like it's real or you know, but might not be. And so there's a whole host of attacks that are now again much easier. It's sort of a different magical product experience that we might all think is actually less magical or the result is. But those have also become much easier as well.

Yeah. So I mean we see some of these at Vanta too. There's a lot of impersonation. It is just easier to, you know, I can think the old way was the joke at Vanta was if you joined the company you would get three text messages from me asking me to buy you gift cards, but things were probably like somewhat misspelled and you probably wondered, you know, does the CEO really need me to go out and buy gift cards? Again it was sort of this, you know, attack that was easy to laugh about. That one still happens, but there's versions of that that are much more compelling. So, CrowdStrike has recently talked about how they had one of their customers, I think it was the CEO, but an executive at one of their customers actually impersonated in this credible way and, you know, go ask employees that I think over video to go do something for them. And so again, it's kind of you can see some of these same attacks, but just way more compelling.

Yeah, I think you're totally right about the, you know, audio quality getting quite good, the video quality quite good. And I think maybe just taking a step back, one of the things that is so exciting about this period is like that technology has gotten so good is opening up lots of new use cases and products. So again, one of those is maybe phishing somebody else's employees and trying to get their information, which probably not a good thing for society or markets or really anyone. But another one is we're seeing companies build like wholly interactive experiences that again help them deliver their products and services even better. And do things, you know, for good in the world rather than again try to try to scam people out of their money. But these technologies can be kind of used for both ways. I feel like this is a lesson we've really learned over the past 10 years of social media is there's a bunch of great things that can come out of new technologies. There's also some things that are less great or less expected. And I think we're really seeing that sort of parable play out in the age of AI as well.

Yeah. So, Vanta was started under the dual mandate of helping companies initially build out their security program and over time understand it and improve it, and then that's all a bunch of hard work, right? And get credit for that hard work of building their security program with the market, with their customers, with executives, with their board. And so, part and parcel, we've always tied the security work we help companies do to again customer outcomes. And so the first place we started doing this were compliance reports. So we help companies get SOC 2, ISO 27001, GDPR, HIPAA, tons of standards like these. We help with the security questionnaires. We help security teams demonstrate all of that hard work they're doing. That helps customers build trust and ultimately grows their company's revenue.

We are in a bunch of different ways. We started building AI in from the ground up across our product. So everything from helping a company decide which security controls are best for them given their maturity, given what their customers are asking for. When there's a, you know, error, vulnerability, how do you fix that? AI is helpful for giving a first pass or a suggestion for the team to take action on. And then a lot of the process of these audits or requirements is just a lot of documentation. Back to the security questionnaires I talked about at the beginning, there's, you know, you have to do the work and then you have to tell people about how you did the work. When I talk to security professionals, they usually really like doing the work, that's why they got into the job. They generally don't like having to tell people about the work in a bunch of different forms. So, you know, an audit, a screenshot, a security questionnaire, and a conversation. And so, one of the places we've seen AI be really effective is again to take all that work that was done, summarize it, and put it in the right format. So the teams can focus on actually doing the work.

So it is definitely generative AI models, kind of the foundation models plus some post-training and a ton of kind of quality improvements to make sure there's high accuracy. And security compliance is a place where I think there is less tolerance for creativity and true generation and, you know, more desire for accuracy. But basically take these models and not observe what people do but observe the outputs. So what have they done, like what was there before, what did they do, look at the output rather than kind of the work itself, but then take those outputs and say, okay, maybe you went and I don't know, changed a bunch of configurations in your cloud infrastructure in a way that's more secure. Okay, we can take that new configuration, you can turn that into a policy document. So there's a standard kind of document written in sort of legalese or compliance that describes what there is. Take that configuration, translate it to answers for the security questionnaire. So the next time you're asked about your cloud infrastructure configuration, you got the answer there direct from, you know, the reality of what's in the system live. And then finally take some of, again, take that same information, map it to a bunch of compliance frameworks. And so if you're going through an audit or going for a particular certification, all that information is there up to date. And again, no one had to kind of keep it up to date or remember, okay, after I went and changed all those configurations, now I need to go update the documentation. Documentation is just there for you.

I think we're, so not a, not a, I don't think we're doing anything that like other leading AI companies aren't doing honestly, but a couple of things. A bunch of post-training, a bunch of prompting around like, you know, just do not hallucinate. There's parameters inside some of these models to set where you can sort of tell it like how creative do you want it to be and we generally say please don't be creative at all. And then we just keep humans in the loop on both sides actually. There's Vanta team on the post-training, we have this golden data set and so you're constantly kind of checking and making sure that from the data we see, not customer data but data we generate, how are we doing on that and it's a metric we keep track of. And then also our customers in the loop. So sometimes I describe this and people think, oh, you just fill out the questionnaire and then send it back to the customer and no one sees it and that would be a magical experience if these things worked, but given the domain we're in, that is not a magical experience, that's a very scary experience. And so to that, you know, the way it works is we'll take a, we'll try to answer all the questions. We put it in front of the end user and say, 'Hey, what do you think of these?' and actually then look at that score and, you know, they can go change things, they can go send it, they can go about their day and then our system gets smarter because we're learning from the places where, you know, our end user accepted an answer versus rejected one. So humans in the loop in a couple of places given the domain we operate in.