How Safe Is AI-Assisted Coding? | Tariq Shaukat (Sonar)

Sure. Well, I'm Tariq Shaukat. I'm CEO here at Sonar. Pleasure to be on the podcast, so thank you for having me. I joined Sonar about a year and a quarter ago as a co-CEO with our founder, and then more recently as CEO as he stepped more into the founder role as opposed to the co-CEO role. Prior to being here, I've got a long history inside of various parts of the tech world, both as a user of technology on the enterprise side as well as somebody working in the tech space. I was prior to here president of Bumble and helped take it public the last couple of years, and prior to that was president of the cloud group at Google for a number of years, helping to get that really off the ground and scaled to the substantial scale that it's at now.

Sonarsource.com, exactly right. So Sonar is, we believe, the leader in code quality and security. We started off really 15, almost 16 years ago now, as an open source project building a product called SonarQube that really inspects code as developers are writing that code. It inspects it for what we refer to as quality in general, to make sure it's clean. What that means is we look for bugs, we look for maintainability issues, we look for stylistic issues, we look for really anything that could be wrong with the code as you're writing it, including security issues as well. We do this in 30 different programming languages, we do this globally, and it's turned into not just the open source project—which is still available and a lot—we have 400,000 organizations who use our product, over 7 million developers use our products. So we've got both that open source but also commercial editions where we've got some of the largest enterprises in the world all the way down to two-person startups using our products to help improve the quality and security of the code they're writing so that their software is better and more maintainable and has less tech debt.

There's still a good part of the product that is deterministic and rules-based, and we actually think that that is a real plus in a GenAI world. Because one thing that GenAI and all non-deterministic systems have is, by definition, you can't quite predict the outcome that you're going to get exactly. For things like quality control, it is helpful to be able to have repeatability and certainty as you are doing the verification and various analyses. So the history of the company has been in deterministic, rules-based analysis of code. We have been supplementing that recently with a couple of different things. One is looking at how AI in general, GenAI in particular, can be used in some of these gray areas. There's some types of problems that deterministic systems are really good at, and there's others that are a little bit more gray where you need a little bit more judgment, you need to take into account more context. And we're starting to look at how can we use AI to really supplement what we've historically been doing, so that's certainly a key piece. And then the second is, our historical business has been in finding issues in code, and it's up to the developer to fix the issues. And what we are using GenAI for is to help suggest fixes for the issues that we uncover.

Yeah, yeah. So SonarQube can run in the IDE. We have an IDE plugin for a number of the most popular IDEs, the development environments that are out there. And then we run either on-premise or in the cloud as you're starting to do your pull requests. And really the source of the code doesn't matter to us, right? What we look at is the code itself. And so I would say the majority of our customers now, anyone who's using Copilot as an example, or Codium or CodeWhisperer, more likely than not they're actually taking the output of Copilot and running it through SonarQube. We've actually recently launched a specific workflow for this that we call AI Code Assurance, which is meant to—we don't see ourselves as a code generator, this is not the business that we're in. We're happy to work with Google and Microsoft and Amazon and Codium and Cody and all of these other players that are out there. What we really see ourselves in is providing the workflow and the assurance that the code that you get out is high-quality code, that it's not going to cause you maintainability issues down the line, that it's not riddled with bugs, and that it's not riddled with security issues. And so we really work in tandem with any of the code generators that are out there.

Yeah, it's a much more integrated workflow than that. We fundamentally view ourselves as being a developer assist tool, for lack of a better way of saying it. So meaning we want to work really with developers to improve their workflow, improve their productivity, cut down on a lot of the nuisance that they have and non-value-added work that they do. And so everyone I've seen is using some level of code review for the code that is being generated by any of the code generators, including Codium. What we see is the industry sort of norm at the moment is only 30% of suggestions coming from Copilot as an example are actually put into—are actually accepted by the developer. And so it's clear, I think most people recognize that there are issues in the code that comes out just like there would be with a human. And there are different types of issues—in some cases you don't have as many stylistic issues with GenAI, it's pretty good at the spell-check types of things, you know, normally put a semicolon where you should have put a colon and that sort of thing. But you do have some more complex issues—sometimes it makes up variable names, sometimes it doesn't operate in the context of your codebase correctly, things like that. And so that's where we really, as you say, we sort of take the output, run it through our analyses. We do that in the IDE and then a more complex analysis as you're issuing the pull requests themselves. But it's not quite as manual as a copy-paste, copy-paste. We've actually done a lot of work with GitHub and others to really integrate into the workflows that they have and be kind of seamless for the developer, or as seamless as you can be for the developer.

They're just different plugins inside of the same IDE, that's right.

It depends on the type of analysis that you're running, right? Inside of the IDE, we have an IDE extension for SonarQube that will do, for all intents and purposes, continuous inspection of the code. Now there are some types of analyses that require just more computation, more time—the more sophisticated analyses of the code—so that doesn't happen right as you're writing the code. But a lot of the analyses do. And our philosophy is, as much as possible, not even—there's this expression that's used of 'shifting left,' we actually try and talk about how we 'start left.' You want to start as early as possible with the developer and catch issues while they're being created. So there is a lot of the analysis that is being run fairly continuously in the background. We do a lot of work to make sure it doesn't slow down the workflow, that it's not distracting to the developer, we try and optimize the number of true positives and minimize the number of false positives, things like that in the IDE. But then really where the next level of analysis happens is when you're issuing the pull request, the merge request, right, and you want to actually put the code into your main code base. And that's where a lot of people would do code reviews, and we essentially do an automated version of that. We identify all the issues that we think the code has, we'll tell you how severe we think the issues are, if you click into the issue we'll try and explain the issue to you as much as possible, like why we think this is an issue, and so there's an education component to it also. And then with our AI Code Fix, for certain issues where we have high confidence, we will suggest, oh, if you accepted this piece of code to replace that piece of code, that would fix the issue. But it really happens—we try and have it happen as early as possible in the development process, in the IDE and at that pull request phase.

Yeah. At the moment we're doing a lot of pre-training if you will of the model, so that the acceptance rates we're seeing are about double the acceptance rates of your standard code generator, because we have a lot more context than a typical effort to create new code has. We know what the issue is, we know what was causing it, we can kind of use that to create the prompt and put it into the LLMs in a specific way. So it's not pre-training exactly, but it really is a higher-fidelity way of generating the response. And you can absolutely have the option of running the analysis again. What we find is that most people at that point then subject to human code review as well, because all good code reviews have an automated component and a kind of expertise component to it. And rather than getting stuck in this do-loop that goes over and over and over again, we find that developers just, to see the fix, they look at it, they agree to accept it or not, and then they go on with their process. So you can, but it's not the typical way.

Yeah, I think there's actually a couple of different aspects to this. One of them is, you're absolutely right. GenAI code has bugs, it has stylistic issues, it's got maintainability issues, it has security issues that are getting introduced, and in many cases it's actually introducing new vectors on which you can have a problem. There are new ways of potentially causing security issues inside of your codebase using GenAI that really didn't exist as much in the more traditional world. So you do have—GenAI code is not perfect. And it's not for English language search things either, right? This weekend I was trying to write something for a board and I said, 'oh ChatGPT, give me a bunch of sources,' and it turned out it made up three of the sources. And thank goodness I double-checked the URLs that it gave me. So it's a good lesson learned that the way the math works on GenAI systems is it is going to have hallucinations—they're a feature, not a bug, in many cases—and that does have real-world implications on your code base, on your security, and on the reliability of the software that you're shipping. But I'd say that's a known problem. It may be not a fully appreciated problem, but it's a known problem. You can look at the research, there's a lot of research that's been published recently that talks about this issue. The part that we think is as big a problem and maybe a bigger problem is that while people, if you push them on it, they understand that there's issues, there is this accountability crisis—I really think 'crisis' is starting to be the real word here—of who's actually responsible for the AI-generated code. And it's very easy to say, 'oh, the developer that's using the code generator is,' but it's almost human nature—you don't hire a software developer, they didn't go to school, they didn't learn how to write code to become a copy editor for AI. One of my customers, a large financial institution, told us a story a couple of months ago now of—they were starting to have a minor outage a week because of AI-generated code. And it's not because the systems are flawed—the systems are actually quite good—they're just not perfect. They have this one example of—they figured out there was this line of code that was buggy, they traced it back to the developer who checked it in, they went to the developer and said, 'hey, so what happened here, let's do the postmortem.' And the response was, 'well, that's not my code.' 'What do you mean? You checked it in.' 'Well yeah, but the AI gave me the code and I just accepted it.' And I'm not at all saying that developers are being lazy or not doing their job—it's that we're asking them to do a job that really is not what they signed up for, not what they're trained to do. And humans, unless this is your skill set, you're not trained to sit there and look at hundreds of lines of code that are being written by AI and double-checking it. And so this is why we think having some automated checks, tools like SonarQube, this AI Assurance workflow we've put in place, is so important, because you have this accountability crisis and the solution to that is to make sure you've got really robust workflows and you're doing rigorous code reviews. And by the way, most developers who are listening would say 'duh, of course this is what you should do'—this is like best practice in software development in general. And that's true, but I think there's been this misperception in a way of 'this will be easy in the AI world,' and I actually am finding that it is substantially harder for the reasons that I said just a minute ago.

Yeah, and code reviews will miss things as well, because we're only human, and I guess at this point we're all also only GenAI. And there's going to be bugs in the system, if you will. I think part of the question is how do you—we find that organizations that employ rigorous quality gates, which is something our product offers, which means that you just can't go forward until you resolve these issues, have substantially better reliability and security and developer productivity actually than people who don't. Because there's a lot of judgment that comes into code reviews—not only is this an issue, but is it an issue that needs to stop production? This is the key thing. There's always a trade-off, there's always a time pressure. Part of the reason we want to shift left or start left as much as possible is because when you're at the point where the product's supposed to ship in two hours and you just find a bug, now you're the person holding this up. This is not a position anyone wants to be in. And you cut corners, knowingly or not knowingly—I don't think anyone would describe it as cutting corners, but you do, because that's just life, that's the way time pressures and things like that operate. So the earlier you can detect these issues, the more likely you are to fix them and the cheaper it is to fix them as well.

Well, I think there are certain types of problems where the deterministic model works really well, and if it ain't broke, don't fix it. There's other areas where it doesn't work that well, and these are problems that have not been addressable in the past. And so I think those are areas where new techniques are really good. And so our approach is very pragmatic, almost transactional. If this problem can be solved deterministically, we like the predictability of it, we like the cost of it, we like the efficiency of it—there's a lot to like about that. If the quality can be improved, or if it can't be solved with deterministic models, then how do we supplement those models with other approaches? And it could be GenAI, it could be other more probabilistic analytical approaches as well. I think fundamentally I believe that a lot of these systems are going to end up being a hybrid of deterministic and non-deterministic approaches that are brought together, where the problem itself determines what type of solution is being used.

Yeah, so to tackle the second part first, and I'll talk about the first part. We view GitHub and GitLab and BitBucket and all of these different players as really good partners. We do a lot to integrate with them. There are certain things that we're really good at—we've got over a decade of experience, we've got deep IP in—we understand the nuances of this sort of code analysis, particularly for quality and security, we think as well or better than anybody out there. And so to us the question is absolutely, if we can integrate this so it is completely seamless for a Codium user or for a Copilot user or a Gemini user or whatever it is, we would love to do that, and that is what we are doing. So that is very much part of the strategy. We think that there's some pieces of IP and context that we have that's pretty unique that helps you—as I mentioned—solve some of these problems earlier. So you can throw brute-force analytics at certain problems, but in some cases really understanding exactly what's causing the issue and having that full set of know-how helps you get to the answer faster than just the brute-force piece. And so that's why we find that our code fix suggestions have a 2x higher acceptance rate than your sort of generic code-generated fixes. It's because we can supply it with that much more detail of the analytics of your codebase and things like that. And by the way, the same thing holds true on the first part. There is very little inside of Sonar that makes us likely to be the best at generating code from scratch. We're not good at foundation models, and none of these players are building their own foundation models to my understanding—maybe Microsoft is—but we're not good at foundation models, we don't have proprietary data sets for code model training. Most of them are trained on the same code bases of open source and competitive coding competitions and things like that. So it really comes down to—it's a different problem than what we're trying to solve. What we believe we're really good at is taking a billion lines of code, which is the size of the code base of a lot of our customers, and really decomposing it—deconstructing is a better word—and figuring out how do we make sense of it so we can find these problems. And that's a different, we think equally valuable, but different problem than creating code from scratch.

Yeah, you can. This is not just for your new code that's being written. Most of our customers apply this—they kind of manage their code using SonarQube. And we've got some customers who do this on two billion lines of code, and then there's others who do it on 10 million lines of code or 100,000 lines of code. So it really depends on the size of your code base. But as you think about the challenge the code generation has, fitting into the code base of a large health insurer, something like that, you need to not only understand how does Java work and how do you solve this one-line problem that you're trying to solve—you have to figure out, okay, what's the structure of the code, what's the architecture been, what's the syntax you use, what's the style that you use, things like this that are kind of unique to every customer. And so that's one of the challenges that I think a lot of the code generation companies are trying to get at as they launch fine-tuning and things like that. So you can fine-tune on your codebase and make it that much more tailored for you. We think that's super important, because otherwise you're going to end up with a hobbyist tool or an autocomplete. Tailoring to your specific codebase is really important. And then analyzing the code that's generated in the context of your specific codebase is really important. In part, to analyze the new code well, we have to understand all of your codebase, because that's what determines if it's a good fit. It's not just a circle, but it's the right size circle to fit into your codebase. And you can use it to reduce your tech debt that you've got, because it'll show you here's all the issues that you have and you can slowly work your way through those issues as well.

I mean, without getting into all of the sort of math behind it—you've got your repos that hold your code, right? You've got your different branches and sub-branches and all of that. And essentially, to make it fast, if you just tried brute-force to get your way through a billion lines of code, it's going to take you a long time. So part of what we do, part of what others in the space do, is think about how do you deconstruct the code into pieces that are analyzable, then how do you add it all back up together for lack of a better way of saying it, and that makes the analytical problem much more tractable. In most cases, it's not—it's not one piece of software that has a billion lines of code. It's in thousands of different software programs. You have your online banking system, then you have your core banking system, then you've got your risk management system or whatever. So there's kind of natural structure to the codebase that we use and you analyze it both in parallel and sequentially in different ways. One of our offerings is, we call it Data Center Edition, and part of the value of the Data Center Edition is high availability, because you have multiple instances of SonarQube working in tandem. But it's also high performance, because you can actually break down your codebase—you can essentially apply more and more instances, right? So analyze more in parallel, which makes things faster and higher performance.

So it is certainly flagging them, and it will provide you with metadata around them in most cases. So the simplest example is, do we think it's high, medium, or low severity—what's the criticality? We launched this thing we call the Clean Code Taxonomy, gosh, nine months ago maybe, 10 months ago, which is really a taxonomy for describing code problems. Is this a responsibility problem, and what does responsible coding mean? It would be things like, is it inefficient from a power consumption standpoint, things like that, or is it a maintainability issue. So we provide a lot of metadata around this so you can categorize the issues. There's some companies we've worked with who've said no issues should move forward, and they take a very, very strict approach. There's others who just say, understandably, no high-severity issues can move forward but low and medium severity is fine. And this is where that workflow element of what we do is so important, because you can define your quality profile—what is good for you as a company. And by the way, it may vary. Like to the example I was using earlier, your wire transfer system—if you're a bank, that thing can't go down. You have to notify regulators if it goes down. You lose millions of dollars if it goes down. You're probably going to have pretty strict quality gates for that. For your mobile app that takes care of your internal knowledge base for employees, probably okay if it goes down every once in a while—at least the regulators don't get involved. So you may set your quality profiles and quality gates differently. So part of it is we are giving you a huge amount of data about the type of problems that you have, how would we help you prioritize it essentially. And then for certain types of problems where we think a fix can be generated—and this is a really important point for us—we will suggest the fix, but we have to believe that we're not just giving noise to the developer. We pride ourselves on high-signal, low-noise ratios. And there's a lot of tools out there that will just suggest a fix for everything. And imagine, to use a non-developer example, imagine if you were in Google Docs and it kept suggesting to you a whole bunch of grammar fixes that are not actual grammar fixes, that are actual mistakes. Pretty soon you would tune it out. The developer would go, 'oh, this is an overwhelming list of things I need to look at, everything has the red squiggle underneath it, I don't want to deal with it.' So we put a huge amount of effort—and we're not perfect by any means—put a huge amount of effort into how do we try and give confidence that if we identify an issue, you're going to think it's a real issue. Otherwise, and we see this happen in a lot of cases in our industry, there's long lists of stuff that gets identified and nobody ever deals with it. It just sits there because you tune it out and developers think this is useless, and that's the negative spiral we're trying to avoid.

I think the future is sort of a mosaic of different approaches. I don't think this is a world in which GenAI is going to be the be-all and end-all. And I'll use the example that you gave—maybe GenAI can help with design and architecture work, but fundamentally somebody has to sit there and think about how is the health insurance system going to work, what are the design and architecture rules that you're going to put in place, how are you going to construct this billion-line code base, how are you going to maintain and migrate it, how when you're going to move part of it into the cloud, how do you take pieces of it and do it intelligently. I don't want to say AI can never help with that because there's already AI tools that try to help with that, but these are real hard problems that I think require deep understanding and focused effort. And I think those are areas where there's judgment calls that are going to need to be made, not just the analytics. And I think this is an area where we see that human developers are going to spend a lot more time obsessing about design and architecture of their software systems, because only when you've got really well-thought-through design and architecture can GenAI be really effective at building the code that you need for that. Otherwise you end up with a spaghetti mess. And it's just—even we acquired a company recently called Structure101 that helps us do the type of analyses that we have historically done, but on design and architecture points as well. So you can define this is what your architecture looks like, and we can tell you when is this part of the codebase violating your architecture, things like this. I think software development is made up of thousands and thousands of problems in order to create a codebase, and simply—writing the code is a pretty small part of it. Again, you think about writing a novel—you have to have your storyline, you need to know who the characters are, etc. You can't just put in 'write me a book.' I mean, I'm sure it'll write a book, it probably will not send the wire transfer that you need to have sent, right? If it does that. And so I think a lot of the work that's going to be required here is do the design work, do the architecture, really have the AI be a partner with you in constructing this all together. And you're exactly right—the size and complexity of most, let's call it, Global 2000 code bases is extraordinary. And it's only going to get more complex, because GenAI is going to write more code and humans are going to write more code and companies are going to run more and more on software. And so really this is why we think thinking through these workflows is so important, because you have to have the governance—and governance sounds really boring, most people don't want to talk about governance, etc.—but you really have to have really well-documented governance and controls in place so this doesn't get out of control.

Yeah, COBOL is one of the languages we support—actually up to 30 now. And there's a decent number of companies who really care about COBOL, because it is harder and harder to find really good COBOL developers. So as they have to make changes to that codebase, they need the quality assurance and things like that that we provide. You know, this is—I think this is an evergreen area. Codebases, not to sound too metaphysical about it, are living systems. You're changing your codebase—the stat that I think I've seen is that 20% of any codebase changes every year. So you think about that—if you're a large bank, you may have a billion lines of code, but you've got 20,000 developers, 30,000 developers. So they're actually changing that codebase, and I think the 20% sounds probably about right. But inside of that codebase there's a 20% that changes frequently, and then there's code that, hey, it's working, nobody's ever touched it, and it doesn't pencil from an ROI standpoint to try and refactor it. I get into all these conversations about why don't you rewrite your COBOL in Java or something like that, and the answer is in part what I said earlier—if it ain't broke, don't fix it. So I think every large Global 2000 company has got a large amount of tech debt. And part of the approach that we advocate for is to make sure you've got a rigorous process for remediating tech debt as you're creating new code. Because I don't think you need to do a gut renovation of your codebase. You don't need to say, 'all right, stop everything, I've got all this tech debt, I'm going to go and clean it all up and throw it all out and rewrite it.' It's just not practical for people to do that. However, saying, 'hey, we're going to make adjustments as we see the opportunity to do so—I'm touching this piece of code, oh, I can make this better, this better, and this better, in addition to the new thing I'm writing'—that is something we think people should be doing. So continuous maintenance of your codebase. And it's one of the use cases you've got for SonarQube—as you are working on a particular piece of your code, it'll show you not just the issues with your new code, it'll show you the issues with your existing code as well. And it gives you the opportunity to kind of continuously improve your codebase. But you're right, it is kind of a mess of organic growth and evolution over many, many years. I don't think this was an intelligent design problem. I think in most cases this was an evolutionary process with all the mess that goes with that.

You know, we—I don't have—most of the work, because most of what we do is on-premise, right? So we give you the software, you run it yourself—we don't have a lot of the telemetry like that. The best proxy we have for this is on our cloud offering. So SonarQube, our SaaS offering for SonarQube, which you can find at sonarcloud.io, we actually let you analyze open-source projects for free, basically. So anyone can put their open-source project in there and we will analyze it and show you all the issues. And it's a fascinating thing to look at—packages or open-source projects that people use every day, we can show you: are these an A-grade package from a reliability and a security standpoint, or do they have different issues? And it really is kind of all over the map. I wouldn't tell you there's like a typical pattern that you see as you look at these open-source packages. And I think it gets to—developers and developer teams are different. Some of them have really cared about stylistic consistency, and I know a couple of companies that are just militant—you join the company and you will go through a month of training on their style guide. Maybe a month is too much, but they will be rigorously indoctrinated in how they write code. There's others where it's the wild west, and as long as you get it done—as long as it works, it's fine and we'll worry about it later. And so you really see the personality, if you will, of different companies in their codebases. But as a result, there's no one type of problem, which actually means that you need to really understand the character of your development organization.

We do, and we find issues with our own codebase as well, as you'd imagine. And so we use the most stringent version of SonarQube of the workflow I talked about—we call it 'clean as you code,' where we insist that all issues are resolved before you go through with the pull request. So this is, I'll call it, a highly opinionated workflow where we've got this super strict quality gate we've set up and that's what all of our developers have to go through. It's good—it makes them understand. You kind of want to eat your own dog food, as the expression—it lets us do that. And I think historically our developers, because they've been users, they've had a lot of empathy for the people we're building for, whether those are open-source developers in the open-source community, or whether that is a developer at a large financial institution or retailer or whatever.

So the major—so we have what we call a Community Edition version of SonarQube. It's fully functional, you can download it, you can use it. It'll only tackle certain languages and certain types of problems, but it is widely used. It mostly focuses on, call it, stylistic and maintainability and those sorts of issues—doesn't get into security as much, as an example. So, and that is open source. Despite the fact it's open source, it's sort of a strange open source model that the majority—actually, the entirety of the contributor base almost, we have one or two small exceptions to this, are Sonar developers. So we don't have a lot of outside contributors for this. Open source has not been a development philosophy, although if people want to contribute, we are excited to have them. It's just not been the motion that's been established for us. Open source is about providing transparency, visibility, and hopefully engendering confidence within the user base that they understand how the machine is working. And this is something we're very committed to moving forward—we are going to be providing that level of visibility and kind of clarity as much as possible for our community.

So first of all, it's been a great conversation and we've covered lots of different topics. I really appreciate the curiosity built into the dialogue here. You know, our historical—the kind of legacy of the company has been on this issue identification. We think we're really good at it. We think we're the de facto standard at the quality analysis of code, or code quality analysis, and this is an area we're going to keep investing in to be the leader. We've got this orchestration, for lack of a better word, this workflow element that more and more companies find to be really valuable, because you can control not just for quality but for things like unit test coverage and anything that you want to put in place as either guidelines or prescriptive measures for your development team, you can kind of do in a very organic way. And so that's the kind of history of Sonar, and it's an area that we're going to keep investing in—how do we get better at security, how do we get better at new languages as they come out, things like this. The part that we think is additive to that and really exciting, and we're just in the first innings of, is the how do you really fix the issues that are being identified. Every CISO I talk to right now is overwhelmed by lists of vulnerabilities, lists of issues that are coming out. And the question that they're all asking is how can we help them triage these lists better. To my earlier point, not everything's a real issue and not everything has to be addressed, but today they really don't know how to do it, how to triage well—they don't have a lot of tools that help them with that. And then where we can help with fixes, let's help with fixes. I think Stripe put out a survey a couple years ago that estimated 50% of a developer's time was toil—just like, nobody really—it's debugging, it's documentation, it's necessary—I don't want to say it's not necessary, you can't just stop doing it—but it is not what anyone enjoys. It's not quote-unquote productive, in the lean sense it's sort of like waste in a way. And I think the more we can start helping developers be more effective by cutting out the toil, the better. A lot of it gets into that debugging area, the code context issues that you were talking about, things like that. So we think there's a big role for us to play with AI as that continues to evolve.

If you went back three, four years ago, I think a lot of people still manage their code bases on-premise, or you know, it's very sensitive data and people were just a little hesitant on this. I'm not exactly sure why—having come from Google Cloud, I found it a little surprising to be honest when I arrived how few of our customers were actually asking for a cloud solution. Now this is changing, and we just launched our enterprise-grade SaaS offering, and this was in August of this year, so pretty new. The demand has actually been through the roof now, which is great to see. I think there's a lot of people who want to be out of the managing-your-own-software business. And it sounds like we're having this conversation in a way in 2018, right? So this is not new news. But for us, this is something that people had not had their developer tool chain operating this way. Now there's more and more demand for it. So for our new business, we see it at, you know, 30% and growing moving forward. But historically, it's not been the dominant use case for us.