All right. Thank you everybody for coming out. I know we're the last thing between you and lunch, so I appreciate you coming to hear us. Ladies, if modern systems can fail in ways that are a little harder to predict, right? What does it mean to talk about reliability in 2026? And is the definition of that changing as systems grow more complicated?
When I think about how software has evolved, and the speed at which we're able to produce it, I think that it is going to take all of our best habits and best practices and things that we know that we should have done all along, and bring it to the forefront, right? When you, I think when you think about quality or reliability, what does good software mean? Sorry, say again. What does good software mean? What does it mean to build good software? What does it mean for software to be doing its job? These are questions that you kind of have to front-load in order to point an agent at it and in order to get that implementation landing where you want. And the nature of the question doesn't change. You still need to have some software that delivers outcomes to your users, but it is changing how we engage with that question and when and using those questions to drive the actual implementation of the software.
Hi, I'm Jennifer Tejada with PagerDuty. I would say that reliability's become a CEO topic, a boardroom topic. Reliability is really about trying to prevent failure and reduce fragility from your entire environment. And these failures, even though we see them all the time, the nature of failure is changing because AI breaks differently than software did. I'd also say that reliability and resilience are both required for a business and a brand to succeed. Reliability is about prevention, resilience is how you respond when something that you maybe could anticipate but can't predict goes horribly, horribly wrong. And we used to talk about downtime, but now it's less about downtime and more about disruption, degradation. Did the brand experience you were expecting to have, did it work the way you thought it would? And if not, you know, how do you take that out on the brand? So I think from a business perspective, it's certainly changing. And as systems become more and more complex, it becomes difficult to manage.
Absolutely. All right, so this session is called Reliability at the Speed of Change. To each of you, what's the hardest part of that tension? Is it the nature of the change or is it the speed?
I think it's both. I really think that speed really matters. Sometimes speed is associated with competitive advantage, sometimes it's about how quickly you can compress the time between an idea and how you monetize that idea in the marketplace and capture demand. And at the same time, you know, we're all trying to identify and mitigate the risks that again could put your brand at risk, could put that customer experience at risk. And we're seeing problems kind of in three different ways. There's new and novel problems that you've never seen before, that can be the way a model drifts or an application or an agent works in a way that you didn't intend. There are some that are partially understood, so you recognize them, you can leverage AI to use patterns to recognize previous incidents. And then there's well-understood issues that I think are becoming easier and easier to manage. The issue is that we don't, again, while we can anticipate that these problems are going to come along, we can't predict when they happen. And what we're seeing leaders in large enterprises talk about is this idea of bringing together technology resilience, operational resilience, and human resilience. Because when you have a major failure, even though we can automate a lot of the detection and the triage and the resolution, there's still human stress that comes along with that failure and some of these failures are more and more material to the business.
I think what you're describing, that piece of sort of known known pieces that can be automated, those were the pieces that were never the interesting ways of systems failing in the first place. And I think that in the age of AI when we're either building or wrapping LLM capabilities or we're introducing code generation into our systems, we are now well and truly in a world where we have to embrace that uncertainty. We have to embrace the non-determinism. We have to acknowledge that code maybe never was, but certainly now is not the source of truth about what's happening in your system. And you can only operate on what are the contracts that we have with our colleagues, with our customers around our software. How do we check for that and what do we do when those contracts are violated?
Yeah, I think trust is the primary currency of this economy that is emerging right now and I love what you said. I think uncertainty is the only certainty and if you don't love uncertainty, this is maybe not the industry to be in because every day there is a new challenge, also a lot of new opportunities.
Absolutely. How do you go about building that trust and earning that trust? Do you have, I mean, I imagine you're having meetings talking about this all day long.
Yeah, trust is built in droplets and lost in buckets. And I think you have to constantly think about it like an ongoing relationship as opposed to something that you do once and then hope it continues. And I think, you know, consumers, when a branded experience fails them, they don't care if it was the infrastructure, the network connection, the agent, the LLM drifting. They just dislike that the brand let them down. And a lot of investment goes into building that brand. And so I think when we think about trust, it's really the holistic piece of it. It's the technology, it's the operations, it's the humans in the loop, it's how it all comes together. I know Matt Garman from AWS was here yesterday and he was talking about one of the benefits of AI being the fact that salespeople can spend more time with customers and less time doing a lot of the prep work and some of the administrative work that they have to do. Why is that important? Because people still buy things from people and they care about your strategy, your roadmap, you know, how you respond when something doesn't go the way you expected. So I do think trust continues to be incredibly important.
I love the example of the salesperson because I think that as everything is changing about how we're building the software and then delivering the software, to keep up with the change requires you to be conscious of how you built trust to begin with. What were the implicit things we used to rely on to build that trust? Maybe, you know, you used to offer a very complex B2B software and you earned trust with your users as they leveled up at proficiency in that software. Well, if you're changing your UI, you're changing that foundation of that trust, them saying, 'I know the software, I know how it works, I trust that when I push this lever it will do this thing.' When you replace that with a chatbox or you replace that with an agentic experience, how do you rebuild that trust? What do you replace it with? What do you augment it with? Do you make the agent always show its work? Is that tedious? Is that not trust-building? It's so dependent on every organization and software, and it requires that internal understanding of how did it work to begin with? And if I move this person's cheese, how will they adapt? How do we evolve?
I feel like in the developer community, transparency's always been important. You know, I think you're right. Our users do not like the black box. It's just not a thing. And so, to the extent that you can be open, and we talk a lot about blameless postmortems, learning from an incident, how important it is to learn from an incident, prevents those failures from happening again, to try and remove that fragility from your systems. But also, like when you have a failure, to be transparent about it, here's what happened, here's why we think it happened, here's what we're doing to prevent it from happening again, here's how we understand it impacted you. That goes so much further. I mean, I remember an incident that we experienced, and the next time I saw the executive at this very large financial services company, the first thing they said was we had one of the best incident debriefings we've ever had with a supplier, and we really appreciated it. And I sort of thought to myself, yeah, I hope so, because we're PagerDuty, we should be good at those kinds of briefings. But it was amazing that that was the first thing they said, because it underpins trust. You know, you can't hide from these things, no matter how catastrophic or challenging or difficult they may be.
Yeah. I'd like to stick with incidents for a minute, because this is as much a people and culture thing as it is a technical thing, right? So, what can you say about teams that handle incidents well?
I think when a team handles an incident well, they're looking at it as a learning opportunity. My dad worked in the healthcare industry and he would talk about how when you're in the ER, you get to exercise skills that you don't in your general practice and you're under time pressure and you're meeting a patient that you've never seen before. There are a lot of unknowns. There's a different level of teamwork and orchestration and coordination that comes into it and I think major incident response is a bit like that. You know, you're in there together under time pressure, potentially under business pressure to try and solve for something and the more everybody can focus on what can we learn, can we leverage AI to learn faster to get to an answer sooner and then afterwards again, being brutally honest about what happened, what could we have done differently, how can this make us stronger and more resilient in the future? I think those things are really important.
I think a good incident is one that doesn't turn up again very shortly and about 70% of major incidents we see on our platform are repeat incidents. They're similar dependencies failing at the same time that haven't been addressed. I think that your question about teams kind of has the answer embedded in it, right? It is about humans, humans and agents now, but it's about collaboration, it's about communication, it's about creating an environment where connections can be made that may not have been obvious initially, right? If it was obvious initially, you wouldn't be having an incident review. And I think that if you listen to teams, if you go to an SRE conference or you go to one of these conferences where people are really talking about the sociotechnical practices of what does it take for a team to show up well to incidents, it's all about psychological safety and how do you enable people to bring context that you might not have known to look for. I think it's really interesting to think about that in an era where there's a real temptation to say, 'Well, all of this we can automate. All of this we can just hand off to this agent that will take in all the telemetry and make all the decisions.' There's such a mismatch between humans trying to solve this problem, talking about very subjective human things, and this technical temptation. And when I look at that gap, I just think, 'Gosh, I really hope that this agentic solution that's promising the world is really thinking through how to augment with the context that only lives in a human's brain.' How do you enable people to bring intuition that maybe has not been captured in data and bits and bytes on disk that an agent has access to? I think that the human-agent collaboration is going to continue to be a really rich sort of exploration surface area because it's inherently a collaboration problem in the unknown.
