Welcome to Fort Knox. I'm John Ford here with AWS CEO Matt Garman and Accenture CEO Julie. Welcome to both of you. Want to talk about AI and particularly agentic AI in 2025, but we're almost in 2026. And Matt, I know you just got done with re:Invent last week. Had some big announcements, including Nova Forge. I'm interested in customization. It's a theme we've talked about when it comes to infrastructure, but what you're doing with Nova Forge, I think, is important. So tell me how that fits into the overall agentic narrative that we saw play out this year.
Yeah, one of the things that we heard when we talked to customers about what they're looking for and where the gaps are in the previous AI stack that they had was really how they take their mission-critical AI and the knowledge about their company and their industry and actually get the real value out of that. And what we found through much of our science is that if you can take that data and inject it early into a model and have the model really learn a customer's data, a customer's industry, it actually is better. It performs better for them. And as they're trying to drive real mission-critical agentic workflows, knowing the business, knowing the proprietary data of that particular customer makes a huge difference. And even after they go and try to reinforce it after the fact and distill it down, actually the model performs better for them.
How much better? Do you have like a percentage or range?
We're still seeing that, but we've seen an example. We worked with Reddit and they were seeing 30 to 40% better performance than the absolute best model they've ever had, and it was like the first, we'd only been customizing for about a week or two with them. So really meaningfully better performance on accuracy, on understanding the business, on getting the tone. And as we see through agentic where customers really start to drive mission-critical workloads where it's not just at the periphery but really is at the core of what their business is trying to do, that deep understanding of that customer's data and their workflows is super important. And we're very excited that Nova Forge is this new model that can unlock that.
Julie, how important is that customization, especially because you guys deal with a huge breadth of customers and it's a multi-cloud world? There's lots of different options on how they might approach AI. Where does this effort fit into what your customers are demanding and the path that they're going to have to take to get the sorts of results that they need to be competitive?
Well, first of all, this is one of many different building blocks. And the AWS strategy of providing building blocks, think of them as Legos that you can put together to then build solutions, is really important to clients because each client wants to have a different result. They want to be differentiated. They want to do something different in the market. And as Matt said, on a client's own data is really important. But I think one of the things that you just heard from Matt around the importance of data and workflows really goes to then how do you make all of this successful, because you start with models and great tech, but how you make it successful ultimately is actually changing the enterprise to use it. So for example, we're working with VMO2 and AWS where we're helping them transform their service organization. So they started with great tech, many different building blocks from AWS. Then we used all of our knowledge about telecom, and they had a lot of people complaining about their service, and that meant understanding customers, why do they call in, what do they need to solve, and then using the technology, the understanding of the customer, to change to get a different outcome. In fact, complaints are down 65%, NPS is up. But it requires great tech, lots of different tech put together, plus a deep industry understanding, a deep customer understanding, and then in this case scale, right? 10,000 advisors moved to that platform. And that's how you get real outcomes. And so think of 2026 as the year that agents will deliver big outcomes.
Matt, how are customers going to calculate and understand the return on investment on this? Not just on the infrastructure, but on the decision to build with Nova Forge versus take what's off the shelf and say, 'Okay, well, I've got some efficiency that I can take now. We're going to have to figure out how to optimize over time.' It's really a strategic investment to say, 'No, all right, we're going to invest in building this because we believe that this is going to lead to that much better outcomes over time.'
Yeah, there's a couple things there. I think number one is that we can deliver absolute lower cost and better performance, and so that makes the ROI calculation relatively easy. I think what Julie said is exactly right, that their team plus our team together can go help customers really transform this. But from our viewpoint, we really want customers to use the absolute best model. And if off-the-shelf models are delivering the absolute best performance, it's fantastic, and it's why platforms like Bedrock have a whole bunch of model choices. But we do think that customized model is something that customers have wanted to do for a long time and just hasn't been possible. They've tried to use open weights models to try to get to that customized place, but they're not quite good enough because you have to do it at the end of the process and you don't have full access to the training data where you can mix that in. And so I think we can deliver really differentiated performance. We've already shown differentiated performance with a handful of customers that we had in private beta before we launched this last week. And we're quite bullish that the outcomes will be great. But it doesn't take away that if particular workloads, I think there's going to be a mix of different models. And so sometimes customers are going to want customized models and sometimes they're going to want the off-the-shelf models that deliver the best performance for them, and we support both.
Okay. Julie, you mentioned that you use Agent Core within your own organization, I believe, and I'm wondering how you look at it as a strategic advantage. How much of it is Accenture is able to perhaps go after clients of a different size, maybe smaller, because it doesn't take as much human power to manage that? How much of it is just being able to work more efficiently with clients? How much of it is the opportunity to help clients build and customize what they might specifically need?
Well, in many cases, it's all of the above. So we specifically use Agent Core to create some of our services agents where we want to move quickly. It has the governance and security out of the box, and we're able to deliver a very clear ROI. And the answer to your question about ROI, John, by the way, is it's not about the tech itself, it's about how you use it. You have to build the business case, and the number one piece of advice I give CEOs is whatever that business case is needs to be embedded in your actual KPIs that you run the business. So when we invest in AI at Accenture, it becomes part of the financial plan and any other metrics like if it's speed to market, for example. So that's how you get ROI.
Can you break that down a little bit for me? If you're saying, 'Hey, the business case is that we're going to improve margins this much by cutting costs in this area,' then you have to measure executive success and overall organizational performance by actually getting that value.
Absolutely. So when we set our financial plan, if we're expecting to improve costs, then the margin that we're expecting that part of the business to deliver reflects that assumption. And too often these projects around AI become separate initiatives, and then the CFO says, 'Wait a minute, I'm not seeing it in part of the P&L.' And so that's where Accenture and AWS come together, where we create the whole transformation, the tech plus the process and everything, what the business case should be, and then help our clients be able to embed that business case into their P&L expectations or other metrics. There's a lot of different kinds of business outcomes. And that's why this partnership is so important. And when we talk about 2026 being the year that agents transform and deliver value, it's because all of those things, and by the way, that all starts with great clients because clients have to have the courage to change and the confidence to be able to say we're going to commit to outcomes in our plan.
Matt, when so much of this is about culture and process, how do you make AWS the best at it?
