Welcome back to You Can With AI, a seven-part series on artificial intelligence from KPMG that is part exploration, part blueprint, and focused on helping you and your organization fully leverage the AI opportunity. One of the biggest differences between consumer and enterprise AI is the complexity of the systems that enterprise AI has to interact with. To be successful, enterprise AI requires more than just great technology. It requires trust and governance systems. To discuss those systems, we're joined by Kelly Combmes, managing director in KPMG's digital solutions and architecture team and a US solutions leader for trusted AI.
All right, Kelly, welcome to You Can With AI. How are you doing?
Great. Thank you, Nathaniel. Happy to be here.
Yeah. Now, this is another one of the conversations that I think is hugely relevant for companies and folks who are listening who are really thinking about operationalizing AI. Right? This is where the rubber meets the road. We're no longer just talking about theoretical possibilities. We're talking about the nitty-gritty and often like the sort of less heralded parts of transformation, but parts that ultimately are going to be hugely deterministic in whether you know new initiatives and new approaches succeed or fail. So, I'm really excited to have this conversation. Maybe to kick us off though, could you share just a little bit about the perspective that you're bringing to the conversation, what you're spending your days on, and kind of how it shapes your take on AI, trust, and governance?
Yeah, thanks Nathaniel. I live and breathe this topic. So I've been focused on AI governance since KPMG launched our first framework around AI governance in 2018. And I've been focused on the methodology for governing AI, adoption of AI, how Gen AI has come into play and is changing sort of the nature of AI capabilities and really accelerating the pace of adoption. And I've originally focused within the more highly regulated industries. So working with some of the tech conglomerates or the providers of AI capabilities as well as financial services. But I really serve all industries and I've seen all flavors of AI governance. So I'm really excited to talk about what does practical application look like and mean for a number of organizations whether they're highly regulated or not. And so I look forward to this discussion. I also focus on building and enabling AI capabilities, organizing your data and identifying the right functional use cases to apply AI and Gen AI.
That's great. I mean, one of the things that I love about that perspective is you're sort of firmly, you know, have a foot in the operational world of AI and understanding how this is impacting businesses, but also thinking about these trust and governance issues which are going to shape, you know, over the long term how it gets integrated. So I love that combination.
Let's start by, you know, setting up I think one of the things that makes this topic in particular relative to all the things that we'll cover across the course of the You Can With AI series is that this is not a topic that is just the province of companies internally and their internal decision-making. There is obviously that aspect of it, thinking about culture and ethics and trust frameworks and the way that you want to run your business, but there's also questions that are about how governance issues are going to interact with regulations and laws and frameworks. And one of the things that we were discussing before we started recording is that right now we're in a bit of a limbo land, an in-between period as relates to the frameworks and the regulatory standing for this particularly in the US. Talk to us a little bit about what that means for companies that are trying to dig into these issues and actually figure out how to handle and address them.
Yeah, I think probably one of the most common questions I get asked is where is AI regulation headed? And so I always look into my crystal ball and try to say where are we headed? And we're in this land of we don't know for sure. But there are some key pieces of frameworks as well as regulatory pieces that are serving as anchors that can guide an AI governance framework and areas of focus that organizations should think about. So, first I'll say there's no prescriptive federal legislation specific to how to govern AI or what's expected when you use AI in the context of making decisions for end users or consumers. So in the absence of that we see a little bit on the state and local side where some states have said in high-risk scenarios you must have auditability transparency, you must consider aspects such as bias and discrimination, and you must surface up transparency in how you made a decision if you used an AI solution that impacts an end consumer. So we see pockets of that as well as the EU AI Act, which is sort of like the GDPR for AI, that is very conservative and again takes a high-risk use case approach on what are the responsibilities an organization has when developing, designing, and deploying AI solutions. So what we've been doing for a number of years is trying to keep a pulse on where are we seeing these pockets of legislation and regulations pop up, what are the common themes across them, and then designing and anchoring around a set of common principles such as fairness, transparency, accountability, and making sure that our clients are designing their frameworks with those common principles in mind while we keep a pulse on where we think AI regulation is headed. And that's probably the best we can do today while we continue to see legislation evolve is how do we best prepare and how do we create a governance framework that's aligned to our organization's values and the speed and the pace that we're using AI internally for our own employees and embedded within our products and services to our clients and consumers.
And so you touched on this I think quite a bit already, but I'd love to hear more about what it means or what are the parts of a governance framework, right? When you're talking with a client, especially if they're starting from a relatively sort of zero state when it comes to AI governance, what are sort of the pillars? What are the anchors? What are the parts of a governance program that they need to be thinking about?
Yeah. Well, first I'll say some are just starting and that's common and that's just, you know, where some organizations are at. Others have already started, but with Gen AI coming into the picture, it's also a good way to refresh is our governance framework covering the right aspects now that we see a lot more AI as a service or third-party AI. So whether you're just getting started or you're saying given the emerging capabilities out there, agentic workflow and Gen AI in general, we need to re-evaluate is our framework covering the right components now is a good time to start thinking about that. When we think about what are the components of a governance framework, again many organizations have an AI policy or are starting to create policies, procedures, standards on what's the permissible use or expectations we have for AI. There's a lot of work also being done around roles and responsibilities because AI is truly cross-functional. It's one of those technologies that impacts legal, third-party risk management, the functions in and of themselves, data, technology. So aligning on who's involved in what aspects and where and how do we mitigate risk across those roles and responsibilities is really important and how do we set up the right constructs and forums given the emerging nature of AI and Gen AI so that we can continue to have the right stakeholders involved to evolve our processes, our frameworks, our procedures is really important. So we see a lot of AI risk working groups in tandem with how does the risk working group align to the AI strategy and the vision of where the capabilities are headed in the organization. And then maybe the fourth core component of AI governance that I would say is this idea of how do we start to instill technical controls in how we architect and use these solutions. So think of are there ways that we can control the user's experience to make the output more accurate or make them understand how information is being given to them. Are there ways we can look for and detect more proactively data leakage in some of these solutions. People injecting bad prompts and assisting them with what's the right way to ask a prompt to get the right output or a more accurate output align with how you want to use the information. So this idea of can we instill technical controls not just processes and procedures and manual methods but how do we architect AI to be more automated in nature and also assist the user and how they can understand and consume the information.
And so presumably this is in the same way that AI is both an internal change force as well as an external facing force. I imagine that governance frameworks also need to consider both sort of policies and procedures as relates to internal use and how employees are doing their jobs and how work is structured as well as external facing and how the company is interfacing with its customers. Are those sort of fundamentally different areas of exploration? Are they all integrated? How are people thinking about where those two meet or diverge?
Yeah, I would equate it to what's the level of risk. So, a lot of organizations are using these tools for general productivity in relatively lower risk areas. You know, we're willing to use it as a way to augment human activity, not replace it, but assist in how we work so we can work more efficiently in areas where we're touching our products or services or our consumers. Those tend to be areas that organizations think are more risky because the impact of something going wrong is greater. And so we talk a lot about how do you put the right governance and more governance when the risk warrants it. So right-sizing governance processes and procedures. Not everything is treated equally but you set a foundation of expectations and a policy and processes for everything including internal use. And then as you start to introduce more risk because of the data or because of who you're impacting or if it's external facing, how do you layer in additional processes or technical controls commensurate with that risk? And so that's an area we like to focus on because it also helps say we don't need to treat everything the same or in an equal way. Let's focus our efforts and our resources where we have the biggest exposure potentially.
And how much is there or is there I guess a tension between top-down desire for governance frameworks with natural employee experimentation or you know desire to try new things where do those ever run afoul of one another? I guess
I think there's a ton of tension there and you know we often work with who's driving the AI capabilities in the organization. It's typically a CIO, a COO, a head of data science, and their whole job is to build cool things and put them in the hands of employees, you know, and then you've got your risk function hat on saying, how do we do them in a way that's responsible? And in some ways, we're concerned about giving AI solutions or specifically Gen AI to users that are not skilled in understanding how to interpret the results or how to interact with AI. So with Gen AI, I think we've started to introduce a little bit of fear and rightly so a lot of excitement around what Gen AI can do in that we're bringing data science closer to business users. And so we have a responsibility to educate those users on what expectations we have for them and upskill them and how can they use it to augment their job for productivity reasons but also not expose us to additional security or data or privacy related risks. So there's a tension there. We're often building foundational governance and starting to test and learn with the tools in lower risk areas. And as we upskill the organization, as we create more knowledge around Gen AI and its capabilities, then we start to experiment with adding more complex or higher risk areas based on the pace and how quickly the organization wants to move. At the end of the day, there has to be both this idea of we need to allow experimentation. We need to create avenues to innovate. If we truly want to adopt these technologies, we need to put them in the hands of everyone strategically, but at the same time, we need to set guardrails and technical controls to monitor are people doing the things we're expecting them to do.
It's interesting. So you a couple times you mentioned upskilling and my next question was actually going to be how much you think that upskilling and the organization's sort of responsibilities to its employees vis-a-vis supporting their exploration of and learning around these technologies actually should be part of this governance conversation not just sort of floating off somewhere in left field and in L&D but actually kind of a core part of the consideration as we're designing these governance frameworks.
I think upskilling is absolutely essential. You know, here at KPMG, we have digital and data literacy pathways based on your role, what level of proficiency you want with data and AI and insights. I think it's vital to create training on the capabilities and set expectations. If it's part of your strategy to adopt these tools and really change your ways of working, there has to be both the how do I use the tool in parallel to what are the expectations and the responsibilities I have when I'm using the tools. So a perfect example of that is training on how to use Copilot or on how to do good prompting. And then as you use the tools, there's certain training requirements on the policies, the procedures. When I log into our chat at KPMG, which we call AIQ, there's a splash page reminding me each time I log in what our policies and procedures are around using the tool, data, how that information can be shared, etc. So I think it's important to have the skills training on capabilities and what are the right ways to ask the right questions to increase usage of the tool while reminding them along the way that hey we have these policy expectations and you have to safeguard information and data and understand what role human in the loop plays in oversight around the output of these tools.
Yeah, it's really interesting if you look at one of the things that has been most consistent in surveys of employee Gen AI usage really for the last year, year and a half is that there's a large amount of Ethan Mollick called it secret cyborgs basically people who are using these tools and not really reporting it in. And one of the things that it's felt like to me is that the behavior has stayed the same but the reasoning for it might have shifted. I think a lot of it in the early days was just not being sure exactly what was and wasn't allowed. I think now in many cases it's perhaps an encroaching sort of slowness or a gap between the availability of tools and sort of someone's general life versus what's made available to them at the workplace. But what's interesting to me is that in either case, governance actually becomes potentially a solve for that rather than an additional barrier. I think you know this is a classic sort of misconception I think is rules equal sort of less ability to do things whereas in this case my instinct is that organizations that have better articulation of what is and isn't expected and ground their employees in that probably have a lot more experimentation and sort of just usage because people aren't questioning what they can and can't do.
