In the eyes of the world, everyone on Earth, AI became the GPT. AI governance naturally evolves alongside it. Clearly, people are not focusing on the controls or on the brake, if you will. They're just pushing the gas pedal.
Data and AI governance teams often get a bad rep in trying to prevent disasters. They often become the team that says no.
Governance is a control mechanism, right, over a management layer. Most people associate the word governance or control with a negative. To control a system, you need first to have visibility on the system, right? You need to have transparency. You need to know what's out there.
I want to know how to change unhealthy cross-team conflict between builders and governors into healthy enablement in order to ship things safely. Our guest is Stijn Christiaens. He's the chief data citizen and co-founder at AI governance platform Collibra. Stijn spent 18 years at Collibra having been COO and CTO. Currently, he runs data strategy, data infrastructure, and product evangelism.
Don't complicate it. Make it practical. Make it about not talking, but about rolling up the sleeves. If you want an LLM to perform better, you got to give it better data. Better quality data, better quality context makes the model do better things.
Let's find out how to do AI governance the right way.
Hi Stijn. Welcome to the show.
What's up Richie? Good to be back. How are you?
Doing great, thank you. So to begin with, I need some disaster stories. Tell me, what's the biggest problem you've seen from poor AI governance?
Oh, disaster stories. I got to flip through my list of them. So I think there's a whole wide range of them. I think we've all seen the examples like chatbots recommending a competitor's products or people getting a policy and then following the policy from that chatbot, turning out that is a completely made-up policy, but the airline or the insurance company still has to sort it out. And then it gets really dodgy in my view because what I learned recently about Claude, the cloud bot, is that it's powerful because it has all the Linux commands. Now, Linux is a very nice operating system with the bash operator. So, you can do really a lot of things, especially if that machine is connected to something else like the internet. But the horror story is there that people actually do this. People actually will have taken AI from the internet and run it on a machine that can do whatever, that's root. And then, you know, that leads into all sorts of horror stories. I mean the most recent social media phenomenon heat was around the social network for robots where they started talking about conquering the world and turns out 80% of them were people in the first place.
Yeah. Yeah. Yeah. So I think the horror stories are all over and we'll see more quite frankly because clearly people are not focusing on the controls or on the brake if you will. They're just pushing the gas pedal. They're excited to play with the technology. The technology is evolving. So, we'll see more horror stories come out. But we'll learn as people, as society, as organizations. We'll learn how to use this new technology widget better.
Absolutely. And I think there's quite a wide variety there. So, I think a lot of the chatbot cases have been sort of fairly well publicized over the last few years, like the idea of chatbots saying things to customers that they shouldn't have said and then the company being liable for that. But this Claude bot thing that's new, so these sort of computer use tools where they can take over your computer. I mean yeah there's a lot of security risks there. Maybe we'll dive into that more in more depth later but for now I also need some motivation. So can you talk me through any success stories you've seen? So do you know about any companies where they've put a lot of effort into AI governance and it's paid off?
Yes, of course. We've got a number of them on Collibra's website where we literally advertise the stories. We have worked with for example McDonald's or Siemens or other companies. So the organizations that are really serious about working with this AI stuff, they know that they have to. This is not, you know, I put AI in my company and I push a button and tomorrow everything's solved. They know this is a new technology, it's a platform shift. So they're going to have to roll this out across functions, departments, lines of business. They're going to recreate new business. And they know that, just like with the mobile phone or the cloud before, they know that you need to put some controls around this, right? You need to have some maturity with respect. I mean, yes, experiment with the technology to learn what it can do, but also experiment to learn how you need to do it well. So that liability that you were talking about earlier doesn't start to become a problem.
With governance. That's one of the things where it seems like a lot of it's about avoiding liability, avoiding or reducing risks and avoiding problems. Are there any kind of positive business metrics that come out of it? Like can you talk about, I don't know, improvements in revenue, improvements in productivity, all this kind of, are there any more positive metrics rather than just avoiding negative things?
Oh absolutely. Yeah, I mean I want to be clear about that Richie, governance, whether it's data governance or AI governance or even corporate governance, governance is a control mechanism right over a management layer and it has the purpose of making that management layer for data, for AI, for corporations run better and run well. That's it. So imagine any place where that control function is gone, well yeah, of course it's going to bring in negatives potentially because not definitely but for sure something's going to happen if you take away control function but ultimately it's about making it run well, right? Making it run as well as it can. So yeah I think there's even MIT research that shows that there's a measurement between how well you do data, how well you do AI, just is a percentage point like I don't know 10% or whatever it was better in your business. So that's a net positive which makes sense. You're instead of doing an experiment with an AI just like that, doing it well and learning how to master that thing for your business is just going to give you more positive output. It's just that most people associate the word governance or control with a negative, you know, they associate it with, for example, compliance or regulations and laws and fines and liabilities to your point. But I think that's a perception issue. Now perception is reality. So you have to recognize that people see it sometimes like this but show them now this is about the risks and avoiding them as well as it is about making our business work better and executives are starting to pick that up right there's taking the learnings themselves.
Yeah think about if you avoid doing stupid stuff that's actually a pretty good competitive advantage then. So yeah I like the idea do less dumb stuff and that's going to help your business. It feels like AI governance is a relatively new field but data governance of course is much older. Can you talk me through what the similarities between data governance and AI governance are and where they differ.
Yeah. So I mean we can speak about this topic alone for a very long time right and what I want to say Richie is that data governance has reached some level of maturity curve. But I wouldn't say, you know, maybe if you want to put it in human terms it's maybe left home for university or something like this right so it still needs to get to the peak of its career right so that maturity curve will continue a little bit and there's going to be evolution there in the data governance requirements still as well because you'll have new data storage and warehousing and movement technology so that will come with new requirements and you'll have new laws and regulation and policies that will come with new requirements so it will evolve a little bit but you can clearly see it's further advanced on the maturity curve. Now when you look at AI governance then I would say you're much more in a moving target area and the moving target is on the one hand tied to the underlying technology for example if you would look at AI governance topics when GPT just came out and by the way there's AI governance topics pre-GPT as well right but I like to make GPT like a BC/AC moment because AI got redefined. Before AI was like machine learning and you do some data science. You take data, train a model on it and it makes predictions and after GPT came out in the eyes of the world, everyone on Earth, AI became the GPT right so in that first year what AI governance was was around use cases, you know, organizations were trying, oh, somebody's going to upload our IP in the LLM so we got to put protections around the use cases, what are we allowed to use it for and what not and which one are we allowed how to use and which one not. So that use cases was sort of the first step of AI governance and that doesn't end there right that's going to continue. But then the second year we would I think we're going back now to last year, agents became a thing right because they took the LLM engine if you will and then realized well to make an engine run as a plane or as a car we need to put something around it like a steering wheel maybe and so that's the harness of the agent. So then AI governance naturally evolves alongside it and then AI governance became also so use cases but also about for example having an agent registry. So you know which agents quote unquote live in your environment and then inevitably an organization will not have one agent they will have multiple. So then you can sort of predict that the multi-agent system will come up from the olden research days and people are already talking about swarms right so then you'll have AI governance requirements that have to do with that whole system where the traceability of which agents are important but also how they interact with one another and also how the whole system as a whole acts so you have AI governance requirements that's a moving target there and across all of them and then this is where you start to go into the overlap of data governance across all of them it's going to need to be clarified which agent or which LLM or which agent system uses which data and what you start to see there Richie is that the topic of context is sort of bubbling up in the world as well and context is essentially everything that you give to the agent so that it's informed to make its decision right the semantics the context of the business policies regulations but also data sets so there you'll see that there's some overlap between data and AI governance because let's take a simple topic like quality will become important because everyone knows if you want an LLM or a non-pre-trained model to perform better then you got to give it better data, better quality data, better quality context makes the model do better things. That's proven mathematically I believe even so that's clearly an overlap topic between these areas and then when you mention governance it'll take 10 seconds and somebody brings up lineage so if you look at similarities and differences there in a data governance world a lineage requirement was very much about how does my data hop from its source into its ETL and then its target in the warehouse for example and it gets a lot more complicated but was more about how the data moved and at which points it moved through and which controls you could put on it. Whereas if you look in that word lineage in an AI governance world people even start talking about provenance but ultimately you start talking about well how does one agent connect to another one and which data flows in between them. And make no mistake Richie, people will set up these agent systems with separation of concerns right not all agents will be allowed to do all things and will be allowed to touch all data right because you want to box up an agent in a controllable unit so that you can actually manage that overall system so that lineage if you will of which agent and which data and how they all interconnect that will be like an AI governance requirement and it'll look a little bit like your data governance lineage but it will also come with new things. So I would say a lot of similarities including on the organizational side which is to say what you saw 10 years ago in data governance organizationally you saw people wondering who needs to do this and why and how do we need to do this and how do we measure that we're making progress on this topic which have all sort of been sorted out right there's a lot more clarity in the maturity curve on data governance but if you look at AI governance discussions today you'll see those same questions bubble up. So for AI governance is that the lawyers the privacy lawyers have to look at this or is it the head of technology or the CIO who's responsible or is it our compliance team or is it the chief AI officer, the chief data so you see that confusion happen including why we need to do it when we need to do it how we need to do it and how will we measure success and of course the security people pop up as well. So a lot of similarities and a lot of differences as well including differences on well the maturity curve means the organization is figuring out how to do this new thing.
That's I have to say that was incredibly thorough answer. I'm like I shouldn't be surprised but there's a lot of things to think about with governance. So I guess let's before we get overwhelmed maybe let's figure out like where do you get started then? So, one of the last things you mentioned was that there's going to be a lot of different leadership teams involved. So, you can have like legal, you can have IT, security, like the data team needs to be involved. There's a lot needs to go on there. So, talk me through what sort of organizational structure do you need to put in place in order to govern your AI effectively?
Well, I think most of the organization will likely already be in place. Let's unpack this with the simplest example, right Richie? Let's do it the way that probably a lot of people will do it. Some CEO or board members like we need to do something with AI and then they find somebody in the company who does something with AI. Not like oh first we need to think about X Y no just I need somebody to do. So you start developing a prototype chatbot agent. Let's forget about that for a second what it is exactly, but they're just doing something with AI, right? And now they didn't take into account any of the other parts of the organization. They just did it. Because that was their assignment. And now a problem occurs, right? Oh, there was a prompt injection on the chatbot. And now, oh, it did something bad. We gave it root access on the Claude bot or whatever and it deleted all folders of files or it dropped the table in the database because it will do those things, right? And then you know that's when out of the woodworks based on the incident you'll see security people come in and they'll be like well we need to protect ourselves against new sorts of attacks. So are we going to do that? Then obviously you'll see the privacy or the compliance lawyers come in into the woodworks because they'll say well you were building this chatbot or agent which model did you use because now we need to start assessing this model whether we think it's okay for our company to use that model and not another one. And obviously you'll see some technology leaders swoop in and say well we're standardized on hyperscaler such and such or frontier lab such and such. So you should have known not to use that one right so instead come to the dark side and use company approved platform for which we're already paying credits and tokens and whatnot. And then inevitably as that chatbot so now you have some of those stakeholders that surfaced and then inevitably at some point in time the chatbot doesn't know or the agent doesn't know anything anymore. It's lacking context so it needs to be fed with additional data. somebody gives it a RAG architecture or some other setup, it says, 'Oh, but we have that data in the source system or in our fantastic data lake or delta or data warehouse or whatever consolidation place of data the organization has' and then you'll see the data boss come out say hey well if we're going to feed that into you we need to make sure that this is done appropriately and put a quality control in place or whatever else. So now that person surfaces too and then of course at some point in time then it starts touching multiple parts of the organization sales marketing or different lines of business and people will start to realize well if that agent operates like this it actually generates an organizational cross functional problem. How will we solve this? Well we'll assign a steward, AI steward, agent steward, whatever name they give it. They can call it AI Ninja for all I care and they'll form some sort of committee that maybe on a strategic level on a quarterly basis reviews well what kind of issues did we experience cross functionally with this new AI widget and how will we decide to prioritize solving that together because you need that senior leadership. So you see many of those roles and functions I believe are already in place and I would recommend you tap into them. you make them friends from the get-go rather than first have a problem and then have them be angry at you and then make friends with them. It's fine too, right? Both ways work. But on top of that, you'll inevitably have, you're seeing that in the market just like we saw before with other functions. You'll have some people opining that there needs to be now a chief AI boss and maybe that rolls everything up, right? Maybe it's chief AI, chief data, chief data analytics, chief AI and data analytics officers, whatever it's called, but that function is starting to materialize or in some cases even consolidate I would say. Many pieces are already in place and a new leadership function will probably grow or evolve from an existing one and maybe that's a temporary one right because the way I see it Richie is that when you're dealing with a platform shift organizations and business will start to operate differently right so job responsibilities will shift business processes will shift three years down the line this is now business as usual just like the cloud is or the internet or the mobile phone or the desktop is all business as usual now and then maybe that role turns out to be a temporary transformative role rather than a permanent one or maybe it just rolls up to the CIO again or the CTO for example. So many in place and some new ones probably transformative but maybe permanent will start to materialize.
Okay. I say when you were naming your bot for controlling things I was like you called it Stuart. Why do you call it Stuart or is the AI steward agent? I like that. But it's interesting that last point about maybe it's only going to be like this chief AI officer is only going to be a temporary role like having someone coordinate these activities. That's interesting because I think a lot of companies are just starting to think oh well do I need to hire a chief AI officer? But you're saying maybe it might be better to do this in a federated manner and have AI responsibility within different departments rather than like try and centralize it across your whole business.
I think it already sits everywhere Richie because if I look at the AI phenomenon, it captivates everyone's attention no matter your job or country or department or company or industry you work at. It's keeping all of us busy even people who are not working even yet. So but in that sense I think it's already everywhere in all functions and then putting like a leadership role on top can help to corral efforts or can help drive focus on it so that all of these individual efforts maybe go in the same direction. But if you say okay I'm an organization and I don't have that and I'll hire one from outside to help change our business. Well, you're not necessarily setting that person up for success either because maybe that person doesn't know anything about your organization. So to be effective, they will need to work with the organization that's in place because they'll need to extend it or transform it. And if they're an external candidate, okay, maybe they have the AI chops, but then they still need a little bit of the business chops in the company or the organization as well. Of course.
I think it's a problem with a lot of these sort of technical roles. I mean, it's been a problem with data science for a long time. It's like you need business skills, you need the technical skills as well and the ability to communicate in both ways and having all those different skills in one person is often a bit of a challenge.
Yeah. No, it's a team sport in my view just like what you saw in data, people often refer to it as a team sport and data and AI we know goes hand in hand. So AI in that sense is also a team sport. I mean I gave the examples earlier with various roles involved even if you're whether you're a large organization or a small organization the same thing applies and maybe in a smaller organization more of the roles are bundled into one but then you're going to start hitting a point where this is a white raven type scenario right and then it's such a unique profile with that company of skills and 10 years of GPT experience right if that's even possible it is by the way but that'll be a handful of people worldwide and you might not be able to afford their multi-million dollar salary either. Right. So I think it'll be indeed a mix of people collaborating to make this a successful initiative.
Absolutely. Actually, do you want to expand on that thought? Like if you're interested in a career in AI governance, what sort of skill set do you need? Like what are the technical skills? What are the soft skills?
Well, anything governance is always going to come with a lot of people skills, right? So a lot of people program management, project management, communication, even change management skills are definitely going to come in handy. Technical skills I think that's a developing topic. If I break down technical I don't want to say just technical coding technical but coding in its technical in its broad sense because you'll need legal AI technical skills. For example, you need to be able to comb through let's say the EU AI Act a little bit and try to map how those external requirements can be executed upon pragmatically inside your organization right so that requires some translation but it also requires for you to understand how AI functions a little bit I mean you don't need to know the exact workings of a neural network but you need to know how that system operates and which risks it produces right so I think there's some legal. You need to be able to at least read it, right? And interpret it a little bit. I think you need also data skills for sure, right? Because I mean, again, not the deepest, but inevitably the chatbot, the agent, it will need to eat up data in some way. That doesn't mean you need to be the ETL developer necessarily, right? There's all sorts of low code, no code self-service connectors to say, okay, my agent connects to my CRM or whatever it is, right? But you need to be able to piece those together. And maybe if you're the AI governance person, maybe then you're not doing that yourself, but maybe you need to drive a more technical person to help you do that. So you need to understand at least the requirements of what's needed there. And I think then there's also business skills you need to learn. But if you are going into AI governance right now it's a great time because you're looking at the start of a discipline and if history repeats which it doesn't but it echoes then you're looking at at least a 10 year window in which you can develop that career and because it's a new discipline maybe anyone who applies themselves a little bit in the short term quickly knows more than most others. So yeah, it's a great time to get started and this podcast and the DataCamp learnings are a perfect place to start increasing those skills.
Absolutely. Oh, thanks for the plug. That's wonderful. But yeah, I like that there's quite a wide variety of ways in. So you can go through the more technical angle, you can go through the more sort of program management, project management angle, or even the legal angle there.
Now on legal angle which I would be cautious also because if it's too legal and too compliance one of the pitfalls you have there is that it becomes a checkbox skill. Like you say okay but I've checked the model we've assessed it and it's fine so my work is done. But if that activity is not connected to the actual agent system that runs and does...
Things then, yeah, you have the checkbox, but the problem still exists. So I would say if you're in that angle, make sure that you understand the actual operations of the system well enough and that you're plugged into that, then it's not just a checkbox exercise because that won't be enough.
Yes, this is a common problem where you have a governance team that's kind of the team that says no and they can be a blocker on other teams and you can end up with a sort of bad culture where different teams are fighting each other. Do you have a sense of how you can avoid that situation? Like, how do you stop governance just being a blocker on other teams?
Yeah, that perception is real. Right. So people look at governance of, oh, there's the police or the ones who say no, right? But on the other side of the fence, when you bring up the word governance, you also hear things, well, that's important but somebody else has to do it, it's important but for later. So there's sort of a mixed bag. But to avoid that kind of situation where increasingly you're becoming opposed to one another, I think you need to lean in and understand the other party. So let's say you're in the governance unit or maybe that's you're the governance unit and now you found somebody in the organization who's like, oh, I'm doing an AI project, my boss told me and he gave me a lot of budget so I'm running and I'm racing and I don't want the governance guy to come break me, right? So you can already assume that they'll take a bit of a stance like, don't get in my way because it's their little project and essentially anyone who comes from the outside, there's a risk that they're going to make it muddy and more complicated and slow it down, right? So, I don't think that's necessarily a threat from just a governance person, but from anyone who comes into that, it's my project, leave me alone. So, I think you need to just openly go and be curious about what they're doing and why they're doing that because honestly, Richie, if you find out that this project that they're running is just some sandbox, maybe you don't need to be involved, right? Then you can just say, hey, I love this project. Great work. Can I just look over your shoulder and learn from it from time to time because it'll inform me of how we can do governance right and make many more of these initiatives. I mean, you're doing the experiment so let's learn from that and I'll do the learning, you don't need to do anything extra, don't worry about it. So you're sort of looking for what need exists on the party that you need to collaborate with because maybe you discover that the project is an experiment or it's wider and then detect, well, they hit a point where they do need interaction with the lawyer or interaction with the security person or interaction with the data person and then you're like, well, no problem, I've done this many times, let me find you the right person right away, let me give you here, look at this button, click this button and then you get access to the data, somebody will approve it. So you're actually looking where they are having problems and seeing how you can help them and that changes the whole game, right? You're not the person who's going to send them to a three-month governance course in Siberia. You're the person that's going to help them with their needs and respect the constraints or the boundaries of what their project or program actually is and you're helping them achieve those goals. But the perception is real, right, Richie? People look at, if we compare it with racing, people often look at, I'm the one who's driving the wheel, I'm hitting the gas, right? And governance is the one who's doing the brakes. But that's a naive perception, right? Because really governance is around the track that is there, the road, which signs are on the road, which curves are coming up on the road and how those curves are indicated, what the rules of the road are, you know, you can drive 50 or 100 or whatever. And even on a racetrack, your car, yes, it has a brake because that brake allows you to slow down a little bit at times so you don't hit the wall or you can take the curve in a better way and win the race as opposed to crashing out in the first kilometer of riding. So governance is more about the rules for the whole system that make the whole system work at its most performant way for the long term, not just for the next month. I hope that helps give a bit of a different perspective of governance.
Absolutely. And I suppose I should have guessed that the solution to having different teams not fight with each other is just more communication between those teams. So have the governance team talk to the people who are doing things with AI and once you find out what they're doing you can have a dialogue and that's going to resolve some of the miscommunication, some of the tensions there.
