Unlocking the Practical Power of AI to Drive Enterprise Innovation with Grainger CTO Jonny LeRoy

Yeah, sure. Great to meet you and great to be here. I've been at Granger, this industrial supply company, for the last just over four years. Working backwards, before that I spent around 15 years at a tech consulting company called ThoughtWorks, who actually joined in London but they took me out to San Francisco pretty soon after that. I spent most of my time there bridging between some bigger enterprise companies, sort of retail, financial services, healthcare, some of the big well-known tech companies, and some interesting startups. A large part of my role there was really trying to bridge between those and help startups as they were hitting architectural and organizational scaling points. I used to joke that some of my work was really helping startups grow up and helping enterprises loosen up, so how they can sort of find the sweet spot between them. In many ways, that's what I've been trying to do at Granger: get some of that technology startup culture into a large 97-year-old enterprise. Before that, I had a bit of a background in the London startup scene, was CTO of a startup in the late 90s, early 2000s. I had a weird background: I was self-taught because I made the strange choice to study Latin and Greek literature, language, and philosophy at university, and detoured via the law, but I finally found my way into computers while teaching myself while working in a pub in a good English tradition.

So we're a 97-year-old company. We're a distributor of industrial supply products, and we sell mainly into maintenance, repair, and operations organizations. I'll really simplify that by saying largely we sell into the basements of big buildings, whether it be an airport, a hotel, a manufacturing plant, a hospital. We're selling into the folks who keep that building and those operations running. In fact, the folks whose job it is to not be seen, because if they're seen running around, something's going wrong. So we're trying to make sure that they're successful, whether it's making sure they've got the right safety supplies, the gloves, the glasses, goggles you need, whether it's the hand tools, material handling, or in some areas more complex high-end tools. But our job is to make sure they have all of those products that are needed to keep their companies working. That leads up to our overall mission: we keep the world working. That's what we do as an organization. About just over $16 billion of revenue, three-quarters of that comes through digital channels.

Yeah, I can lay out a few of sort of historically what we've done. Weirdly, although we're 97 years old, we were founded around new technologies. Those technologies were electric motors and pumps. Mr. Granger was helping people have the information they needed to work out how to repair and fix them and keep them running. Then through newer technologies, we were early into e-commerce, had a whole set of e-commerce sites, early into ERP, so we ended up with a lot of SaaS for better or worse. We were early in some of the distribution center automation with goods-to-person mechanics, so there's lots of big robots and some newer robots coming into our distribution centers and supply chain. Now we've been leaning into more of the digital tooling, and really going through that transformation over the last five, six years, and that's a lot of what I've been driving. That's beginning to segue into how we think about ML and AI. I might just say a couple of things on what we're focused on. We've got this quite old landscape we're trying to modernize. As we're using our new technology, we're beginning to use more and more custom software, custom built software that we can tailor to our needs, and we're trying to focus that on areas where we think we can and should be differentiated. We sort of earn the right to have our own software there. We really focused a lot of that around understanding our products and understanding our customers really well, so we're building custom software in those areas. One thread is build software to drive advantage for us as a company. Another big thread is modernize all our older systems as we're going, and try to do those two things at once, almost the walk and chew gum strategy. To enable all of that, we're trying to grow our talent and ways of working and culture to make sure that's a long-lived transformation.

Yeah, there's definitely some hype. There's definitely some hype. Teasing through that is interesting, but there's actually a bunch of value and we've got some value that we're seeing in production. The phrase that I've stolen from others is I think if you look at there's a spectrum from some of the sort of breathless stuff you'll hear on Twitter or X talking about these sort of godlike AGIs, the other end of the spectrum is you've got access to a bunch of interns. On that spectrum of deity to intern, we're a couple of clicks beyond you've got a set of interns. So you can do a lot of really good stuff with semi-unlimited interns, but you've got to be careful about where you apply that, the structures you put in and around it so that actually drives a fair amount of how we're using it. We can talk more about that in a bit about actually looking at smaller steps in processes to apply AI to drive improvement, and then the guardrails you'd put around it. In the same way, interns are very energetic, very smart, but they don't always come up with the right answers, so you need some ways of checking their work before you push it out the door, and you need to do the same with AI. That's one big area I think about. There's more to say about how the runway LLMs have without some other architectural pieces coming in, but maybe that's a future bit of the conversation.

Yeah, and we're doing all the sort of standard basic stuff on the guardrails around governance and so on, and picking the right partners to make sure that our data is well protected. But in terms of where to use AI in that sort of intern model, we're a big company, we really believe in continuous improvement. We have a lot of the sort of lean manufacturing background, and that works for our supply chain. So a lot of our processes we're working out how do we improve them. You might have a multi-step process. We've been looking for areas where one step in the process is quite painful or low quality and we think maybe addressable by AI. A couple of examples I can talk about. One for our customer intelligence: as new customers are coming in or they're hitting certain spend thresholds, we'll do some research on them to try to work out are they going to be a big customer, what sort of potential do they have. Based on that, we can work out how we market to them, whether we apply sellers, whether we send them one of our awesome catalogs, or how we approach them. One important step is working out what industry they're in. Sounds surprisingly simple, but there are these industry NAICS codes, and it can take 20 or 30 minutes to manually go and research per company. We've put in a step that seems to be working quite well now of having a large language model go off, do the research, come back with a recommendation of an industry or two with a couple of bits of proof, here's the links to go check out if you don't trust us. Really reducing that step from 20 or 30 minutes to two or three minutes. When you scale that out to our sort of multiple millions of customers, that's actually really quite impactful. That's one small step that again you could throw a bunch of interns at, and so it's similar, you've got guardrails bounded around the edge of it, but it has a really quite dramatic improvement. Another similar one is as we're taking on new customer facilities, often we'll go in and someone else will have been managing or they'll have been self-managing their supply room or their tool crib where they're storing all of the products that they want us to look after. They'll either have a spreadsheet of a list of all the products with sort of weird and wonderfully named products in there, or we'll go in and sort of walk around and look at the labels. The process of trying to work out what on earth is this, specifically what was this tool, and you've got a really sort of obscure bit of text describing it, they might be really compressed the power or the size or the color or the brand of that tool. We'll tend to sort of throw that into systems to try to say do we have a direct product match or do we have ones that are similar. We've noticed that when some of those strings, bits of text that we're searching on fail or get poor matches, we now try to expand them out with a large language model to work out actually if we expand that out to this was 12 inch, whatever it was 100 watts, whatever the brand is, push that back into the matching algorithm, we get much better results. That one seems to be working fairly well, but it's a tiny little step. We think there are so many of those potential areas all the way across the business that if we take that continuous improvement approach, understand the process and understand the inputs and outputs, we can actually really see if we're having an impact or not. Those are just a couple of examples of areas that are kind of behind the scenes but seem a little odd but actually having tangible impact for us.

I think you've got to dive in and get started now. I'm a big believer that in technology, the right way to approach is you take big problems and you break them down into small problems, and then you deal with small problems one by one by one. Some of the art comes in actually how you break problems down. But we're trying to do a lot of sort of incremental small release pieces. For example, we've been doing a lot of work to support customer service agents. There's lots of stuff that isn't quite ready there, and we're waiting to see if some of the big vendors, will it sell for us or SAP or someone else give us the tooling we want, but that wasn't there yet, so we wanted to get started. We took a small set: we took the text channel where customers can text with our customer service agents, which is the smallest channel for us. We took a thin slice of people who were looking for product information out of that small channel and started peeling those requests off to see if we can provide better information back to our customer service agents. This was important for us to start small and we can learn and grow. It was also important to have humans in the loop there, partly for the AI guardrails piece, but actually because of our business strategy, we operate this high-touch service model. We try to go the extra mile to make our customers' problems go away. We're not just shipping them products in the nice red and white boxes; our job is actually to make sure that they can keep their operations running. So we need to be one step ahead of that, make sure that they never run out of the product they need, we know what they're going to need, we get it to them ahead of time, we make it easy for them to procure and interact with us. To do that, we deploy a mixture of people, humans, our team members, and technology. So really we want our customers to be interacting with people first, and so we're trying to augment them with these extra powers. What was interesting as we were beginning to spin up this LLM gen piece to start helping with product selection is first up we thought well we can just give the customer an answer and the customer service agent will push it through, but we started realizing that the right thing was to slow down and work out the right next questions to ask. So a customer says 'Hey, I need a drill.' The answer isn't 'Hey, we got a drill' because we've got thousands of drills. It's 'Okay, what industry and what use is it for? What sort of power do you need? What sort of protections do you need?' So this digital assistant was actually recommending better questions for our customer service agents, so that guided the customer to good answers. The feedback from those customer service agents was they loved it because it made them seem smarter because they got to ask smart questions about the products, about their industry. So we started small and that's been growing, seems to be successful. That's rolled out across that channel. We're now looking at other problem areas we can look at and potentially going to other channels like email or voice, but they have their own channel challenges that we'll need to address as we get there.

Yeah, and I think that's one of the lessons learned as we've been doing more and more custom software delivery. That's one of the major things I came in to help us bring: a culture of how to build our own software and what's different there from rolling out large off-the-shelf systems, which are largely big bang, you buy the thing, install the thing, integrate it, and hope it works. With custom, getting to these really small iterations, continuous delivery, well tested, pushing out small pieces and learning about the customer feedback and the resilience and scalability of that software at the same time. That mindset, that test and learn rapid mindset that we'd really sort of internalized for software delivery, is really helpful as you're pushing out some of these AI systems as well. So that mentality and that culture was already in place, and so we're just leaning on that.

We're almost like a dating site: we're dating customers with products that they need. So we need to understand our customers and their industries, understand the products and the supplies, and how to get those together. I think AI is just one extra tool in helping us do that. We do see a virtuous cycle between the human efforts of researching these products, talking to the manufacturers, working out from a merchandising perspective what's most important about these products and these assortments. That actually generates better data for us. A lot of the custom software we're building is to empower whether it's our merchandising agents or customer intelligence people to do their work, but that spits out better data. That data then feeds into potential machine learning AI systems, we can get that into motion, get that in front of customers, and improve that data and feed it back. So I think we understand the sort of major forces or direction we're going, and I see that being that's not changing: understand products, understand customers, bring them together super easily. Where AI applies in there, it's hard to see places where it doesn't apply. If you take this sort of sliced approach of looking at small steps inside existing processes that you understand well and how can you improve a bunch of them, there are so many processes across the organization that could be improved in that way. I think we'll see a ton of those. There may well then be some of those sort of next-step leaps where you realize that a whole process is maybe slightly unnecessary, you can leapfrog over that, and that's where some of the sort of innovation or next-level thinking comes in. That is actually one of the challenges or opportunities we're looking at in our technology transformation efforts: challenging ourselves and our business partners to think about how might we want to operate the business if we were unconstrained by our current technology. That's a big part. I partner with our chief product officer, Brian Walker. A lot of his job is to go drive that question repeatedly until he actually gets an answer he likes, because often people are so conditioned by the software they've been using for the last decade or so to think that's the only way to operate the business. Pushing them to think differently about what do we want to do for the customer, what's the value that the customer gets, get a little creative and curious about how we might do that differently, and then backing that up by building the software or training the systems that can actually support that different way of operating. So that's one portion of the transformation journey that we're trying to be on.

Yeah, there's a handful. I'll start with some of the products we're seeing. We see great products, we've been using a fair amount of the Microsoft Copilot tools. Excellent in some areas, but in some areas they're not there yet, and that's just the state of the technology. To be honest, they're kind of priced like a product but probably operate more like a feature, so that's one of those things to wait and navigate on. But we have a good relationship with Microsoft, so we're working through that. So those are areas where they're not as amazing as they could be. There are some use cases that are great: summarizing large PowerPoints is excellent, I get so many of them and I can get a good idea of what's happening in there. Creating PowerPoints from a sketch I've scribbled out, I've struggled to make that work well. Similar on the software development side, we see a lot of benefits. There's a lot that helps with coding. There's a fair amount that actually helps upstream with information discovery. We had a team who as part of a hackathon put together a gen-based search across Slack and GitHub and Jira and Confluence and various other places, and you're saying 'How does this API work? What does this error message mean? What was the decision of this last architecture decision record?' It's really good at providing that information really quickly, and those are some of the slow pieces of the software engineering process. So there are pieces there that are great. I see good demos of very small applications generating code, but generating it for much larger applications I think we're still a fairly long way off. So I'm more bearish on that. As I mentioned, the progress of large language models I think is beginning to taper, the benefit we get. I'm ready to be surprised by the next models that come out. I'm also a little worried about the energy usage and the sort of running out of data to train on, because neither of those are unlimited. What makes me think that maybe the design we have for large language models right now isn't the final or perfect design is the amount of power they need. Our brains can do as much or more and they operate on what, 20 watts, rather than the amounts of gigawatts that we're looking at investing in right now to train AI. So I think there are more advances to the architecture that's needed there. I think we'll need another iteration on the architecture, some of these models that we're using, before we really get to more interesting pieces.

There's this old phrase in tech: choose boring technology. That's great advice for anyone. Actually sometimes the boring simple technology is the right one. So if 3.5 is the boring technology, maybe just pick that. If that does what you need at a better price point and a better latency, then maybe that's the one to use. Actually, as we've been building out our customer service digital assistant, we've been doing some arbitrage across models because while the sort of four generation models are higher quality, latency is a lot worse or more variable. So we've actually got a couple of steps in the process and we can throw one set of things at an earlier model that's cheaper and faster, and then for more complex pieces then sort of upgrade which model. So that sort of model arbitrage, even sort of AI FinOps, working out how to manage the finances of which models you're using, that's going to become more and more of a thing.

Yeah, and I would say the innovation in a vacuum of 'Hey, I've got a decent idea to put a startup and put a thin wrapper around ChatGPT' maybe is a little thin or doesn't get traction. We really believe in coupling innovation with a sort of tech innovation with some real understanding of the business problem. So I think being able to go deep into a business domain is the key. You couple the tech and say actually what's a very specific problem we're trying to solve in supply chain or in marketing or in understanding our customers, and apply it there. We've done a bunch of that on computer vision, that's now traditional AI, I guess. But I think that's where the more interesting innovation happens: getting into understanding a business area and whether that's the semantic layer or the ontology of that business. That's an area that we'll definitely see more movement on.

Yeah, and I think for us, we feel like we have the rest of the scaffolding of what we need for the other systems that can actually then sort of process before and after that, and the standard work of the people and the relationships to slot that into. But I think you were touching it just before, and then on some of the testing or the scoring of models, we've been putting a fair amount of effort into our ML Ops, ML platform team. I'm a big believer in testing and test-driven development, and we're having to rethink that for ML and AI. In fact, some of the tests are really sort of data-driven scoring, coming up with what's a good scoring mechanism to work out if we're giving good answers to a search or to a chat conversation, so that as you're changing models or upgrading models, you've got some notion of are you better or worse or have you done something weird. So we've almost got these sort of algorithmic tests that we're now putting in place.

Oh, ask my boss. No, I'd probably break it down into three things. If you did ask my boss, he would probably say: one, are we driving tangible value, measurable value for the business on the key strategic areas that we think are important? That's one. Second one is, do we by some measure are we getting better in terms of effectiveness or productivity? There's all sorts of traps of trying to measure productivity in technology, but are we getting more for what we're putting in and spending? And then the final one is a sort of team member one: are our team members engaged and happy, and do they feel like this is a place they can do their best work? I think if you're doing well across those three dimensions, you're probably in a decent place.

Well, I first became a CTO in the late 90s, and I was a CTO of myself, so the problems there were different. As a CTO now, pick which decisions you actually make. I think a fair amount about how decisions are made and where and when and by who they're made in an organization. Actually, one of my jobs now is to make as few decisions as possible, so really try to design the organization so that you can push decision making down closer to the work. The flip side of that is people have to know when they can't make a decision, they've got to bring it back up. Understanding how quickly is this a rapid fire two-way door decision or is this a more complex one that's better to circle that decision. Then how you make decisions visible, use these architecture decision records so we can broadcast decisions we're making and have people weigh in on them. But the fewer decisions I have to make, the better. In fact, maybe that's a key metric: how many decisions per day do I need to make, or have I set up the organization to be good at making decisions.

The non-data science work that goes around it. Having good data, getting the data into the right places, you still have to feed it into applications, you have to test those applications and get them out, you have to operate them, you have to secure them. There's a rush to hire data scientists and ML scientists. We're decomposing actually a fair amount of the roles and skills around that and looking at what are some of the non-PhD dependent roles that are easier to cross-skill or train people into. So yeah, data science is the tip of the iceberg. There's a fair amount underwater, and there's a whole bunch of skills and grunt work that needs to be done. Some of that may be AI addressable well over time, and there's some opportunities for startups to address some of that. But don't underestimate the size of the iceberg under the water.

I was a big fan of 'Zen and the Art of Motorcycle Maintenance' because for me that was real insight into while I was repairing motorbikes, I saw it very much as similar work to debug software, and some of the mentality you need behind that and some of the joy of that as well. It had a good slice of good West Coast hippie before I moved out to the West Coast.

Well, I actually love some of the robots we have in our distribution centers. They're kind of cool, and some of them have got names. Our favorite one is called Tuna, and he runs around. But there's these little robots that we like partly because they're helping our supply chain get better, partly because robots are really cool, but partly because their power consumption is way lower as well. We do a whole bunch on carbon neutrality and trying to improve our energy footprint, and these are way lower energy than some of the previous conveyance systems we had. So it's really interesting stuff happening there.

I think that at the last possible moment, humanity will find a way to address the climate crisis through technology. I used to hope that we could all live simple sustainable lives, but I actually think we need some more aggressive technology interventions. It'll be a weird and wonderful future, but I think we will need to, and I think we will, and I really hope we'll be successful.