I'm Matt Slaughter. I have the honor of serving as dean here at the Tuck School. And it's wonderful that you've all braved this wintry day. As we all know, it's been a little wintry here in recent days and weeks, but we love four seasons here at the Tuck School of Business at Dartmouth. And we're very fortunate to have a discussion here. I'll make the introductions here in a second. About our world of AI. As I think a lot of us know, the phrase artificial intelligence was coined here at Dartmouth. There was a conference convened 70 years ago this summer by a handful of academics, one of whom was here at Dartmouth, and that phrase was coined. It kind of existed as I'm a poet, so I'm going to give my you guys correct as we need to. Larry or Allan. In the world of technology and innovation, we had a lot of technology innovations with AI kind of percolating in the background. We had this thing called the IT revolution. Personal computers were invented. Our mobile phones were invented. Artificial intelligence was percolating along with large language models and other things. And as we all know, suddenly we get to December of 2022. This thing called OpenAI. They put into the public domain a free version of their chatbot. And here we are today with market valuations and all sorts of innovations.
Yeah. Unbelievable. Right. And some people foresee it's going to be the fastest and most sustained productivity growth in the history of humanity. And as John Maynard Keynes waxed lyrical about 100 years ago, we might end up with human beings working about 16 hours a week and it'll be leisure and all good kitty cats and puppy dogs. And some are foretelling that it's going to end up being like Cyberdyne Systems in the Terminator movie. So where we get to is going to depend on, in our lexicon of talk, hopefully we have some wise decisive leaders going to help us make the wise choices and we're going to have a conversation about that here. I'm going to do introductions. Patrick Wheeler here, a terrific colleague, executive director for the Glassmeyer-McNamee Center for Digital Strategies. Professor Alva Taylor and senior associate dean Alva Taylor is the faculty director for the center and they're hosting this talk on behalf of Tuck and behalf of all of Dartmouth. So we're here with two leaders in the world of technology and business more generally. I'll introduce each of them. They've known each other, I think, professionally, but also through a board role for a warm call or a cold call I'm going to make it. How long have you known each other?
Mostly from the board. Yeah. I mean, we knew of each other. We actually...
Yeah. I Yeah. Yeah. My office was at 101 Main Street in Cambridge back when it wasn't... It was only $15 a square foot, not a hundred. And Allan was right below me and we didn't know each other but I always saw a car that said Pega on the license plate and then he said I saw a BMW I thought was really nice looking. So we knew each other and if the floor would have given way I would have fallen right on Allen onto his desk. Yeah. So but anyway...
Thank goodness for architecture because we may not be here.
All right. Well, that was Larry Weber speaking and Alan Trefler is with him. And so, let me say a little bit about each of them from their extensive bios. I'll turn it over Patrick to start the conversation. I think we're going to... you're going to moderate some initial questions and we're just going to open up to the group here. Okay, great. Larry Weber is a globally known expert in public relations and marketing services. We at Tuck, Alva, I and others have known Larry for some time. He's passionate about the convergence of technology, the internet, the web, and communications. And he's the founder and chairman of Racepoint Global, which is a firm that has thought about brand building and marketing and technology in the global economy for many moons. He recently wrote a book that came out, I guess now technically two years ago, 18 months ago, called A New Age of Reason: Harnessing the Power of Tech for Good. So hopefully you're going to share some insights on that, Larry, with us. He's authored, I believe, seven books on marketing, on technology.
People only read half of them. So...
That's okay. That's okay. So you know my mom says that about anything that I read.
Oh, it's 500 a batting average.
Yeah, that's a very good batting average. And he also serves on the... he's had many board roles but currently serves on the board of directors of Pega Systems and thus the connection thanks to them being neighbors in Cambridge and not having fallen on him of Alan Trefler. Allan's a member of the great Dartmouth class of 1977 as I learned a moment ago. A native of Brooklyn, Mass. still lives there. Found Dartmouth in a college shirts as a lot of young adults do and found his way here. He's here sitting today because he's the founder and CEO of Pega Systems which is the enterprise transformation company for which he's the visionary leader. He's the technology change agent and we're going to talk in particular I think your remarks are going to focus on thinking about agentic AI going from having conversations with your chatbot to actually creating autonomous systems that AI might enable to do. I think as a lot of us are learning and maybe have lived, a lot of leading organizations that they're trying to figure out what AI might mean are realizing you can create these autonomous agents and have them go out and execute tasks and creative things. And I'm struck at how many organizations I'm finding they're saying our managers and leaders were talking about not just who are the human beings that are part of their teams, but how many agentic AI team members do they have as well. How are they motivated? How are they evaluated? And what that implies for the future of work. But Allan actually in his earlier life when he was at Dartmouth he was a chess master and so was... his expertise in chess when here at Dartmouth he combined with his engineering background and he got interested in AI and did a bunch of technologies things here when he was at Dartmouth. He went out into the world of work he decided to found this company at the age of 27 and here we are a couple of years later and he's still leading and successfully helping this company bring us into the age of AI whatever that's going to mean so on behalf of all of Tuck and all Dartmouth, please join me in offering a warm welcome to our two guest speakers.
Patrick, I'll turn it over to you.
Thank you very much, Matt. Thank you. Appreciate it so much. Welcome, welcome to Tuck. Larry, welcome back. You know, we spent quite a bit of time together. It's been really, you know, great to have you come back as well. You know, speaking of returns, you were, you know, one of the places we want to start is if we could, you just returned from Davos where outside of some of the other big conversations that were probably happening in Europe at that time, AI was a topic of significance this year at Davos. Would love it if you would be able to just share a little bit about what that conversation looked like on the ground from your time there if you wouldn't mind.
Total confusion, gibberish. People are I think really struggling to internalize what all of this means and how it's going to affect their businesses and the people and all the elements. And I think there's a lot of good reason for people to wonder because many many things are up in the air still.
And you know so I think one of the things that you've been able to do if you wouldn't mind just sharing a little bit about kind of Pega and Larry as well feel free to as well a little bit about kind of how you're navigating that shift right because it's been a core part of your strategy if you wouldn't mind to share a little bit more color on that. That'd be great.
Sure. So, Pega is a business school. So, we're close to 6,000 people. We'll do 1.7 billion this year. And we've been through massive shifts in technology. The first generation of the technology we built ran on something called a DEC VAX or IBM CICS. Both of which could qualify as cautionary tales. We've gone through lots of changes and I think part of having not just survived but been able to build a business to do that is trying to get ahead and really think about what's real, what's hype, how can you apply it. What our customers do, just give you a little context, is they do typically pretty sophisticated workflows where you have a need to do something. Like for example, if you ever had a charge on your credit card that you wanted to dispute and you called your bank, there's about an 80% chance that banks involved are using one of our systems to take the input, figure out what to do, go talk to Mastercard and Visa and whoever to rectify or not rectify as the case may be, that sort of thing. And those sorts of workflows are all over businesses and that's what we go after. And we've been using AI to create a statistical foundation to call machine learning to be able to find patterns in math. And even though no one's talking about that these days, everyone only is talking about generative AI, I still think that this mathematical statistical AI is tremendously important and operates really very differently. And it's also they're both now very important to the way.
Yeah, I think that that determinism that you get from that from different versus probabilistic versus deterministic AI we often hear talked about quite a bit too, right? With the difference between generative and more traditional forms of machine learning.
Yeah, it's interesting because parts of generative are entirely reliable and parts of it for example translation has gotten to the point where it can really if I as you can with our system were to talk to it and say hey I charged something last Wednesday and I got billed for the wrong amount. AI can, you know, with enormous reliability figure out that next Wednesday was a particular date and translate that to what the system needs. The places that I think people are getting themselves in trouble and they are getting themselves in trouble is when they're doing reasoning at runtime in response to the person when they really should have done it in advance to sort of create the playbook as to how they want. That's really how we've tried to carve out an alternative path to minor competitors like Microsoft and Salesforce.
I was just going to say to give it some context for the audience the competitors are Salesforce, ServiceNow, Microsoft so not... but they have a heritage of and I've been around software for over 40 years now and the Pega system for business processes and we love this guy look... yeah but no but it's you can tell you know so for business people you have to understand you want to be close to engineering excellence as an outcome for business and that's what it's about. So they have a heritage of that.
So yeah and I think what we're seeing right is that shift where more business knowledge is coming in because of some of the capabilities of this technology. You know how is that shifting in your business? Are you seeing more sort of like you know more folks that are on the business side versus the engineering side engage more frequently with your software?
Well add to that question what is the future of software which is something he can talk about and will there be a lot of engineers helping there?
The future of software is changing. It's changing a lot and it's changing at a pace that I would not have expected four years ago here. It's you know some of the change I think could be very positive. Some of it's going to be highly disruptive. If you wanted to just ask a computer, you know, ask ChatGPT or Gemini or Claude, you guys I know like Anthropic. The quality of the code that they can generate is actually pretty remarkable in terms of being able to take a problem and to solve it. The things that people sometimes aren't aware of is that the LLMs are incredibly sensitive to nuances in the data and the context that are available and they can actually decide to do things. If they decide that a customer was rude or badly treated, they might do something that would fall outside the guidelines of what a business would actually want to do, particularly in a regulated industry. So we use this idea of workflows to do two things. First, to harness the LLMs at what we call design time. So we built a capability that's just taken off in the market called Blueprint. It's available on pega.com. It's free. Anybody can use it and it's I think one of the most advanced uses of generative AI that I have seen and it's been really highly regarded by analysts. What Blueprint does is it lets you have a dialogue around a business problem you want to solve and then it goes and uses a language model technology called a vector database to get our intellectual property about how we handle workflows. How do you assemble a workflow? How do you organize that? And it pulls our stuff in. Then it goes out to the internet itself and it says what are the best ways to do this? And then it allows the customer to upload documents like manuals or looks at the website etc. And it basically tries to figure out what are the workflows that I need to run this business. And having done that, it will show them to you. So you can look at them and say, 'Yeah, I actually like these workflows.' Or you can say, 'No, that doesn't make sense.' Or, 'I'd really want an approval here, or I've got a different way I want to do this calculation.' It walks you through the analysis of how you want a set of business processes to run. But then at runtime, when you're actually dealing with a real person to solve an issue, it doesn't do the reasoning that candidly all our competitors do. Our competitors are doing reasoning at runtime. They're talking to the LLM. And I think it's fine to brainstorm with the LLM, but I think you want to brainstorm with the LLM in a context. And in our situation, the context becomes those workflows. So, we bind the LLM at runtime to the workflows that have been designed and pre-approved. And that gives us a real reliability at runtime. If it's repeatable, you can not just ask the LLM what it did, you can ask the LLM what it is going to do. And that to us is our distinction and why I wear my 'no prompts' button here which is that you really want to depend on the prompts when you're designing but then having designed it you really want to use the language models very narrowly just to do the semantics and just to do like the language translation. Does that make sense?
It does. It's really... and to clarify with the audience too, right? I think workflows and enterprise software are sort of why we have like it's been that the history of enterprise software is about building workflows. But what we're saying is and this is a significant shift. So I want to make sure we for the audience also clarify right and reiterate that instead of everyone gets the same workflow and we predetermine what that software looks like. We're building with the customers for more specific to what they need solved.
And we can build with the power of the language model at design time. We can build greater variations so you don't have a lot of places the workflows are very simplistic. They basically say, 'Oh, you're going to get a problem. All right, route it to somebody. Let them solve the problem.' That to me is not, by the way, that's the way a lot of it works. That to me is not really what I would call workflow. Workflow would be okay, you get a problem reported. There are 62 categories of problems. Each goes through a different set of steps. You know, if you think about a help desk at your lab, you know, sometimes you got to reboot the machine, sometimes you got to reinstall a piece of software, update something. Those different workflows we make explicit with the power of the LLM because otherwise it's really too hard to do all that. But then at runtime, we're basically just saying, okay, let's pick something that we know works. And in the 99% of the cases where the thing you're fixing has been fixed by somebody before in life, it's going to operate with complete reliability. And then...
It's such a significant shift because, you know, as folks here, we have many folks that have been consultants as well and worked at many software companies know, right? Like that is what we pay a lot of these folks to do is get your company to transition to the workflow that the software dictates to your company, not the other way around, right? But a lot of times those workflows are either too rigid or not specific enough or aren't integrated correctly. With the LLM now you can actually fix those problems and so the workflows become way more capable. It's just such a different more kind of customized, you know, we used to always trade off customization with kind of how durable is this now when we want to upgrade, how do we want to do this, how much cost we want to spend on customization long term in the software world. And that's always been this trade-off going back decades. It's a pretty significant, remarkable pivot to do this.
Well, being able to use the LLM to do the building, but still have the workflows to depend on means you can go back to it, you know, a month later and say, you know, I don't like the way that workflow is working. I'm going to change it. The alternative to that is to have everything running on prompts and you have to go figure out what part of the prompts you're going to change to try to get the outcome you want. And we have a strongly held belief that that's going to be harder and less reliable, though it's very popular these days.
Not to mention the sophistication of the end user comes into play there as well.
So they have to figure out how to... you're going to have a range. You're going to have folks that are very more capable and less capable using this. Are you finding more folks... are your customers are more of their employees using your software then it's kind of opening it up and making it more kind of available.
Yeah. The problem with our software historically that when we saw at the end of 2022 as you said ChatGPT hit candidly so much better than we expected. You know when I was at Dartmouth neural networks were known but it was a failed technology. Nobody was doing neural network but you know why don't we teach for little experiments and stuff but it just was not capable enough. But what happened in 2012 to 2014 as a result of the Nvidia hardware, there was some experimentation done where people said, 'Oh my god, this stuff is so much better at matrix math that we can actually maybe get these neural networks to work.' And then by the time you go into like four or five years ago, they were really working. And they were working better than anyone could expect. And you know it blossomed into what we see out there. It's actually I actually find it really sort of threatening as a human being because I had thought that one of the things that might make us special, you know the sort of closer to a god as it were is the power of language. And turns out all you need are 1.5 trillion connections and you know the silicon can do it. And so it's a little spooky actually to tell you the truth.
It's pretty wild. Yeah. And it's worked out pretty well for Nvidia these days as well.
Larry, you think a lot about this kind of historical kind of waves of technology, you know, from your perspective, right? You've seen this come and go. You've seen AI winters of past. How are you thinking about this moment and sort of this opportunity because you not only do you have that perspective, but you're also sitting inside one of these organizations by being a board member at Pega and working with other leading companies around the world.
Well, having launched so many companies over 40 years, I always separate a bit business from the technology a little bit. And for the first time in this case is I think it's a time where this is about business strategy, the use of what's available here and it's not necessarily just buying technology anymore. It's about dealing with how do I create competitive advantage in a world that's changing so quickly and so fast that business ends up becoming just about two things in my opinion quality and speed. And so the more you can have software tools and I put AI in as a software tool by the way, but how you can use those to create quality and speed as competitive advantage is very important and also this is a time and I'm sure Alan would agree this is a time similar that I saw at the web and we were lucky enough to introduce HTML which is technically the first browser I guess but for Tim Berners-Lee and people most people just didn't know what to do with it that was just very... and that was another moment that was strategic. You know you finally saw a guy like Bill Gates just say stop 1995 we have to change everything and do this and I think we're at that kind of moment from a business perspective not completely a technology perspective that if you don't learn and apply especially agentic integration into what you're doing you're going to be far less competitive in this world and it's also a time that I was hoping for before I was not working anymore was a time where humanity and technology were coming together, not going further apart. And I actually believe that this whole movement is going to create more of a serious human connection with... and one word that when I wrote my last book that a lot of experts Google, Microsoft that I talked to, Meta, Pega, they kept using the word adjacency and this idea of human thought and reasoning in connection with the use of these new tools was going to be extremely important especially in vertical categories like healthcare and agriculture and so forth. So, you know, I'm sure Tuck's teaching this and Thayer's teaching it, but it's I was saying on the way up here, it's someone was a genius by putting engineering and business schools next to one another. Because I think that's what's happening right now.
And that integration has been really fast, right? To see that sort of blurring of that line has been incredibly fast-paced.
Well, it's wonderful because the outcomes are going to be huge. I mean we don't even know will their iPhones go away who knows things like that.
Voice is an interface, other interfaces, yeah.
So you know as we think about that one of the things you mentioned is some of is decision-making too right how are we making decisions. Talk a little bit both of you maybe about this moment of decision-making. You know I think what we've observed and there's been kind of conflicting research on this is hey are folks turning decision-making over to AI in too great of a way or are they using it to help them make better decisions?
I think Allan's the best one to answer that especially with the applications in serious stuff like finance and areas where Pega has deep history.
Yeah. How do you think about that when you're kind of building the software of where do those decisions lie and how do we manage that effectively?
Well, we're trying to draw the line where there's an analysis or a sort of evaluation phase where the AI is extremely aggressive at recommending things and showing you possibilities, but that a person should be able to choose which of the variations of this it either might want to override or it might want to choose between two choices. There's a concept in chess which actually I find is one of the few things from chess that I've been able to apply to business called candidate moves. And when a chess master looks at a position you train yourself not to just think about the position to come up with a move you train yourself to come up with candidates what are five things that I'm going to analyze and then with some discipline hopefully you actually go through and analyze each one of the five ones. People who don't play strongly usually will sort of obsess about it and settle in on a thing and then do it. And it's really easy to miss a candidate that's better. What the AI enables you to do in decision-making is it enables you to generate slews of candidates and actually it can even help you evaluate them, but then you have that opportunity to really pick between a much more creative palette than you could come up with on yourself. And I think used that way the AI is contributing to the decision-making not entirely making it.
Yeah. And still have you know cling to a little bit of value as human beings in terms of... it. It's interesting right because you can imagine over time that that becomes advantageous for some businesses to try to move towards more the AI making that decision as you mentioned before Larry about speed being an important here, right? In some ways that human in that decision-making loop is the slowest piece, right, of if the candidates come up really quickly and we can do some analysis. Do you expect that to be tension in the market?
Well, I think it's going to depend on the problem domain. So for the problem domains that we focused on, what we're saying is hey you really want to do this analysis as much as possible in advance because most people want a playbook that describes how they handle certain types of things. They don't want to be making it up from scratch every single time an interaction happens. And for our types of businesses, I think that's really good. It has the other advantage this by the way is more important in Europe than it seems to be in the US these days. The other advantage of not burning the amount of electricity that this reasoning when our stuff reasons we're just burning the kilowatts like anybody else. But every time I've reasoned something through and come up with a pattern that I've codified, the subsequent executions of that happen thousands of times faster. They happen on a regular CPU as opposed to GPU. And so I think there's a lot of advantages, but for some types of problems, you can't do that because it's just too dynamic. You know, if we get Elon's humanoid robots running around, they're going to be doing this constant decisioning on the fly. And that's, you know, kind of what...
Very much on the edge. Yeah. To do.
That's fascinating. So, you know, this brings up the question also though of, you know, as we have more democratization even within the enterprise of who's building workflows and building systems, how are we ensuring proper governance and oversight of some of those decisions as we build?
Well, there are things you can do and you can actually build into the various technologies to look for predictable patterns and to look for certain ethical behaviors and sort of judge that from the outputs and the outcomes that you're rejecting.
A big part of it is there transparency in terms of why a decision was made. One of the fundamental issues with neural networks which sit on these things, you don't really know and you almost can't know exactly how it made that decision, right? That's why I favor, you know, workflows over the black box of a neural network. And that's not a bad thing. It's inherent to its power, but it also, I think, means you need to select it in certain situations. So, you wouldn't use the neural network to make a decision in a situation where you might run afoul of housing loss or run afoul of other sorts of things, which the neural network could easily decide this is a more better probabilistic outcome, but it's just precluded. And figuring out how to get that balance to work is very tricky with prompts.
Yeah. It's important to have somebody in the room that understands the ethics around software applications and so forth and Pega does. But so if people are interested in this category, I think of all companies, even though it's big, IBM has been doing a lot of thoughtful work on guardrails and ethical use of AI and software. And I'm not big on governments understanding anything technical, but the UK government actually is doing some interesting things in sort of ethical use. And you know, there's been some companies that have, you know, in America especially, when a category screws up, we make a chief of something always. So when marketing didn't work, we got CMOs all the time. And you know, something else, we always got chief. But maybe it's time to have chief ethical officers or chief, you know, something that helps make a moral corporation.
I think I can program an LLM to do that role.
I mean, in some ways that's the joke, right? That's some of the idea, right, behind a constitutional, you know, model of AI theory, but in practice, it's a little bit more challenging.
Well, I think also the people who are worrying about LLMs and neural networks needing governance and somehow thinking they'll get it need to realize that that ship has sailed.
Yeah. It's gone. It's long gone. Yeah. It turned out to be too easy.
Really? I mean, maybe we shouldn't be amazed because we have billions of people who learn to speak and so now we have... Hello.
Maybe it was just never as mysterious as we thought. But these things are extremely powerful. They're just getting more powerful. And having said that, I don't really believe that we're going to get the, you know, the Skynet coming, right? I don't think we will either.
Well, and the metaverse. How about that? Anyway, but the confusing part sometimes is that we're living in a time where truths, you know, at least to the everyday person, you know, what is real, what's not, what is AI created? And that's where you forget that it's been around seven decades and it's been developed.
Where it's really going to be very valuable I believe is in some of the boring applications. So, you know, so finishing the supply chain, you know, like you can take a photo and instead of having a high school kid count how many half gallons of oatmeal Starbucks needs, that can all be done very quickly, you know, in a photograph. I was having a problem with my boiler during the last snowstorm and it was putting out this weird message and I, you know, went down and I took, you know, Google Lens, right, and Gemini and I just snapped a picture and I said, 'What's going on?' And it told me exactly what was going on. And then it told me...
Now don't tell your plumber that.
Actually, I did tell him that when he came because Gemini couldn't fix it.
Right. Right. Right. That's true. Yeah. Tell me what was... Yeah, plumbers are safe. That's actually...
That's true. That's true. Yeah.
Um, you know, I think to this point about the power and governance and sort of where these lines are, I think we've seen in the last week and a half, a really strange phenomenon where, you know, all the fears that we sometimes see from folks of AI and of technology. We also have a cohort of people that decided I'm going to turn over every possible thing, you know, that I own, all my passwords, my entire kind of computer to autonomous agents like Claudebot and or now Mulbot or whatever it's called today, OpenClaw. You know, what are your thoughts on that? I mean, there's sort of a it's a fascinating phenomenon, but it's also sort of terrifying from a governance perspective, from a security perspective.
