I'll point out to you underlying GPU pricing which is what all of these things run on has doubled in the last 6 months.
So it's getting more expensive to use these tools and there's going to be a huge motivation then from everybody who's using them to say I need to optimize. I need to use each one but in the most economic way possible. I'll use the analogy in a very crude way. An automobile is an automobile at some gross level if you step back. I guess you could take your kids to school in an 18-wheeler every morning. You could go milk shopping in an 18-wheeler. Mhm. Then you'd ask yourself, is it really the most effective vehicle for that?
This is Masters of Scale. I'm Bob Safian, your host. IBM is playing a distinctive role in the AI race. Not building AI models, but betting on how best to use them and on what comes after them. In this conversation recorded in front of a live audience as part of New York Tech Week at IBM's Manhattan HQ, we dig into why Arvin thinks most enterprises are using an 18-wheeler for every task. Plus, what kind of risk-taking businesses need to take right now, how to think about cost versus benefits when implementing AI, IBM's big bet on quantum computing, and much, much more. Please welcome to the stage IBM chairman and CEO Arvind Krishna.
Arvin, first of all, thank you for hosting us at your house. We're together as part of New York Tech Week. IBM is a New York institution. It is a global institution. As a company, it's had to continually reinvent itself, from mainframes to PCs and consulting and cloud and now AI and on the cusp of quantum computing. How do you think about staying fresh as tech moves? And are there things about IBM's legacy that are an advantage versus what isn't an advantage?
Yeah, look, the advantage is client intimacy. Knowing your clients. The advantages are around trust. I don't think we have knowingly ever done anything wrong with a client's IP or data or people. Those are advantages. I think our people are incredibly technically adept. I would say they're experts in the areas where they have spent time and energy. So those are all the advantages. Now technology keeps changing and I'll be the first to acknowledge sometimes we are really good at predicting where it's going to go and we kind of get ahead of the wave and do that. You mentioned a few of those. The mainframe certainly but this is now 60 years ago was one of them. I would say the IBM PC may have been another one of them. I think embracing Java in the internet era was another great one. And then you sometimes miss them.
It's not for actually a lack of knowledge. You miss them because the business model doesn't align. You don't quite know how to get the investments and returns to come together. I would say public cloud was one of them kindly. Mhm.
That we missed. I would actually turn around and say despite inventing the IBM PC, client server was another one that we missed. So I think it comes into you could probably miss some and you kind of know you're going to miss some sometimes and you know you're going to I think because if you don't take risk you're never going to succeed. So part of it is hey I'm getting a lot from that one. I kind of want to keep my focus there and kind of if you're not two three years ahead of the wave you're going to miss the wave. Mhm. But the whole point then is can you do enough of those where you can be with the wave as opposed to way behind.
Yeah, because it moves so fast nowadays that if you're two three years behind you're not going to catch up and so right now we are very focused on hybrid cloud that is sort of our answer to the cloud movement and AI where I think that our play is going to be much more of similar to a hybrid play how do we help our enterprise clients take advantage of it fully.
I mean the first mover advantage is this sort of catchphrase in techland and we I was thinking about you AI and IBM Watson and you were sort of ahead in some ways but you didn't maybe make the splash that you wanted. Was that a missed moment or is it more that you know you were too early or the tech wasn't quite ready or.
There's always all those things. I think when we won Jeopardy with Watson, I think it woke the world up because I think for the first time in a long time, AI started doing something people thought it could not do. Unfortunately, it woke the world up completely in the sense that a number of other companies started investing very heavily. Now, we had an advantage. We might have been able to succeed, but I'm not putting on 2020 vision looking backwards. The mistakes we made were actually much more of a strategic nature. As opposed to creating building blocks, we wanted to create solutions in verticals that I think is a mistake. As technology shows in the beginning, it's too quickly to the application. We went too quickly and we wanted to make a monolithic application. Mistake number one. Mistake number two, we picked a domain which is perhaps the hardest of them all which is health. Mistake number three, how many IBM sell to doctors and how many IBM deal with the FDA? Like none. So you picked the wrong solution set in an industry you know nothing about and with a customer you know nothing about. Other than that it was pretty good.
So as you think about IBM today and its role in the AI ecosystem like what is it? I mean you're not trying to be OpenAI or Anthropic, you're not trying to be Google or Microsoft. Are you ahead, are you behind? How do you think about all that?
So we are not going to be a hyperscaler which was two of the four you mentioned and we're not a foundation model provider. I'll say something provocative. I think foundation models are going to become commodities by the way not that far out. Is it a year, is it 2 years, is it 3 years? Commodities doesn't mean that they don't have value. Gold is a commodity. So is iron. So commodities means that there is very little switching cost to go from one to the other. Second, I think that right now the token price on all of these is going to go way up. It just has to to justify the capital investments. You put those two together and there's going to be a huge motivation then from everybody who's using them to say I need to optimize. I need to use each one but in the most economic way possible. Our role is to a let our enterprise clients do that. Two do it in a way that is safe and right now there is very little demand for on premise or smaller models which are effectively then 100th of the cost to run. So I'll use the analogy in a very crude way. In the end, an automobile, which I'll include trucks into it, is an automobile at some gross level. If you step back, I guess you could, for those in the suburbs, take your kids to school in an 18-wheeler every morning. You could go milk shopping in an 18-wheeler. Then you'd ask yourself, is it really the most effective vehicle for that? But if you're moving homes, which you do every seven years on average in the country, it is the most effective vehicle for that. I think right now we're using the 18-wheeler for everything.
And this is the transition that we're going to.
And this is the transition that I'll predict will happen within 24 months. I'm not sure it'll happen within 12 months. I'll point out to you underlying GPU pricing, which is what all of these things run on, has doubled in the last 6 months on a per hour basis. So it's getting more expensive to use these tools and right now when you're pre-public it's okay to lose money because you're gearing towards number of customers. I think you've seen this before, right? It used to be called eyeballs and then it suddenly became the economics are important. So I think we're right maybe a year from that point.
I mean you said something at the IBM Think event a few weeks ago. You said it's day zero of the AI revolution and I think for a lot of folks it feels further along than that. I mean you've got trillion dollar AI companies. A lot of business leaders worry that they're falling behind. Does day zero mean like you're not too far behind? You don't have to rush too much? What do you mean by that?
So, first let me be clear because I can sound a little bit cynical about the economics and I actually am. That said, I think AI is an incredible productivity tool. I think those who don't take advantage of it will be perpetually disadvantaged compared to those who do. So, let me begin by saying that it's going to optimize how you market. It's going to optimize how you write code. It's going to optimize enterprise operations. It's going to optimize how you sell. It's going to optimize how you get your daily work done. So there is no question about it. It's going to make a profound and deep impact on all of those things. By day zero, I mean it's time to sit down, take it seriously. You're not in the experimentation phase. So this is not like you're in high school. Day zero, the race is about to start. Put yourself in the blocks and start sprinting. But by that I mean take three, four, five things, not a hundred, and learn how to do them at scale because that'll teach you how do you get all your change management done. How do you get your data organized? How do you really get people motivated to change a process? So do a few things at scale, learn how to do that really well. Now do 10 and then give yourself the confidence to do the next 20.
I mean, there's this expression that's used to talk about the economy these days that's a K-shaped economy. You know, some households do great and some don't as well. And I sometimes get the sense that when it comes to AI, we're sort of having K-shaped businesses that like the tech companies, folks like you are super excited and then there are a bunch of other companies and these may be clients of yours I don't know that are like falling behind.
Yeah. So unfortunately I think corporate performance is even more of a differentiated K. If you look at corporate performance actually it tends to be a 2080 rule more of a power law. Then the K is really more of a 50/50 I think if I follow the economist correctly here it's more of a 2080. And so 20% get it. They go forward. They're jumping into it. They kind of are going to get their returns and 80% are either not getting a return or don't quite know what to do. If you're in that 80, figure out what is it that you should do and it probably doesn't matter where you start as long as you're starting to try to do it at scale to make a real difference to your bottom line.
So what you're saying though is like you don't have to know what to do like it's better to pick something and go than to just be like I'm not sure what to do. Like I have a client walk up to me and say, 'Look, I get it that I need to do it, but I don't have the right people in my team. Can you give me a deep AI expert, somebody who's kind of done their PhD in AI?' And I looked at them and I said, 'Actually, I recommend we give you somebody from a domain who doesn't really know the depth of AI, but who understands the difference AI could make to your domain. So, you don't need to know what to do because those domain experts exist in every company. Find that 20 or 30% of them who are motivated to say I want to learn a new way to do things.' So I think curiosity, a willingness to adapt is more important. I think we're getting hung up on I need to know AI like a PhD in computer science. I think that's the wrong thing because that's for the inventors of AI. That's not needed for the deployers of AI.
Find that 20 or 30% of them who are motivated to say I want to learn a new way to do things. So I think curiosity, a willingness to adapt is more important. I think we're getting hung up on I need to know AI like a PhD in computer science. I think that's the wrong thing because that's for the inventors of AI. That's not needed for the deployers of AI. There's also this idea that AI is going to save me a lot of money. It's going to be very efficient for me and I think for a lot of businesses when they start implementing those results don't necessarily come. Now you you guys have talked about that you've unlocked four and a half billion dollars of efficiency from AI. You're doing something. But there are also these hidden costs as you mentioned tokens like how do you balance what are the costs versus the efficiency and what you should be expecting.
Yeah. So this was my point of scaling. I would probably turn around and say that for our first six months to a year, we were probably spending more than we were saving because if you think about you're putting a couple of hundred engineers to work at it, that's an incremental cost. The underlying infrastructure aka the tokens if you're doing it on public is an added cost. There's opportunity cost also of not doing other things with these people that could have resulted in revenue. That's a cost. Now once we learn a rinse and repeat method that you're not doing it across two or three but across 10 or 20 when you're saving a billion dollars a year well that's a lot more than the cost of a couple of hundred people after year two we were definitely getting a return that was 10x compared to what we were spending and now at year four we'll be I think over 5 billion from a baseline of 22 spent so that's not an incremental five over last year that's an incremental saving compared to our year-end 22 spending. So that is tremendous. That is more than enough to offset any extra expense. And there was a CEO I was talking to about some of these issues, the ineffective the inexact nature of some of the outputs you get from AI, right? I don't want to call them hallucinations, but the things that don't go the way you want.
So unlike humans, right? That's so unlike humans.
Well, what he was saying was that the money that he was saving by having his engineers use AI that on the few cases where it was wrong, he had to spend so much trying to find what was wrong and fix it that he wasn't actually coming out ahead. Yeah. So, I actually think that that is an edge case of how you use AI that I think is actually wrong. I think that you should try to use AI in a case where you're not going to have to go undo 6 months of work or undo having spent hundreds of millions. Take customer service. If it gives a wrong answer, you got to undo one customer service answer. Then you can put all kinds of evaluations and checks. So AI can check itself to make sure that you're not like way off in the wild. Okay, it may be slightly off, but you're not way off. So you can put checks and balances and this is the sophistication of how you use it. So when we use it for example for our software developers to help them code I don't think they realize it. We actually have checks built in to make sure that what it is suggesting is not absolutely crazy. Right. But do you don't you I mean as with a human worker you have to expect that sometimes it will go wrong. It will go wrong. I kind of turn around. If I look at customer service, I think the stat which would be a good one is 85% of the time the human people get it right and 15% of the time and I say of course humans get angry, humans get pissed off, humans may not like the tone of the person on the other end if it's a call. Humans are sometimes overconfident. I'm sure we all remember things perfectly, right? I mean perfect. You and I do. I'm not sure. Have never told us that we are completely in the wrong and remember it completely. So humans have all those too. I think AI is at least if you keep it constrained to some extent it's probably 95% correct. So the eval I'm talking about is more like saying when you think you're way off punt it to a human. Don't try to venture into the underconfident range.
