I know that you guys have been using the T-word a lot the last day or two in your conversations, the tokens. But maybe we could kind of pivot a little bit away from AI. I guess it's a little bit rooted in AI, but there's been this big conversation set in the last couple days about cryptography and some of the risks to cryptography. I think it took root after the OpenAI math drop, but there's more conversations now about this big quantum initiative and how that might kind of take us into an era where we need to kind of start to think about adopting quantum cryptography. But Arvind, maybe we could start and talk a little bit about where things stand today from your view.
Sure. So, a little bit of what David just touched on is DARPA's been running a competition called QBI or Quantum Benchmarking Initiative. A couple of hundred companies joined it a couple of years ago. I think it's down to maybe 13 are left. So, that tells you kind of what's maybe real. I'm calling it maybe not quite real. Some of it is real. And so, their goal is who can build a quantum computer that works. Let's sort of put the technology side out of it. And the debates are is it 2028, 2029, 2033? And sort of if you pick the middle point, let's call it 2030, well, it will break today's cryptography. That's not a supposition, that's just a reality. And if that's going to happen, then what's the cryptography that you got to use to protect yourself against that? Unfortunately for some of you, what's even worse, if you have an encrypted hard disk today that gets stolen, somebody could keep it for 5 years and break it at that point. The good news is post-quantum cryptography is understood, it's well-known, the code is written, but people have been too lazy to get to it. But I think the wake-up moment is now.
Are the solutions that we understand quantum-proof or quantum-resistant? Because it's sort of based on our basic understanding today of what quantum probably looks like.
Yeah. No. It is based on understanding at a theoretical level of what quantum could do. So, they are quantum-proof in the same level of rigor that today's cryptography cannot be broken by today's classical computers. That's actually not a theorem-proof level answer. It's more based on complexity theory in computer science, which is grounded at the end on some assumptions, not on...
So, if you look at where we are today, and that 2030 midpoint, can you just walk us through the evolutionary moments that you need to see, where we all have to start paying attention and say, "Okay, how do we move?"
Yes. So, today you got toy quantum computers. I'll call them toys because they can do certain problems, but they are rife with errors. They have no way to scale fully yet. So, question one will be, "Who's the first one who can demonstrate, not a simulation, not on the side, but some level of error correction or mitigation on a running quantum computer?" Step one. And so, I'll predict you'll see somebody make claims by next year. Then, you should get third parties to evaluate are those claims real. You'll find one or two will make progress by 2027. Error correction, not at scale, but error correction.
Question two will be, "Can you run these things at some level of scale?" Not yet complete scale. I call it gate computations for the lack of a better word. Today, most people are somewhere around 10,000. Which if you think of a normal computer is not very much. When your iPhone does a couple of billion per minute.
So, gate computations, is somebody scaling from 10,000? Are they adding a zero each year? That's the second milestone I would put. You put those two steps together and now if you put a zero per year, that means you could be at 10 million by 2029. That's remarkable. That can do some things that will surprise the hell out of everybody.
Does this reference kind of this notion of a logical qubit being kind of an error-corrected operating qubit? Is that a term you've adopted, qubits, and that everyone kind of has the same milestone, the same metric?
I don't love the term, but I think it's a fair term to just explain to people to say, "Hey, if I say I'm building a computer has got 10,000 qubits and they say, 'Yeah, but they're all noisy.' You're right. So, if I begin to error correct, it in some sense decreases the effective capacity of the machine. So, a logical then says, 'Okay, what's the effective capacity of this machine?' is I think a reasonable way to look at it. So, that's a term that while I don't love it, because it's not like I've taken these four things and it's like a DRAM cell and that's it. It's actually the whole machine is error corrected, but it's got some divide. So, 10,000 physical, maybe 200 logical.
Nikesh, you have red teams that think about what is at the frontier and pushing it. Let's say that you start to see these ooms, these orders of magnitude start to compound. If you had to guess, where would your red team say are the obvious honeypots? Where do you point that if you were a bad actor?
If you think from the perspective of everybody here has pretty much crypto keys running across the entire IT infrastructure. It's not one thing. Everything is running with crypto keys which are encrypted. Every vendor in our space has keys at each end. So, the first thing that's going to happen is whatever's not quantum-proof is going to be breakable. So, our view is you can't wait for 2029 or 2030 because it's not like one day all of us are going to go out and say, 'Oh, let's go burn $5 trillion of IT infrastructure and let's go build it again in 1 year because now we need quantum-safe infrastructure.' So, you should be replacing your IT stack today with quantum-safe or quantum-proof infrastructure right now. Now, there are technologies, there are capabilities which are already out in the market where you can start, you know, wrapping post-quantum-ready keys around traffic as a stopgap. So, you can do that, but the time to—if he says it's 2029-30, we're already late.
We're 3 years away and I'm pretty sure everybody in this room has got a few hundred million dollars of IT spend that is earmarked for this year and I'm guessing 50% of your spend is on infrastructure that is not quantum-safe.
There was a pretty senior AI researcher that said we don't need it by 2029. Well, I mean, his view was actually AI models today can get us there before quantum computers get us there. Do you have a view on this? And I guess, you know, and I guess it's rooted in this idea that on the frontiers of number theory you can see AI start to develop models and proofs that are not apparent to humans today that allow us to rethink how we can effectively do code-breaking. And perhaps the OpenAI math dump this week is an early indication that this might actually be a good prediction that he made.
Since you geeked a little bit by going down that direction, you'll excuse me if I just geek a little bit. It's not going to be possible. At the end of the day, this goes back to stuff that people like Richard Feynman did. You cannot simulate something on a different architecture and expect a more efficient result. Right. So, if you take that as a thesis, quantum computers do not operate in arithmetic, right, which is what all of classical computers, including GPUs, do. Could you have an AI algorithm discover a new quantum algorithm? Absolutely. Could it run it effectively and efficiently on today's hardware? Yes, it's not possible. So, you need quantum hardware to run these things to get the positive advantage that the quantum algorithm has. Otherwise, it'll be slower by an order of a thousand or ten thousand. So, could you prove that it's a workable algorithm? Yes. Could you get that benefit? Unlikely.
I think the argument is using, and I know we're geeking for a minute, but just using classical computing, you can actually have a different path, arithmetic path. For example, if you had a better predictor of the distribution of prime numbers, you know, like at the frontiers of number theory that we've not really resolved, we can start to have classical models that can start to break cryptography.
It is possible. The question is how long will it take because that's always the question. Yeah. So, I think it is going to be possible, but if it takes 10 years to break one key, most people are going to rest in peace. That's not a practical answer. That's I think where the debate is going to lie, but it won't prove out until it's done.
Let me move the conversation to something that's happening currently. I would love to get both of your takes on the following concept. There's open source, there's closed source, there's frontier, there's pacing the frontier, there's now effectively very reasonable alternatives that are open weight at a minimum. Your philosophy around the entire spectrum, and just give us a sense of how we should think about models and capabilities over the next couple of years. Nikesh, do you want to start?
Sure. Look, I think earlier in the day somebody said, 'Don't restrict your teams in terms of capability in the experimentation phase. Don't start trying to cost optimize. Don't start trying to stitch your own in that phase' because the teams who work for us, 20,000 people, have just discovered AI in the last 12 to 18 months, and it's not fully diffused across the organization. So, if you make it harder for them to adopt, if you make it harder for them to understand, it's going to take longer for us to be ready. So, right now, we're saying, 'Go use the best model you can find, the easiest way to use it, so you can understand and validate the things that these things can do, whether it's from an efficiency perspective, revenue perspective, or product development perspective.' That's kind of I'll call it phase one. But, in phase one, what we've discovered is there's different horses for different courses, right? What people don't understand, you know, there was a moment when everybody thought that cybersecurity was going to get consumed by AI models and, you know, our lives are over because Mythos is going to solve every problem. Well, guess what? We inspect all traffic at an endpoint with the cost of $20 a year. Right? Hundreds of megabytes of traffic per endpoint at the cost of $20 to $40 a year because machine learning is running on your endpoint, your compute, and we run it on a 20, 30, 40 megabyte payload. Now, if you want to take all that traffic, run that through OpenAI using tokens for every employee in your company, sounds like the sound of revenue. Like, OpenAI would probably generate all their revenue in 1 year from one company.
So, that's not a practical example of how models do it. So, what you do is you take a small language model, you compress it, you make it task-specific, you keep training it until it gets to 99.9% accuracy and say, 'Instead of using machine learning algorithms, we're going to use generative AI, small language models, and they're going to do that.' So, the point is, what we're doing is we're trying to figure out what task requires what level of intelligence, and what is the cost of that intelligence for that particular task, and you'll suddenly discover you don't need a Ferrari for every task. And by the way, the customers don't want to pay for a Ferrari for every task. I'll give you the flip side. Flip side is let's assume you're undergoing a breach. You're being attacked, you know you're being attacked. At that point in time every customer in the room says, 'Don't worry. Use whichever model you need to stop the breach.' Because they know the business impact of that breach is in the billions of dollars. At that point in time I can run agent swarms, I can do all kinds of stuff that you want from a defensive perspective. So there I may want to use the most up-to-date frontier LLM and say, 'Go think about it, reason the hell out of it and figure out how to stop this breach, how to make this happen.' So what we're discovering is you're going to end up using frontier LLMs in unique cases where the business value that is going to get disrupted justifies it. We can do drug discovery if we want. I think, for most practical use cases in enterprises, we're going to end up training some version of an open source model. And I think, when it comes to even in peacetime for us for example, running a defender model we're more than likely going to take an open weight model, use our own proprietary data, train it, manage the weights, because that's what gives us the auditability, the explainability, and the open system where we can actually explain how that happened.
I completely agree, but I'm going to take a slightly different lens to get to the same exact end conclusions. Let's take economics. So today $20 for a million tokens for probably generation minus one, 50 if you want to go to the most extreme. We could take one of the Chinese open weight models, take a Kimi K2, something 800 billion parameters; and if you run your own GPUs with a little bit of optimization but I'm talking like a week of engineering work you could drop the cost to about a dollar to two dollars for a million tokens. Which enterprise in the world will not accept a 10 times or in this case a 40 times cheaper answer for effectively what can do 95% of the work. So, I think economics is important. The second is going to be at the end of the day, you just talked previously about chain of thought, not just the end tokens. Well, effectively then you're giving away your IP if that chain of thought can be captured and logged and audited by somebody else. So, brand and IP is going to drive people to say, 'I need to run the models on my own.' Well, you can't run a frontier model on your own because none of them will give it to you. So, then you're back to open weight. And the third is geopolitics. The best open weight models today are Chinese. By the way, I think there's very little danger if you're putting them in a sandbox and running them by yourself. Sure, if you use them from a cloud, that's different. But I think that over the next year or two, I'm very doubtful that there will not be great models that come up in the west that are also open weight and that are as capable as the Chinese. The gap in these things is 6 to 12 months at the most. And so, all those three drive me to conclude I don't know whether it'll be 75% to 90%, but that's the range I would put of all token usages will go to open weight models for the reasons I mentioned added to the reasons that Nikesh mentioned.
And the token consumption that's left over, is that because you actually need it or is that because it's a default behavior and people are too lazy to change?
Okay. So, I'm going to talk about developer behavior and since I was one, I'm making fun of myself in a younger age. Developers are lazy. They're going to use the most expensive answer unless you put a control on it.
And so, that is going to take it there. So, I would say if experimentation phase you allow that to prove it works and now I got to put it on a cheaper answer, it's probably fine if it gets me there a lot faster. But doing it for production usage is where I am skeptical that it's of value except for a tiny number of use cases.
Nikesh, you were brought in as one of the small circle of people for Mythos to kind of preview it, be it their partner. I'm sure you're now doing that with OpenAI as well. When you're sitting with these guys, is part of you thinking they're looking at my business and they're like, 'Wow, this is a great business'? Do you think that they're... And what is the actual technical boundaries that you expect to be respected? And the reason I ask this is we've talked to a couple of other CEOs. I think Marc Benioff does the best example of this where, you know, we've talked about ZDR and he's been incredibly aggressive about making sure there's zero data retention and doing that on behalf of his customers. What's so interesting about the ZDR and all of this stuff is like none of the models say that they won't generate synthetic data. It's literally not in there. You guys should look. There's all these outs. There's all these trapdoors. So, I'm just wondering like when you're in the room, what are you thinking? When you're in the room, you come out and say these guys are not going to get into my business.
They're not going to be able to get into my business.
Because if you think about when Mythos came out, right? We've been paying billions of dollars in coding agents. Well, guess what? Not everybody's got ZDR in coding. There's tons of people out there using Claude Code who have no issue about that data training the models. So, we are training these models with end users doing coding. These models are getting better and better doing coding. Well, guess what? If you teach a model what good looks like, it also can figure out what bad code looks like. So, the first thing that Mythos did, it figured out what bad code looks like. And then if you let it loose against your code, it'll tell you what is bad. What was fascinating is when it did that, even then it was hallucinating 30% of the time. Right? But the problem is the reason it's perfect for attackers is they just need one vulnerability. They don't need 10 vulnerabilities. If you're a defender, you need to find every vulnerability and patch it. So, all the people in this room, I think, have been through this journey in the last 12 months is that everybody here has gone, tested them whether with Mythos or with some cousin of Mythos or some open source malware they discovered. Every malware they have to then be tested, understood, you know, figured out. And I would guess in this room, if I show of hands, people say that number is roughly 60 to 80% have been real and 20% have been hallucinations. So, from that perspective, the models are great to be able to tell you what the vulnerabilities are, but I don't even think anybody in this room, if you ask them to raise their hand, will tell you that they're not going to even use Claude Code to patch the vulnerability without having to test it again.
I have a very random question, but it makes me think. So, you're engaged to defend companies, but it now looks basically like warfare. Like both of you guys, you know, it's like warfare. And so, is there something that you can learn from how war has been fought or how war is being fought? And do you do that to actually inform how the technology should behave? I don't know if this question makes sense, but...
I think to the extent that you can be better together, you can collaborate. Like Arvind runs a program, we're now saying we're all finding vulnerabilities, why don't we pool our vulnerabilities and find the defense ones? As opposed to every company in the room trying to go test all the open source vulnerabilities they have, figure out how to patch them, go patch them individually. So, then you take some trusted parties, give us your vulnerabilities, we'll patch them, we'll distribute the patches so you can all be safe at the same time. It's kind of like how cybersecurity works today. If you find an attack technique in a different part of the world, we can take that attack technique, write machine learning code, and ingest that into our product and make sure it gets available to every customer.
Is AI-generated code error-free?
So, look, but first on your war question. So, what Nikesh talked about is you can have powerful alliances where the alliance effectively can become more powerful than the attacker. That's technique one. Technique two in warfare what has been the most effective is asymmetry. And so often an attacker is cheaper than the defender, so the defender runs out of money and at some point then concedes. Or can you make defense equally cheap? In which case you can basically have symmetric warfare and then you have a much better chance of success. And the third which has been the way all wars have been won is when you get technological superiority on one side. So go back to World War II, go back to Korea, go back to Iraq, go back to anyone. If you get technological superiority on one side, you tend to win. And so our goal is can we get technological superiority and that's what Nikesh is referring to. Can we find vulnerabilities, can we patch them, can we put boundaries around them that makes it really expensive for the other side to now try to find a way through? So that's what I would answer there. Look, at the end will agentic code be superior to human written code? Absolutely. We can find that. We use agents and coding agents now for probably half the code that is written inside our place and it is superior to what was being written before. But the reason it's superior is the agents can also do evals and basically make sure that everything they can find is actually fixed before it ever goes out. All that said, we're only finding all the flaws that we know how to find. And so we get into this weird loop in computer science. Okay, do I...
Fully properly verifiable?
It's not fully verifiable because that is a task that is going to be almost computationally infeasible right now. Now, I'm not going to hold out hope that it doesn't become what you were talking about in number theory and all that. This is a task in which maybe we'll find a way to actually get to be able to verify the correctness of these things. At least where the inputs are limited. If the input can be anything then I think it's really hard.
Yeah. Let me ask the implications of the points that you just made. It seems obvious to me that there isn't a world where China and the United States or the West or whomever, a bunch of international countries all hold hands and say let's all stop AI development. That being the case, it becomes a necessity that there's an arms race, that there's a defensive or offensive superiority requisite, that there's a compute capacity requisite, that there are all of these needs that arise to protect industry, to protect our economy, to protect our jobs. Does that argument win the day with everyone that's trying to stop AI and the political conversations and the social stuff about AI needs to stop and we need to stop doing this? Does this point about the inevitability of one side having it requiring the other side to defend it change the conversation? I'd love for you guys to share any anecdotes you've had with senior party officials who might not be in power today but might be coming into power and whether they're going to kind of have a view change around this because of that point.
