Well, thank you very much. It's very good to be with you today. I'm here to share some thoughts about AI and the infrastructure world. I think the most obvious thing to see in the relationship between AI and infrastructure today is just this extraordinary, insatiable demand that the world has for tokens.
If you look at the consumption of tokens, it's growing faster than any other era metric, anything that we saw in the PC revolution, the internet revolution, the cloud revolution. The number of tokens being consumed is literally doubling every three months or so. So we're demanding that our infrastructure produce trillions of tokens and the growth shows no letting up.
If anything, the current developments in AI where we have agents performing intelligent tasks inside organizations, not only handling coding but all sorts of business processes, just generates more token usage. Because first of all, we're finding that we can break the model usage down into a collection of models. So we have multiple models cooperating with each other. But then we need to check on these models. So we need to establish juries of other models that are checking the correctness of the models and checking them for compliance and other regulatory constraints. So it seems to be that we've got a self-feeding machine here that the more tokens that we request, the more tokens are needed.
And the obvious solution to this insatiable demand is to actually not try and recentralize everything and put everything into a few large hyperscale clouds, but in fact to move the compute out to the edges, into data centers, and eventually all the way down to individual devices.
Now, in addition to that, there's this whole issue of sovereignty. In other words, people are concerned about where their computing is being executed and where the data is stored and where the data is flowing to. Because people don't realize that as we move to more sophisticated usage of AI and the prompts grow larger, larger amounts of data are actually flowing out of the organization into the inference engines wherever they are and coming back again. So you couple that with geopolitical risks and the risks of vendor lock-in if you become dependent upon a single underlying model provider. There are good reasons to want to move the inference tasks out into a much more distributed structure, which is going to drive demand for infrastructure that can support AI and inference in many more locations and many more settings around the world.
Now, the reason that this is actually something that is feasible is because even though the landscape has been dominated to date by these large so-called frontier models that are executed in the proprietary clouds of a few vendors, the smaller open-weight models are dramatically increasing in terms of their performance and catching up with the frontier models.
So these smaller open-weight models that can be executed, downloaded from Hugging Face and executed in general-purpose infrastructure, are becoming very, very effective. Just this morning I went and had a look at the LiveCodeBench leaderboard, and this is a coding benchmark that is developed by MIT and Stanford and other universities. And most importantly, they don't disclose all of the benchmark because otherwise vendors simply take the benchmark and train their models to be very good on that benchmark. So the individual model providers can't see exactly what's going on inside the benchmark.
Now, all of the models on the leaderboard, all 30 or 40 of them, when it comes to the so-called easy task, are all scoring above 98%. In other words, they're all as good as each other. More interestingly, when you look at the hard column, this is where they're being tested on harder coding techniques, the top scoring model, which is one of the OpenAI models, scores 63%. But if you go down to number eight, you'll see the Neotron 32 billion parameter model scoring 46%. Now, if you think that's not very impressive, if you go further down to number 20, you'll find Claude Sonnet 4, one of the most popular models in the world, scoring only 26%.
So we have an open-weight model, 32 billion parameters, downloadable from Hugging Face, that's eminently suitable to be moved out to more distributed infrastructure. And of course, when you look at the cost spread amongst these models, it's pretty dramatic. You can achieve much greater cost savings, much more efficient execution of these inference tasks when you move out to these open-weight models deployed upon distributed infrastructure.
So today we also have the situation when it comes to building infrastructure for AI and inference that Nvidia really rules the roost and you have to pay Nvidia prices obviously for the GPUs involved. But I think that's going to change as well. We've got new architectures coming. First of all, we're seeing the arrival of unified memory architectures. Best example of this is Apple's M4 silicon, which means that on a relatively high-end but still very cost-effective laptop, you can now execute easily 32 billion parameter models and get very good performance. You see Qualcomm is following with similar unified architectures, and in fact the x86 ecosystem is going to follow up later this year and next year with unified memory architectures. These are architectures where the CPU and the GPU eat out of the same memory, so you don't have to bear the memory cost twice in the CPU complex.
Perhaps even more interesting, we're starting to see the arrival of new so-called TPUs from people like Google and others. So these are dedicated architectures that have been developed specifically for generative AI inference. And while they're going to be initially deployed in proprietary settings inside the clouds of the hyperscalers and eventually coming all the way down to your phone, it's inevitable that merchant silicon will follow and we're going to have a greater choice in the architecture that we can use to support this movement towards distributed compute.
Now, of course, it isn't enough just to set up low-level infrastructure. This is not about virtual machines and Linux containers. You've got to make it easy. Otherwise, you're not going to win the war against the hyperscalers. We need to work with infrastructure vendors who can provide a complete solution so that it's very easy to set up these new open-weight models in particular and construct the agent frameworks on top of that that allow people to work at a higher level. So it's highly advisable in my opinion to be working with someone like Virtuoso who can give you essentially AI-ready infrastructure, not just the low-level primitives but also the frameworks that you're going to need to provide ease of provisioning and management. So I look forward to seeing Virtuoso introduce its new capabilities here and help with this democratization of AI. Thank you very much.
