So, we're at an AI conference and you're an HPC guy. So, what are you doing here? Right? Did you get the wrong directions, the wrong conference? What's up?
True. Well, I started Next Silicon eight years ago and we went after supercomputing, running physical simulation, fluid dynamics, everything that is nuclear related, security, defense, and those things. And now there is AI, and AI is all you get. Yeah, that's the sexy new one. No one cares about HPC, but in the foundation level.
Even though that's the route that Jensen and all of GPUs followed as well.
So that's the point. I mean, even GPUs obviously started with graphics, but then they moved to supercomputing with the big machine running up, and HPC and AI in the foundation are exactly the same mathematical principle, and even AI workloads are way simpler. So we have tackled the hardest problem about running HPC code, which is more complicated. You have so many different kernels, so many different simulations that you need to run, and from there going to AI is kind of a simple one. Let me give you an example. When we designed our chip, we had the Maverick chip. That's the name of our accelerator. We had a data flow that can crunch the parallel section of the code. But in HPC, you have a lot of serial code. So instead of going to the CPU side, we built our own CPU. We have built our own RISC-V chip inside the chip to accelerate those workloads. So it's kind of entangled together. And the point I want to say is that yes, we are in HPC and we can rock in AI and machine learning acceleration, tackling the big thinking model, the agentic one, open source models like GLM, Kimi, Qwen. And speaking of CPUs, I mean, you had a vision a long time ago when the entire world thought to itself, well, GPUs are going to take everything.
And we are obviously doing our own CPU, but we are seeing that things change now. So how do you see the CPU play in the world of AI?
Well, having only done 14 generations of microprocessors all the way back to the 286, 386, 486 where I cut my teeth. I've been a CPU guy a long time. A couple of years ago, I was having this interesting conversation with Jensen about the CPU to GPU ratio. Okay, we still need some CPUs, but it's like one CPU and eight GPUs, one CPU and 16 GPUs. It was like, what is that balance of compute requirements across the two? And it was always drifting in that direction even though I never believed that would be the case. CPUs are about the workload, not about the architecture. How do you run the workload? And now as we've seen with the Gent multimodel environments, there's a whole lot more affinity toward the CPU portion of the workload. Now people instead of 1 to 16, it's 1:2 or even in some cases one. So the CPU is sexy again. And for a CPU guy forever, we're like, okay, that's pretty good. That's been a meaningful benefit to Intel over the recent period where there was a lot more demand for that balance between CPU and GPU to be much more one-to-two. And obviously, since I've been a CPU guy forever, I'm happy about that. But it's about the workload. It's never about the underlying architecture. It's how do you execute the workload? The software is on top, the hardware is underneath. And how do you think about that, Eli? The balance between those two, and what are you doing to make it easier to address them both?
So I mean we have seen recently a trend that people say well let's build a custom ASIC, let's build a chip that can do only one thing. I mean today the current big models, the big LLMs are obviously transformers, so okay, let's build a chip that can run Llama. That was like a thinking. But we are seeing innovation moving forward. We are seeing a lot of envelopes around the attention heads, multi-latent attention, sparse attention, and those computations keep changing. And what you need, as you say about the software and hardware, you need to have a flexible hardware. You need a general purpose type of hardware that you can throw anything on it. Whatever kernel, FFT. Yeah. I mean sometimes like Mamba models, you need to work on it in the frequency space and not... And with us, what we have done, that's the biggest plus of Next Silicon: the fact that we have built our architecture from day one to be software driven, to have the ability to take whatever code section that you want, understand them, place them on a data flow for the acceleration, and the serial code on the CPU, on the RISC-V CPU. That was day one, and this is future proof. I mean, whatever model will come next, it's something that we can accelerate. And I mean, have you seen any in the history in the past eight years since the volume any architecture that says let's accelerate one thing and one thing alone and it was successful?
Yeah, and you know, generally you see the special purpose things emerge, right? There's a period of time where they're interesting, and you go all the way back to like floating-point processors, if we could be so mundane, where they were specialized, or media processors and specialized, but eventually they're just absorbed into the architecture of the machine because the workloads continue to evolve over time. And I mean, look at some of the conversations today. We have a training farm, but that's different than the pre-... you know, the G code, and it's different than the prefill, right? So we're taking the same workload and carving it up into more specialization, right? For it, that's not a scalable architecture, right? You end up with these islands of compute over time at scale, and fundamentally the workloads continue to evolve.
Absolutely. I mean, training is a workload that is kind of different because you can batch a lot of things together, but now everyone wants to talk about prefill. This is decode in the inference side, because prefill is very similar to training, and the decode you have the serial part that calls attention that is kind of serial, and they say okay, we need something else that is more memory bound. But I mean, here Yann LeCun always pitching the model should keep learning like us humans that keep learning. Yeah, it really is pretty, you know, to say that we have training and then inferencing. Inferencing is becoming decode and prefill, right? It's like, no, I want models that are constantly training, learning, becoming more customized into environments. So there shouldn't be that separation in workload. And obviously, we're now in what, year seven of transformer? There will be things beyond it as well. And we need architectures that accommodate those evolving algorithmic spaces and new models over time.
