The Pi Cast Episode 200: Raspberry Pi CEO Eben Upton talks Raspberry Pi 5 16GB

I think they're going well. I mean, as you know, we had a real new product introduction tear in Q4 last year. I think we kind of exhausted our capacity to invent, ship, write about new products. And there was certainly a sense of exhaustion also out in the wild. You know, when you get to a point where you are launching more than one product a week, when you start to run out of weeks in the year, perhaps you're releasing as many products as you should. So it's been kind of nice. We started the year with a couple of product releases last week, but it's now just good to slow down a little bit and enjoy the products in the world, particularly the 500 monitor, which I'm sure we'll talk about a little bit.

We talk about them as generics sometimes, these products. Ah, thank you, thank you, Les. They are products, as you say, which are useful even if you don't have a Raspberry Pi, because often we talk about, 'This is cute.' You know, you see the two little teeth, the two little hook marks in the when you fold the foot all the way up like that. That was Philip Colligan, who runs the Raspberry Pi Foundation, that was his son Dylan's idea. He was like, and this is a product that's been developed, I don't know how old Dylan is now, I think he's like 16 now, and this is a product that's been developed so long. He was 10, 11 years old when we first talked about it. And when we showed him a concept of the monitor, he said, 'Wow, I love it, but one thing I want to be able to do is bang a couple of nails into the wall of my bedroom and then just hang it on the wall.' And so those little teeth marks there are a Dylan thing. And we, he got one for Christmas, so Philip came into the shop and bought him one for a Christmas present, which was kind of cute. It's been enormously long in gestation. I remember John C. Hughes, our industrial designer, I remember John prototyping this in Lego during lockdown one, so April of 2020. I remember he had his son's Lego kit and he had red and white bricks, and he made a little that kickstand structure. It went through a lot of iteration before we figured out that we could do it all using friction. And I remember he had several different Lego models of how we might have something which was stiffened with a cross piece, like an A-shape, rather than being stabilized by friction.

Yeah, because you do get those monitors, you know, the kind of portable monitors. This kind of goal of building a five-volt portable monitor, it is a — the ones that you see on eBay often have that kind of surface-type fabric case where you have the kind of notched fabric, which is good as far as it goes, but it is fundamentally something if you knock it a little bit, it's gonna fall out. Oh, it's a big source of frustration. We review a lot of portable monitors and we see that all the time. It was actually funny the other day, a company that makes monitors, that does nothing but make monitors, came to our office showing us their monitor, their $400 monitor, which had an origami case, not a kickstand, and they said, 'Oh, don't worry, this is fine.' And as they're setting it up, it's flopping over. Uh, yeah, and this is supposed to be a classroom product. It's actually got — the history of it is there was a government tender in Argentina, I think in 2017, maybe 2018, for off-grid computing. And it was apparently with the Pi we had a solution for the compute element of off-grid computing, but we didn't have a solution for the display element, because all monitors tend to require AC power. And so this is where the idea of building a five-volt monitor came from, but it's also where the idea of having to have an object which is very physically robust. And you, I don't know if you've taken yours apart, but it has a big stamped aluminum sheet inside that provides it with torsional rigidity. And so that concept, and also the concept of wanting to have a very high packing density, so you see the thin structure at the top and then the thick structure at the bottom, and they're designed so they nest together. You can pair them up, and they nest together about an inch, you can get about two of them into an inch. You could get roughly 25 to 30 per centimeter case if you pile them up. It's slightly annoying that the bulge is slightly more than half the height of the monitor to fit the VESA bracket, but it is still designed so you can put a lot of them in a case and then take them out into the wild and use them.

Definitely. So nice community projects for the monitor where they've been 3D printing brackets to fit on the back of the monitor to contain a Pi as well, and they did it in the same color PLA as the monitor, which was a very nice touch. Yeah, I mean, that's what's nice about these products, is they are designed to be kind of quite hackable. We always try not to close off. I haven't seen anyone put a Pi inside. I was kind of expecting probably to have seen a Zero 2, because there was half of that — you know, one half of the kickstand, the back contains the motherboard, and the other half is basically empty off of the speaker, so it's quite a lot of internal space. I don't think there's enough internal depth to fit a full-height Pi, so you probably would need to strip connectors off if you wanted to fit a full Raspberry Pi in there. But I suspect we'll see some of those sorts of hacks over time.

Yeah, an update on that. Jeff Geerling put something out earlier today, I think, where he's hacking around with Intel and Nvidia GPUs because he's been very AMD. The limitations of the PCI Express root complex even on Raspberry Pi 5 — Pi 5 has a much more sophisticated PCI Express root complex than before, but even there, there are some limitations. The cards kind of push right into the corners of the PCI Express specification, and the AMD ones seem to push a little less hard into the corners than the Nvidia and Intel ones. But yeah, if he's made some progress, I think it's one of those where you'd want to probably make it easy, but by default, I think there's a gap between 'by default' and 'make it easy'. So I think one thing we're doing a lot with a number of commercial partners at the moment is making it easy for people to install particular pieces of software, not necessarily to promote to being a default element of the distribution, but doing the kind of one-touch install where you just — so you don't have to install that. I think that might be a sensible home for that. If that became available in a packaged form which was very easy for people to use and understand, that might be quite a nice middle ground between 'go build it yourself' and 'this is now a supported element of the operating system.' I don't think right now there is even a really easy way to attach the card. I think the dock is using one of the PINE boards, on boards that break out PCIe, it's a bit of a teetering stack. It's a fragile object. I have wondered in the past whether the sort of compute module object, which is the minimal, you kind of imagine a world where you have graphics cards that you plug into PCIe, but there are operations, Ethereum and AI operations, where what you actually want is the minimum PC that can host a graphics card. And I have wondered a little bit about whether maybe someone might build a little object which has a compute module on it and a PCI Express connector, and almost when you look on the table, you're seeing a graphics card with a computer plugged into the side of it, rather than the other way around. And whether that would be a viable host, because some of these applications require a lot of host memory bandwidth, wide PCI Express, and a lot of host memory bandwidth, where some of them really just require enough host to boot the system, and then by and large, the workload is contained almost entirely within the card.

I think the two that we called out obviously are LLM applications. You do see places where people are able to get respectable LLM performance out of a Raspberry Pi 5, except they are limited — there's a hard limit. There's a number of coefficients. People trying to use the 7B model, like the Llama 7B, and then you find you run out of space. And now of course there's a cliff if you go to swap, so it kind of moves that cliff out a little bit for LLM applications. It's not really worth moving it out much further, because then you become memory bandwidth limited, you just don't have the instruction throughput to get the token throughput out of the LLM. But it's a meaningful shove of the cliff for some of those applications. We've had computational fluid dynamics applications where there's a notion of a sensible amount of memory to provision each core. That was not supposed to happen — this is the problem with being this — we are being very eco-friendly. Well, there we are, this is Raspberry Pi being very eco-friendly — if you sit still in any room, Raspberry Pi... I'm sure there's a button I could have pressed to prevent it from happening, but I clearly pressed it. So you have computational fluid dynamics applications where you like, how much memory should I provision? And two gigabytes is not the right answer, four gigabytes turns out to be the right answer. So we have a number of units with some friends who do at Silverstone, who do CFD, who build clusters for CFD. We're looking at that. I think there's also that kind of containerization thing where you want to take a lot of workloads and containerize them and have a container host. And there, your expectation is probably heavier on memory, because you want to give each container a hard-coded amount of memory. You're probably heavier on memory than you are on CPU, because generally you rely on some containers not all of your containers running at maximum CPU utilization all the time, but they will all run at maximum memory utilization all the time. So that containerization world is also pretty relevant.

I think what we said at launch is there is a 64 gig aperture in the architecture, but to get to 64 gig you need several things to happen. To get beyond eight gig, you need something special to happen for each power of two beyond eight. And the three things that have to happen are: bite mode, where you have a die that only gives you one bite, so effectively the package we used on the 16 gig product has eight 16-gigabit dies in it. Each die gives you a bite-wide stripe of one rank, so you have four dies together to make rank zero and four dies together to make rank one. So in order to do more than that, bite mode got you from eight to 16. If you want to get another factor of two, that would be if you went to quad rank, if you had four chip selects. Now the controller does support quad rank, but with some technical limitation I think means we would never deploy it, or would be very unlikely to deploy it. And then the last one is large individual dies. So if you imagine your DRAM vendor is prepared to sell you 32-gigabit dies instead of 16-gigabit dies, you can put eight of those together to make a 32-gigabit dual-rank product. I don't think either of those are going to eventuate, so I suspect my current estimate is that the 16 gigabyte is the largest Pi 5.

Yes, that is that. That's all that — yeah, there have always been part numbers set aside for those products. That makes a lot of sense.