Dylan Patel on CPU bottleneck
From Dylan Patel (SemiAnalysis): The Datacenter in 2026: CPUs, RL Environments & Agent-Driven Workloads · · Daytona and SemiAnalysis
“Whereas before you would have many GPU servers per CPU server and so you'd be like 100 megawatts of GPUs would be served by even like one megawatt or less of CPUs nowadays the ratio is getting much much closer both for RL training and for inference agentic inference. So then you've just seen everyone run out of CPUs. Amazon's volumes on CPUs of CPU servers they're installing have 3x year on year right from this year versus last year. And so there's just no capacity anywhere and that's causing a lot of instability.”
On , Dylan Patel, Founder, CEO, and Chief Analyst at SemiAnalysis, spoke about CPU bottleneck during Dylan Patel (SemiAnalysis): The Datacenter in 2026: CPUs, RL Environments & Agent-Driven Workloads on Daytona and SemiAnalysis.
About Dylan Patel
Dylan Patel, founder and CEO of SemiAnalysis, has been speaking at several industry events in early 2026 about AI infrastructure, benchmarking, and market dynamics. At an Aria Networks launch event in April, Patel stated that AI inference demand has grown so rapidly that the rental price of three-year-old H100 GPUs has risen from around $160-170 per hour to over $240 per hour in six months, with no spare capacity available. He also discussed the InferenceX project, which he described as a free and open-source benchmarking effort with over a thousand GPUs donated by companies including OpenAI, Microsoft, and Nvidia. In a March interview at the Daytona Compute Conference, Patel said that hyperscalers like Google, Amazon, and Microsoft were slow to move into AI, creating an opportunity for "NeoClouds" that could skip complex legacy software. He also noted that the entire cloud market had run out of CPUs, with Amazon's CPU server installations tripling year-over-year. In an April interview with Patrick O'Shaughnessy, Patel said his firm's AI token spend had skyrocketed from tens of thousands of dollars annually to $7 million, driven by non-technical staff using AI for coding. He stated that "ideas are cheap and plentiful but execution is very easy," and warned that people who do not use more tokens, generate value from them, and capture that value will "never escape the permanent underclass." Patel also predicted a "large scale protest against Anthropic and AI," citing a Pew survey that he said showed AI is less popular than politicians. In a panel at the Beyond Summit, Patel asserted that vendor benchmark claims are "lies, impossible to achieve," and that "if you're not pissing off people with your benchmark, then you're not testing something useful."