Ion Stoica on AI infrastructure stack
From Distributed systems to AI platforms with Mark Russinovich & Ion Stoica | BRK227 · · Microsoft Developer
“I think that what I is there is some stock which is used by companies. There is, you know, it's like, and you know, you see, you know, you have at the lower level Kubernetes or Slurm under the hood. If you want to move because everyone now has to want to have a freedom to be make move across regions or from region to region. Maybe you have something like Sky Pilot. On top of that, you have to have some framework like like Ray or Pathways or something like that, which helps you to distribute this kind of AI workloads. Then obviously you have Pythorch and so forth. So I think you can put together this kind of stack and then you have VLLM or HLANG or whatever. And so our DLLM is our inference layer. And then post training you have kind of veril sky rail and things like that, right? So you can create a stack and you have a stack, I think a pretty good stack.”
On , Ion Stoica, Cofounder at Databricks, spoke about AI infrastructure stack during Distributed systems to AI platforms with Mark Russinovich & Ion Stoica | BRK227 on Microsoft Developer.
About Ion Stoica
Ion Stoica, cofounder of Databricks and executive chair of Anyscale, spoke at a June 2026 conference about reliability as a major barrier to enterprise AI adoption. He argued that AI systems, particularly those using large language models, lack clear specifications and are difficult to debug because they function as black boxes. Stoica noted that moving an AI feature from prototype to production requires 10 to 50 times more resources than prototyping, and he cited a paper from Stanford and comments by Dario Amodei to support his view that ensuring AI agents are safe, reliable, and predictable is a central challenge. He also discussed LM Arena, a platform he helped create that has hosted over 250 million conversations and evaluates more than 700 models, and presented research showing that excessive use of emojis in AI responses is negatively correlated with user preference. In a May 2026 conversation with UC Berkeley professor Shankar Sastry, Stoica reflected on the founding of Databricks and the current state of AI. He described the proprietary model landscape as inefficient in its use of human capital, with engineers having little incentive to share knowledge. Stoica also commented on humanoid robots, saying the premise is "huge" but that it will take time for the technology to become profitable. He called for universities to adapt to the changing AI landscape and argued that federal taxpayer money should serve as "patient capital" for long-term research, rather than forcing the capitalist system to be more patient.