How big do you think this room is? Good morning. Welcome to CES. Welcome to NVIDIA Live. Setting the stage, we are live at the Fountain Blue theater here in Las Vegas. In just about an hour, Jensen will take the stage to share NVIDIA's vision of where AI is headed. And this panel has the daunting task of warming up this really smart audience before the most legendary keynote speaker takes off. What we will do is first start with introductions and then we will have a panel of industry experts to talk us through a wide range of topics within the AI ecosystem. So starting with myself, I'm Vivek Arya. I'm a semiconductor analyst at Bank of America Securities and I'm joined by
I'm Sarah Guo. I'm the founder of AI native venture firm Conviction.
And my name is Mark Lipacis, senior semiconductor analyst from Evercore ISI.
Great. Thank you Mark. So in this first session we will talk about AI infrastructure, where we are coming from, where we are headed, and we are joined by Sridhar Ramaswamy who is the CEO of Snowflake. Before I bring in Sridhar, let me just kind of walk you through a memory lane. AI is not the first large deployment in technology infrastructure. We have had the buildout of the internet, the rollout of fiber optic capacity throughout the globe. We have had the rise of cloud computing. We have had a big infrastructure cycle in 3G, in 4G, in 5G. Before I joined Merrill Lynch I used to work in industry and I have often lived through the boom and bust of some of those cycles. So that's why it's fun for us to contrast what we are seeing now versus what we have seen before. And you know, it's always risky to say it's different this time but it is different this time for the reason that in the last 3 years almost 800 plus billion dollars has been spent on AI infrastructure and if I look at our forecast close to $600 billion will be spent on AI infrastructure in 2026. So that's a lot of funding. The question of where is this money going, what is the return on investment, this bubble word comes up a lot. In fact I was reading Bloomberg yesterday and they mentioned that just in the month of November there were 12,000 articles with the word bubble in them. So in my view we actually have a bubble in the media headlines that talk about where AI has been going. From my perspective there are three things that make this infrastructure cycle different. Number one is the fact that there has been seamless adoption. Number two is high utilization and number three is the amount of funding and the amount of very well financed capital that is going into this infrastructure. So if you go through these three topics quickly, first of all, seamless adoption. The day ChatGPT was launched in November 22, every person with a PC or a mobile device and that's close to 5 billion plus users were able to access that service immediately. It was seamless unlike some of the prior cycles, the buildout of the original internet where yes, a lot of fiber optics was deployed but the end users were not ready. And that is partly why that buildout stopped in early stages. There was a lot of capacity deployed in the core but the usage only happened over a much longer period of time. So number one is seamless adoption. Number two is the fact that we are seeing very high utilization of this infrastructure. Back when I was working in industry we would talk about the amount of fiber that was being deployed and then there was a new interesting term that came up, dark fiber, which is kind of an oxymoron because the reason you put fiber is because you want to light it up. So there was all this dark and underused fiber that was in the ground but what you don't hear is dark compute. What you see is all the compute infrastructure that is being deployed is being utilized very effectively. Even six, seven year old GPUs are being fully utilized, other custom chips are being fully utilized. So that I think is another very important difference and to us if there is one metric that people should watch for is the utilization of this infrastructure. I think that's a far more meaningful metric to measure the success of this buildout. And then the third and final aspect is that unlike the prior cycles, there were a lot of companies that were let's say not as well funded. This time the funding is being done by companies who are generating exceptional amount of free cash flow. If you look at the amount of capex that has gone into the ground, that's only two-thirds of the cash flow of operations from these companies. We think their free cash flow continues to stay positive but that's another thing to watch out for as another leading indicator to see the success of this cycle. So we think this is the creation of brand new infrastructure. So that's the state of the union but let's talk about where we are going and for that I'm really delighted to bring in Sridhar Ramaswamy, CEO of Snowflake. So maybe let's talk to us about the collaboration you have with NVIDIA. How did that come about? What are you doing and what are the emerging use cases that you see in the enterprise?
Thank you. I live entirely in the software space. So I'm a natural complement to some of the folks that we have on the panel here. And back to your point about high utilization, I think what's very unique about this moment is the many different places where AI is creating value even with the old chips for example. We built a search engine this was three, four years ago entirely on ATMs which were already not even the top leading models in the world. And so we see a lot of our customers run very large data processing jobs for example on smart models running on previous generations of infrastructure. They get utilized. On the other hand at the very top foundation models that are leading ones are doing exceptionally well because of their applicability in areas like coding agents where they are unleashing an enormous amount of productivity. And now you're talking about people that are making easy $200,000, $300,000 every year. And so they're not going to hesitate about spending on a SaaS service that's going to make them more efficient. I think that's why AI is pretty unique in terms of how much value that it's creating. We have launched a bunch of data agents on top of Snowflake. I in fact I have one running right on my phone and anytime I want information about a customer I just turn to it, how are we doing with this particular customer, do we have a relationship, is it up, is it down. That kind of access is something that every CEO wants to get at the latest information about how their company is doing and I think these are the many reasons why you're seeing such excitement in the space. Our relationship with NVIDIA goes back many years. Jensen has actually been an honored guest in two I think of our last three Snowflake summits. We collaborate with them very deeply. Obviously all of our AI products including Snowflake Intelligence run on top of NVIDIA chips. We used to train pretty large foundation models once upon a time. We don't quite do that anymore. Those were done on NVIDIA chips as well. We also collaborate with them in the nitty-gritty detail. What are the right embedding models that are going to make search products work much more effectively? And I think the big prize that we are both after is in the future when GPUs can be applied not just to specialized inference on top of foundation models, but also to things like data computation, which is what we specialize in. So I think we're very much in the early innings of how are GPUs, how is accelerated computing going to impact a lot of things that Snowflake does.
No, I think I'm glad you brought those points up and I think that kind of ties to the intro that what we are seeing is kind of very strong seamless adoption. And this is being used in kind of mission critical infrastructure. This is not build it and they will come. But I'm sure there are certain obstacles, there are certain enterprises who are still kind of resistant to the adoption of AI. What do you hear from them Sridhar and what do you think it'll take to get them over the hump of adopting this new tool?
Look, practical problems are always going to be there. Anytime you have a new computing paradigm data is going to be sent over to a new class of chips. All reasonable customers, your employer included, have a lot of questions about hey what's happening with this data. Who owns this data for example. At Snowflake the simplest answer that I give every customer is your data is your data. We process it on your behalf. We do nothing else with it. There are questions like that. Part of what people are very finicky with is data sovereignty. They don't like it when data moves from one location to the other. In a consumer service you can, ChatGPT can get away with not really telling you where your request is going to be handled. It can be in the US, it can be in Europe, it can be in Japan. It doesn't matter to you as a consumer, but as an enterprise, that's not the viewpoint that you take. You care a lot about where's the data resident, how does sovereignty come in. We often end up dealing with things like micro shortages. Back to your point about full GPU utilization where we have customers in Germany that don't quite have capacity right in Germany and we have to go negotiate with them about is it okay to send that traffic to Sweden or not. So there are these kinds of practical problems that are going to come up and driving change through an organization. When it comes to day-to-day how are our jobs going to get done, can you convince people to do them differently? There's no easy answer to that. We have a great sales agent on top of our own data built on top of Snowflake. But it's an entirely different story for me to tell each and every one of my sellers, you need to be demoing this on your phone every single day. That's a new muscle. They have to learn it. And as leaders, we have to come up with a whole set of techniques for how do we drive usage in a positive way within every organization. That kind of social change is just hard. It takes time.
In this transition, we have the inbuilt distribution of everybody having a smartphone, a laptop. And so you have this massive prosumer adoption that's really happened. And I think that the enterprise businesses who know how to do change management with these large organizations are really going to benefit this next year from consumers and prosumers having experienced the power of AI for themselves. And so I think Snowflake and NVIDIA and companies that have those relationships with customers are really well suited for this next year.
That's right. And we very much think about things like agentic technologies as natural extensions of what we did as a data platform. Why do people bring data to Snowflake? It's because it's easier to manage, easier to get it AI ready, easier to get value from it. And an agentic platform fits right on top of that. I basically tell our CIO or CEO that we deal with, we can put that information in your fingertips. It's an easy sell.
Right. Got it. The one other thing I find very fascinating about this rollout is how it's suddenly becoming more global in nature also. It's like you follow where power is being deployed, that's where AI is being deployed and we are seeing infrastructure come up in the Middle East, we are seeing infrastructure come up in Asia. How is that changing Sridhar your kind of go to market, if there is more and more computing that is available right across the globe, how does that change your strategy?
So, Snowflake runs on top of the hyperscalers and we have benefited from effectively infinite compute capacity when it came to regular computing. That's what has defined Snowflake success because we often rent CPUs on demand. We don't have to make pre-commitments. And so that model actually has carried over on the GPU side. I don't worry for example as much about bubbles because we are not making large commitments. We are not buying hardware. We are not setting up data centers. Neither are our customers. You don't need to make pre-commits in order to use AI and Snowflake, in order to use Snowflake Intelligence. It's actually very beneficial for there to be large deployments everywhere because as I said data sovereignty is very much an issue in the geopolitical environment that we live in. A customer in Germany wants no part of that data going over to some other continent. So I think from that perspective this buildout is very helpful that it is more and more decentralized and demand for that is going to keep going up.
And how do you see this evolution between open and closed models taking shape? Does that influence what you do at Snowflake?
It does only in an incidental way at least right now. And the reason for that is the frontier models, the four odd companies that make these amazing models are so much better than everyone else when it comes to key applications like agent tool calling, like coding agents and so there is very much a demand for that. On the other hand, as things mature, people want to run them at massive scale, which is where open models become more and more important. The one thing that we all need to remember now as a country, as a hemisphere, is that open models end up influencing developer mindsets enormously. At the end of the day, it's only the few thousand engineers that work at OpenAI that have access to the latest and greatest that OpenAI has. And it's a small environment. What we have seen repeatedly is the power of open platforms to literally attract thousands, millions of developers to work on them. And that can be a positive cycle as well. Even in areas like data center development, which you would not think of as being very amenable to open technologies, Meta, Facebook, the company did pretty well in open sourcing some of their plans because they wanted an environment in which more and more people adopted that. So I think open source has a huge influence on just developer mindset. This is the reason why closed companies like OpenAI, they actually release open models because they want to be part of the developer ecosystem.
No, I think that's a very good point in terms of expanding adoption and that's one thing that I also find fascinating about this industry is that in some ways the growth and the disruption is being led by startups if you will, OpenAI or Anthropic or xAI, and they're challenging these really large incumbents which I would imagine is healthy in some ways.
Good thing for society. All these large public hyperscalers, they were startups themselves. They challenged the status quo when they were coming up. So I'm sure it's not an easy fight, many important decisions have to be made but it's very interesting to look at it from the outside.
Maybe Sridhar one last question from my side. Talk to us about where you are seeing adoption by way of end markets. Which end markets are more open to adopting AI, which verticals are kind of lagging behind in their adoption.
That's actually a fascinating topic because it's very broad-based. Absolutely folks in the financial sector especially people like asset managers they tend to lean into technology because they have less regulatory scrutiny say compared to a big bank which just has a lot more to answer to. So the financial sector always leads the way in terms of using, adopting AI technologies. I would say the sleeper hit in this world has actually been healthcare. You don't think of doctors as rushing over to use the latest tools but a lot of what they do is tedious work. Taking down notes after every single visit making sure that they are accurate. And all of us have jokes about doctors' inscrutable handwriting. And I think areas like that are also seeing a fair amount of adoption. It's really broad-based this time in terms of the value that people are seeing from AI and a lot of it similar to Google search has been because of the presence of something like ChatGPT that you and I and the 7 billion people in the world that have access to phones can all get to. I think that is really driving a broad-based adoption. And I think what's also really interesting is that unlike Google search which effectively put out enterprise search, there was no enterprise search once Google succeeded. What I think is unique about this moment is there are all of these interesting applications like coding agents that are thriving more in an enterprise context. And ChatGPT isn't really just sweeping the field. So I think that's the other interesting point in terms of the simultaneous growth of both consumer AI as well as enterprise AI.
No and I see from our enterprise also that there's a lot of so-called dark data that is sitting there, that if it is ingested properly and kind of curated properly the amount of insights that can be generated from that. So is the missing link Sridhar the kind of the productizing of that, yes we have the kind of the basic underlying infrastructure in place, is that how do we take this to actually making a product that is easy to just click and drag and drop and use. Is that the part that we should be looking forward to?
Well so that's an ad for Snowflake because that's what we do. We make data AI ready but much more we also help people put governance on the data. At the end of the day again for a company like Bank of America, unauthorized access to data is a very serious problem. Regulators take a very dim view of stuff like that. But part of what companies like Snowflake do is they bring along a governance framework. They bring along long expertise in making sure that the right data is accessed by the right person at the right time. But back to you, I've met lots of customers who will basically tell me stuff like, yeah, we have 20 years worth of contracts sitting in some SharePoint repository. And they generally email some person and say, hey, how about that contract from 8 years ago? They'll come back to me after a week. That's the value that's in front of a bunch of companies to create. And that's what's exciting about this moment.
Right, Sridhar. Thank you so much for speaking with us. It's a great segue into talking about the foundation and the data infrastructure to now the generation of agents and AI applications further up the stack.
Well, thank you folks for having me.
Thank you. I appreciate your insights.
Mark, what's your takeaway here?
Well, I think for me what's fascinating is the expression that you use, early innings. And you know, I think about other technology transitions. You think about the iPhone started off with six apps. I remember when the PC came out and replaced the mini computer, we were wondering what are we going to use these things for? Like are we going to put our recipes on there? Then you had word processing and then you had spreadsheets and Adobe and all these useful apps. And from my standpoint like how do you invest to support demand for workloads that don't even exist yet? And it goes to the point that Vivek started out with, the utilization of these processors that are getting deployed is as huge as the spending seems is 100%. We hear that and the forecast for capex just continues to go higher and higher.
Absolutely. If anybody has dark compute as Vivek mentioned maybe you can just share it with Harj and Shiv and me and our portfolio companies. We'll happily take it and use it. I'm super excited to talk to the founders of Abridge and CodeRabbit. Now, thank you guys for joining us. And to talk a little bit about two areas where we've just seen a massive surge in adoption of AI within verticals and horizontal functions, healthcare and coding. Healthcare, as Sridhar just mentioned, has been a bit of a surprise for those of us who have been looking at healthcare and investing in it for a long time. A bit of a sleeper category. Not the fastest adopter but has proven differently and coding I think has just been the breakout category for the last year. So Shiv let's start with you. Can you first just tell us a little bit about what Abridge does and sort of how the capability advances in reasoning and agents have been important to you guys for the last year.
Yeah absolutely. So we're an AI company in healthcare and we're focused on automating as much as we possibly can in such a way that we can unburden clinicians from clerical work so that they can focus on the patients in front of them. And just to sort of unpack that problem and that opportunity just a little bit more, there was an article published in a journal a couple years ago, the Journal of General Internal Medicine, that suggested that doctors need 30 hours a day to get all of their work done. And so that's how much opportunity there is for AI to actually seize this moment and for these agentic systems to actually help them focus less on clerical work and a lot more on the person in front of them. The split right now is 80/20. 80% clerical work and 20% face-to-face actual clinical reasoning and thinking and creative work. We think AI can flip the script.
I think all of us have had the experience with an amazing doctor who's spending 80% of his or her time typing in notes when I need them to reason about my case or my kid's case or whatever it is. And when you think about the capability advances this past year, what are the things you're most excited about that Abridge is working on?
Yeah, I think that the general playbook that we've gained a lot of conviction on is that we have to do everything we can to fit ourselves into workflows. It's not the other way around. It's not the workflows fitting into these new sort of AI systems. We have to design systems that fit into a pretty complex workflow when you think about enterprise healthcare. So it's not just checking off the boxes in terms of privacy and security. It's also thinking about latency. It's also thinking about all the artifacts that we have to create from medical encounters. And so the playbook that we're getting a lot of success from involves distillation and it involves fine-tuning. It involves post training. We're at scale now. We probably touch 80 to 100 million lives over this next year. And all the edits that we get on a daily basis, we can learn from so that we can be that much better for every single clinician in every single specialty in every single setting. So, that's what I'm most excited about is like now we're on to this new sort of inning where we're learning from these edits where we're continually getting better and we're also gaining conviction on certain workflows actually being able to go all the way to the end zone.
Harj, can you talk a little bit about what CodeRabbit focuses on? I think people know that coding agents and coding AI has made rapid progress over this past year but especially for people who are not developers who don't run development teams maybe how does this idea of complex workflows apply to you guys.
That's right. Coding as you all know has been one of the biggest use cases for generative AI in terms of token usage. It started with a lot of simple tab completion to now fully agentic systems. Now you can think of CodeRabbit as using generative AI to review all that code now that the volume of code has increased significantly. So we provide a critical trust layer that sits between your coding agents and your production allowing the organizations to enforce guardrails on what goes into production as you vibe code. We started around a couple of years back but this has been a pretty crazy growth trajectory for us, used by tens of thousands of organizations, hundreds of thousands of developers and so on. And we clearly see that in the new world of agent AI as the models are getting better like with Opus 4.5 or you've got Gemini 3 and GPT-5.2, the software agents are really getting great in terms of going from a high level prompt to some sort of feature or any application that you're building. And what happens in this new world is that two things become super critical. As the inner loop gets solved the outer loops are where the bottleneck emerges. One is how do you review all that work, that's CodeRabbit, and then the other problem is how do you specify your intent on what needs to be built, how do you describe your destination where you want to go.
Harj, when you talk about developing an AI for coding, what does it do for the average coder or the excellent coder? How do you think about the value add to the different strata of coders?
No, that's a great question. What we have seen is now with generative AI the barrier to get into software development has been lowered. You're just going from natural language English to any kind of language code that you want to generate. Earlier the bottleneck was hey I don't know TypeScript, I don't know Rust or any of these languages and that used to be the barrier. Now that's been significantly lowered. At the same time generative AI doesn't make a 10x developer become 100x. It takes all the great examples that it has been trained on in the open source and brings those examples to more of an average developer or someone who's just starting out and makes them good enough with code. So that bottleneck has now been removed. So a lot more people are now getting into coding. So a lot of people fear that maybe the coding jobs are going away but that's actually not what's happening. We are seeing a lot more people are now getting into coding. At the same time we still have to see whether someone who's been 10x can become 100x. I mean of course now they're getting more volume of code getting generated but then the review and other things are still a bottleneck.
Shiv, you work in an area where all the decisions and the data is really mission-critical. I'm thinking about this idea of needing review once you have more speed and other bottlenecks being removed in different fields. How do you think about building trust in the healthcare sector where users are rightfully pretty risk-averse?
Yeah, absolutely. Trust is table stakes and I think it's some combination of transparency, reliability and credibility. And so from the transparency standpoint I think being able to build trust in healthcare it's a pretty high bar and there's auditability that's involved. There's some amount of transparency involved in terms of how we benchmark our models and how we share those metrics with our health systems. But there's also a lot of work around evals and being able to actually deploy something and then in a pretty organic way share with the health system partner like this is what we're seeing actually. And we've created any number of LLM judges for example that will actually look at the outputs and judge it against their own, the health systems' metrics, their rubrics so that they can understand how well anything is performing. So I think it's not just about that, it's also about credibility. In healthcare it's about publishing, it's about actually partnering with folks and being arm's length distance from the researchers who are actually going to do some semblance of a maybe a randomized control trial to actually assess what is the actual impact here for us. I think if there is a mantra that we repeat in the company is that we want to save people time. We want to help people be more present with each other. We also want to save the system money. We know that healthcare needs deflationary economics and there's just so much opportunity for AI to assist, to augment and to automate and to actually bring people closer together and that's something that I think we need as a society more than anything else. But then there's like a third act. Saving lives. There's how do you actually help clinicians make better decisions like you were pointing to with your family members, how do you help them actually operate at the top of their license, at the top of their own creative stack. And so the stakes get higher and higher as we go farther on this journey because getting into that world, that space of clinical decision support involves a lot more hoops, there's a lot more red tape but it makes sense.
So one of the big buzzwords for this past year that there's a lot of controversy around is just agents and agentic capabilities, especially on more complex long-running tasks. Like both of you face. You're in the real world. You deal with, let's say, grouchy users. You deal with perhaps grouchy, but certainly very careful users. Is agent reliability a solved problem today?
Yeah, it does. I think it depends how we invoke agents versus how we invoke agentic systems. Probably depends on what kind of use case we're targeting. In healthcare, there are high-frequency low-stakes workflows where an agent can with the appropriate harness for example in the background with access to all the appropriate data with the right guardrails in place actually get jobs done and do it behind the scenes perhaps without a human in the loop. But then there are tasks, there are jobs that should involve a human especially in healthcare where maybe chemotherapy is involved or some new diagnostic or therapeutic or some really important high-stakes decision is involved. There should be a human involved. And so that's where a lot of our early sort of products have focused is those sort of experiences where we have to fit into a system where humans are there to check the output and be that last mile.
Yeah, I think in the coding side it's been challenging and constant evolution and people are still learning. Most of the use cases where we've seen strong product market fit for these agents has been very interactive use cases, human being in the loop and having a constant chat session with something like Claude Code or some of these agents that have now emerged. And the challenge there is as soon as you go into background agents where they can run for a longer horizon, the reliability is very very low. And you would see that in every organization, out of 10 developers maybe two or three are really crushing it with their Cursor usage or Claude Code usage while the rest of the engineers are still struggling with the right prompting techniques and it becomes really challenging because unlike your self-driving cars where you put in your destination it takes you there, in coding you're describing your destination and if your description is not accurate the agent may run for 30 minutes but you will end up with a lot of slop. And that's where the challenges have been around human in the loop. How frequently those agents should come back with counter questions. And another thing has been the tool calling formats. There's been a lot of discussion around how do you act in the real world, like sandboxes for instance, can we generate code to do everything. There's a vague philosophy right now that for every action there should be code generation behind it to make it more accurate. So those kind of things people are still figuring out.
One question Sarah if I may. I'm curious what is the biggest barrier to adoption in healthcare because that seems like such a natural adopter of this technology. Is it that the underlying technology infrastructure is acceptable, that's not the bottleneck? Is it the delivery of that? Is it that there are certain decision makers who have a natural, to your point whether it's a regulatory aspect, whether it's the privacy aspect. So what needs to be solved for there to be much broader and faster adoption?
Yeah, it's a great question. I think it's not technical. It's really change management and that's the biggest challenge. Healthcare is an industry that for a lot of good reasons has moved slow, but that's not this moment right now with AI. Healthcare is actually moving faster arguably than any other industry in taking in AI. And a majority of doctors in this country are using AI in some way, shape or form every single day that they're seeing patients. So I think the real challenge as we start to or continue to scale but also get into new use cases that AI can take on is going to be around change management. It's going to be around putting new solutions in front of users, getting them used to these very very fast feedback loops. The ground is shifting so quickly. Every couple weeks there's some big improvement. For us, for example, we do a lot more context engineering than we used to. We pull data from not just a medical record system, but maybe from a payer system, maybe from textbooks, and we're engineering that together with a conversation to create output that's compliant, output that's also clinically useful. And maybe last week's user didn't benefit from that. So, how do we get them to retry that solution next week and the week after? And so, I think that's probably the biggest challenge. At the same time, I think it is a pretty historic moment right now where it's moving very quickly in healthcare.
You know, one question I'd have for you guys running these scaling AI companies is a change in the environment that's happened over this past year that a lot of folks in our portfolio are taking advantage of is increasing availability of competitive open-source models. What does that, how does that factor into your product roadmap or how you guys think about cost and model vendors.
