Welcome everyone. It's a great pleasure to be here again. And it's really a great pleasure to introduce Andrew, someone I had the privilege of working with while at Stanford. Andrew is the founder of Deep Learning AI. He's a managing general partner of AI Fund. He's an executive chairman of Landing AI. But before that, he built from the ground up Google Brain, he led a 1,300-person AI team at Baidu, and among countless awards and recognitions is also a Time 100 most influential person in AI. But Andrew is not just a pioneer in machine learning, he's also transformed how millions learn about AI. Think about it, 8 million people have taken your classes through Coursera, which you co-founded along with Daphne Koller in 2012.
Um, thank you. It's very exciting. Over the years, it has been amazing, exciting to see Andrew's dedication into making AI accessible and helping accelerate the adoption of responsible AI worldwide. Please welcome a dear colleague, Andrew.
Thank you.
So Andrew, let me go back a few years in the past. We were both full-time professors at Stanford. This was during the last AI winter, and over the years, AI has evolved from a niche topic into mainstream. From your own perspective, how much has the role of academic research and corporate research evolved since then?
So I find that there's a lot of buzz and hype about corporate research, and that's fine. What's underappreciated is that academic research at the great institutions like Stanford and Berkeley and MIT and many others, and CMU and many others, is also extremely important. I think the corporates have larger marketing arms and so their research tends to be amplified, but there is lots of great research done at the leading academic institutions as well. As you just saw at Stanford this morning, one of the really nice things about academia is as a neutral party, academics get to talk to all of the leading corporations. And so there's a lot of knowledge that is still centralized in academic research. And we now live in a world where there's so many opportunities. Corporates are doing great stuff, big companies doing great stuff, startups doing great stuff. It's just a world of more opportunities than, you know, people with the time and bandwidth and skill to pursue.
Great. Yeah, it's really amazing how the whole landscape has changed over the past years. And even in my own research organizations, we've been looking very deep in understanding how we should reposition ourselves with respect to this new environment that is emerging, both from academia and from the business, the enterprise. So you've been vocal in the past about how AI is overhyped, and in particular AGI. Why do you think it's overhyped, and what do you think we should do to provide a more balanced view of what is hyped and what is reality?
So with the rise of GenAI, maybe you know, two, three years ago, because the technology was so new, there was a handful of businesses that almost got away with saying pretty much anything. And at that time, the broader public media, traditional media, social media were not good at fact-checking some of these companies. And so their PR efforts for fundraising, publicity, whatever, wound up distorting the public view of an accurate technical understanding of AI. One thing to mention is OpenAI's AGI. So for various incentives, I think, different, so the original definition of AI, they know of, is AI that could do any intellectual task a human can. I still think we're very far from that, right? Maybe decades, I don't know, quite far from that precise definition. But in the last couple years, there have been so many alternative definitions of AGI that it is now a term that many different people mean many different things by AGI. And when that happens, the term kind of loses meaning because you can't use it and have a common reference what this term actually means. So I think, as Silvio knows, as many of you know, to the developers writing software, we just don't use that term in our daily lives because they're busy building software, which is exciting. I find that the use of AGI, it is used in maybe the non-technical, you know, mass media. It is used by companies that do know what they're doing to fundraise, right, to generate buzz and excitement. But outside of that, I think when the term's lost its meaning, it's hard to use it in a productive way. So I find that most of us, we just, you know, don't focus on that and just focus on the very real, very tangible, very exciting work of building AI agents or building other things that matter.
So do you think that the type of benchmarks that are used for evaluating AI or AGI, it should be redefined, should be restructured? Maybe we're using the wrong benchmarks.
Yeah, I actually don't know of any good benchmarks for AGI. You can take a benchmark and slap an AGI-ish logo on it, but that doesn't really make it, you know, a good benchmark for AGI, especially when different people mean different things by it. I think it's been interesting seeing where the buckets of value in AI emerge, and this drives the benchmarks as well. Right. So for example, what I see today, well, in an earlier era, Google dominated horizontal information search, then there was lots of room for vertical players for, you know, social media, and then like travel, Wikipedia, for retail, and lots of other verticals. And what we see now is buckets of value for AI. I think ChatGPT has, you know, runaway momentum for the consumer journey from research, or I think Gemini, great channel, Anthropic, you know, still some competition. And then the verticals are still emerging, be it coding where Claude Code works great, but I think OpenAI Codex made a lot of progress just in the last month or so as well, Gemini worth keeping an eye on. And I think where the other verticals are creating a lot of value is still emerging. I do think there'll be more verticals this time round than in the internet era because AI can do more, the set of things we would tackle, the verticals you can make a big dent in is bigger. But with that horizontal information discovery and lots of verticals to be built out, that landscape is emerging. And I find that as the horizontal and the verticals come into focus, that in turn drives the benchmarks. So what's the best coding benchmark, what's the best information discovery benchmark, what's the best, you know, medical benchmark. But all of this, you notice I'm talking about a lot of applications which are wildly exciting. Nowhere did I use the word AGI, and I'm not even sure what the AGI benchmark would be at this point.
It's actually very interesting what you're saying about that, how those benchmarks and those evaluations should be really aligned with domains and use cases. So at Salesforce, we introduced this term EGI, which stands for Enterprise General Intelligence, which is really targeting an AI that is focused on enterprise use cases, which are much easier to define and to use for evaluating how effective, how accurate, and how consistent are those tools for enterprise, for business. So Andrew, we know that you actually helped coin the concept of the term agentic AI. This came from, we were discussing this a bit earlier, came from situations where there was a bit of a confusion in terms of what is the definition of an agent. But now we are seeing that this wave of agentic systems that have been really successfully used for planning, for actions, and are getting integrated into the fabrics of the enterprises. So what excites you the most about how organizations and teams are using those agentic systems to accelerate innovation?
Yeah. So when I coined the term agentic AI, like almost, I didn't think we'd be sitting in the agentic theater with all the lasers. So I said, so a couple years ago, almost two years ago, I saw there to be this exciting agent movement, but I saw a lot of researchers and developers arguing, is this an agent or not? And there was my definition of agent, your definition of agent. Someone would write a paper saying this is an agent, but say, no, that's not really an agent. And I felt that debate, which was consuming, you know, bandwidth, was not productive. And so I started to put forward, hey, let's not argue what's an agent or not. I think there are degrees of agenticness, where there are highly autonomous agents that can plan, take multiple sets of actions, and there are, you know, less autonomous agents that only carry a couple steps of usually large language model-based inference and action taking. But if we take an inclusive, embracing view, let's just say that is a, was it adverb, adjective, sorry, and say that different degrees of agenticness, then let's just call all this agentic and get on with the work of building it rather than arguing what it is and isn't. So I started to use that term, encourage others to use it. What I did not expect was that around last summer, a bunch of marketers would get a hold of this term and use it as a sticker and slap it on everything in sight. The interesting thing is even though the buzz and hype about agentic AI has taken off like that, the amount of actual productive use cases has also taken off. Maybe not as fast as the hype, but also incredibly quickly. And so I find that the technology for building agentic AI is still evolving. And just last week, DeepLearning.AI announced a new course on agentic AI where I tried to teach the best practices as I understand it on building these systems. But I think a lot of the work that lies ahead of us is, there are so many business workflows or individual workflows or personal workflows that it is now feasible to implement through these agentic AI techniques. And so I think, I know some businesses are saying, oh, wait a few years, we have AGI, then all problems will be solved. I'm actually not a huge fan of that view. Instead, I think a lot of the important work that lies ahead of us that we should start doing now is to take these really valuable business process workflows and figure out piece by piece how to implement it into agentic AI workflows, and that will be very valuable.
So in fact, actually, I remember reading some of your articles on the emergence and importance of creating agent orchestrators that can actually manage and handle this kind of large set of specialized agents. This is actually something that resonates very well also with my view of how we should be organizing, we should build this workforce of agents, and also kind of aligns with the new narrative around Agentforce that was just introduced this year at Dreamforce. So how is this evolving, this kind of hierarchy of agents? Do you see that building different layers, or how do you see that happening?
Yeah. So the rise of agentic orchestration layers, including, you know, Salesforce has its own tools, the whole world of agents, there are also tools like LangGraph, CrewAI, Disclosure made small investments in these last two. I think the good news is it's becoming easier than ever to build, you know, complex agentic workflows. But it turns out that going through the intellectual exercise of what to build, how to implement it, how to do the evals and the error analysis to efficiently drive its performance, all that is still hard. And maybe one of the lessons I've learned working with my friend Kirsty Tan and Landing AI, as we work with a lot of large businesses, one of the biggest challenges is figuring out the data layer. It turns out that until recently, a lot of the value of a lot of the data engine was on structured data, right, the large tables of numbers, because that's all our computers could process with machine learning or traditional analytics or BI or whatever. But with GenAI, we now have much better tools to process unstructured data, so text, images, audio, video, and PDF files. And so what I'm seeing a lot of businesses do is rethink their data architecture to manage not just the structured data but also the unstructured data. And additionally, there's a lot more value now because AI can handle more types of data, there's a lot more value to building the orchestrators on top of the different, say, data lakes or different data silos to bring together these different data sources in order to connect the dots. So for example, if you have your sales data, can you correlate that with something that went viral on social media, or with the weather that for certain industries drives sales. So if you have a data architecture that takes structured data, unstructured data, and have a layer that lets you connect the dots to drive analytics, that's very valuable. And so what I'm seeing, and I'm actually talking to a lot of large businesses about, is how do we rearchitect this data layer to say build on top of Salesforce, build on top of, you know, your other existing data sources to make the business AI-ready, to get the data to AI or to humans for that valuable processing. Actually very large projects I think many businesses will end up doing.
