AI Dev 26 x SF: Andrew Ng: The Future of Software Engineering

It's really nice to see all of you here. In addition to the AI prompting for everyone course announcement, there's one other significant announcement I want to make in this presentation today, and you'll be the first people in the world to hear about this. I'll do that toward the end of this presentation. What I want to do today is share with you what I'm seeing with the future of software engineering as it relates to AI coding and touch on Context Hub and a new thing that I announced a little bit to share with you more. I think how we can hopefully help developers navigate to this future.

A year ago at the first AI Dev, I presented a slide like this where I talked about thinking of software as assembling many building blocks to create these wonderful things. So it turns out that I often think of software tools, frameworks, APIs as akin to Lego bricks. And if all I have is one type of white colored Lego brick, you can build some stuff, but not that interesting. If you mix in some black Lego bricks, you can build more interesting things. Mix in some blue ones, mix in some red and yellow ones, and so on. And as the number of distinct building blocks you have grows, the way you can combine them to create interesting software grows combinatorially.

Over the last year, the rise of AI coding agents has made it far more effective for all of us to quickly put these building blocks together to create amazingly cool stuff. And this includes both AI building blocks like large language models, you can prompt, RAG, agentic workflows, and many more as well as non-AI building blocks like UI components, persistence of databases, identity or auth managers and so on. And it turns out because of AI coding, these building blocks are proliferating at the speed like we've never seen. There are more and more building blocks every single day, both AI and non-AI for all of us to use. And this is wonderful especially when AI coding lets us get AI agents to assemble these golden apps for us.

Now when I speak with different teams in startups, academia, also very large companies, one question that's often discussed and debated is how much should we use AI coding agents? And there's actually one popped up poll question which is I'm not going to ask this question but I see many people ask how much of your coding is AI versus human. And you know there'll be some teams that'll say oh yes my team does 80% of our coding via AI and you know that's pretty good. My own coding is not 80% AI, it's actually pretty much 100% AI and I want to share with you why I think this even last gap makes a big difference. Oh, by the way, in case there's anyone here, there's 0% AI is okay, too. But I encourage you to increase that percentage, right?

But what I see among teams is if 80% of your code is AI, then in terms of time spent, it actually looks like this, right? AI coding time, human to review that 20% of code just takes forever. Whereas a team that's 100% AI coding just gets a lot more done in a short time. So I find that this is true not just for writing code but also reviewing code and I know that a lot of teams also have AI write code but have a human review every single line of code but for myself and a lot of my teams we find that if I got to review the code then I become the bottleneck and it also doesn't work. And I'm not saying you know if you're working for NASA building and you need to write 50 lines of code for a spaceship go ahead write it by hand it's totally fine so this is not a religion where we should never be allowed to write code by hand. But what I'm seeing is many of the frontier teams are trending toward 100% or almost 100% of code written by AI. And this creates a real acceleration that getting only 80% of the way there feels very different from both for writing and reviewing code.

Now as coding is done by AI there are fascinating second order effects. I think last July in the Batch newsletter I wrote about the product management bottleneck and it was this observation that when I'm trying to build stuff you know we'll often have an idea, build a prototype that's software engineering, and then go to users, get feedback that's PM or product management work, use those ideas to come and refine the software. So AI coding has made that 10 or 100 times faster and so this means that deciding what to build or the product management work becomes the new bottleneck rather than the actual building. So what I start to see like starting a year, year and a half ago was this very strange trend where we used to have product manager to engineering ratios of you know 1 PM would keep eight engineers busy right these ratios take a grain of salt but 1:8 PM to engineer, 1:7 common in many companies but the ratios start to trend toward you know 1:2 was really weird and then one which is even more weird. And what I'm seeing is rather than one engineer and one PM the only thing that can move even faster is you take those two people and collapse them into a single human and I find that engineers that shape products or product managers that can code can move really fast and that speed is a huge advantage. But not everything can be done by a single person team and so I find that for more and more AI native teams the trend is that very small teams they're all kind of generalist where the engineers can do some PM work the PMs can do some engineering work and everyone kind of knows almost everything and a small team can move really fast.

I know that in some Silicon Valley circles it's trendy to talk about the job apocalypse. This idea that AI will take all jobs or almost all jobs and there'll be rioting in the streets. Frankly I'm not really seeing that and in fact with this profile of engineer the future AI engineer has a very bright future. When my teams are hiring for AI engineers, these are what we look for, right? First have to be able to use coding agents effectively, be it Claude Code, Gemini CLI, Open Codex, Open Code or something else. And I find that having a robust knowledge of the amazing building blocks out there, which is a challenge because there's so many of them, so and they change so quickly, that is a challenge. But knowing these building blocks let you assemble things very quickly. I'll come back to the building blocks a few times this presentation. And then lastly, I find that people with generalist skills, be it basic product management or other things as well, can build much faster. So there's a combination of skills on the ability to how to build things as well as the ability to know what to build. And I'm seeing massive unmet demand for a lot more engineers with anywhere near this mix of skill. Frankly, even my teams, we can't find enough of these people.

And this is why and maybe going beyond the product management bottleneck, I want to take that one step further which is when software engineering speeds up 10 or 100 times x everything else seems slow in comparison. So in addition to the product management bottleneck, I'm seeing that there's often a design bottleneck because you design something and she implements it or even better the designer just implements it in code without going through a design tool like you know Figma and that even speeds up. And one thing that used to be painful but it's even more painful now is the legal compliance bottleneck. Right? If you spend three months writing code and legal takes a month to sign off, it's okay. But when you take a day writing code, they got to wait a month for legal. It's like boy. And even marketing bottleneck for many of our teams. We write code and ship products so fast that marketing has a hard time standing to keep up with what these engineers are doing to figure out how to tell people about it. So I'm seeing design bottlenecks, legal compliance bottlenecks, marketing bottlenecks, sales bottlenecks, a whole whole bunch of other things. And this is why I find that increasingly small teams of people that includes engineers but also generalists can move really fast. And a piece of good news is it turns out that an engineer that's not a marketer but that knows how to prompt AI well. Maybe they're not an amazing marketer. AI actually makes really weird marketing decisions. But you know they could do some basic marketing work. And in fact, if a team needs software, product design, legal, and marketing, and it's a team of two, by definition, these two people have to have some skills in all of these functional areas. And I find that smart AI native teams, they can use AI to help with some of these other functions can move really fast. Not advising anyone to be a lawyer but frankly I often have AI you know deal with legal stuff as a first job and then take it to a real lawyer to sign off before I launch something but I find this AI native team moves really fast.

Something I don't have time to talk about is how to structure multiple AI native teams to work together because again not everything can be done by a 10 person team but if you have a 100 person team or many AI native teams with limited communication and clear API boundaries that also helps to scale beyond a single team. Well to come back to the work that needs to be done to create software writing software code is just a small piece of it that's accelerating rapidly but I just don't see the AI job apocalypse happening anytime soon. I find that the business media often has to be more accurate than the popular press because the popular press can tell exciting stories, but the business media is paid to get it right. And I've been encouraged that over the last month or two, a lot of the business media has been getting the story right that this AI job apocalypse now is missing. Texan recall job covers, jobless being delayed. A Federal Reserve Bank a small highlight sorry if we can't read it highlighted few see a need to reduce their workforce from study by the Federal Reserve Bank of Philadelphia. So actually and to be clear I know a lot of people are feeling job insecurity at all levels in seniority I think we need to do something to address that and I know many people have been laid off because of overhiring from the pandemic or hiring from zero interest rates period. So, I'm not saying the job situation is perfect. There's a lot of it that's not that pretty, but the job apocalypse is possible at the top in some circles. I just don't see it happening. Which is why I'm eager to keep on investing in all of us becoming better at building these.

Now, one of the hardest things to do, exciting, most important hardest is as we're becoming better and better builders, each of us, how to master these building blocks. I want to share with you a framework of parallel skill development. Here's what I mean. We talk a lot about how our coding agents are becoming more capable and this includes agent skills like the Anthropic agent skill center work but more generally our coding agents are just getting better and better. And then there's the education or training world that I've been a part of for a long time that focuses on building human skills. But looking into the future, I think we increasingly need parallel skill development. Where as our coding agents become more capable, our people, we as people need the complementary skills to help us to drive the coding agents in the appropriate way so that we can partner together to build new things. So I want to briefly touch on two things in the rest of this presentation. One is Context Hub which I'll talk about before I spend a couple minutes on. And the second which is the announcement I made toward the end is Code Dream.

So Context Hub. It turns out building blocks are proliferating so fast that most of our coding agents which have a knowledge cut off date you know far in the distant past sometimes a year sometimes a month old will often hallucinate or use deprecated APIs or not know about the latest tools we want to use. So it turns out even today if you ask Claude Code to call the OpenAI API it will use the chat completions API which is older deprecated and call older models because most of the training data on the internet uses this older API even though the newest API has been out for a long time. Context Hub is a tool that some colleagues, Rohit Prasad, senior fellow and I built in order to give your AI agent most up-to-date documentation. And it turns out that if you run Context Hub with Claude Code, it will generate code with the newer responses API. Maybe just to show you really briefly, I don't know, chub search openai. So this returns so these are packages and then chubget get this actually open chat pipe this more and so this will fetch the latest documentation on OpenAI that knows about GP 5.5 has the knows about the responses API and so on and this is a tool that you are welcome to use but is actually built for your coding agent to use. And so this generates like okay a 600 line markdown file right kind of long for a human to read or you're welcome to read it to give your LLM the context it needs to make this and other API calls correctly. So I pray many of you too, right? I've had my coding agents hallucinate API calls to definitely, you know, OpenAI, Gemini calls make bad calls to various database services. There's just so many tools that coding agents don't know about. But feeding on this latest context, I think can be very helpful.

But beyond that, I want to share with you a project that we've been working on to help with human skill development. For a long time, you know, my team's been working on a variety of online courses and please keep on taking online courses. I think an important part of how we can all stay up to date with the building blocks and other tools and we've been exploring one other alternative experience that I'm announcing here for the first time today and that's a project we've been working on called Code Dream. The Wi-Fi here is not great, so I'm going to see if a live demo works. It well may not but let's see if this okay welcome to code. This is not a course. It's a conversation where I want to show you how to use a coding agent together with tool called Context Hub to write code using up to date API documentation. I say this is a conversation because I'd like you to imagine we're on a video and you can interrupt me at any time using that little microphone button down below. The Code Dream team has created a voice agent that has information needs to try to answer your questions as a woman. So feel free to stop me if there's anything you like to chat about or if you like you can just lean back and watch the presentation.

Here's the idea. All right. What are we going to learn about in this conversation? We're going to learn how to use Context Hub to keep coding agents up to date with the latest information. Then we'll build a joke generator app together. So this is not a slide, right? I just click into this and type. Let's tell a dad joke about bad Wi-Fi. Why did the computer show up late? Where it was bad connection. Okay, not bad. Could be worse. And the other thing about this being JavaScript is you can also copy paste from these slides. So for example, this is a slide with instructions on how to install Context Hub and how to prompt Open Code and so on. So actually copy I'm hitting command C and then go into the Code Dream build environment. And this is a terminal in my browser where I can you know paste that command. I just copy paste it from my slide to run npm install chub and then I can do things like start up Open Code which is open source version of a coding agent kind of like you know alternative to Claude Code, Gemini, Open Codex and then I don't know writes python whatever right and so this is a learning practice environment that lets you practice using modern coding agents with tools like Context Hub.

And so we have been working on this Code Dream build environment for quite some time and I'm excited about exploring this concept where in view of taking a course you can come on to a video conversation and have a conversation with me or an AI version of me to try to gain new building blocks and code in this more modern AI coding way. And I am happy to announce that Code Dream is available as of today in preview. So please check it out. And the first experience will show you how to use Context Hub and give us feedback. We're eager to keep on working to make this better.

So just to wrap up, coding agents are allowing us to assemble building blocks quickly to create software and that acceleration is real, but many of the downstream implications about where the new bottlenecks are are still being worked out. And I'm eager to help our entire community keep moving forward because the job openings are there. The job populace is not going to be nearly as bad as people are hyping it up to be. And I think it's important that we keep on learning these building blocks and learning these skills. And for parallel skill development, coding agents are working well, but give them access to the latest building blocks. Given the right context, Context Hub will help. And for human skill development, I'm excited also to share with you the Code Dream conversations. And all of you in this room are the very first people in the world that I've shared this with outside our team. And I hope you will check this out and that'll help you to master some new building blocks and master some new skills. So, with that, really excited to be here. Thank you for all coming and thank you all very much.