What's happening right now is a big deal. This is the biggest launch since ChatGPT. It's called Muse. Has the tech world buzzing. Five million downloads in 22 days. Surpassing ChatGPT. And he built it.
Most people are using AI to answer questions, basically glorified Google search. This is something totally different.
It's an AI agent, built for everyday life. We believe that Muse will make you money. One guy had Muse help plan a weekend away with his wife. It can make phone calls. It can do tasks for you. But for it to work, you have to trust it — first with your personal information, then to do what you actually want it to do, and then that it's not gonna, well, get out of control, all at a time when the most important AI leaders are warning us that AI is getting out of control. And I urge you to urge me to stop.
If you understand what's happening right now with Muse, you'll understand the biggest things happening in AI. So what can it actually do? When should we trust it? And what do we need to do to make sure this goes right?
And we're at the moment of, like, peak insanity. Why are all these billionaires obsessed with restaurant reservations? We have a shared interest in making sure this technology does not go horribly wrong. Is that something that's happening in some private room somewhere? The degree to which certain personal animosities are actually defining how they're developing this really important technology is insane to me.
Amazing. Let's do this. Okay. Welcome. How have you been?
I'm good. I'm really good. How have you been?
It's been kind of insane, yeah. What does it feel like?
We didn't expect that it would be such a hit, so I feel amazing for the whole team and everyone who worked so hard on it, 'cause it's really come together and become a thing. So that's what I'm excited to talk about.
I know you know all about the show already. But this isn't a traditional interview. Like, I'm not gonna ask about financials. I'm not gonna ask about management style. You just launched one of the most popular consumer tools of all time, and so the only thing that I wanna know is what's the future that you're working toward with that tool, and how can this audience that's really optimistic and really wants to be part of making sure that that future go right — how can they feel like they're part of the decisions that are being made?
Yeah. That's what we do. One of the reasons I wanted to talk to you was I've always appreciated that you're really blunt. Like, I honestly think that people need straight talk about this. I think that they feel confused, and I wanna get at what's really happening, what have you built, how are people using it, how are they not, what are the real challenges. I think one other interesting thing is, given how popular Muse has become, I want to use it as a way to talk about other things in AI.
Awesome. Okay. Let's get into it. Muse is the fastest-growing consumer tool since ChatGPT. Faster than ChatGPT, I think, and it feels like this is the moment when we're moving maybe from chatbots to agents. But there have been a lot of other popular agents, mostly in code, and I'm curious, why do you think this is happening for Muse right now? Why is Muse exploding like this?
I think one of the mysteries of AI so far this year, for most of the year, has been that agents were happening for engineers and software engineers and coders, and then nobody else in the world was experiencing it, knew what it was, understood exactly what was happening. And so for most of this year, you've kind of had this, like, two dialogues almost, where people who do code and could see how powerful AI was becoming and how useful it was becoming were like, 'Oh my gosh, this is incredible.' And then everyone else was kind of like, 'What are you talking about? I just still use a chatbot. Like, it's a little bit better than Google, but what are you guys all freaked out about?' And one of the things that we were really excited to do is, how do we take the magic that engineers and coders and developers are seeing and bring that to everyone?
One of the things that has happened over the course of the past year is, like, the AI models started out as pretty janky agents, and you kind of had to struggle through them to get them to work, and then they slowly got better and better and better. And most of the industry's been really focused on making them just better and better at coding. But we really felt this was a big gap, because most people in the world, they don't know how to code. They probably never even want to code. But they still wanna feel the experience of agents and having AI that is able to do more for you and actually start taking on more end-to-end workflows and take on more tasks for you. So that was really the goal behind Muse, and I think we spent a lot of time polishing all the little details to make sure that it would just work for people, because we knew that, like, developers can kind of withstand a lot of jank and a lot of stuff breaking, and they can kind of deal with all of that, because, you know, they're used to dealing with, you know, unripe technology. But for everyone else, you have to make something that's really smooth and really works for them.
That also helps explain the big gap, I think, between what it feels like for most people to actually use AI, when they were using a chatbot in particular, and the conversation around AI that it's gonna take everyone's jobs, it's really scary. It's, you know, there's a macro conversation happening, and then there's an experience that people are having. And it just seems like those things are really, really far apart. And to your point, Muse is giving more people the opportunity to have the insight that computer engineers had, you know, call it a year ago. Is that fair?
I think that's true, yeah.
There are also moments in history that I think feel similar. Would you say it's like the mobile phone moment for agents? There are, you know, billions of people brought online by mobile phones that never had PCs because it was easy and simple to use, and they didn't, you know.
I actually hadn't thought about that until you just mentioned it, but I think it's a great analogy, because these models, they're improving really fast, and they're improving really, really quickly. And it's really important that we, as those models get better, that the superpowers that they lend can be given to literally everyone all around the world. The part that I like about the analogy is so much of it is just about figuring out how to package it all together into a form factor that will resonate with people, and that's what we spent a lot of time trying to craft and get right and make perfect.
Which also gets to one of the interesting initial reactions that I've heard of Muse, which is, is everybody's life more complicated than mine? Like, I wake up, I go to work, I play with my kids, I have one restaurant reservation a month, I plan one vacation with my spouse. Like, why are all these billionaires obsessed with restaurant reservations? Why do I need an assistant? How would you answer that?
Whenever I explain Muse to someone who knows nothing about it, I just say, like, Muse can help save you money and save you time, and everybody in the world wants to save money and wants to save time. Depending on what your lifestyle is, that can mean a different thing to you. I think almost everyone has, like, subscriptions that they shouldn't be paying for or subscriptions that are really expensive, like cable bills or insurance, and for a lot of people their magic moment with Muse is just saving money on things that they shouldn't be spending money on or they shouldn't be spending so much money on, or finding gift cards that they hadn't spent, or, you know, the list goes on and on and on. I think it's awesome that so many people are experiencing that.
I think that's great because these are things that, I think for most people, they're just so boring that you would never get to actually handling them or dealing with them, but, you know, your agent, Muse, can just do that for you. And then there's saving time, which I think for lots of people there are always experiences that are just way more frustrating than they should be — like going to the DMV is one example. We actually, when we built Muse, we specifically tested it to see if it could deal with the DMV website, because that was, like, the pinnacle we thought of the most frustrating experience for any human, is dealing with the DMV. So we needed to have Muse be able to do well in that situation.
We've seen people use it for, like, navigating the healthcare system is a big one. Medical claims is a huge one that most people just — it's so complicated and annoying that it's just incredibly frustrating. We've seen people use it to help run their small business. We've seen a lot of people use it to help run their social media even, and I think the medical system is a really good example 'cause it's so hard to understand. And I think, you know, one of the best ways to put it is, I think the cases where it's a huge lifesaver are the cases where otherwise you would be at your computer, you'd have to Google like 50 different things, you'd have to read and go to a hundred different websites and try to figure out what the hell is going on, and it's just this incredibly frustrating experience — or you could have your Muse do it.
I also think about how many things I suspect are made difficult and time-consuming on purpose in order to frustrate me so much that I don't get to the end? How many claims are actually designed to be that hard so that I give up? And I wonder what effect more people having more patience because they have support will have on that kind of systems engineering.
One of the little memes that I made of the little Muse was Muse in Hood, which is like a Muse Robin Hood. But I think there's a real truth to this, which is there are lots of things that exist in the world that are, to your point, meant to be hard — canceling subscriptions is one of the best examples, where there's some subscriptions where you have to click on one thing and click on another thing, and then click on the little piece of text, and then not click the big red button, but click the other button. And that's just the perfect example where there's lots of cases where things are designed to be hard for people, and I think this is giving people the tools to be able to just do those things seamlessly and easily.
I got access to the calling, and I had it make a call this morning, which it did successfully. What does it sound like on the other end?
I think it's clear to the other end that it is an AI, and I think if you look, read the — we show users the transcript. Like, if you see the transcript, the beginning of the call I always think is really funny, because it's like, 'Hi, I'm so-and-so. I'm Cleo's assistant, and this call is being recorded.' And I think it's just always this very funny beginning of a call: 'Hi, I'm Haley, calling on Cleo Abrams' behalf. I'm transcribing our call for notes.'
What are some examples of when it's messed up for you, or categories you feel like it still struggles with?
One thing that we're always trying to improve is the memory of these agents, right? And this is still one of the more unsolved problems, I think, for a lot of these agents — the memory of the agents is not like that of a human. Sometimes it can feel like it's like that, but it is really different. So that's one area where we're always trying to improve. We always wanna figure out how to make the agents better. And then we want them to be able to continue doing more and more complex tasks.
I find this really funny, but there's a lot of people who will just ask their Muse, 'I want you to make me $10,000.' And obviously not everyone can just ask their Muse to make them $10,000, but I think these kinds of more amorphous, nebulous tasks — we want Muse to be able to at least build a plan and figure out, and, you know, you're gonna have to do your part if you wanna make $10,000. But I think we wanna enable the agent to do more and more complex and esoteric tasks to help the users.
There are some things where I feel emotionally, even if Muse could do them, I would prefer that they didn't. So for example, if I'm texting a friend and I find out that their Muse agent is responding to me, I would be offended by that. Are there categories of things that you think explicitly Muse should not do that are just extremely deprioritized — not because they're technically that hard, but because socially, not the point?
A lot of what we care about is having Muse kind of clear the way such that you can spend more time on the things that you want to do, and the things that you enjoy doing, and the things that are valuable to you. We generally speaking want to figure out how to make someone's Muse a different entity than them. That's why it's like a little cute guy. And we want to, over time, continue making that more and more clear because, to your point, it is socially weird if you think you're talking to your friend and it just happens to be the little Muse guy behind the curtain.
Speaking of what they look like — I've visited humanoid robots before, and one of the things they spend a lot of time thinking about is how to make it cute so that you like them and you want them in your home, or at least you're not afraid of them. What did Muse look like in previous iterations, and how did you decide on that particular guy?
I think this is one of the most shocking things. What Muse looks like now is what he looked like in the very first prototype. It's kind of one of these rare things where the very first version was just perfect, and then it just grew on all of us, and we ended up going with it. We had lots of debates about it, which in retrospect will be some of my funniest meetings — like, a bunch of serious adults getting in a room arguing about this cute little character. But yeah, I mean, we were like, is it not serious enough? Is it too playful? Does he need to be more colorful? These were all things that we asked the questions about, and we looked at them. We were like, 'No, he's perfect.'
What is the big vision here? I sometimes struggle when talking to AI leaders to push people to think about more than, like, five years out. But what is the vision here? Is it that Muse becomes my layer on everything on the internet? Like, map this out for me, if it goes the way that you hope it does.
What we really want Muse to be over time is kind of this general manager, so to speak, that helps you accomplish all the things you want out of life. So, like, I think one of the tragedies of the modern world is that most people have wants and dreams that can range from really big ideas — like, hey, I want to start a company, or I want to cure disease, or I wanna save the planet — all the way to small wants that are like, hey, I wanna spend more time with my family and I want to eat healthier. And there's a whole range, but for most people, I think there's just so many barriers and burdens in the way.
What I really hope it accomplishes is helping every single one of the eight billion people all around the world be able to accomplish their wants and dreams. And I think this is, like, if we are able to accomplish that, the world will seem insane. That's what we really want Muse to be, and what we think it can be if we keep working at it.
You know, it's really nice to hear you talk about giving people time back, because for a really long time I have felt, and I think a lot of people have felt, like so much of the financial incentive of most tech products that I use, especially the ones that are ad-backed, is to take my time, is to distract me from the things that I want. Muse is free right now. It probably will have ads in the future. Is there a tension between giving people their time back and having an ads-based business? Or are there other incentives about, like, Muse steering me down a path that isn't quite what I said I wanted because it's ad-backed? How do you add ads to a product like this?
First off, I think right now it's free. There are subscription tiers. I think we think that there's plenty of business models that are really aligned with the user — for example, if they're buying things using Muse, or Muse is saving them money, or Muse is making them money, figuring out a way to take some small fee on those kinds of transactions. So I wouldn't say we're spending that much time thinking about ads necessarily right now.
That being said, I don't think ad-based models are necessarily synonymous with you needing to take all of your time. I think Google Ads are a really good example of this, where when I go to Google, my dream is to get in and get out, and I don't think Google's really trying to stop me from getting in and getting out. But they've built something obviously pretty incredible, and so yeah, I don't think it's a necessity by any means. I think, again, if we actually accomplish this thing that I'm saying — which is that people are using Muse to really do the things that they want to do and, like, really, in a non-corny way, accomplish their dreams — I think there's gonna be a lot of ways to make that sustainable.
So when I think about mapping out a future to get there, to this optimistic vision, one of the most important things is what do we need to get right, what challenges do we need to avoid in order to make that true. I want to hear how you use Muse and how you've dealt with these challenges at Muse as an example of larger questions in AI. I think everything that I'm about to say applies to basically every AI product, and I would be curious if you disagree with that. I think there are basically three levels here. Level one is privacy. So you say, 'I download Muse.' And then I get to the point where it asks to be connected to my Gmail, and I say, 'Whoa, whoa, whoa, I'm not sure that I wanna do that. Like, nope, nope, I'm out.' That, at the lowest level, is a concern about the relationship between the user and Muse itself, and therefore Meta. So what is Meta going to do with my information? Could hackers get that information? And in some countries, can governments get that information somehow? I know that you've spent a lot of time thinking about this. I'd like to get as detailed as we possibly can here for a lay audience. What is the sentinel agent? What is Muse Confidential and Secure VM? How do you think about this in terms of what can Muse see and not see, and things people should know about this first level of privacy?
Muse is built on top of the secure VM, the Muse Secure VM. And that is a completely sandboxed virtual machine — or think about it as a mini sandbox computer that is sort of isolated. The information that you share with Muse is stored in that little secure computer. It is not stored elsewhere. And that little secure computer is sort of like your home, your Muse's home, so to speak. Inside that secure VM, we have a very, pretty sophisticated security architecture, because, you know, we knew that privacy and security, frankly speaking, were gonna be some of the most important things for most consumers to get them to adopt it.
So as you mentioned, we built a sentinel, this sort of architecture with a sentinel agent. So the way it works is that the model or the AI agent on the inside, as it's going about and doing its work, whenever it wants to communicate outwards — so it wants to take any information and send it out, either to fill it out on a website, or send a message, or make a phone call, or any case where it would take information from outside of that virtual machine — there's a separate agent, the sentinel, that's watching and making sure. It's kind of like almost like a traffic cop or something. It's just making sure that there's nothing — it's saying, 'Oh, whoa, whoa, whoa, that information, like, that's your credit card number or Social Security number,' or whatever it might be. Like, do not share that. That should stay inside.
And then the last thing that we do is a lot of human-in-the-loop verifications. So whenever Muse wants to connect to a new website or send information to a new website, there will be a little dialogue that pops up, which asks the user, like, 'Hey, are you okay with Muse contacting this website or sending information to this website,' and what not. And we — and some people have said we're really annoying with it, that we are constantly asking people to approve where their Muse is sending information. But we just wanna make sure that people know exactly what their Muse is interacting with and understand that really deeply.
Then in addition to that, we are building, and will release soon, what's called the confidential VM, which will be an even more secure and private offering, where it's even more encrypted, it's even more secure, and it's actually designed by the creator, the person who built WhatsApp encryption and helped build other encryption on other major messaging platforms.
Moxie Marlinspike, exactly, who's I would say one of the real luminaries in the industry about security and privacy. And that's something that will be an additional offering that we are going to be releasing soon.
Why launch without it? Why launch first with secure as opposed to waiting for confidential? I think I got that right, right? Secure is the one now. Confidential's coming later.
Secure is the one now, confidential's the one later. There's a lot of technical challenges to get the confidential VM working right, and even when it's released, there are going to be trade-offs associated with the confidential VM. Because of how it's architected and because of various technical trade-offs you have to make, it might be a little bit slower, it might have other challenges. So we wanted to get a great experience that we knew already we'd invested a ton of resources into safety and security. You know, we actually delayed the launch of Muse for many months to get the safety and security exactly right before ultimately releasing it. So we feel really, really good about what we have released. That being said, we're always trying to push the envelope on how to make it even more secure, and we'll be coming out with a confidential VM soon.
