The stakes are becoming ever higher in the global AI race. Companies spending vast sums on deals of all kinds, including data centers and people. The biggest brains in the field can now reportedly command pay packages of up to $200 million. Leading AI for one of the biggest companies of all, Microsoft, is Mustafa Suleyman. He's now trying to give Microsoft's AI autonomous capability and is also thinking about the next frontier, which he says is superintelligence. You'll hear him explain all of that to me. But as ever on The Mishal Husain Show, this conversation is about the person as well as the work.
Mustafa Suleyman, thank you for being with us.
Thank you for having me, Mishal.
I'd love to start by getting your insight into a world which is your everyday world, but which is still developing for most of us. And to understand what are the uses of AI that are in your life that the rest of us might not be using yet.
It's funny you ask, because yesterday night I stayed up far too late watching a film and afterwards I added to a table that I've made in Copilot, which basically records all the films that I love, films that I want to watch, lists them by date, describes what they were about. I add my personal notes to it, gives me a link to the film poster, and I can keep just saying, okay, now add this film that I want to watch, what would be a similar one that I want to watch. So just like you might ask an assistant to organize your life, and I think that people will be surprised, like the more obscure, the more creative, the more challenging you think the task is that you're going to ask, you know, your AI or Copilot the better.
But have you used it to do autonomous tasks? So, for example, has it booked tickets for you or bought a gift for you? Because I know this is one of the promises of Copilot, it's just not available in my region. So I haven't been able to try it out for myself yet.
We're still experimenting with it. It can do it, it doesn't always get it right. So it's in like a sort of dev mode. So it's not like generally available just yet. But when it does work, it is the most magical thing you've ever seen.
Okay, what are the mistakes it's made that have created problems for you, bought a present for the wrong person?
Well, it can buy the wrong thing, but obviously you can intervene and it will always ask you permission before it takes the next action and stuff. So it's quite safe. It's just one of those things with technology, you know, it promises so much and it's always sort of just around the corner. You know, I remember like the allure of getting a Discman. And I had this dream that I was going to be, you know, back in the 90s, like rollerblading down the street. And then I got one and obviously it didn't have any kind of vibration proof, you know, shock proof technology. And so it would skip every time, like I turned it on its side or tried to, you know, put it in my back pocket. And so it's a funny thing technology, it's still magical and amazing, but it's always just got a little bit further to go. And I think in this case it's a while yet before it's everyday.
And I guess that given everything you've seen, you're one of the founders of DeepMind and you've worked at Google and had your own AI company Inflection, before you came to Microsoft, does that mean that when you see these hiccups or these stages of development, you have faith? You don't get frustrated because you remember the moment when back in 2016, when your AI model beat the world champion at Go, which was a huge moment for AI.
Yeah, I mean, I think I'm very stoic about these things. Like, I know that it's going to work in the next six months or 12 months or maybe the worst case, 18 months. But, you know, I think we've got a pretty good rhythm now of picking things which are not too far away but are genuinely transformative. And also, you know, it's important to remember how magical it is today. It can generate images. It can generate videos. I mean, it is already, you know, sort of superhuman, magical. And I think I'm just one of those people that's still inspired by what it can do at any given moment.
By this time next year, could I be buying my Christmas presents using an autonomous AI agent?
Yeah, I'm pretty certain that you will be. Yeah, I think it's highly, highly likely.
So, speaking of magical things, the recent announcement that you made at Microsoft about superintelligence, that's a term which has really crept into the public debate thanks to you and others in the last few months. When you use the term superintelligence, what does it mean to you?
I mean, superintelligence in the industry today means an AI system that can learn any new task and perform better than all humans combined at all tasks. So it is a very high bar and at the moment it comes with a great deal of risk. It is not clear. In fact it's very uncertain how we would contain and so limit its capabilities and align, so get a system like that to care a lot about humans and want us to succeed and flourish. It's very unclear how we would contain and align a system that is so much more powerful than us. And so the framing that I prefer is one of a humanist superintelligence, one that is always in our corner on our team aligned to human interests. And until we can prove that it will remain safe, we won't continue to develop a system that has the potential to run away from us and to get out of the box. And I think that that's an obvious thing to say. Like that feels like a statement of fact, like everybody should agree to that. And yet, you know, I think it's kind of a novel position in the industry at the moment.
So is that how you're trying to set Microsoft apart? A lot of people are working on superintelligence. Are you trying to make Microsoft distinctive by saying we will always use it through a humanist lens?
Yes, that is our position. You know, Microsoft is a company that has been around for 50 years. It is very careful as highly trusted 90% of the S&P 500 users to provide email and operating systems and every day productivity tools, which are now among the best in the world with Copilot. And, you know, I think that we've got that reputation because the company has been careful and we're going to continue to be careful. And setting out a vision of humanist superintelligence is part of that program.
So what does that mean for your rivals in that field, some of whom you work closely with, like OpenAI, which I'm sure we'll talk more about, but does it mean that they're the Wild West and you're the moral ones?
I think that everybody has to decide what they stand for and how they operate. And I don't want to judge how they're operating right now. I don't see any evidence of large scale mass harm. I don't see any indication that these things are improving themselves or operating autonomously. But we all predict a time in the next five years, maybe the next ten years, where these capabilities do start to emerge. And systems like this could set their own goals or they could improve their own code or they could act autonomously. Those things are capabilities that I've clearly outlined as increasing the level of risk. And therefore we have to approach them with caution, with more transparency and audit, with more government engagement, and make proactive declarations about how close we are to those three capabilities. I think that's obvious.
And does that mean that you won't be releasing a superintelligence tool until you're confident that it can be controlled?
Yeah, I think that's right. I mean, I think for sure the containment and alignment are necessary prerequisites, they're red lines and I think everybody in the industry has to sign up to that idea. I mean, nobody wants to cause mass harm. I mean, fundamentally, even though we all disagree and clearly everybody is committed to, you know, the survival of our species and, you know, the kind of, I would hope, the kind of flourishing and wellbeing of everybody. So, you know, that's the discussion that we're trying to push now and require everybody to ask themselves in the industry, you know, are they building a humanist superintelligence? Does it really stand for people and for our species first? Yeah.
I'm just wondering how much confidence we can really have in that, given that things do go wrong. And I don't just mean I mean humans do wrong things or things just go wrong for other reasons. And the recent example that Microsoft had to deal with was having to cease and disable some services that were being used by the Israeli Ministry of Defense after reporting by The Guardian that suggested they could be being used for mass surveillance.
That was very good reporting by The Guardian. We were very grateful for it. And as soon as we became aware of it, we actually made all the necessary changes, removed the IDF from those servers. And I think, you know, they were clearly not in compliance with our terms of service. And there's an ongoing investigation internally. But I guess the broader point is that it's hard to have confidence of controls and checks and balances and uses.
I mean, these are huge and complicated systems. They carry a lot of risk. And the most that we can do is make sure that we are auditing them and we're removing actors that violate our terms of service as quickly as possible.
So are the first uses of superintelligence going to be in the medical field?
I think so. I think that this is probably the most exciting application of superintelligence. I think we now have systems that can diagnose any rare condition that is found in the literature significantly better than human performance and do so with fewer tests and interventions. And that's the key thing, is like often it's actually quite expensive to run, you know, medical intervention tests before you get to a diagnosis. And often they can be unnecessary too, and they take time. And then in that process, you know, the condition can often worsen. In our case, our system does this significantly more cheaply with fewer tests and with higher accuracy. We're just putting it through independent peer review at the moment and soon there'll be clinical trials of it too. So I think this is very, very, very exciting.
And did you push for that focus yourself? Because you've been running Microsoft AI for a while, about 18 months now. Superintelligence was a big recent announcement, and I'm conscious, as I hear you talk about it, that your mother was a nurse and you, unusually for senior people in tech, you've worked in the public sector as well.
Yeah. I mean, this is an area that's very important to me. Obviously, my mum was an NHS nurse and you know, I am just a big believer that technology is here to serve us, like it should, you know, make our lives better, should make us more comfortable. You know, one day I think it is going to help us to live longer. I think it's going to, you know, give us the option to work less if we choose to. I do think it's going to produce abundance, and I think that we have to make conscious decisions to use it for those applications first. And that's why, you know, even back at DeepMind, I ran our health unit and, you know, I'm very passionate about this.
Abundance is such a big promise. Tell me what you mean by it, because most of the time we hear about AI destroying jobs. But are you saying that the work will be done by AI increasingly, and therefore we humans won't have to work as much?
I mean, I think it is inevitable that at some point over the next 20 or 30 years, machines are going to be more capable than humans at doing most work, and that might come much sooner. And so we have to decide as a society what our purpose is, and we have to be very thoughtful about the rate of introduction of new machines because we have to make sure that that rate of displacement is counterbalanced with a mechanism to fund people and to support people through a massive transition for which there is probably not going to be other jobs on the other side that are more competitive than, you know, what a machine can do. And so this is a real big step change in what it is to organize society.
But do you believe in the idea of a universal basic income, that that's what AI, being very productive, making the economy more productive could unlock?
Yeah, I mean, I think I've long been on record saying that. I think that is both inevitable and very desirable. I mean, we already live in a world of abundance. It's just poorly distributed. We already have more calories to consume. We have incredible amount of food wastage. It's just unevenly distributed. And now that value isn't just manifested in atoms like food and cars and physical things. It's manifested in digital goods, ideas, knowledge, intelligence and that's actually great news because that can proliferate. It can spread extremely quickly around the entire world. I mean, these are LLMs and chatbots have been the fastest spreading technology in history. I mean, you know, basically 2 billion annual users in the space of three years. And that's actually amazing because it's delivering knowledge to every single person who wants to use it on their phone in the current sort of format and device they already use. So I think that we have no problem with getting technology out and available to everybody because there's going to be massive competitive forces to reduce the cost of experiencing an AI. I think the challenge we're going to have to figure out is how we tax and redistribute so that the transition is a healthy one.
Tell me more about where these ideas and your beliefs came from or come from. Your mother was a nurse, but the family you grew up in, in London in the 1980s and 1990s, how would you describe it?
Pretty working class. Yeah. My dad was a mini cab driver and yeah my mum worked in the NHS. And yeah, I mean, we were, you know, fairly regular, like kind of unremarkable, like my parents didn't super value you know, education stuff. They always thought I should go get a trade. And my mum would often say to me, you know, you should be a carpenter or electrician, you know, leaves for 16 and stuff. And yeah, so I don't know. It comes from a place of just experiencing, you know, the kind of rougher end of things a little bit and then having just a desire to try to do the best that we can do with the short life that we have.
Your dad had come to the UK from Syria. Did your parents meet in England?
And what happened when you were 16, Mustafa? Because I read that when your parents split up, you and your brother, your younger brother, were pretty much left to fend for yourselves.
That's true, yeah. I don't know where you hear that, but, yeah. Me and my younger brother did live on our own for a few years. Yeah.
I think you mentioned this in a Guardian interview a couple of years ago when your book was coming out, but I'm really curious to know more about what it taught you. And how you dealt with it at the time?
I think that when you're that age, you're just sort of precocious and overconfident and fearless, and I didn't really notice it at the time. You know, we were, you know, we had everything that we needed and it worked out pretty well.
And this is at a crucial time, the last couple of years of school. And you did do well in those last couple of years of school because you ended up getting to Oxford.
Yeah, I was very lucky because I had, you know, great teachers and good mentors and I went to a very good school. You could do exams to get in. So I studied really hard when I was ten years old for the entrance exams and it was essentially like going to a private school. I mean, it was basically everyone was like very studious and got great grades and stuff. And yeah, I was lucky enough to get into Oxford and, you know, that was an amazing experience.
Have you thought about what your life would have been like in that period in your mid-teens had it not been for that school?
Yeah, I haven't thought much about that. It would have been pretty, pretty different.
And then you got to Oxford and you did actually a subject pretty close to what my son is doing there right now. He's doing theology. You did philosophy and theology. And then tragically to my ears as a parent, you dropped out.
Yeah. Well I was very frustrated and eager to change the world and get stuff done. And so I actually dropped out and I helped to start a telephone counselling service for young British Muslims called Muslim Youth Helpline, which I worked on for two or three years as a volunteer, and it was a secular, non-religious, non directional, non-judgmental, just peer to peer listening service for young British Muslims that post-9/11 were dealing with a lot of I guess identity crisis would be the way that we would describe it in hindsight. But at the time it was just like, you know, lack of like connection to community, family, parents, a lot of bullying and so on and so forth. And so, yeah, I worked on that pretty much full time for two or three years.
And had you experienced anti-Muslim sentiment and hatred yourself at that time, given the period we're talking about after 9/11?
A little bit, yeah. I mean, I think that it was more just not so much me, but I think a lot of people felt that, you know, we weren't sort of British enough and people were figuring out how to both live their cultural religious identities in the context of families that often were first generation, didn't really speak the language, didn't know how to navigate the system. And then the slight increase kind of skepticism that we were, you know, sort of mostly terrorists or whatever. And I think there was just this like general kind of, you know, fear and exclusion. And actually most of the time, you know, that was dealt with by just having kind and supportive conversation and having somebody be available on the other end of a telephone. Yeah, someone to talk to.
I know the work of the helpline, actually. So I know exactly the world that you're talking about. Did you think hard about dropping out of university? And have you regretted it at any point since?
I haven't regretted it, and I probably could have thought harder about it. My tutor did encourage me to, like, switch subjects or change college and actually had me go meet one of his friends at a different college to go and talk about doing history or PPE. But I think at that point I was just so full of energy and like eager to start things. I started a business at the same time doing electronic point of sale, selling, you know, sort of PDAs into restaurants to try and do restaurant booking and stuff and doing their networking and so on. It was a very early days of the internet, so it was a bit early, but I did that for about a year or two as well. So I was just like I had too much energy, I think, for sitting around thinking in the dreamy spires.
Well, you found the outlets for it because then, of course, you went on to co-found DeepMind with Demis Hassabis. And now here you are in this circle of power, right? One of a small group of people who are making decisions on a technology that is already changing all our lives and will continue to do so. How conscious are you of that power?
Very conscious, I take it very seriously. And it's a great responsibility. And, you know, I think that this is a moment when decisions that we make may have very lasting consequences. And I read history and I look back at, you know, times, for example, in the social media revolution where I think there was it was very clear that there were potential harms that maybe fell on deaf ears for too long. Or I look back at, you know, the electric car revolution that could have taken off way back in the 20s or, you know, you look at the kind of smoking or oil. So I'm very sensitive to those things because it's very clear that, like these things will cause harm. And, you know, I think that we have to be very, very careful about how we deploy them and how they're introduced into the world.
Do you talk about this with your peers, with people like Sam Altman, who I know you're close to?
Yes. I mean, I think everybody in the industry does. And I think there's definitely a group of the CEOs, obviously, Sam and Dario from Anthropic were co-founders, me and Demis were co-founders. You know, so we all know each other very, very well. And I think on the whole, you know, I think everybody is genuinely committed to trying to find the right path through. It's also very competitive. So there's obviously a lot of that dynamic too.
Yeah. Do you recognize the term broligarchy?
I haven't heard that before, but I can figure out what it means.
Yeah, I guess that's true. And it is very male centric, although it should be said, Mira Murati, who's the ex-CTO of OpenAI, is one of the best people in the field. Or tech bros, I'm sure you've heard that one. I don't know if you've seen Jessie Armstrong's Mountainhead, have you?
It is all about a group of people, one of whom is clearly Elon Musk, another one who looks like Peter Thiel or sounds like Peter Thiel. And it's well, I recommended it, we interviewed him not long ago here. But, you know, it's quite a sobering portrait, his impression of life in this quite rarefied, very powerful circle.
Yeah. And I think part of the challenge is that we all spend a lot of time in Silicon Valley where the skies are blue and, you know, life is very peaceful. I try to travel a lot. I just came back from China a couple of weeks ago where I was in five or six different cities and, you know, just staggering to get out of the bubble and see how this technology is being developed on, you know, the other side of the world. And, you know, both the pace of innovation, but also, you know, I think the thoughtfulness of some of the regulatory stuff. I mean, I think they are definitely thinking, you know, pretty carefully about it as well. So it's impressive. And I think just yeah, I also come back to London a lot and make sure that I spend time outside of Silicon Valley.
Yeah. So if you have a different impression to someone like Jesse Armstrong or others about your world, can I ask you just to because you know these key players, can you describe some of them in one word, like if I say to you, Sam Altman, what word first comes into your mind?
Oh my God. Yeah, I guess courageous.
Well, you know, he's obviously sort of growing his data center fleet very, very aggressively. And I think he's you know, he may well turn out to be, you know, one of the great entrepreneurs of our generation. But, you know, he's certainly achieved a lot. And I think now he's building data centers at a faster rate than I think anyone in the industry. And if he can pull it off, it will be pretty dramatic.
