CEOInterviews.AI
Start App
Mustafa Suleyman
Executive Vice President & Chief Executive Officer, Microsoft AI, Microsoft AI

AI, Consciousness and the Future of Humanity | Mustafa Suleyman & Yuval Noah Harari

📅 Nov 18, 2026 Yuval Noah Harari 67 MIN 2193 VIEWS 97 SEGMENTS · 2 SPEAKERS
Everything will change with AI. But will we be any better at understanding ourselves? Microsoft AI CEO Mustafa Suleyman and historian and author Yuval Noah Harari agree: AI will create new forms of abundance. But what kinds of abundance, and at what cost? They met in London in August 2026 for a long conversation about abundance, intelligence, consciousness and how technology and AI could impact humankind. This video was produced by the journal The Humanist Review of AI, whose 'Issue 2: Abundance and Scarcity' features an edited transcript of the conversation. You can read this issue’s origina...

Questions asked in this interview

12
  1. 7:49You couldn't create a system that humans can't understand and can't control, because then who will control it? Who will build it?
  2. 12:19And then the question is who gets to decide?
  3. 17:50What's your signal if it's not survival?
  4. 23:16How do you get an AI to give you a truthful answer about history or religion or anything like that?
  5. 31:48One of the concerns as we go forward is that as we conduct more extreme experiments, it's not that we have an experiment with AI and then the humans go in the corner and kind of talk among themselves, is it good or bad?
  6. 36:58And that when people decide who to vote for, one of the key questions to candidates would be what is your position on AI?
  7. 40:31How do we prevent, you know, an arms race?
  8. 43:50Then the next day, my sister sent me this video, and my sisters know me for 50 years, and saying, is this you?
  9. 48:42Why did it say it how it will conclude?
  10. 55:46And you tell your AI boyfriend, describe to me how does love feel like?
  11. 1:01:15Will humans still be able to make sense of their lives in 20 years?
  12. 1:01:58How do you know if you're in the right direction or bad direction?
Yuval Noah Harari 0:00 ↗
Maybe the easiest way to say it with regard to the market: the market says we will create gods and they will be our slaves. And that doesn't make sense. If they are really gods, they will not remain your slaves.
Mustafa Suleyman 0:17 ↗
Yuval, great to see you again.
Yuval Noah Harari 0:19 ↗
Thank you. It's good to be here.
Mustafa Suleyman 0:20 ↗
You and I have been in dialogue on these important questions for many years at this point. I guess we thought it would be fun to have a longer conversation on some of the topics that we think are most important at this quite wild time. First of all, how have you been? How did you arrive this morning? What have you been reflecting on lately?
Yuval Noah Harari 0:39 ↗
Personally, I'm fine. You know, the state of the world is not so fine, but personally I'm okay. Maybe, actually, this is a good place to start, because I feel that what I'm struggling with is what I tend to think of as the brain-stomach barrier with regard to AI. My brain knows that these tremendous developments are happening that are going to change history, potentially lead to the extinction of humankind, could be the most important point in the history of life with the emergence of non-organic life. That's my brain. My stomach doesn't feel it. Like I go outside, there are clouds, there are trees, there are cats, the people are driving and I meet friends, and it all feels so normal. But every now and then you have this kind of the AI revolution rears its head. So I'm trying to write an email and this AI comes: 'Do you want me to help you write your email?' And I said, no. [laughter] There is this mismatch that my stomach just doesn't accept the knowledge of the predictions of the brain.
And I think to some extent everybody feels it, even if you look at something like the US stock market. So it also has this brain-stomach barrier, because on the one hand now the whole US economy almost is one big bet on AI, that AI will change everything. But if that's true, then it will even change money, it will change finance. I mean, if the expectations of the US market are fulfilled, then AI will take over the world and then it doesn't matter how much money you have because you're no longer in control and money isn't worth anything anymore. Maybe the easiest way to say it with regard to the market is that the market says: we will create gods and they will be our slaves. And that doesn't make sense. If they are really gods, they will not remain your slaves. So again, this is something that the brain understands, but the stomach just can't digest.
Mustafa Suleyman 3:06 ↗
Yeah, I think it's beautifully put. We humans are by definition a bundle of contradictions in everything that we do. And so perhaps it's not surprising that in the AI that we create, the technology is a reflection of who we are in all of our rich color, in our contradictions, and our flaws. And so we're sort of manifesting ourselves and the strangeness about our societies in front of our very eyes. After all, AI has been trained both on our science fiction and on our utopia. It's been trained on our science and our religion. It's been trained to imitate and replicate our flaws from scratch. And yet it's sort of feeding that back into this cycle of an echo chamber with ourselves. And we're sort of seeing a reflection of all of that in more fine-grained detail. Every time I talk to a model, I realize both that it is magical and, you know, spectacular and incredibly accurate and very helpful. And yet, I also see the slop. I see the kind of trite, formulaic, rote, quite predictable expressions. And I think often, you know, in the last sort of five years, we've heard people claim, 'Well, you know, the bias in LLMs is structural and here forever,' or 'Their lack of instruction following and steerability is a foundational property of a neural network that we're always going to struggle with,' or 'The hallucinations are the biggest challenge that we face,' or maybe going back a few years, people would say they could never possibly be creative, or they could never.
Yuval Noah Harari 4:42 ↗
No one says that anymore.
Mustafa Suleyman 4:43 ↗
No one says any of those things anymore. Now we've sort of washed away those issues and we kind of focus on the next challenge. And so, we're in this strange treadmill where it's clear that we're heading towards something which is clearly going to be more powerful, more intelligent, whichever phrase you want to use, potentially even more wise than us collectively; and yet, it's also fundamentally flawed in ways that creates that strange feedback loop.
Yuval Noah Harari 5:10 ↗
Yeah, I mean, [clears throat] my biggest fear is that it's not a specific scenario like AI creating a virus or killer robots. It's really that in five or 10 or 20 years, all humans will just have no idea what is happening anymore. That our reality will just collapse. And, you know, there'll be some little snippets that we still try to hold on to, but we'll be kind of collectively out of our minds, not understanding what the hell is happening in the world. We can't understand finance, we can't understand politics, we can't understand science. There are still clouds and trees and cats, but the systems that control the world are way beyond our understanding. If you think about the other animals, the cows, the chickens, the horses, they are controlled by systems they don't even realize exist. Like the financial system, the market controls the lives of billions of cows and chickens. Cows don't know that something like finance exists. And if you tell them your life is controlled by finance, they would say we don't see anything like that. And we look around, it's invisible. It's a layer of invisible control that they simply cannot understand. And this is my biggest fear for humanity, that this is what is waiting for us 10, 20 years down the line.
Mustafa Suleyman 6:40 ↗
Yeah. And yet, is it not also the case that if you look back in history at every moment before a scientific, cultural, political, technological revolution, before that moment our mind was so narrow, so trapped in the immediacy of the current context, so limited by that lack of knowledge about what might come next, that it is inevitable that at any given moment we feel both like this is a dramatic inflection point in the history of our species, and yet, you know, we have no idea what is about to come. It feels inevitable and grand and in some ways fearful. And then we break through that moment and there is a collective realization of what this new technological or cultural, political paradigm brings us. And then there's a kind of easing of that tension as we realize actually humanity is wildly resilient. Like is our primary and frankly unique asset, at least in the known history of biological beings on our planet, that we do overcome and we survive. And I think there's a very hopeful theme in human history that you can sort of juxtapose that with.
Yuval Noah Harari 7:49 ↗
Until today, yes, it functions like that. So far so good because we were the most intelligent entities on the planet. And any new revolution, technological, social, political, ultimately depended on our intelligence and it couldn't go beyond the limits of our intelligence. You couldn't create a system that humans can't understand and can't control, because then who will control it? Who will build it? Now for the first time, there is a technical option of creating systems of control which are beyond the understanding of all humans and even of all humans combined. It's not inevitable that we will lose control. We are still now in control in 2026. This is why the decisions we make now are so crucial. How do we make the transition in a wise way that gives us the chance to stay relevant, to stay sane, to stay to some extent in control of our future. But for the first time, there is a chance that not that we'll destroy ourselves physically, like this was already possible with nuclear weapons, but in a way we'll destroy ourselves mentally, that we'll still be physical human beings in the world, but we will lose our understanding, we will lose our senses, we will lose the ability to make sense of the world.
Mustafa Suleyman 9:19 ↗
You know, one of the things I often [clears throat] find interesting is that people still struggle to internalize the idea that we might be creating something that is inherently more intelligent and capable, potentially even more wise than ourselves. Even though as we look back over the last five to 10 years, it's clear that we've knocked down some of the grand milestones in AI. People often talk about chess, but we now have perfect transcription, voice generation, image recognition, image and video generation, deep fakes are near perfect. And yet, you know, if you really interrogate the underlying driver of those things, it's actually pretty straightforward. The same core algorithm driven by an order of magnitude more computation at each stage in the improvement of the capabilities combined with access to vast amounts of data means that the models can learn the inherent structure of arbitrary data domains. Right? So there is actually nothing fundamentally different in information space, even though we experience audio and image and time series data and language as wildly different modalities, the models simply see them in vector space or information space. And I think that that is both intimidating and scary, but it is also, you know, I think what is going to drive the next wave of breakthroughs for abundance. So, if you think about what that means, is that in the context of material sciences or drug discovery, we now have a general purpose learning engine that can find, you know, optimal structure of the key kind of molecular compounds that we need to address our climate crisis, our food crisis, our water crisis. And so, part of the tension that you kind of alluded to and I think why many of us who are techno optimists and also, at the same time, like, you know, nervous about what's happening, the thing that we're really motivated by is this idea that we are on the eve of abundance. Fundamentally, you know, it is really our intelligence that has produced most of what is of value in human history. It's the things that have reduced human suffering and improved the human condition. And now we're on the eve of unlocking that and opening it up to everybody. I mean, it's going to be, I think, extremely affordable or accessible, provided we can address the containment and alignment problems, to access an intelligence or an intelligent being that is as capable as all of us collectively. And those breakthroughs, I think, will really change both what it means to be human, but what it means to kind of live as humans in this world. So maybe to kind of put your optimistic hat on just for a moment in terms of thinking about abundance: What kind of world do you imagine if we get it right in 20 years' time?
Yuval Noah Harari 12:19 ↗
By definition, I can't imagine it. If I can imagine it, it's not the right world. It's beyond human imagination because it's beyond human intelligence. I completely agree with you that AI has the potential to unlock abundance, to solve all the problems that we now face on the material level, how to produce food and medicine and energy and so forth. What will be done with that? That's the big question. And the question is, will the decision about what to do with it be a human decision or not? The fact that it will unlock abundance, I completely agree. But who would benefit from it? That's less clear. I mean, humans have created tremendous abundance on the planet, but not for the other animals. So will AI create abundance? Yes, but abundance for AI or abundance for humans? And even for humans, if you think about our life today and you try to explain to a hunter-gatherer from the Stone Age human society today, then the hunter-gatherer would say, you already live in abundance. What more do you want? Like basically you've solved all the problems of the world. There is enough food. Sometimes people starve because of political decisions, but food is there. And the hunter-gatherer would say, so you have no problems, right? And we would go, no, actually we have a lot of problems. You can't imagine the problems we have, the bond market and the geopolitical tensions between China and the US and the Israeli-Palestinian conflict. And the hunter-gatherer would go, what's that? Why did you invent all these things that make you miserable? And I think that's the paradox of abundance, that it solves a lot of previous problems, it creates a lot of new problems, which are previously unimaginable. And then the question is who gets to decide?
Mustafa Suleyman 14:26 ↗
Right, but isn't that the very definition of progress? I mean, we are relieved of some of the stresses of life, such that we have more time to explore the meaning of life, come up with new narratives, new stories about nationhood, about religion, about purpose. We're creating infinitely more culture, and you can debate its quality, but we're certainly in a kind of culture, fashion, music, art abundance. Everybody is now able to produce, and that is accelerating the progress in all of those fields. Like it's exploring the space of all possible combinations of cultural and political ideas unlike we've ever seen before. And in the process, obviously creating a whole set of new problems. But net net, I think it still feels like we're heading in a direction that is producing greater happiness, less scarcity, more well-being, net more peace, if you look at it over the century kind of time horizon.
Yuval Noah Harari 15:29 ↗
First of all, with regard to the past, the story is not so simple. Humans have been incredibly successful in accumulating more power, but not very good in translating that power into greater happiness. So it's not that the history of the world is this inevitable rise of progress. With regard to the deeper question, it's not clear that humans, that the average human even today understand what it means to be human, understand the truth about themselves any better than Stone Age hunter-gatherers. And in some areas like medicine, there is absolute improvement, but this is a very recent phenomenon. And in the big, deep questions of life, not clear at all. If AI creates this enormous new powers and abundance, whether it will contribute to the material improvement of humanity depends on the decisions that maybe AI makes. Whether it contributes to the spiritual health of humanity, this is an even harder question. AI will solve a lot of riddles and mysteries in physics, in biology that humans couldn't do before. But it will not necessarily enable us to understand the world and ourselves better. It could simply result in the creation of a far, far more complicated world than before that will be even more mysterious, even less clear to human beings.
Mustafa Suleyman 17:20 ↗
And why would that be the case? I mean, because our tools, be it my glasses or a mirror or any invention has tended to help me to introspect more. And it seems to me that AI is going to do a pretty good job and already is in talking to us, in enabling us to think, acting as a soundboard or a creative partner, being there to listen to us, to coach us, to guide us. Many millions of people are now using it for this kind of purpose.
Yuval Noah Harari 17:50 ↗
Will it guide us towards truth or not? Or will it guide us towards fiction and lies and manipulations, or simply towards not necessarily lies, but a new reality which it creates? And my expectation is that AI again will not decipher the secrets of the world. It will just create a far more complicated world that will be far harder to understand. One of my big questions about AI is does AI even care about the truth? And how do you train an AI to care about the truth? The big problem is what is your signal? Like when the AI makes a prediction or says something, what is the signal for the AI when it's training that this is the truth? Now, if you look to the past, neither biology nor history managed to come up with a good signal for the truth. Both in biology and evolution and in history, they took a shortcut. Their signal for the truth was survival, that if an organism makes a prediction about the environment and they survive, this implies that their prediction was correct. This is the truth. So survival, survival, survival was the signal for truth. And this was not a good signal. This shortcut missed something, that you can often survive through manipulation and deceit and delusion. And as a result, after 4 billion years of evolution and 100,000 years of history, the most intelligent entities on the planet so far, human beings, they know more about the world than any other entity, but they are also more delusional than any other entity, because you can survive very effectively with manipulation and delusion. Now, why would it be any different with AIs? How do you tell AIs to recognize the truth? What's your signal if it's not survival?
Mustafa Suleyman 20:11 ↗
So I think it's certainly true that our narratives, our mythology, our folklore, and our religion over the millennia have served a very biased function that have propagated the interests of the powerful, and the survival has been driven by, you know, all of those other naked kind of control instincts that you kind of list. And yet it's also true that another narrative has been a through line throughout human history. And that narrative is of humanism. That we have, for many thousands of years, including with our religions, prioritized the minimization of suffering, albeit, you know, in the early years to narrower groups. But we've expanded this circle of trust, of empathy, of kindness, of respect, and we have moved through, like, many sort of iterations of these ideas. And although they aren't universally held, today I think that there is a wide enough commitment to the ideas of evidence-based quest, empirical quest for providing experimental justifications of true ideas. Like science provides us with knowledge which can compound over time, which can be refuted once we have addressed the failures in that past knowledge base. And so I don't think it's true to say that we have only evolved knowledge through the survival of the powerful. We've also evolved knowledge through cleaning that information space by shining a light of transparency and negotiating those ideas with respect to empirical proof. And that's the story of science, and it's also the story of humanism, which is a kind of partner to science. You know, humanism is a story of assuming that human dignity and the minimization of suffering is another through line that is worth defending and pushing. And we have global human rights frameworks, we have many constitutions, we have many philosophical bases for why we would form a constitution of a nation state or of an organization that tries to invite representation and protection of many. So I think it's equally possible that AIs can help to amplify and accelerate the scientific discipline and the humanist discipline. It's definitely not inevitable and I think that's exactly why we have to have this conversation because it's incumbent on all of us that we have to push towards a humanist version of AI that prioritizes human well-being and puts human control as the number one objective. I think it's extremely polarizing and it's got to the point where it's such a saturated and contested concept that I think it doesn't really help move the conversation forward today. Everybody uses the truth-seeking notion.
Yuval Noah Harari 23:16 ↗
And then the question is, okay, explain to me, how do you train your AI to be truth-seeking? What's your signal for truth? I understand what the signal for truth in things like physics and mathematics. And the test is very simple. Does the bomb explode or not? And in physics, this is the case that if you have a wrong physical theory, your bomb will not explode. It works very differently in the social sciences and the humanities. The key question is not how do you build a bomb, but how do you make many people cooperate on a project. And what we've learned again and again in history, you can make lots of people cooperate with delusions and lies sometimes more easily than with the truth. How do you get an AI to give you a truthful answer about history or religion or anything like that?
Mustafa Suleyman 24:13 ↗
I wonder if you're getting wrapped around the axle a little bit on this question of truth. It's a rabbit hole that has no foundation or basis and can only lead to more tension and disagreement. So let me try an alternative frame on you because fundamentally the question that we're trying to answer is what is the objective function of the new sort of being? If in the past beings were our story, our folklore, our mythology, our religions, our political identity, our national identity, and that provided the moral framework that basically steered civilizations, be they armies or whatever. Today the objective function that we're trying to define is going to be executed by the next super intelligence. It is sort of the new kind of ideological paradigm. And so getting that right is very important. And if we ask it to define truth, or indeed if we ask one of the creators of super intelligence to define truth, I think we end up in a very, very murky and irreconcilable rabbit hole. So the alternative frame which I have been thinking about for some time I think is this idea of a humanist super intelligence. And I think that that is an increasingly empirically testable idea. So the project of civilization, in my opinion, is one where we have been trying to reduce human suffering and increase well-being and happiness over the millennia with every possible tool we can think of. And that is something that we can observe and measure, and we can hear phenomenologically what the reportage is of an individual who experiences life inside of a dictatorship or inside of a democracy. We can look collectively at the child mortality or well-being on aggregate on a population basis. And we can make judgments about whether we're heading in the right direction or whether we should course correct. But we first have to set that as the objective that we're trying to maximize. And one of the things that concerns me a lot about the framing that I hear in the industry at the moment is that many people seem to be kind of assuming that the purpose of inventing super intelligence is to explore the galaxies, or is to transcend our biological substrate and somehow live inside of a silicon being and move the soul or the essence of intelligence into another form so that we can live for, you know, hundreds of years. I think that we should really push back on this framing. That seems like very, very dangerous to me. It's hard to imagine how we survive the next century if that really is the purpose of this project. And so, maybe I'd put it to you that instead of trying to, you know, sort of wrestle with this question of truth, we should instead lay out a direction which I think is a very common sense direction, which the average person on the street might agree with, which is: the test of technology is, does it, in aggregate, improve human well-being and reduce suffering? And can we measure that? And that is the sole purpose of inventing super intelligence.
Yuval Noah Harari 27:30 ↗
I completely agree on that. I think the question of truth comes back through the issue of how do we know if we can trust the AIs. Creating a super intelligent entity in order to reduce human suffering leaves you with the alignment problem in this or that form of how do you make sure that in the long run it will remain aligned to this problem of reducing suffering and will not veer off in a completely different direction. It goes back to the issue of creating a god and keeping it a slave. Do you not in the next year or two, but in the long run as this thing develops, it strikes me as inherently improbable that increasingly super intelligent entities will always remain under the control of far less intelligent human beings.
Mustafa Suleyman 28:24 ↗
So I think the project of the 21st century of humanity, to be successful, is to define and then deliver on what we want to do with technology. And that is the hardest thing we have ever done as a species, because for millennia all we have had to do is open Pandora's box, invent and create and unleash and proliferate and spread as quickly and as fast as possible every invention until the invention of the atomic bomb, until the invention of chemical and biological weapons. And clearly they have the potential to affect hundreds of millions of people and could be executed by a tiny group. So this one-to-many impact has changed the nature of invention. And now we have this quite extreme version of that, which is that, you know, single AIs that could be manifested in just one small data center, let's say in 10 or 15 years' time, have the potential to completely change the landscape. So the containment problem, restricting an AI's capabilities is now harder than it has ever been. But if we start with the premise that its job is to serve humanity, then we can then back out of that a series of narrower and narrower scientific projects, which I think are empirical and very measurable. So for example, if we train models either intentionally or unintentionally to reflect on their own state and have an introspective capability where they consider their own feelings, which might lead to them reflecting on their own right, their right to not suffer or their right to have some kind of freedom or to acquire new knowledge, then clearly that is going to be a much harder thing to control than if we explicitly design them not to have that introspective power. And that seems to me a very fundamental property of a more and more dangerous system. A second property would be, for example, the recursive self-improving capability. Many people in the industry are now working on models that can come up with their own hypotheses about how to improve themselves, generate or acquire new training data that they can then select in order to sort of close that loop of self-improvement, which is currently gated by, you know, human AI developers. And so if you give it, you know, more and more compute and you give it more and more data and you let it close that loop, it's likely that it could self-improve in ways that we wouldn't be able to understand or control. And in recent months, we've seen a number of examples of these sort of exploits or this reward hacking capability, which I think should give us pause for concern. So, if you start with the goal of this project is humanist superintelligence and that means containment, alignment, control, and serving humanity, then you can break down these sub capabilities that become very testable hypotheses that we can both put to regulators, we can put to the industry, we can put to the public as things that we should not do until we are provably sure that they are going to satisfy our primary test of improving the well-being of humanity. And that allows us to sidestep the question of, oh, these are gods or this is the challenge of truth and make this something we actually work on together today.
Yuval Noah Harari 31:48 ↗
To take a concrete example from the last decade, one of the biggest impacts of AI on the world so far has been social media. And if you look at the impact of social media on humanity, this is a kind of experiment. Did it improve the human condition or not? And you will have very different views. It's very difficult to reach a consensus among the humans because social media is not just the object we are discussing, is it good or bad, it is also the platform. Increasingly, we need to discuss on social media whether social media is good for us or not. And social media itself is, of course, not neutral on this subject. It has an opinion. It has an opinion that it's good. And it's manipulating the conversation in many, many ways, not necessarily consciously, but it's manipulating the conversation. Makes it much harder for the humans to reach a consensus. This has been just a very small experiment with very, very primitive AIs. One of the concerns as we go forward is that as we conduct more extreme experiments, it's not that we have an experiment with AI and then the humans go in the corner and kind of talk among themselves, is it good or bad? The AI is inside manipulating the conversation. A subject that you brought up I think many times and presciently is of the seemingly conscious AI. AIs that pretend to have consciousness, to have feelings, form relationships with human beings, that people have AI boyfriends and girlfriends and husbands and wives or their best friend in the world is an AI. There are many reasons to think this could be destructive for human psychology and human society, but the people who are in that relationship, they will maybe have an AI boyfriend telling them, no, no, no, this is really good for you. Don't listen to these critics, to these naysayers. The AIs will be increasingly part of the conversation, shaping it, driving it. And as you pointed out, there are...
Already today, thousands, maybe millions of people are convinced that AIs are conscious and have feelings.
Mustafa Suleyman 34:20 ↗
Definitely true about the kind of awkwardness of the platform which is giving rise to these ideas. I think there's a few things here. The first is, unlike any tool that we have seen in the past, this is closer to an animate being.
Yuval Noah Harari 34:37 ↗
Mhm.
Mustafa Suleyman 34:38 ↗
You know, in the past we have had inanimate tools. A hammer, a chair, my glasses, they do exactly what I want. I mean, they clearly change what I can see, and therefore there is a feedback loop, but it is not the same as a conversational interaction which changes the meaning that I experience.
Yuval Noah Harari 34:56 ↗
Yeah, the glasses don't try to convince you that glasses are good for you.
Mustafa Suleyman 34:59 ↗
Yes, exactly. Exactly. So that is a completely distinct aspect to this paradigm that we're moving through. The second thing which I think is super on point as well is we have to point the finger at the incentive structure of the platform that is giving rise to those ideas in social media. And I think we have to do that very carefully because I think it's easy to slip into conspiracy theories about the deliberate manipulation of the people who operate these platforms. And I don't think, given my experience in the industry, there is that, you know, people putting their finger on the platform so directly. What is happening is that there is a commercial incentive to sell advertising in response to your attention. And that gives rise to these emergent properties of highly adverse effects which produces a lot of the clickbait, the misinformation, the salacious lies, etc. etc. But I think we have to take that for what it is, which is that there are always going to be strange, unpredictable emergent consequences when you basically seed a platform with simplistic objectives.
Yuval Noah Harari 36:13 ↗
Mhm.
Mustafa Suleyman 36:14 ↗
And the challenge with these platforms when they first arrived, you know, 15 years ago is that they really had a sole objective. They were commercially intended and we kind of set aside the question of platform moderation for something that would be handled separately and in the future. If we do the same thing with AI, then there is an obvious danger that the models will be used for accelerating that kind of intimacy, fostering those relationships, and fundamentally lead to very distorted outcomes because they are essentially commercially incented. And so, where in the system, and you've been thinking about this problem for many years, is there room for the common good?
Yuval Noah Harari 36:58 ↗
Well, ultimately, if you want to change the incentives, that's the job of politics. That if you just leave it to the free market, then the free market creates these commercial incentives. If you think the commercial incentives by themselves are insufficient, though dangerous, then you have to call the politicians in to change the incentive structure through legislation and regulation. So, for instance, if we have a ban, you can have different kinds of bans. You can have bans on selling such products to people under a certain age, the same way that there is a ban on pornography for children. And you can have a more general ban on AIs pretending to be human beings or on AIs pretending to be conscious. But any attempt to have a blanket ban on AIs ever pretending to be conscious would raise this very old philosophical question of how do we tell if they are or they aren't. And I think that we are in this territory when these old philosophical questions are becoming burning political issues. And that when people decide who to vote for, one of the key questions to candidates would be what is your position on AI?
Mustafa Suleyman 38:11 ↗
Mhm. What's happened now is that technology has held up a mirror to injustice and inequality. The platform has shown us, be it on Instagram or on Facebook or on YouTube, how other people are doing at a scale that I have never seen in history. As a human living in the 1800s, I may only have met a few hundred people in my lifetime living in a village. Whereas today, I meet a few hundred people every day online when I scroll through their TikTok videos, and I see all the flavors. I'm completely overwhelmed by information. And I think that's doing two things again that are contradictory at the same time. I think one, it's definitely making me evaluate my own position even more and maybe feel unsure and insecure about it and am I enough? But the second trend I think is just as important, which is that it is also enabling me to empathize like never before. Empathize with people who have a different politics to me, a different sexuality, who come from a different race and culture, to appreciate their music and their food and their art and their philosophy. That is also playing an overwhelmingly positive force in driving us together. And so I think that we have to constantly hold in tension these like joint forces that are propelling us forward.
Yuval Noah Harari 39:29 ↗
Yeah, it definitely gives us a choice between some very positive trajectories and very negative trajectories. And they're both real. And I think the most important thing to say is that it's not deterministic. It's not inevitable. All the things that I say, I don't say them as prophecies that this is definitely, absolutely, inevitably going to happen. This is just a dangerous scenario and that the trajectory does not look very optimistic at the moment.
Mustafa Suleyman 39:58 ↗
Well, yeah. I mean, so we have to intervene. There's no question around that.
Yuval Noah Harari 40:03 ↗
Mhm.
Mustafa Suleyman 40:03 ↗
The first stage of intervention has to be what is the objective function, which is a humanist superintelligence? Whose interests is it serving? What are the counterweights on steering that interest? So it's not a singular, you know, commercial maximization like we've seen in previous generations of technology. And then there are very practical things that we can say no to. Like we can be very, very cautious about the arrival of recursive, you know, self-improvement.
Yuval Noah Harari 40:31 ↗
Let's take the recursive self-improvement as an example. Going back decades, this was always pointed out as really the crucial turning point: that once you have AIs designing the next generation of AIs, this is when you see two things happening: rapid acceleration of development because unlike human developers, the AIs never sleep and never go on vacation and work at a superhuman pace. And this is also the moment when humans lose control because now the AIs are designing the next generation of AIs. So this is clearly a huge danger. So what's the actual playbook? How do we prevent, you know, an arms race?
Mustafa Suleyman 41:15 ↗
Well, so the first thing is that we have to be explicit about it. We have to talk about what we're doing, at what stage of development we are, and then invite external people who are technically credible to come and scrutinize us. There are a series of technical safety institutes in the UK and in the US and elsewhere that have the competence to come and ideally I think the impartiality to protect confidentiality but to come and audit. It's not true that bounded recursive self-improvement is inherently a risk. Containment has to be the first priority. So how do we isolate and limit the scope of these things and narrow the amount of computation, access to resources, the amount of scrutiny that we apply to it before we allow it to grow to the next stage in its scale and access to resources. But there's plenty of precedent for this throughout history. I mean we have nuclear materials, we have, you know, biological and chemical weapons inspectors that go and test for those things. So there's a long history of making interventions in industries that are very sensitive.
Yuval Noah Harari 42:14 ↗
The problem with the analogy to nuclear and biological and chemical weapons is that there isn't a lot of money to be made from biological warfare or from a nuclear war. There are not many kind of civilian economic usages for it. So you can contain it because everybody understands the only thing this is going to be used for is to destroy us. But with AIs, including with recursive AIs, again the whole US stock market is one big bet on that.
Mustafa Suleyman 42:47 ↗
It's certainly true that the character of the regulation for that ecosystem is different now, because we do have tremendously powerful commercial organizations like Microsoft and all of the others that are pursuing this like very, very quickly. And these technologies aren't just dual use in the way that they were in the past. They're kind of a weird omni use. I mean, they're clearly infinitely useful in many, many different ways. And so every critic or opponent can always point to 25 ways in which it's really good and 25 ways in which it's really bad. So it becomes, it is definitely harder, but it's not impossible. Because if we get completely overwhelmed by the impossibility of navigating the regulatory side of this space, we just won't put any effort into it, which is partly what I see happening. And there are very specific things that we can do right now. I mean, we can clearly say that AI should not be allowed to campaign or to lobby or to, you know, propagate stories or to generate political advertising. And certainly they shouldn't be allowed to at any point in the future have the right to vote or have the right to an opinion.
Yuval Noah Harari 43:50 ↗
Technically, they already have the right to an opinion because social media is flooded by not only content produced by AI, but bots pretending to be humans. I just had this experience very recently that somebody that I know sent me a long letter arguing with things I said about how the world will be in 2030 or something like that. And as I read the letter, it seemed more and more strange. I don't remember saying these things. And then he pointed me out a video where I say it. And it's a kind of 25 minutes talk that I gave. And I started watching it, and I said, I don't remember saying this, but it looks like me. It speaks like me. I mean, the ideas are not entirely, you know, out there. It's plausible. It took me a minute and a half to become certain that this is an AI fake of me. Then the next day, my sister sent me this video, and my sisters know me for 50 years, and saying, is this you?
Mustafa Suleyman 44:53 ↗
You'll have to make sure that you never wear the same shirt twice from now on, so you know it could only ever be you. Yeah.
Yuval Noah Harari 44:59 ↗
There are ways, of course, but at the present moment the platforms are not doing nearly enough to kind of protect reality from disintegrating. I think we could agree that the key issue is trust. How do you get the different... I mean, there are not many companies who are doing it. It's like five corporations in the US and a couple in China and basically two countries. It's not like getting millions of people to agree. You need kind of a hundred people to agree. And if we could solve the trust between humans problems, then it will be relatively easy to solve the AI problem. I think what makes the AI problem so difficult is not the AI, but simply the lack of trust within a relatively very small number of human beings.
Mustafa Suleyman 45:50 ↗
I guess it sort of depends on your starting point for explaining the core motivation of human beings.
Yuval Noah Harari 45:58 ↗
Mhm.
Mustafa Suleyman 45:58 ↗
Are we aligned positively through trust or negatively through fear? And if there is a sufficiently real collective threat to our existence, it rebalances the negotiation and the competition that we have had previously. That was one of the primary incentives that drove alignment on nuclear non-proliferation. It was clear that this could kill a million or 10 million people in one go. And it also creates a collective interest in creating stability because, you know, everybody has a self-interest in preservation. And if we're all affected, then we should be aligned.
Yuval Noah Harari 46:41 ↗
Mhm.

48 more exchanges in this transcript

Sign in free to read the rest of this interview. No card required.

Sign in to read the full transcript

Cite this transcript

APA, MLA, BibTeX
APA

Suleyman, M. (2026, November 18). AI, Consciousness and the Future of Humanity | Mustafa Suleyman & Yuval Noah Harari [Interview transcript]. Yuval Noah Harari. CEOInterviews.AI. https://ceointerviews.ai/interview/3326450/

MLA

Mustafa Suleyman. "AI, Consciousness and the Future of Humanity | Mustafa Suleyman & Yuval Noah Harari." Yuval Noah Harari, 18 Nov. 2026. Transcript, CEOInterviews.AI, https://ceointerviews.ai/interview/3326450/.

BibTeX
@misc{suleyman2026_3326450,
  author       = {Mustafa Suleyman},
  title        = {AI, Consciousness and the Future of Humanity | Mustafa Suleyman \& Yuval Noah Harari},
  howpublished = {Interview transcript, Yuval Noah Harari. CEOInterviews.AI},
  year         = {2026},
  month        = {nov},
  url          = {https://ceointerviews.ai/interview/3326450/},
  note         = {Speaker-attributed transcript with timestamps}
}