Beyond Boundaries: Redefining Humanity in the Age of Innovation | Yuval Noah Harari

Hello everybody. And what I want to talk to you today about is the power to shape the world and the power to shape our future. About your power as leaders and as innovators, but also about the power of the human species as a whole. And the big question we are facing is whether AI is taking away humanity's power to control our future, or perhaps AI is giving humanity more power than ever before. And also, can we even speak about humanity, about a single humanity anymore, or perhaps humanity in the era of AI is splitting into different groups, each with a radically different future. But before we get to AI, we have to say a few words about politics. The relatively stable liberal order of the early 21st century is collapsing. Instead of a cooperative order based on universal values, global institutions, and international laws, we are entering into a super competitive chaotic world in which only one thing matters. And this one thing is brute force. Leaders like President Trump envision the world as a collection of fortresses, with each country separated from the others by high walls. Financial walls, like tariffs, military walls, cultural walls, and physical walls. This vision may have certain advantages, but it fails to address one key problem. World fortresses are seldom friendly. Thousands of years of history teach us that each national fortress would probably want a bit more security, prosperity, and territory for itself at the expense of the neighbors. And in the absence of universal values, global institutions, and international laws, how would rival fortresses resolve their disputes? For millennia, the vision of the world as a collection of fortresses always led to exactly the same outcome. A never-ending cycle of wars with powerful empires conquering the weaker neighbors, and eventually fighting each other for supremacy. This is already beginning to happen all around us. Wars of conquest, which were taboo since 1945, are coming back. Russia already sustained a million casualties in its attempt to conquer Ukraine and reestablish the Russian Empire. The Trump administration has already expressed imperial ambitions regarding Greenland, Panama, and even Canada, and seems to support Israel's conquest of Gaza. China, too, has its eyes on overseas expansion. The collapse of the old order is frightening, but it is also, of course, an opportunity to shape a better order. And the big question is, who will be the shapers? In the old liberal order, the ideal was that all humans would be shapers. It didn't matter whether you came from the USA or Brazil, whether you're white or black, a man or a woman, straight or gay, you had equal human dignity, and therefore, in theory at least, your views and interests counted for something. Sure, this liberal ideal was never fully realized. Ideals are not reality. But the liberal world was nevertheless a huge improvement over previous historical models. In the new post-liberal chaotic world, the ideal of equal human dignity is completely and openly abandoned. Instead of all humans trying to cooperate to shape a world that will be good for all humans, the current mantras are me first and might is right. A few powerful people in a handful of countries think that they have the right to shape the world for everyone. If you remember the meeting earlier this year between President Trump and President Zelensky at the White House, whenever Zelensky talked about things like justice or peace or territorial sovereignty of nations, President Trump interrupted him and said, you don't have the cards. You don't have the cards. As we are meeting here in Sao Paulo to talk about the future of humanity, there are two questions for each of you to ponder. First, do you have any cards? Are there any cards in Sao Paulo? Or are all the cards in Washington, Beijing, and Moscow in the hands of people like Donald Trump, Elon Musk, Xi Jinping, and Vladimir Putin? The second question is perhaps even more important. Do you want to live in a world where the only thing that matters is cards? Is life ultimately just a card game with winners and losers? Now, I could have given the exact same speech that I gave now in 1939 or in 1914 or in countless other historical moments when humanity was on the brink of a major crisis. But this time, it's really different because as we ask ourselves, who will shape the new world of the 21st century? We need to take into account not only human shapers. For the first time in history, there are also non-human shapers on the planet, AIs. We might be quite close, maybe a few years, quite close to the moment when an algorithm would tell all humans, including Trump and Putin, you don't have the cards. As we enter the era of AI, I want to briefly explore here three questions. What is AI? What is the threat of AI? And how can all of you, business leaders, entrepreneurs, and innovators help humanity not just survive, but also flourish in the era of AI? So let's start. What is AI? There is so much hype around this term, AI, that the term is now inaccurately applied to almost every machine, and it is becoming difficult to know what it really means. So let me be very, very clear. AI does not mean automation. AI means agency. AI is not a tool in our hands. AI is an independent agent capable of acting and learning and changing on its own. To be an AI, it is not enough for a machine to act independently, automatically. It must also have the capacity to learn and change by itself, to make decisions by itself, and to invent new ideas by itself. As a simple example, consider a coffee machine. If you press a button and the machine automatically makes you an espresso according to a pre-programmed procedure, this is not AI. The machine doesn't learn or create anything new by itself. But suppose that as you approach the machine, even before you press any button, the machine says to you, I've been monitoring you for the last few weeks. Based on everything I've learned about you and based on your facial expression right now, I predict that you would like to have a double espresso with one spoon of sugar. So I've already made you such a cup. Here, take it. That's an AI. It learned something by itself and decided something by itself. And it is really AI if the following day the machine announces, I've now invented a new drink that I call Bespresso, and I think you would like it even better than espresso. So here, try it out. I made you a cup. The confusion about AI results in part from the term itself. The term AI has been invented decades ago as an acronym for artificial intelligence. But it is inaccurate to think about AI as an artificial intelligence if by artificial we mean something that is entirely designed and controlled by humans. AI is not artificial in this sense. So it's better perhaps to think about AI as an acronym for alien intelligence. Alien not in the sense that it's coming from outer space. It doesn't. Alien in the sense that this is a non-human intelligence. It's not even an organic intelligence. It operates on completely different inorganic lines. It therefore is able to make decisions and to invent ideas that would never occur to human beings or to any other organic animal. A famous example of this was given already back in 2016 when the AI AlphaGo defeated the human Go champion, Lee Sedol in Seoul. This game became justly famous not just because AlphaGo defeated the human champion. It was the way AlphaGo defeated Lee Sedol which shocked the world. Because AlphaGo, in order to win, invented new strategies for how to play the game that never occurred to any human player in thousands of years of the history of Go. Now as long as AI invents new ways to play games or new kinds of coffee, it doesn't seem very important. But very soon, AI will invent new military strategies, new kinds of money and even entire ideologies and religions. Consider money and finance more generally. If you recall the 2007-2008 financial crisis, you might remember that it was largely caused or started by something called CDOs, Collateralized Debt Obligations. CDOs were financial devices invented by a handful of mathematicians and investment wizards. These financial devices were so complex that they were almost unintelligible to most humans including politicians and regulators. And this led to an oversight failure and a global catastrophe. Now what happens if we now allow or even encourage AIs to invent new financial devices that will be orders of magnitude more complex than CDOs and that will be utterly unintelligible to any human being? Such devices could greatly improve financial efficiency and contribute to economic growth and they could become the bedrock of the global financial system. But what would happen to human politics when no voter or politician is able to understand and regulate finance anymore? And what happens if after a few years of unprecedented boom, the financial system suddenly busts and there isn't a single human being on earth that is able to make sense of what is happening? Now consider the implications for something like religion, especially let's take as an example religions based on texts like Christianity, Islam and Judaism. For thousands of years, Judaism granted ultimate authority not to human beings but to texts like the Bible and the Talmud. Flesh and blood rabbis nevertheless had immense power within Jewish society for the simple reason that their holy texts could never speak for themselves. They were mute. Whenever you had a question about the text, you needed a human interpreter. Rabbis are humans who know the way around many Jewish texts from the Bible to the latest commentaries on the Talmud. This is the source of their authority. Now over the centuries, so many Jewish texts have been composed that not even the most learned rabbi is able to read all these texts in his or her lifetime, let alone remember everything written there. But of course, this is something that an AI can easily accomplish. AI could quickly read all the Jewish texts ever composed, remember every word in every single one of these texts and also find patterns in the texts that have so far escaped human notice, human attention. And AI could therefore use its unprecedented scriptural understanding and knowledge to produce completely new interpretations of the holy word. And this of course is true of Christianity and Islam as well. In effect, AI can start functioning as a holy text that talks back to the faithful without any need of a human mediation. What happens to a religion of texts when the texts start talking? Now, since Judaism invests, as I said, authority in the texts, a flesh and blood rabbi cannot just ignore the scriptural arguments of an AI. And human rabbis will very soon find it very difficult to compete with AI's mastery of the Jewish canon. Even if AI is not granted formal rabbinical status, it is likely that at least some humans, including even some rabbis, would turn to an AI rabbi for scriptural guidance and inspiration. The result might be surprising new developments in very old religions or the emergence of entirely new religions. Now, these kinds of developments have enormous positive potential. They can help humanity in countless ways from inventing new medicines to preventing catastrophic climate change. But of course, AI also poses many threats. So let's move on to the second question of what is the threat of AI? The basic problem with AI is not that it is evil. The problem is that it is alien and therefore unpredictable and untrustworthy. At the heart of the AI race, there is a paradox of trust. Humans find it difficult to trust one another, especially as the world is becoming a collection of hostile fortresses. But the people who develop AI somehow manage to believe that we should, that we could trust the AIs. When I meet the people who lead the AI revolution, I normally ask them two questions. First, I ask them, why are you moving so fast when there are such huge risks? And the response almost everybody gives is the same. They say, we agree. We know that there are big dangers and it would be best to proceed with more care. However, if we slow down while our competitors in the other company or in the other countries won't slow down, they will win the AI race and the world will be dominated by the most ruthless people. We cannot trust our human competitors so we must move as fast as possible. Then I ask the second question. Do you think you could trust the super intelligent AIs that you're developing? And the same people who just told me they can't trust their human competitors suddenly assure me that they could trust the super intelligent AIs. And this is such a paradox because we have thousands of years of experience with human beings. We have a broad understanding of human psychology and biology, of the human craving for power, of the tricks humans use to gain power and also of the forces that keep the human pursuit of power in check. We have also made considerable progress in finding ways to build trust between humans. A hundred thousand years ago, people lived in tiny bands of just a few dozen individuals and couldn't trust anybody outside their band. Today, there are nations like Brazil with hundreds of millions of citizens and there are even bigger networks of cooperation linking all eight billion humans on the planet. Complete strangers often grow the food we eat and invent and develop the medicines that keep us healthy. Of course, we are far from completely solving the problem of human trust and there are still many tensions and conflicts and wars in the world, but with humans, at least we understand the challenge we are facing. In contrast, we have almost no experience with super intelligent AIs. We have just created the very first prototypes. We already know that even primitive AIs can lie, manipulate and adopt goals and strategies that were not foreseen by the human developers. We have no idea what will happen when millions of super intelligent AIs interact with millions of humans and it is even more difficult to predict what will happen when millions of super intelligent AIs interact with one another, creating AI societies, AI tribes. What kind of goals will they develop? What kind of tricks might they invent? True, at present, it is humans who develop the AIs so we can try to design them in a way that will make them safe for humanity, but remember that a machine is an AI only if it is capable of learning and changing by itself, making decisions by itself and inventing new ideas by itself. So no matter how human designs them originally, the AIs can then change in a radical and unpredictable manner and it's a huge gamble to simply trust that these super intelligent alien intelligences will always remain our obedient servants. The humans who despair of trusting other humans yet think they can trust these alien intelligences may be making a very, very big mistake. Now, I know that it's sometimes hard for people to understand how AIs might escape human control. Due to the influence of science fiction, when we talk about AIs escaping our control, people tend to imagine a scenario of killer robots running in the streets, shooting people. While something like this is actually already happening in some places like Ukraine and like Gaza, this is nevertheless the wrong image. If you want to understand how AIs can escape our control, don't think about killer robots, think about business corporations. What are corporations? Corporations like Microsoft or SpaceX or Petrobras are non-human agents. Though they don't have a body or a mind, these corporations can open bank accounts, can hire people, fire people, lobby politicians, and do many, many other things. Now, previously, this corporate agency was just a legal fiction, a fantasy of lawyers. Why? Because in practice, every decision made by Microsoft or Petrobras was actually made by some human being, not by the corporation. Every decision was made by some human executive, human shareholder, human engineer. But with AI, corporate decisions will increasingly be made by a non-human intelligence, a non-human agent. Imagine a corporation that has no human executives and no human shareholders, only AI executives and shareholders. This corporation can make billions and reshape the world. A practical question already today in 2025 is do we want, as a society, to allow such things? This is a practical question. Do we, as a society, allow AIs to open bank accounts, to manage corporations, to trade and invest, to hire people, to lobby politicians? So let us move to the last question. How can humanity flourish in the era of AI? To survive and flourish, we need to remember two lessons from history. First, power isn't happiness. We humans have been extremely good in acquiring power, but so far we have not been very successful in translating power into happiness. That's why today we are thousands of times more powerful than in the Stone Age, but humans are not thousands of times more happy. That's also why the most powerful people in the world, like Trump and Musk and Xi and Putin, are definitely not the happiest people in the world. Secondly, to survive and flourish, we should remember also that intelligence isn't wisdom. Humans are the most intelligent animals on the planet, and at the same time, we are also the most deluded animals on the planet. No other animal can believe the kind of nonsense that humans can believe. For example, my home region of the Middle East is burning in a horrendous war, fueled in part by the most ridiculous fantasies you can imagine. Humans there, on all sides of the conflict, seriously believe that the creator of the universe commanded them to fight and kill other humans, and they also seriously believe that the creator of the universe will reward them for their violence with everlasting joy and bliss in heaven. No chimpanzee will ever believe such nonsense, but many highly intelligent humans believe it. So power isn't happiness, and intelligence isn't wisdom. AI will be more powerful and more intelligent than us, but the power of AI doesn't mean it will bring happiness, and the intelligence of AI doesn't mean that AI will be free from delusions. In fact, a super intelligence might very well be super deluded. Therefore, the big challenge for business leaders and innovators is not how to create more powerful and more intelligent technology. Rather, the challenge is how to develop the wisdom necessary to use all this power and intelligence for good purposes. How to do it? The answer is quite simple. Build more trust between humans before developing super intelligent AIs. Do you know why planet Earth is nevertheless ruled by humans and not by chimpanzees? Not because we are more intelligent than the chimpanzees. Humans rule the world because we know better than any other animal how to build trust with strangers and cooperate in very, very large numbers. We have developed this ability for thousands of years. Unfortunately, just when we need human trust more than ever before, many of our most powerful leaders are busy destroying this trust. Instead of strengthening connections between all humans, they are busy building walls between humans. If humanity is divided, AI will escape our control and will enslave or even annihilate us. On the other hand, if humanity cooperates, we can together develop AI and other technologies in a safe and responsible way. Think about AI as humanity's common child. When we educate children, we know that children learn from what we do, not from what we say. If you tell your child, don't lie, and then the child observes you lying, the child will also lie. Similarly, if you're a tech tycoon developing a super intelligent AI and you're telling the AI, don't lie, and the AI then observes you lying, the AI will also lie. And if you tell the AI, be compassionate to people, but the AI observes you engaged in a ruthless pursuit of power, the AI will also grow up to pursue power ruthlessly without any compassion. This is why we must first solve the problem of human trust and only then develop super intelligent AI. If AI is developed in a cooperative way by compassionate and trustworthy humans, then the AI too is much more likely to grow up to be cooperative, compassionate, and trustworthy. And all of you have a role to play in that. You are leaders and innovators. Amidst the growing political chaos in the world, the business leaders and the tech innovators still have very considerable power to shape the world. So use your power to build trust between humans. Use your power to build a world which isn't just a card game between ruthless warlords. Build a world in which the dignity of everybody still counts. Thank you.