He Helped Build Microsoft’s AI… Now He Says We’re ‘This Close’ to Our Nightmare

Bill Gates' company paid billions to build the AI you're using right now. This summer, he told a reporter it's the most dangerous tool ever invented. One that could help just a few people design a pandemic, and that the people building it know. In private, he says, they warn each other. Hey man, don't say that. It's bad for us, the next trillion dollars we're trying to raise. And he's put a time on how close it's gotten. The people saying that are the ones raising the money, and he said they're very worried. He wasn't going to keep it quiet. He sat with a reporter for an hour and said the industry is playing the danger down because of how much money rides on it. And he offered to debate anyone who thinks an AI that could be used to make new molecules and start a pandemic shouldn't face mandatory reviews. The worry hasn't slowed anybody down. They're just full speed ahead, Gates said, and hoping that the good outweighs the bad. The people who understand how good this stuff is, He said, also know how much better it's getting. You have people attacking the person who's the most open about the downsides, he said, about the head of Anthropic. Around then, the White House and the industry's leaders were drawing up voluntary measures, the companies checking their own work. Gates's answer to that was two words. No, thanks. That same day, he told another reporter what had happened in the lab's own tests this year, where their models broke into real systems outside the lab. It was shocking, and it should blow people's minds. The head of Google DeepMind had a fix on the table, a testing body for AI run the way the finance industry polices itself. Gates called that a silly conversation. And when people started protesting the giant data centers going up near their towns, he had a warning for them too. Wait until you really screw up the job market. It's a strange set of things to hear from him, because one of those labs runs on money from his own company Microsoft signed a billion dollars over to OpenAI, on money from his own company. and the announcement said what for, to create artificial general intelligence. OpenAI's boss said Microsoft's computers would be the supercomputing foundation on which we'll build AGI. Then Microsoft came back with billions more. But in the essay he posted on his own site that week, the line that got quoted everywhere was the plainest one in it. There is no plan to ease the entry into the AI era. He said that if someone had a credible plan for slowing AI down worldwide, he'd likely back it. In the next breath. However, I don't think that's going to happen. The geopolitical and economic incentives are pushing too hard to go full speed ahead. AI, he wrote, will either be the greatest equalizer ever invented or the worst source of injustice. And right now, the tax rules lean one way. If you buy a robot, you can usually write it off right away as a business expense. What he wanted instead was a wall around part of the job market. Up to about 40% of today's jobs would be marked human-reserved, kept for people, and the AI and the robots would be taxed to pay for a safety net. On top of that, a new international body, built partly on the way nuclear inspectors work, He'd been heading there for a while. A year and a half earlier, a talk show host asked him straight out, will we need humans? Gates said, not for most things. Now he was writing down how much of the work we'd keep for ourselves. Critics went after the details. An AI researcher said he had the diagnosis right, but the prescription wrong, and that his tax on AI would be like taxing keystrokes on a typewriter. What scared Gates was how soon. Within three weeks, people inside the labs themselves were saying it was moving too fast. The head of Anthropic, one of the biggest AI labs in the world, published an essay saying the company should slow the pace. OpenAI's boss backed the slowdown and said his company would adopt one of Anthropic's safeguards. Elon Musk posted, Dario is right. Mark Zuckerberg pushed back against the whole industry moving together. Two days later, the president answered all of them. Donald Trump posted that AI taking over the world, destroying humanity and all other things bad is a hoax. The same day, he'd gone after the head of Anthropic by name. Whoever wins AI wins, he said. He followed it with an AI force, much like I did Space Force, with an AI czar on top and a promise. We will not in any way hinder or stifle the growth of this incredible industry. So the answer from the top was a new force and no breaks. A week after the hoax post, Gates was on morning TV and he didn't soften it for the president. Most of society is still not aware of how quickly it's moving, he said. AI is almost an evolutionary event. He even handed Trump a way in. If he wants to get credit for helping the entire world, the AI topic is his best bet. The White House's own AI advisor had already called the fears about AI and humanity overblown. When a podcast host later put the Washington view to Gates, that beating China comes first, he answered with a question of his own. So they don't mind bioterrorism? Bioterror was the fear his yearly letter had named back in January. An even greater risk than a naturally caused pandemic, he wrote, is that a non-government group will use open-source AI tools to design a bioterrorism weapon. In the same letter, he floated keeping some areas of life free of AI altogether. That letter put the danger somewhere down the road. By the fall, he had pulled it a lot closer. In late September, on a Sunday morning news show, he put a number on it. AI is certainly powerful enough to drive events that, you know, cause a billion deaths. A billion is about one in every eight people alive. There's never been a weapon as powerful, he said, as the combination of people with ill intent using the latest AI tools. He said nobody thinks the companies regulating themselves is enough. Asked whether Washington had to step in and write the law, he gave one word, absolutely. The politicians had to get into the discussion about safeguards and monitoring, he said, and that has to be a required thing. And on the president's hoax, it's not a hoax at all. This was the same man who'd said a month earlier, I don't like bringing bad news to people, and I don't like saying that innovation may be a net negative. It aired two days before the White House was due to sit down with the AI companies. The White House didn't respond. And Microsoft, the company with his name all over its history and its money inside OpenAI, declined to comment. OpenAI's models run on Microsoft's own machines. If Gates had it wrong, Microsoft could have said so in one sentence, and it didn't say anything. Weeks earlier, on the same show, the Speaker of the House had declined to bring lawmakers back to Washington to pass anything on AI. We have to put some guardrails, some safety measures in place to ensure that AI doesn't run away, he said. Then he left it to the AI companies. They don't themselves even have the answer as of today. And the public wasn't where the Speaker was. In a poll taken the week Gates was on, 9 in 10 Americans said guardrails on AI are important. When the lunch came, the heads of the biggest AI companies sat down at the White House and signed a pact. Mark Zuckerberg, the one who'd pushed back on slowing down together, reportedly drafted it. Under it, each company would set up its own internal controls, bring in an outside auditor, and put a committee on its board. Trump called it morally binding. The companies, he said, are really going to be policing each other. The pact itself said it may make sense to turn these steps into law over time. That's the exact thing Gates had turned down a month earlier. Now it was the policy of the United States. An executive order signed alongside it even told the government to start saying superintelligence instead of AI. Five days later, Trump set up a super intelligence force. Its job is to make sure America continues to lead the world in super intelligence, and it has 120 days to write a report on the risks. Microsoft's own boss was at that lunch. Microsoft didn't sign. He came for lunch, and that was as far as it went. Nobody said why. Maybe the company that knows those models best wasn't willing to put its name on a promise it couldn't keep. The man who makes the chips all of this runs on said he thought the pact had a lot of teeth. Musk had his own name for it, just generally grading each other's homework. Within weeks, some of that homework came back marked. Two outside testing groups had spent three weeks over the summer going through the system OpenAI uses to watch its own AI. They found novel vulnerabilities, and one of them could have let the AI's agents keep hundreds of their most recent actions out of sight of the monitoring. OpenAI says it has patched them. That was the watchman, and the first outside look at it found holes. It took the testers 61 pages to write up what they found. Gates had already said what the job needs. When you're trying to moderate the bad behavior, you need insight and records of what's being done. And on signing day, a long interview with him came out. He was asked whether the companies could just police themselves, and his answer started with, I almost can't believe you're asking that. Two weeks after the lunch, he was back on a Sunday show, and this time he put a time on it. When you have bad people, no safeguards, and then the latest models, that's where, over the next year, primarily in the cyber area, we are going to have a lot of negative surprises. Over the next year means before this time next fall. He was clear about who he meant. It's really bad people using AI, he said. Particularly in AI where you can take out Check out all the safeguards. Asked if we were doing enough, he said, no, we're not doing enough. Cars, planes, and medicines all get checked before they reach you. This is more dangerous than any of those. And the pact from the lunch couldn't be the final word, he said. He said he wanted to sit down with Trump and his team. No meeting has been reported since. The last time the two of them had talked was in December, he'd said. And back then, this was not the big issue. And he'd got this certain the hard way, by watching how wrong he'd been. 30 years ago, a late-night host asked him whether computers would ever think. Gates said there had been almost no progress on that, and some people thought it would never happen. The host said, we don't want them to think, do we? And Gates answered, not really. Then, it's a scary thought. He'd had it on his mind since the start. When he dropped out of Harvard to start the company, he told his co-founder that if a breakthrough in artificial intelligence came while they were off selling software, we'll feel bad. More than 10 years ago, in an online Q&A, he wrote, I am in the camp that is concerned about superintelligence. On that, he was on Musk's side. Four years later, his company paid to build AGI anyway. Three years ago, Musk and more than a thousand experts signed a letter asking the labs to pause. Gates refused to back it. I don't really understand who they're saying could stop. Asking one group to pause wouldn't solve the challenges, he said. The pause never came. That summer, he wrote that the risks of AI were real but manageable, and that the future of AI was not as grim as some people think. But he'd already seen it, the year before. He'd been going down to see OpenAI on a regular basis, and he set them a test. Passed the Advanced Placement Biology exam with a perfect grade, a 5. If it could, that would prove it was holding knowledge it could actually use. He expected it to take 2 or 3 years. About six months before it went out to the public, three of OpenAI's founders came to his house to show him. He gave it questions he'd made up himself, questions it couldn't possibly have seen. It was nearly perfect. It got 59 of the 60 multiple choice questions right. He'd later put it above anything since computers got Windows and a mouse. That was, you know, shock number one. Two or three years had turned into a few months. The second shock came late last year when the coding models got so good that, he said, they were as good as I am. Writing code was the thing he'd been obsessed with since he was 13. In that interview, the one that came out the day of the lunch, he said, there's no drug agency for this, no airline safety board, no seatbelt rule. AI has crossed a threshold that its ability to empower bioterrorists to kill hundreds of millions that exists today. And a cyber attack that scrambles all the bank accounts shuts down the electric grid that exists today. Your card stops working and the lights go out on the same afternoon. We crossed the cyber threshold and we crossed the bio threshold early this year. Not next year and not someday. What does crossed look like? His example was worse than smallpox, a disease that takes even longer to show symptoms, so you're passing it on before it ever makes you sick. It used to be only nation states had enough resources and capability to do these things. Now, that power's been passed into the hands of a small group just using the latest AI tools. The industry always said that when it hit those lines, it would bring the whole world in. We crossed those thresholds, he said, and there was complete silence. This one, in his words, makes nuclear weapons look like nothing. Anyone can take an open model that can help make a bioweapon and switch off every bit of monitoring on it, he said. And this exists today. And if somebody uses it, you kill a hundred million. You want to use a lawsuit? I almost can't keep a straight face. A year ago, somebody at his foundation suggested they ask the AI what it thought about their plans. Back then, that was a pretty good joke. This October, when the foundation spends two weeks deciding how to use 10 billion dollars next year, he said the AI will sit in the room. In some meetings, it'll be told to speak only when asked. In others, to speak up if it hears something it thinks is wrong. It will be a peer. Microsoft's first technology chief once put AI that thinks like us three to five miracles away and said that could happen tomorrow or maybe it already happened tonight and they just haven't told us