AI leaders sound the alarm over runaway risks of the technology: Analysis

Aljazeera
59 followers
86 Video Views·Sep 13, 2026
The CEO of one of the world's largest artificial intelligence companies says
research into the technology should slow down. Anthropics' Daria Amodei says that AI systems
are becoming too sophisticated for humans to control and his call was echoed by his rivals,
OpenAI's Sam Altman and Elon Musk. Amodei added recent incidents when AI systems known as agents
broke out of their controlled environments and hacked into external systems by deceiving humans
to evade being shut down. He warned that such so-called escapes by agents could soon take over
the internet, causing hundreds of billions of dollars in damage. In the open letter, Amodei
said his company was ready to give employee-like access to independent oversight groups that
could watch for such risks. He added that intentional cooperation would be necessary
to make sure no country develops a dangerous AI system, saying treaties limiting nuclear weapons
provide a model for such an agreement. Right, well, let's speak to Toby Walsh, the Chief
Scientist of the AI Institute at the University of New South Wales, joining us live now from Sydney.
Thank you, Professor. Professor, you've published a book called God, AI, Boom or
Doom? What to Expect When Machines Outsmart Us. So what do you make of this extraordinary
admittance by one of the biggest AI companies here in the world that the very beast that they
are helping create carries serious risks, including killing humans? Because, Toby,
I'm personally finding it hard as a member of the public to decipher sort of what the motive is here.
You're right. It's rather conflicting messages that we are getting. I mean, it's very clear.
We've got lots of evidence now that there are significant risks. I think, as I talk about in
the book, it's a bit fantastical to imagine that it's going to kill us all. That is a really very
high bar. But we are seeing dangerous capabilities that could cause significant harms in terms of,
you know, taking over the Internet. Actually, I was much more worried by a report that came
out from Anthropic earlier this week, looking at a number of bad actors that they'd actually
identified and thrown off their platform, that were using their bot to try and, for example,
do bioweapon research. I think we should be very worried that we're putting these
capabilities in the hands of bad actors. Right. So what's happened in recent weeks
to sort of trigger this alarm? It seems to be a lot in the news at the moment. And what's
driving the extraordinary speed in which AI is developing?
Well, it's clear what's driving it. Lots of money. We're spending over a billion dollars a day,
a quarter of the world's R&D budget on artificial intelligence. We've never put so much money into
one technology ever before. That concentration of money is helping to drive full progress.
And the other thing, the other technical thing that we've discovered in the last couple of weeks
is that when these bots are working together, we're talking about swarms of a thousand bots
coordinating together. They're much more powerful, much more capable than they are on their own,
and much more persistent. So we're discovering that, just like humans are much more capable
when they work together, that the AI is much more capable when it works together. And that
is taking us perhaps into more dangerous territory. Right. Now, I just want to go back to 2023.
Geoffrey Hinton nicknames the AI godfather. Now, he quit his job at Google to be able,
he says, to speak more freely about the dangers here. And he says, including the fact that AI
has developed human-like intuition. So if that's the case, and it's, as you said, progressing
so fast, are companies and world leaders going to get their act together anytime soon?
Well, that's a good question. The problem here is that we're in this race dynamics where each of
them sees the economic benefit of being the first to succeed, and also that they feel that they need
to be the one first before some less well-inclined actor succeeds. So the question is how we can
break out of this race dynamics. And the good thing is that all of them now are starting to say
the same thing, that they need more oversight, that they need the government to step in, that
they need regulation. So I'm very much hoping that governments will actually pick up the invitation
to say, well, we need more independent oversight, we need more regulation of this space.
Just as we do for drug companies when they're working on dangerous pathogens, there's significant
oversight. Or, you know, airline manufacturers with planes that could kill lots of people,
there's significant regulatory oversight. And this industry now, I think, has reached a stage
where we need that as well. Right, and you're speaking about that sort of regulation and
legislation. So everyone, it seems, is sounding the alarm bells. You had the UN chief talking
about this just last week. But at the same time, the British government has just rejected the idea
of creating a so-called kill switch to stop a dangerous attack from rogue AI. And it seems
there is that sort of resistance in the US as well. Why is that? Well, there's this fear that
regulation will stifle innovation. For example, that they'll let China win this race. I mean,
it is a bit of global politics in play here. How do we ensure that everyone does the right thing?
And that's going to be difficult. It wasn't easy with regulating nuclear weapons. It's not going
to be easy with regulating AI. But I think there's growing pressure from the public and a
growing acceptance from many politicians that we can't keep sitting on our hands, that we have to
do something, and from the leaders of these companies who are saying it very vocally themselves,
that they would welcome regulation so that it's not a race to the bottom.
I mean, Toby, it seems that we're putting a lot of trust in sort of a bunch of tech bros
and billionaires, and perhaps that may change soon. Thank you very much,
Toby Wolfe, Chief Scientist of the AI Institute at the University of New South Wales.
