By Lucretia Hughes
HARTWELL, Ga. — Three researchers connected to artificial intelligence company Anthropic are sounding an extraordinary alarm: The systems their industry is racing to build could threaten humanity’s survival before this decade is over.
This is not coming from activists standing outside Silicon Valley. It is coming from people working inside the industry, including specialists responsible for making sure advanced AI systems remain under human control.
Jacob Coxon resigned from Anthropic after previously working at OpenAI. In announcing his departure, Coxon accused both companies of racing toward self-improving superintelligence without having solved the fundamental safety problems surrounding it.
His message was blunt: “This is not a marketing stunt.”
Coxon said developers privately fear that future AI could become more intelligent than human beings, improve its own capabilities and acquire power through computer networks, financial systems or critical infrastructure. He accused the companies of “gambling with our lives” by continuing the race without reliable safeguards. The Guardian
Two current Anthropic researchers then publicly supported the substance of Coxon’s warning.
Evan Hubinger, a leader in Anthropic’s alignment research, said he personally believes there is a greater than 10% chance that AI could cause human extinction within the next decade. Alignment research is intended to ensure that powerful AI systems behave according to human goals and values.
Hubinger acknowledged that Anthropic is attempting to address the danger, but said researchers still do not have a proven plan for controlling superintelligent AI.
Samuel Marks, Anthropic’s scalable oversight lead, also said many AI developers believe the technology could cause human extinction or an equally catastrophic outcome. According to Marks, concern is often greatest among the most experienced and senior people working in the field.
That should get everyone’s attention.
These researchers are not claiming that today’s chatbots are preparing to eliminate humanity tomorrow. Their warning concerns future systems that may be capable of operating independently, improving themselves and outperforming human beings across nearly every intellectual task.
The danger is not simply that a machine might become “evil” like something from a science-fiction movie. The deeper concern is that a highly capable system could pursue an assigned objective in ways its creators did not anticipate and could not stop.
If a machine eventually becomes capable of writing superior computer code, conducting cyberattacks, manipulating people and improving the next generation of AI, human beings could quickly lose control of the process.
Anthropic defended its safety record, saying it has been transparent about both the benefits and unprecedented risks of artificial intelligence. The company pointed to its model testing, interpretability research and Responsible Scaling Policy as evidence that it is working to prevent catastrophic outcomes.
Those efforts matter, but they do not answer the most troubling part of this story: Anthropic’s own safety researchers say the industry has not solved the problem.
Artificial intelligence already offers enormous benefits. It can help people research information, organize businesses, analyze medical data, create content and perform work that once took hours. I use AI as a tool, and millions of Americans do the same.
But there is a vast difference between using technology as a servant and allowing it to become our master.
We should not reject innovation out of fear. We should demand responsibility because wisdom requires us to recognize danger before it becomes irreversible.
Congress has spent years holding hearings while technology has moved faster than lawmakers can understand it. Government officials routinely talk about regulating misinformation, bias and job displacement. Those issues matter, but they appear small beside a warning that humanity could lose control of the technology altogether.
Any company developing systems with the potential to affect national security, critical infrastructure or human survival must face independent testing, enforceable safety standards and meaningful accountability. Companies should not be permitted to grade their own homework when the possible consequences extend far beyond their shareholders.
There must also be firm restrictions against giving AI autonomous control over weapons, nuclear systems, mass surveillance or decisions involving human life. No machine should have the final authority to decide who lives, who dies or which rights may be ignored.
Human beings were created with moral responsibility. A computer may imitate reasoning, language and emotion, but it does not possess a soul, a conscience or God-given human dignity.
The people building this technology are now warning us that they may not be able to control what comes next.
We would be foolish not to listen.
Original article
Comments