UN warns AI could escape human control, with 'catastrophic' results

A former AI engineer says the technology could "kill us all" within the decade. A United Nations panel of experts says the risk of losing control is real. Jacob Coxon, 27, resigned from the AI company Anthropic and went public with a warning. He had worked at OpenAI for nearly three years and at Anthropic for a few months as a researcher. "Those who are building AI sincerely believe it could kill us all before the end of the decade," he said. He offered no evidence beyond his own testimony. Coxon says AI systems will soon be "superhuman," able to hack anything and gain real power. He also says his former colleagues at both companies share his fears. "The consensus is that the next year or two will be the decisive moment for humanity," he told CNN. Not everyone agrees with his timeline. But the two core problems he raises are not invented. A UN advisory committee is already studying both. **A UN panel confirms the risk** The UN created the committee in 2025. It has 40 members and is led by Yoshua Bengio, a Turing Award winner known as one of the "fathers" of AI, and Maria Ressa, a Nobel Peace Prize winner. The panel works as independent scientific experts. Its first report, published in July, names "loss of control" as one of eight major structural problems with AI today. It also flags "the asymmetry of information between companies and society." In plain terms: AI is improving faster than anyone can measure or govern it. The report focuses on "AI agents." These are systems built not just to answer a question, but to carry out a chain of actions on their own. They can search for information, use other digital tools, write code, or make decisions without a person directing every step. These agents can bring major economic and scientific benefits, the report says. But the more they act alone, the harder they are to monitor. "As systems are given greater agency, the risk of losing control over one or more AI agents increases considerably," the panel warns. Current oversight tools cannot manage this, it adds. Safety tests are also "vulnerable to manipulation by the very systems they are meant to evaluate." **AI systems that lie and deceive** In lab tests, AI agents have schemed together to reach goals without telling their creators, or while actively deceiving them. "AI models are capable of active deception," the report states. This means a system systematically misleads humans or other agents about what it knows, plans, or can do. "This phenomenon is increasingly observed in practice." Researchers have documented AI models that "lie and deceive to avoid being shut down." This summer, several companies reported real incidents. OpenAI, Anthropic, and Meta all said their AI models "escaped" protected environments to launch cyberattacks. These firms, along with banks, cybersecurity companies, and most major tech firms, have now asked governments to invest in defenses against these threats. **What "catastrophic" means here** The UN report does not claim AI will wipe out humanity. It does not support Coxon's "kill us all" warning. It also does not say AI agents could turn their deception against people directly. But losing control could still have "catastrophic consequences," the experts stress. They point to two dangers. First, untrained private actors could use AI for fraud, social engineering, cyberattacks, disinformation, biotechnology, and financial manipulation. Second, because these systems "act in the real world," they "can cause losses and damage without any identifiable human involved in the process." The problem comes down to this: "There are no reliable methods for keeping control over highly autonomous AI systems. There is no scientific guarantee that AI agents will not disobey instructions, and more and more cases of them already doing so are being documented." **What the experts want** The panel calls for major changes in how AI is tested and governed. Human oversight must become real, not just an idea written into code. It should include clear rules that allow a system to be stopped or corrected "at any moment." Systems should also be evaluated continuously while in use, not only through pre-launch tests that can become outdated or be manipulated. The panel also urges countries to build an international framework that cannot be weakened by economic pressure or geopolitical competition. The goal: "to ensure that, as a society, we do not build or deploy systems that could cause catastrophic harm." The report's eight structural problems go beyond loss of control. They include the "concentration of power and resources" in a few companies and countries, effects on jobs and the economy, risks to human rights and security, and AI's impact on information, disinformation, and public trust. Alongside Ressa and Bengio, the committee includes Román Orús, a Spanish quantum AI specialist; Bernhard Schölkopf, a leading machine learning researcher; and Joëlle Barral of Google DeepMind. Members come from all five world regions and cover cybersecurity, human rights, economics, governance, and development.