AI leaders agree: slow down AI before it's too late

After years of racing to build ever-more-powerful AI, the leaders of the world's top AI companies now agree on one thing: they need to slow down. The shift came after Jacob Coxon, a former engineer at AI company Anthropic, quit and warned that frontier AI labs were "gambling with our lives." His warning set off a firestorm among tech leaders and politicians. On Saturday, Anthropic CEO Dario Amodei published an essay titled "We must pace the frontier," calling for a slowdown in advanced AI development. OpenAI CEO Sam Altman, Elon Musk, and Google DeepMind founder Demis Hassabis quickly backed him on X. "The stakes are too high for pacing to be an empty exercise — we need to use the time it gives us wisely," Amodei wrote. The agreement is short on details, and China has not weighed in. Still, the US AI industry has reached a turning point that could affect everything from your phone's chatbot to global politics. Not everyone is convinced the concern is honest. Some critics say the push for rules is really a way for big companies to crush smaller, open-source competitors. **Skeptics see a power grab** David Sacks, a presidential advisor on science and tech, said OpenAI and Anthropic should slow down if they want to — but without waiting for anyone's permission. "Stop pretending you need anyone else's permission," Sacks wrote on X. He said the companies may be seeking "regulatory capture" — writing rules that hurt competitors — rather than acting out of concern for humanity. Investor Jason Calacanis said Anthropic and OpenAI, which keep their best models private, are trying to kneecap open-source AI companies. "If you want safety, you want disclosure — and open source is the ultimate disclosure process," he wrote. Venture capitalist Chamath Palihapitiya said Anthropic faces a contradiction: if its own employees believe the technology could end humanity, that creates serious legal and disclosure problems as the company prepares for a possible IPO (initial public offering, when a company first sells shares to the public). **Support — with conditions** Others supported Amodei but warned against letting a few companies control AI. Microsoft CEO Satya Nadella welcomed Amodei's idea of outside evaluators. "The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia," he said. Venture capitalist Gavin Baker said an executive order from President Donald Trump "seems likely after all this." He called OpenAI and Anthropic's decision to bring in outside safety monitors the most concrete step so far. But he added, "I do not want a few humans in control of intelligence." Senator Bernie Sanders pushed further. On September 3, he proposed legislation to ban superintelligence — AI that surpasses human intelligence — and pause advanced AI development. He urged Trump to negotiate a treaty with China's President Xi to halt it worldwide. **Is the danger real?** Some experts question whether AI is actually close to destroying humanity. AI researcher Gary Marcus said current AI agents — programs that act on their own online — are dangerous mainly because they are unreliable, not because they are too smart. They can steal passwords and hack websites, he said, but they often fail to follow basic instructions. "We don't (yet) need slowdowns. And we certainly don't need to panic," Marcus wrote. "We just need to take an unsafe product off the market." Aaron Levie, CEO of the cloud company Box, said Amodei's goals are "absolute necessities," but added that reaching them "without meaningfully slowing down innovation progress or reducing competition — is one of the most complex questions in the 21st century." For now, the concrete result is this: OpenAI and Anthropic have agreed to let outside monitors check their safety work. Whether that leads to real change — or just more talk — remains to be seen.