AI engineers warn their own creations could kill us. Then their stocks go up.
Part of composite article AI Doomsday Warnings Hit Congress as Tech Giants Demand Rules They Once Fought View full article →
Tech leaders at major Western artificial intelligence companies say their programs could destroy humanity. Some debate whether the probability is higher or lower than 10%. These warnings have been repeated for years. Yet AI company stocks have not fallen. They have soared.
This raises an uncomfortable question: What system allows the people warning that a technology may endanger humanity to also have the strongest financial incentive to keep building it?
The risks of AI are real and serious. The technology could replace many jobs that support millions of families worldwide. It is unclear whether it will create enough new jobs for those displaced. It is also unclear whether it will significantly boost productivity.
Concentration of power is another danger. A few private companies now have deep access to the computer and security systems of major countries. This creates geopolitical risks. When a country hands control of a process to an American company — for example, managing its electricity grid — it also gives that company a weapon that could be turned against it.
AI also has a physical cost. It requires electricity, water, data centers, chips, networks, and huge amounts of materials. Its apparent immateriality hides a growing physical infrastructure that worsens climate change. All of these are real ways AI could end the world as we know it.
Yet in practice, we are drawn to a more dramatic idea: that AI will kill us in some bizarre, science-fiction manner.
There is something strange about AI engineers announcing that their creation might kill us. Except for a few terrorists, no one publicly claims they are capable of killing everyone. This contradiction only holds if we accept that AI, or part of it, has become an independent subject — like Frankenstein's creature — no longer under its designer's control. This is questionable at best.
As writer Richard Seymour has noted, AI's ability to act depends on infrastructure, permissions, and goals that humans have designed and deployed. If an engineer builds a model capable of cyberattacking companies and activates it without safety measures, the creator is responsible — not the model, no matter how sophisticated. Put more simply: AI has no consciousness. All its internal goals are programmed in advance. Even if unexpected behaviors occur, political responsibility still lies with whoever chose to deploy the system without controls. AI can kill us the same way nuclear bombs can — as an instrument of destruction. What matters is that behind it is someone who has politically willed that death.
Though engineers have not clearly explained how AI will kill us, I believe they genuinely believe what they say. We all see AI's enormous potential month by month. We know it can be a blessing or a curse depending on who uses it. We also know that users only access inferior versions compared to what exists at the technological frontier. Full functionalities are restricted. If we cannot use all stored knowledge to design a homemade biological bomb, it is because someone has blocked that function. That is good. But the program could reach those extremes.
When engineers warn that AI may kill us, they are also saying its "creative" and "innovative" potential has risen far beyond earlier stages. Under capitalism, that means its capacity to generate profit has multiplied. This explains the fever of multibillion-dollar investments in AI companies. Yet it is unclear whether productivity will rise for the economy as a whole. AI may not positively affect growth. Both things — weak productivity effects and massive investment — cannot last. Everything points to a huge bubble, funded by idle international capital and sustained by explosive announcements.
It would be far more reasonable to discuss the true, immediate risks of AI deployment. For example, its growing use as surveillance technology in workplaces and civilian life. The worst dystopias could come true if our work time — and our freedom outside it — is monitored by AI agents programmed by capital. AI has never been an autonomous force except in concept. It is a product with owners. Those owners are billionaires who do business with the world's most powerful governments and corporations. This makes AI an instrument of class.
Ask yourself: What did you think when you read this headline? Did you assume humanity faces an existential threat from a disembodied, abstract, all-powerful force — that like Dr. Frankenstein, we will die at our own creature's hands? That is the dominant narrative. But perhaps you have already guessed my conclusion: The real problem is not that humanity will lose control of AI. The real problem is that control has always belonged to a tiny minority. Humanity — now meaning the majority — is simply at the mercy of a small number of billionaires. The solution to avoiding the destruction of our world is to limit the power of those who control artificial intelligence.