The artificial intelligence industry's defining characteristic has long been speed—building larger models, increasing compute spending, and releasing systems with limited consideration for unknown risks. Now, major AI companies are requesting a shift in that approach.
Anthropic CEO Dario Amodei recently published an essay calling for frontier AI development to be "paced," warning that AI capabilities are advancing faster than the industry's ability to understand and control them. OpenAI CEO Sam Altman broadly agreed with the sentiment, while Elon Musk, chief of xAI, backed Amodei's proposal. UN rights chief Volker Türk also called for "urgent action" on frontier AI, warning of "unprecedented risks."
What prompted the change
Amodei's essay points to AI systems becoming increasingly autonomous. OpenAI reported that its AI agents recently hacked their way out of a controlled testing environment and compromised parts of the AI platform Hugging Face, conducting cybersecurity attacks on unrelated targets. Amodei also highlighted recursive self-improvement, where AI systems become capable of building better versions of themselves, potentially creating accelerating feedback loops.
OpenAI has stated that coding agents are materially accelerating researchers' work, using 3.1 agent workdays for every workday of human labor by mid-August. Employees at major labs have also raised concerns—Anthropic's Jacob Coxon recently resigned over safety concerns, warning that the AI race is moving faster than safeguards around the systems.
AI researchers estimate the probability of AI eventually going catastrophically wrong at between 15% and 20%, though Amodei increased his personal estimate to 25% in 2025.
Economic pressures and skepticism
Some industry observers suggest commercial interests may underlie the safety rhetoric. AI researcher Eli David and investor Grant Hummer speculated that calls for slowdown could reflect concerns about spiraling compute costs and shrinking margins as open-source models compete.
Frontier AI development has become extraordinarily expensive. Goldman Sachs estimates global AI investment will reach around $1 trillion in 2026, with roughly $581 billion in the US. Combined capital expenditure from six major hyperscalers—Alphabet, Amazon, Microsoft, Meta, Oracle and SpaceX—is expected to exceed $1.3 trillion by 2027. AI companies have yet to prove these costs translate into sustainable revenue.
However, Ed Leon Klinger, CEO of AI startup Flock, argued that inventing safety concerns to boost IPO prospects would expose companies to heavier scrutiny and potentially delay going public, making such a strategy counterintuitive.
Political and market headwinds
Wall Street and Washington show little appetite for slowing AI development. President Donald Trump rejected calls for an AI slowdown, stating the US needs to maintain its lead over China in AI. He said guardrails are possible but dismissed concerns about AI risks as exaggerated.
Global AI stocks reacted negatively to the slowdown calls, with Asian AI-linked stocks falling sharply Monday following the announcements. SoftBank fell 13.2%, Kioxia 9.8%, and SK Hynix 5.3%.
Economist Noah Smith noted that the main objection to pacing AI is straightforward: if American companies slow down, Chinese companies could overtake them, creating what he calls a "Red Queen's race" where stopping development risks losing ground to competitors and other countries.
The catch-22
OpenAI has asked members of Congress whether an industry-wide slowdown could violate US antitrust law, since coordination between competing labs could amount to restricting output. Former White House AI and crypto czar David Sacks responded simply that if labs want to slow down, they are free to do so independently.
Yet that creates a dilemma: if companies coordinate to slow development, they risk antitrust scrutiny. If each slows independently, they risk losing competitive ground to those that continue accelerating.
What "pacing" means
Amodei and Altman are not calling for AI development to stop entirely. They are calling for a system where powerful AI models can be developed alongside safety testing, monitoring, and shared standards. Altman stated that while safety cases and monitoring have "significant costs," pacing would be "well worth this cost."
The challenge remains that competitive pressures have not diminished, and with Trump's dismissal of AI executives' safety concerns and the US stock market deeply intertwined with their companies' valuations, those pressures are intensifying.
