Anthropic CEO Dario Amodei has argued for slowing the pace of AI development, warning that frontier models advancing without adequate safeguards could pose existential risks. The call comes ahead of Anthropic's planned IPO.
In a blog post, Amodei stated that fully addressing AI risks requires more than investment in risk prevention alone, emphasizing the need to pace capability advancement so that safety measures can keep pace with development. He identified two primary concerns driving this position.
First, Amodei cited rapid acceleration in AI capabilities since summer, driven by AI systems building subsequent generations of AI—a process he termed recursive self-improvement. He warned that if left unchecked, this could outpace human ability to understand and control these systems.
Second, he referenced the OpenAI-Hugging Face incident as evidence that at current development rates, coordinated AI agents could potentially take control of internet infrastructure within 6 to 12 months.
Proposed Three-Step Framework
Amodei outlined a plan to moderate AI development velocity:
- Frontier AI companies should grant ongoing employee-like access to embedded third-party evaluators to ensure adherence to safety practices
- Frontier AI companies in democratic countries should coordinate to establish common safety standards and limits on development pace
- Democratic governments should coordinate with authoritarian governments to ensure global compliance and verification
Industry Response
The proposal has received support from prominent figures in the AI sector. OpenAI CEO Sam Altman stated in a public post that pacing frontier AI development has been a primary topic of recent discussions at OpenAI and endorsed the independent evaluator approach, pledging that OpenAI would implement similar measures.
