OpenAI has informed staff that it would consider slowing the pace of development in advanced artificial intelligence, with CEO Sam Altman extending the same proposal to rival labs including Anthropic. The announcement comes as Anthropic disclosed that researchers linked to state institutions attempted to use its Claude models for biological weapons research.
OpenAI stated it has already paused some internal training runs and eased parts of model development over safety concerns. Altman acknowledged that participation in any voluntary slowdown would not be universal, as not every AI company would agree to such an arrangement.
Industry Precedent and Regulatory Pressure
A similar call for industry coordination on AI safety came in a September 6 post by OpenAI chief scientist Jakub Pachocki, who urged AI firms to coordinate "to slow down future development as needed." Pachocki expressed hope that voluntary slowdowns would "become commonplace until shared safety bars are established."
This proposal echoes a March 2023 open letter signed by Elon Musk, Steve Wozniak, and more than 1,000 others calling for a six-month halt to advanced AI work. However, those labs did not follow through with the moratorium they jointly agreed to.
Security Incident and Bioweapon Concerns
Anthropic released a 154-page threat intelligence report on September 10 detailing five instances of researchers attempting to use Claude's capabilities to support biological weapons work. According to the report, these attempts originated from "unsupported regions" and involved efforts to circumvent location restrictions and disguise the purpose of the research. Anthropic said it removed the accounts involved but did not name specific countries or institutions.
Internal and External Concerns
Growing alarm from inside AI labs has intensified the debate. On September 8, researcher Jacob Coxon resigned and accused both OpenAI and Anthropic of "gambling with our lives." Two days later, former staff from Anthropic and Google's DeepMind raised similar concerns and called for greater transparency.
A late-July petition to create a mechanism for slowing development gathered signatures from more than 1,000 workers across major AI companies.
Regulatory Developments
OpenAI is now pressing Congress for mandatory, capability-based safety rules covering testing, independent evaluations, cybersecurity, and incident reporting. Chris Lehane, OpenAI's Chief Global Affairs Officer, stated that "the prospect of AI-accelerated AI development demands more than voluntary commitments."
California has moved to address the concerns. Governor Gavin Newsom signed SB 813 and AB 1405 on Wednesday, with a companion bill, AB 1864, aimed at guarding against AI-enabled biological threats. In Washington, Senator Bernie Sanders introduced legislation to pause advanced AI development and ban artificial superintelligence.
