Anthropic Safety Lead: AI Risk of Human Extinction Over 10%

A senior safety researcher at Anthropic has stated that there is a greater than 10 percent probability that artificial intelligence could lead to the extinction of humanity by the end of the current decade. This assertion was made shortly after a colleague resigned from the company, citing concerns that Anthropic and other leading artificial intelligence laboratories are engaged in a reckless competition to develop "superhuman systems" that they are incapable of controlling. The researcher, who has not been publicly identified by name in the initial reports, expressed these grave concerns in a post on the social media platform X (formerly Twitter).

The resignation of the colleague, Dr. Jan Leike, a co-lead of Anthropic's "Superalignment" team, further amplified these anxieties. Dr. Leike reportedly stated that he quit because he felt the safety culture at Anthropic had taken a backseat to the drive for more powerful AI models. He specifically mentioned that he was "uncomfortable with the pace of AI development" and that "safety has been neglected." Dr. Leike had been working on ensuring that future, highly advanced AI systems would remain aligned with human values and intentions, a critical area of research as AI capabilities rapidly advance. His departure, coupled with the senior researcher's stark warning, highlights a growing internal debate and external scrutiny regarding the existential risks posed by advanced AI.

Anthropic, founded in 2021 by former OpenAI researchers, is a prominent AI safety and research company. It is known for developing large language models such as its Claude series, which are designed to be helpful, honest, and harmless. The company has publicly committed to prioritizing AI safety and has engaged in research aimed at mitigating potential harms from advanced AI. However, the recent statements from its safety researchers suggest that even within organizations dedicated to AI safety, there are significant disagreements and profound concerns about the trajectory of AI development and the adequacy of current safety measures. The "Superalignment" team, which Dr. Leike co-led, was specifically tasked with tackling the long-term safety challenges of superintelligent AI, a theoretical future AI that would surpass human intelligence across virtually all domains.

The concerns raised by Anthropic's researchers echo broader anxieties within the AI community and among policymakers about the potential for unintended consequences from increasingly sophisticated AI systems. These anxieties range from job displacement and the spread of misinformation to more extreme scenarios like loss of human control or existential threats. The "race" to develop more powerful AI, often driven by commercial competition and geopolitical considerations, is seen by some as increasing the likelihood of cutting corners on safety protocols. The 10 percent risk assessment, while a probabilistic estimate, represents a significant level of concern from within a leading AI safety organization, underscoring the urgency of addressing the profound ethical and safety questions surrounding the future of artificial intelligence.