AI Researcher Claims Anthropic Exit Over Existential Risk

AI researcher Jacob Coxon, who claims to have worked at both OpenAI and Anthropic for three years, has published a viral post on X detailing his resignation from Anthropic due to concerns about the rapid, irresponsible development of artificial intelligence. Coxon asserts that both organizations are "racing straight to self-improving superintelligence and gambling with our lives," a sentiment that has resonated widely, with his post garnering nearly 79 million views. He expressed that the current trajectory of AI development could lead to the creation of "superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources," potentially to the detriment of humanity. Coxon stated that some individuals within the AI field "earnestly believe that it could kill us all by the end of the decade," emphasizing that this is not a hyperbolic marketing tactic but a genuine concern among developers. He elaborated on the internal dynamics at these companies, noting that "at OpenAI, many have not deeply internalized the civilizational stakes." Conversely, he suggested that "at Anthropic, the stakes are well understood, but they are locked in a race to get there first." This competitive drive, according to Coxon, stems from a belief that if they do not lead the development, no one else will act responsibly, despite the inherent risks. Coxon characterized this participation in an AI "arms race" as "a hubristic gamble that should not be launched from a private company’s Slack." He urged other AI developers to critically examine their roles in the potential creation of a tech-based doomsday scenario. In his concluding remarks, Coxon implored lab researchers to contemplate the practical implications of the coming years, questioning whether they wish to initiate a superintelligent reinforcement learning training process that could have catastrophic consequences. The post highlights a growing debate within the AI community regarding the ethical considerations and safety measures surrounding the pursuit of advanced AI capabilities, particularly the development of self-improving superintelligence. The concerns raised by Coxon echo broader discussions about AI safety and the potential for unintended consequences as AI systems become increasingly powerful and autonomous. His experience and public statement bring to the forefront the internal anxieties and ethical dilemmas faced by individuals working at the forefront of AI research and development, especially concerning the long-term implications for human civilization. The viral nature of his post indicates a significant public interest and concern regarding the future of AI and its potential impact on society.