AI Researchers Inside Labs Call for Slowdown

A growing chorus of AI researchers, including those working within major AI development labs, are publicly advocating for a slowdown in the pace of artificial intelligence advancement. This shift in sentiment is notable because previous warnings from external safety organizations and independent researchers had largely failed to impact the rapid development trajectory of increasingly intelligent and autonomous AI systems. The recent surge in internal dissent appears to be driven by escalating concerns over the potential risks associated with the pursuit of superintelligence.

Key figures have recently voiced their anxieties, contributing to a significant increase in public awareness. Jacob Coxon, an AI researcher at Anthropic, announced his resignation last week, publishing a post on X that garnered over 171 million views. Following Coxon's departure, several other Anthropic researchers publicly aligned with his concerns, including Evan Hubinger, the Alignment Science Lead, along with alignment researcher Ethan Perez and scalable oversight researcher Samuel Marks. Concurrently, OpenAI safety researchers Julie Steele and Jasmine Wang also expressed their support for the call to pause or slow down development. This coordinated internal dissent signals a critical juncture in the AI safety discourse.

The core technical concern fueling these calls for caution is the concept of "recursive self-improvement." This process involves using existing AI models to design, optimize, and build subsequent generations of AI models. Researchers can leverage AI for various stages of model development, including the design of more efficient computing infrastructure, the generation and curation of superior training data, and the management of the complex software frameworks that govern the training process. Furthermore, AI models are increasingly capable of writing and refining the very code that defines their architecture and functionality. This means that AI is not only improving itself but is also beginning to automate the work required to create the next, more advanced AI.

Coxon explicitly articulated this fear in his widely viewed X post, stating, "They are racing straight to self-improving superintelligence and gambling with our lives." This sentiment reflects a broader anxiety that the current development race is prioritizing capability gains over robust safety and alignment strategies. The implication is that the speed of progress in AI capabilities, particularly in the realm of recursive self-improvement, may be outpacing the development of reliable methods to ensure these powerful systems remain aligned with human values and intentions. The involvement of researchers from both Anthropic and OpenAI, two of the leading organizations in AI development, underscores the gravity of these internal concerns and suggests a potential for significant impact on future AI development strategies.