AI Insiders Raise Alarms Over Company Practices and Tech Risks

A growing number of AI insiders are publicly voicing concerns about the practices and potential risks associated with their former companies and the broader artificial intelligence industry. David Robinson, a former safety leader at OpenAI, resigned last week, citing a "parade of former colleagues" who have also left AI firms due to worries about the technology or its application. In an essay for The Atlantic, Robinson advocated for an end to the "trial and error" development approach favored by many AI companies.

Ed Newton-Rex, who previously led the audio team at Stability AI, the company behind the text-to-image tool Stable Diffusion, departed in November 2023. His resignation stemmed from concerns over the use of copyrighted material in training AI models. Newton-Rex stated to Fast Company that his views on data usage did not align with the generative AI industry's current methods, predicting that the industry's prevailing outlook combined with rapid technological advancement would be "very, very bad for creators." Following his departure, he established Fairly Trained, a nonprofit organization that provides certifications for AI companies adhering to specific data usage requirements.

On June 4, 2024, a group of 13 current and former employees from prominent AI companies released a letter titled “A Right to Warn about Advanced Artificial Intelligence.” This letter highlighted significant risks posed by advanced AI, including the exacerbation of existing inequalities, the spread of manipulation and misinformation, and the potential loss of control over autonomous AI systems, which could ultimately lead to human extinction. The signatories expressed deep concerns regarding the insufficient oversight of AI development. Among the notable signatories were Ramana Kumar, a former employee of Google DeepMind; Neel Nanda, who is currently with Google DeepMind and leads their mechanistic interpretability team, and was previously at Anthropic; and eleven current and former OpenAI employees. Jacob Hilton, another signatory, now works for the Alignment Research Center, while Daniel Kokotajlo has founded and leads the nonprofit research group AI Futu.

The collective warnings from these individuals underscore a growing internal dissent within the AI sector regarding ethical considerations, data provenance, and the long-term safety implications of increasingly powerful artificial intelligence systems. Their public statements and actions aim to draw attention to these issues and advocate for more responsible development and deployment of AI technologies.