Anthropic’s safety warnings may have just backfired — the government has pulled the plug on its most powerful AI

The U.S. government has halted the deployment of Anthropic's most powerful AI model, Claude 3.5, following safety concerns raised by the company itself. Anthropic had alerted the government to a "narrow potential jailbreak" vulnerability in the model, which it described as a "low-risk" issue. The company expressed disagreement with the government's decision, stating in a blog post that a "narrow potential jailbreak" should not warrant recalling a commercial model used by "hundreds of millions of people." This action by the government marks a significant intervention in the development and deployment of advanced AI technologies, highlighting the increasing scrutiny on AI safety and potential risks. The specific details of the "jailbreak" and the exact number of users affected by the halt have not been fully disclosed. The government's decision underscores a proactive approach to AI regulation, even when the identified risks are deemed minor by the developers.