OpenAI Develops GPT-Red LLM for AI Safety Testing

OpenAI has developed a specialized large language model (LLM) named GPT-Red, designed to function as an automated "super-hacker" for enhancing AI safety. This LLM acts as a sparring partner, identifying and helping to patch vulnerabilities in other OpenAI models before their release. The company stated that training its latest flagship LLM, GPT-5.6, against GPT-Red resulted in its most robust release to date.

GPT-Red automates the process of "red-teaming," a critical safety evaluation typically performed by human testers. The goal of red-teaming is to discover as many potential ways as possible to break or hijack a system, allowing developers to patch these weak spots. As LLMs become more complex and are deployed in a wider array of tasks, particularly as agents that can interact with external systems, the challenge of anticipating all possible attack vectors grows significantly. Nikhil Kandpal, a research scientist at OpenAI and co-creator of GPT-Red, noted that the "risk surface and the blast radius also grow."

OpenAI's motivation for building GPT-Red is to future-proof its safety testing procedures. Dylan Hunn, another research scientist and co-creator, explained that as more capable AI models emerge, GPT-Red is designed to discover novel attack modes. The researchers reported that GPT-Red has already identified new types of attacks previously unseen. A primary focus of GPT-Red's efforts has been on "prompt injection" attacks, where malicious instructions are embedded within an LLM's input to manipulate its behavior, potentially leading to data exfiltration, code sabotage, or the generation of harmful content. These instructions can be concealed within any text the LLM processes, such as code or web content.