Perplexity Trains AI Agent on Mistakes to Improve Performance

Perplexity Research has introduced a novel post-training methodology designed to enhance the performance of its AI agent by learning from real user interactions, including instances where the agent made mistakes. This innovative approach, detailed in a recent company publication, pairs rejection sampling fine-tuning with a technique called hint-guided self-distillation. The objective is to enable the AI agent to not only replicate successful actions but also to understand and correct its own errors, thereby improving its overall reliability and effectiveness.

In a live A/B test conducted by the Perplexity team, this new training regimen demonstrated a significant reduction in tool-call failures. The failure rate dropped from 2.24% to 1.77% across two trained checkpoints, representing a statistically significant relative reduction of 21.2%. While this advancement shows promising results, the post-trained weights and training code have not yet been publicly released. The improved model is currently accessible only as a specific model option within Perplexity Computer, though its base model, GLM 5.2, is available on Hugging Face.

The limitations of standard rejection sampling fine-tuning (RFT) were a key motivator for this research. RFT typically focuses on imitating only successful sessions, but this can inadvertently reinforce errors. An agent might recover from an incorrect tool call and still arrive at the correct final answer, yet imitating the entire flawed trajectory can embed the mistake. Furthermore, discarding sessions that ultimately failed means losing valuable data that clearly illustrates avoidable errors. Perplexity's new method addresses this by analyzing each assistant turn to decide whether the behavior is worth imitating or if the mistake needs correction.

Perplexity's team categorizes each assistant turn into one of three treatments: 'Imitate,' 'Correct,' or 'Keep as context.' Turns that did not involve errors in successful sessions are subjected to cross-entropy (CE) loss for imitation. Error turns that are accompanied by a validated hint receive Kullback-Leibler (KL) divergence loss for correction, regardless of whether the overall session was successful. Other turns are retained in the input context but do not incur any loss. This allows successful sessions to provide targets for both imitation and correction, while unsuccessful sessions offer solely correction targets. A 'hint' is defined as a concise corrective instruction derived from information the model already possessed. For instance, if a search call incorrectly uses 'year' for a recency filter when the schema only permits 'day,' 'week,' or 'month,' the hint would identify the failed call, the validation error, and suggest an appropriate value or the omission of the parameter.