Google Research Unveils R4T for Faster Search Retrieval

Google Research has introduced Retrieve-for-Train (R4T), a new framework designed to address the challenge of returning diverse sets of results for search and recommendation queries, rather than just a single best match. This is crucial for queries like 'camping gear,' which should yield a tent, sleeping bag, stove, and headlamp, instead of multiple similar tents. R4T utilizes reinforcement learning (RL) to efficiently learn how to generate multiple relevant sub-queries from a single broad prompt, a process known as query fan-out.

The research team identified two primary issues with standard query fan-out methods when performed by generic large language models (LLMs) at inference time. The first is 'paraphrastic collapse,' where sub-queries generated are too similar, leading to a homogeneous set of results. For instance, a query like ‘Bohemian festival style’ might produce sub-queries such as ‘bohemian festival fashion’ and ‘festival bohemian clothes,’ which retrieve nearly identical items. The second issue is latency, as autoregressive generation combined with multiple retrieval calls is inherently slow. While 'best-of-N' sampling can improve quality, it significantly multiplies inference costs.

R4T employs a three-step pipeline to overcome these limitations. The first step, 'Fan-out LM training,' involves training a fan-out language model (FOLM) that generates k sub-queries. These sub-queries are then processed by a frozen dense retriever, and a set-level reward function scores the entire retrieved set, promoting diversity. The second step, 'Supervision synthesis,' uses the trained FOLM to sample 128 fan-outs per query at a temperature of 0.9. These samples are converted into (query, target set) training pairs without requiring human labels. For open-ended tasks, the targets are embeddings of the retrieved content, while for compositional tasks, they are embeddings of the sub-queries themselves.

The final step, 'Diffusive retriever training,' trains a diffusion transformer with 53.9 million parameters. This model learns to map a query embedding to a complete set of target embeddings using a variance-exploding formulation within the EDM framework. At inference, this diffusion model generates all embeddings in a single, non-autoregressive pass. Subsequent nearest-neighbor search maps these embeddings to actual database items. This approach significantly accelerates the fan-out process, with the framework demonstrating a 12x to 20x improvement in query fan-out speed. The reward design for tasks like open-ended abstract retrieval (OAR) is crucial for guiding the model towards generating diverse and relevant sets of results, mitigating the problem of reward hacking.