Google Unveils Gemini 1.5 Pro With 1 Million Token Context Window

Google announced Gemini 1.5 Pro, a new flagship model, on February 15, 2024, featuring a breakthrough 1 million token context window. This expansion represents a 100x increase over the previous 10,000 token limit of Gemini 1.0 Pro, enabling the model to process significantly larger amounts of information. The expanded context window allows Gemini 1.5 Pro to analyze and reason over entire codebases, lengthy documents, or hours of video content in a single prompt.

During a demonstration, Google showcased Gemini 1.5 Pro's ability to analyze a 402-page PDF document and a 44-minute silent film, identifying specific details and answering complex questions about their content. The model successfully pinpointed a specific frame in the film where a specific object appeared, demonstrating its advanced comprehension capabilities. This leap in context window size is achieved through a new Mixture-of-Experts (MoE) architecture, which Google states makes the model more efficient and performant.

Gemini 1.5 Pro is currently available in a limited preview for developers and enterprise customers, with plans for broader availability in the coming months. Google highlighted that the MoE architecture allows for more efficient computation, with only a fraction of the model's parameters being activated for any given task. This efficiency is crucial for managing the computational demands of such a large context window. The company also emphasized its commitment to responsible AI development, with safety and ethical considerations integrated into the model's design and deployment.