Google Unveils Gemini 1.5 Pro With 1 Million Token Context Window

Google announced Gemini 1.5 Pro on February 15, 2024, a new iteration of its multimodal large language model that boasts a context window of 1 million tokens. This represents a substantial increase from the 128,000 tokens available in Gemini 1.5 Flash and the 32,000 tokens in the original Gemini 1.0 Pro. The expanded context window allows the model to process and analyze significantly larger amounts of information, including entire books, extensive codebases, or hours of video content, in a single prompt.

During a demonstration, Google showcased Gemini 1.5 Pro's capability to analyze a 402-page PDF document and a 44-minute silent film, extracting relevant information and answering complex questions about both. The model demonstrated an understanding of narrative, character relationships, and specific details within the provided materials. This advanced reasoning capability is attributed to a new Mixture-of-Experts (MoE) architecture, which Google states makes the model more efficient and performant, particularly for long-context tasks.

While the 1 million token context window is currently available to select developers and enterprise customers through a private preview, Google plans to make it more broadly accessible in the coming months. The company also highlighted that Gemini 1.5 Pro maintains strong performance across various benchmarks, including reasoning, coding, and multimodal understanding, despite its expanded context length. This development positions Gemini 1.5 Pro as a powerful tool for applications requiring deep comprehension of extensive data sets, potentially transforming fields like research, education, and content analysis.