Google Launches Gemini 1.5 Pro With 1 Million Token Context

Google announced Gemini 1.5 Pro on February 15, 2024, a new flagship model that significantly expands its context window to 1 million tokens. This advancement allows the AI to process and analyze substantially larger amounts of information in a single prompt, including hours of video, lengthy codebases, or extensive documents. The model is built on the same architecture as Gemini 1.0 but incorporates a Mixture-of-Experts (MoE) approach, which Google states enhances efficiency and performance.

The expanded context window represents a tenfold increase over previous models, enabling unprecedented capabilities in understanding complex and lengthy data. For instance, users can input an entire book or a full-length movie script and ask detailed questions or request summaries. Google demonstrated this by processing a 402-page document and a 11-hour long silent film, showcasing the model's ability to recall specific details from across the entire input.

Gemini 1.5 Pro also features enhanced performance, particularly in multimodal reasoning, which allows it to understand and reason across different types of data, including text, images, audio, and video. The MoE architecture contributes to faster inference times and more efficient resource utilization compared to dense models of similar size. Google highlighted that the model maintains high performance across various benchmarks, including a 5.7x improvement on the MMLU benchmark compared to Gemini 1.0 Pro.

The model is currently available in a limited preview for developers and enterprise customers, with plans for broader availability in the future. Google emphasized its commitment to responsible AI development, with built-in safety features and ongoing research into mitigating potential risks associated with large context windows and advanced AI capabilities. The company aims to make this powerful AI accessible for a wide range of applications, from research and development to creative endeavors and business solutions.