Google Unveils Gemini 1.5 Pro With 1 Million Token Context Window

Google announced Gemini 1.5 Pro on February 15, 2024, a new iteration of its flagship AI model that significantly expands its context window to 1 million tokens. This represents a tenfold increase from the 100,000 tokens available in Gemini 1.0 Pro, allowing the model to process and analyze vastly larger amounts of information in a single input. The expanded context window enables Gemini 1.5 Pro to understand and reason over extensive documents, hours of video, or lengthy codebases.

During a demonstration, Google showcased Gemini 1.5 Pro's ability to analyze a 402-page PDF document, a 44-minute YouTube video, and 21,555 lines of code. The model successfully answered questions and extracted specific information from these diverse and lengthy inputs, highlighting its enhanced comprehension capabilities. This advancement is particularly impactful for tasks requiring the synthesis of information from multiple sources or the deep analysis of complex materials.

Google stated that Gemini 1.5 Pro is built on a new Mixture-of-Experts (MoE) architecture, which contributes to its improved performance and efficiency. While the 1 million token context window will be available to developers in a preview version starting February 15, 2024, Google plans to make it more widely accessible in the future. The company emphasized that this development is a significant step towards more capable and versatile AI systems that can handle real-world, complex data scenarios.