Google Unveils Gemini 1.5 Pro With 1 Million Token Context Window

Google announced Gemini 1.5 Pro on February 15, 2024, a new generation of its flagship AI model that significantly expands its context window to 1 million tokens. This substantial increase allows the model to process and analyze vastly larger amounts of information in a single input, such as entire codebases, lengthy books, or hours of video. The company stated that this capability is now available in a preview version for developers.

This new context window represents a 100x increase compared to the 10,000 tokens available in Gemini 1.0 Pro. Google demonstrated the model's capabilities by having it analyze a 402-page document and a 11-hour-long silent film, identifying specific details within both. The 1 million token context window is built on Google's Mixture-of-Experts (MoE) architecture, which was first introduced with Gemini 1.5 Pro. This architecture allows for more efficient processing of large amounts of data.

In addition to the expanded context window, Gemini 1.5 Pro also offers enhanced performance, with Google claiming it is 2x faster than Gemini 1.0 Pro. The model maintains its strong performance across various benchmarks, including multimodal reasoning, which allows it to understand and process different types of data like text, images, audio, and video. The preview version is being rolled out to developers through the Google AI Studio and Vertex AI platforms, enabling them to experiment with and build applications leveraging this advanced capability. Google has indicated that a wider release and further enhancements are planned for the future.