Mistral AI Launches Robostral Navigate for Robot Navigation

Mistral AI released Robostral Navigate this week, marking its first model specifically designed for embodied navigation in robots. This 8 billion parameter model processes RGB images and plain-language instructions to guide a robot through complex environments, including offices, residential buildings, commercial spaces, and outdoor settings. A key feature of Robostral Navigate is its ability to achieve a 76.6% success rate on R2R-CE validation, navigating previously unseen environments using only a single RGB camera, without relying on depth sensors or LiDAR.

The model operates by taking a single instruction, such as 'Leave the lobby, walk through the corridor, enter the supply room, and stop to face the second shelf,' and executing the entire task autonomously. This capability is particularly significant as it allows robots to navigate live spaces populated with people and unexpected obstacles, elements not present during its training phase. Traditional navigation systems often require multiple sensors or specialized hardware, whereas Robostral Navigate demonstrates efficiency and effectiveness with standard RGB cameras.

Robostral Navigate employs a method called 'pointing' to determine its next movement. Given a task and its observation history, the model predicts the target's image coordinates within the current camera view and the desired orientation upon arrival. This pointing mechanism offers robustness against changes in camera intrinsics and world scale, unlike navigation commands based on metric displacements. When the target is outside the camera's field of view, the model defaults to using displacements within the robot's local coordinate frame, such as 'Move 2 meters forward, 1.5 meters to the left, and turn 25 degrees left.' This dual approach ensures continuous navigation even when visual cues are temporarily unavailable.