NVIDIA DeepStream 9.1 Adds Agentic AI to Vision Analytics

NVIDIA released DeepStream 9.1 this week, a significant update to its streaming analytics toolkit for AI-based video and image understanding. This version introduces agentic AI capabilities, aiming to streamline the development of complex video analytics pipelines. The update addresses a persistent challenge in video analytics: tracking objects across multiple cameras, which traditionally requires manual calibration and intricate calculations.

DeepStream 9.1 integrates two key agentic skills to tackle this problem: Multi-View 3D Tracking (MV3DT) and AutoMagicCalib (AMC). MV3DT enables cross-camera object tracking by projecting detections from multiple calibrated cameras into a shared 3D coordinate system, associating observations of the same object across different views, and assigning a single, consistent object ID. This process involves four stages, starting with object detection using supported models like PeopleNetTransformer, PeopleNet v2.6.3, or RT-DETR 2D.

In addition to MV3DT and AMC, DeepStream 9.1 offers 13 agentic skills for coding agents, accelerating the transition from concept to a functional pipeline. The release also includes support for NVIDIA JetPack 7.2, enhancing compatibility with Jetson Orin and Thor edge devices. Furthermore, the toolkit now features a unified open-source GitHub repository, available under CC-BY-4.0 and Apache-2.0 licenses, promoting broader accessibility and collaboration within the developer community.

DeepStream is built on a GStreamer-based framework and leverages NVIDIA GPUs for multi-stream, multi-model inference. Its pipelines integrate hardware-accelerated decoding and encoding, TensorRT inference, object tracking, and message-broker integration. The inclusion of agentic skills in version 9.1 represents a move towards more autonomous and intelligent vision AI systems, allowing developers to focus on higher-level logic rather than intricate low-level calibration and tracking mechanisms.