NVIDIA TensorRT Model Connect Preview Released

NVIDIA has released TensorRT Model Connect (TRTMC) in public preview, an open-source project designed to streamline the conversion of machine learning models into highly optimized native C++ inference engines. This new tool allows users to take supported Hugging Face or local checkpoints and transform them into end-to-end TensorRT inference artifacts with just two commands, eliminating the need for an intermediate ONNX export step. The output is a versioned `.bundle` artifact that integrates directly with native C++ task APIs. This enables inference to be executed within C++ services, embedded applications, or robotics stacks without requiring PyTorch to be present in the runtime environment. The project is licensed under Apache-2.0 and is structured as a collection of reference implementations tailored to specific model families, rather than a single, generalized converter. NVIDIA has stated that the entire development process for TRTMC, including model implementations, performance tuning, testing, integrations, and documentation, was conducted using OpenAI Codex agents under human oversight and review. The TRTMC is currently available for evaluation and native integration work, with the code being open-source and installable. The initial release wheels are targeted exclusively for Linux aarch64, requiring Python 3.10 or 3.12, glibc 2.39 or newer, and TensorRT 11.1.0.106. Users operating on x86_64 architectures must utilize the Docker source-build path, as x86_64 wheels are not published. NVIDIA identifies the primary beneficiaries of TRTMC as teams that already manage their own inference stacks. This includes NVIDIA-focused startups, companies specializing in robotics and device development, and platform or inference teams within mid-size to large enterprises. Smaller teams primarily deploying Python services may find less immediate benefit. Regulated enterprises are advised to await a formally tagged release before standardizing their operations on TRTMC. The technology is particularly relevant for industries such as robotics and autonomous machines, industrial inspection and manufacturing, automotive in-vehicle compute systems, medical devices, defense and aerospace edge systems, and media processing. These sectors often require inference capabilities to be embedded directly within C++ binaries rather than relying on separate Python servers. Potential applications span on-device text generation, speech recognition and synthesis, optical character recognition (OCR) and document parsing, the creation of embeddings and reranking for retrieval services built in C++, diffusion-based image and video generation, image segmentation, and time-series forecasting.