Google Releases LiteRT.js for Browser AI Inference

Google released LiteRT.js this week, a JavaScript binding for its on-device inference library, LiteRT (formerly TensorFlow Lite). This new tool allows .tflite models to execute directly within web browsers, eliminating the need for server-side processing. Google highlights that keeping inference local enhances user privacy, removes server costs, and achieves ultra-low latency.

LiteRT.js is not a new model format but rather a compilation of Google's native runtime to WebAssembly, exposed to JavaScript. Unlike earlier web AI solutions like TensorFlow.js, which relied on JavaScript-based kernels, LiteRT.js ships the native cross-platform runtime with its existing optimizations. This means performance upgrades, quantization improvements, and hardware optimizations developed for Android, iOS, and desktop are now accessible on the web.

The LiteRT.js runtime supports three backends: CPU, GPU, and NPU. The CPU backend utilizes XNNPACK with multi-thread support. The GPU backend leverages ML Drift through WebGPU, and the NPU backend uses the experimental WebNN API. A key rule is that a model's graph cannot be split across different accelerators; delegation is an all-or-nothing decision per model. If a model cannot be fully delegated to a chosen accelerator, LiteRT.js falls back to WebAssembly execution.

Google reports significant performance gains with LiteRT.js. Compared to other web runtimes, it achieves up to 3x faster inference across CPU and GPU for classical computer vision and audio processing models. When comparing GPU or NPU execution against its own CPU execution, the speedup ranges from 5x to 60x for demanding real-time tasks such as object tracking and audio transcription. These benchmarks were conducted in a controlled browser environment on a 2024 MacBook.