Google Open-Sources PyTorch Backend for Cloud TPUs

Announced at PyTorch Conference 2026 on October 20, 2026, TorchTPU enables existing PyTorch code to target a TPU through a standard torch.device("tpu"). TPUs are now recognized as a native PyTorch device alongside GPUs and CPUs. The Google team recommends TorchTPU for all new workloads, stating it replaces PyTorch/XLA.

TorchTPU is a software stack comprising four open repositories: TorchTPU, vLLM-TorchTPU, SGLang-TorchTPU, and TorchTitan. Available as v0.2.0 on PyPI, it requires Python 3.11+ and is licensed under Apache 2.0. TorchTPU runs on Google Cloud TPUs and has been tested on generations from v5e, scaling up to 8,192 Ironwood (TPU7x) chips. The serving capabilities support FP8, MXFP4, and NVFP4 checkpoints.

Performance metrics for TorchTPU include eager mode for debugging and torch.compile for production throughput. The system achieved a self-reported 52% MFU training for Qwen 3 8B with TorchTitan on Ironwood. In a fine-tuning scenario, out-of-the-box eager mode took 11.49 seconds per step, compared to 2.78 seconds when fully tuned.

TorchTPU functions as an open-source PyTorch backend that allows PyTorch models to run natively on TPUs by plugging into PyTorch’s PrivateUse1 interface. Unlike its predecessor, PyTorch/XLA, which recorded lazy-tensor graphs and required developers to mark compilable regions, TorchTPU handles tensors as ordinary PyTorch tensors without subclasses or wrappers. It supports one process per device, preserves tensor views and in-place mutations, and handles divergent MPMD execution, whereas PyTorch/XLA only supported pure SPMD code.

TorchTPU offers three eager modes: Strict (default), which dispatches each operator asynchronously with standard PyTorch semantics; Debug, which runs synchronously with tensor bounds checking; and DeferAndFuse, which fuses runs of eager operators to reduce HBM traffic. The TorchTPU paper indicates that DeferAndFuse can be 50% or more faster than Strict mode.