●Change8

v2.13.0

Breaking Changes
📦 pytorchView on GitHub →
⚠ 8 breaking✨ 9 features🐛 1 fixes🔧 15 symbols

Summary

PyTorch 2.13 introduces significant performance features like FlexAttention on MPS and the CuTeDSL backend for Inductor, alongside major internal cleanups including the removal of Bazel support and named tensors. This release contains several breaking changes, particularly around C++ APIs and Python interpreter version support.

⚠️ Breaking Changes

  • Stop building CPython 3.13t (free-threaded) binaries. Users on the free-threaded interpreter should move to Python 3.14t.
  • Bare `PyObject` is no longer allowed in operator schemas. Parsing a schema with a bare `PyObject` argument or return type will now raise a schema parse error.
  • Remove Bazel build support. Users building PyTorch with Bazel should migrate to the supported CMake/pip install flow.
  • Enforce C++20 minimum in header guards. C++20 is now required to import PyTorch headers.
  • `StorageImpl`'s built-in copy-on-write (COW) materialization is replaced by a pluggable materializer hook. Out-of-tree backends/extensions calling removed COW symbols (`is_cow()`, `maybe_materialize_cow()`, `cow::materialize_cow_storage()`) will fail to compile. Migrate to the new hook API (`set_materializer()` / `has_materializer()` / `clear_materializer()`).
  • Convert `shared_ptr<Node>` to `intrusive_ptr<Node>` in autograd. The signature of `Tensor.grad_fn()` changes from `std::shared_ptr<Node>` to `c10::intrusive_ptr<Node>`. Construction of C++ autograd functions must use `c10::make_intrusive` instead of `std::shared_ptr` with a custom deleter.
  • The minimum supported NCCL version when building from source is now 2.23. Users building from source against an older NCCL will hit compile errors.
  • Remove named tensors. All associated Python and C++ APIs, including `Tensor.names`, `Tensor.rename()`, and `Tensor.refine_names`, are removed.

Migration Steps

  1. If using free-threaded Python 3.13t, move to Python 3.14t.
  2. If parsing operator schemas, ensure bare `PyObject` is not used as an argument or return type; use specific types or handle schema parsing errors.
  3. If building from source, migrate from Bazel build system to the CMake/pip install flow.
  4. If compiling C++ extensions, ensure your compiler is set to C++20 standard or newer.
  5. If out-of-tree backends/extensions directly called COW entry points on `StorageImpl` (e.g., `is_cow()` or `maybe_materialize_cow()`), migrate to using the pluggable materializer hook API (`set_materializer()` / `has_materializer()` / `clear_materializer()`).
  6. In C++ code accessing `Tensor.grad_fn()`, change usage from `std::shared_ptr<Node>` to `c10::intrusive_ptr<Node>`. When constructing C++ autograd nodes, use `c10::make_intrusive<CustomCppNode>()` instead of `std::shared_ptr` with a custom deleter.
  7. If building from source, ensure the system NCCL installation is version 2.23 or newer.
  8. If using named tensors APIs (e.g., `Tensor.names`, `Tensor.rename()`), update code as these features have been removed.

✨ New Features

  • FlexAttention lands on Apple Silicon (MPS), offering up to ~12x speedup over SDPA on sparse patterns, and gains a deterministic backward path on CUDA.
  • CuTeDSL "Native DSL" backend introduced as a second high-performance code path for Inductor on GPUs, aiming for faster compilation.
  • `nn.LinearCrossEntropyLoss` combines prediction and loss computation to reduce peak GPU memory by up to 4x for large-vocabulary language model training.
  • `torchcomms`, a new communications backend for PyTorch Distributed, improves fault tolerance, scalability, and debuggability.
  • FSDP2 now supports overlapping reduce-scatter and all-gather communications via an opt-in dedicated process group, increasing distributed training throughput.
  • Python 3.15 wheel support added for PyTorch on Linux, including builds compatible with free-threaded 3.15t.
  • ROCm gains AOTriton 0.12b with native HIP CMake.
  • Arm adds Armv9-A `torch.compile` targeting.
  • Intel XPU exposes new device telemetry APIs.

🐛 Bug Fixes

  • Tracked regression: ROCm wheels (`torch==2.13.0+rocm7.2`) break `torch.compile` on CPU in environments without a GPU by raising `RuntimeError: Can't detect vectorized ISA for CPU`. Workaround: run the `+rocm` wheel on a ROCm image, or install a standard CPU/CUDA build for GPU-less environments.

Affected Symbols

PyObjecttorch._C.parse_schemaStorageImplStorageImpl::is_cow()StorageImpl::maybe_materialize_cow()cow::materialize_cow_storage()StorageImpl::set_materializer()StorageImpl::has_materializer()StorageImpl::clear_materializer()Tensor::grad_fnstd::shared_ptr<Node>c10::intrusive_ptr<Node>Tensor.namesTensor.rename()Tensor.refine_names