●Change8

Migrating to Accelerate v1.7.0

Version v1.7.0 introduces 1 breaking change. This guide details how to update your code.

Released: 5/15/2025

1
Breaking Changes
3
Migration Steps
4
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

TorchDynamoPluginAcceleratorattach_layerwise_casting_hookstorch.compile

Breaking Changes

●Issue #1

The logic for setting `self.dynamic` in `TorchDynamoPlugin` now explicitly preserves `None` instead of defaulting to `False` when the `USE_DYNAMIC` environment variable is unset. This aligns behavior with PyTorch documentation for torch.compile. Users relying on the previous default behavior might need to explicitly set `USE_DYNAMIC` if they require `False`.

Migration Steps

  1. 1
    To enable regional compilation, set `use_regional_compilation=True` in your `TorchDynamoPlugin` configuration and pass the plugin when initializing `Accelerator`.
  2. 2
    To use layerwise casting hooks, call `attach_layerwise_casting_hooks(model, storage_dtype=..., compute_dtype=...)` before preparing the model.
  3. 3
    If you rely on the default behavior of `torch.compile` when `USE_DYNAMIC` is unset, be aware that `self.dynamic` will now be `None` instead of `False`.

Release Summary

This release introduces significant performance improvements through regional compilation for torch.compile and adds layerwise casting hooks for memory optimization. It also brings substantial enhancements to FSDP2 support, including enabling `FULL_STATE_DICT` and fixing memory issues.

Need More Details?

View the full release notes and all changes for Accelerate v1.7.0.

View Full Changelog