●Change8

Migrating to BitsAndBytes 0.46.0

Version 0.46.0 introduces 2 breaking changes. This guide details how to update your code.

Released: 5/27/2025

2
Breaking Changes
3
Migration Steps
9
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

bnb.autograd.get_inverse_transform_indicesbnb.autograd.undo_layoutbnb.functional.create_quantile_mapbnb.functional.estimate_quantilesbnb.functional.get_colrow_absmaxbnb.functional.get_row_absmaxbnb.functional.histogram_scatter_add_2dtorch.compileget_cuda_version_tuple

Breaking Changes

●Issue #1

Support for Python 3.8 has been dropped. Users must upgrade to Python 3.9 or newer.

●Issue #2

Support for PyTorch versions older than 2.2.0 has been dropped. Users must upgrade to PyTorch >= 2.2.0.

Migration Steps

  1. 1
    Ensure your Python version is 3.9 or higher.
  2. 2
    Ensure your PyTorch version is 2.2.0 or higher. For best `torch.compile` support, PyTorch 2.6+ is recommended.
  3. 3
    Review usage of deprecated functions in `bnb.autograd` and `bnb.functional` namespaces and update code accordingly.

Release Summary

This release introduces significant improvements for `torch.compile` compatibility with both LLM.int8() and 4bit quantization, alongside a major refactoring to integrate with PyTorch Custom Operators. Support for Python 3.8 and older PyTorch versions has been dropped.

Need More Details?

View the full release notes and all changes for BitsAndBytes 0.46.0.

View Full Changelog