●Change8

0.50.0

Breaking Changes
📦 bitsandbytesView on GitHub →
⚠ 3 breaking✨ 14 features🐛 10 fixes⚡ 2 deprecations🔧 15 symbols

Summary

This release introduces significant performance improvements with new fused 4-bit GEMM kernels for inference on CUDA and ROCm, faster CPU operations, and an enhanced Apple Silicon backend. It also brings stability improvements to ROCm support and adds Windows on ARM CPU support.

⚠️ Breaking Changes

  • Minimum PyTorch version is now 2.4. Users must update PyTorch to at least version 2.4.
  • Removed deprecated APIs: the `research` module, non-blockwise (`block_wise=False`) optimizers, and legacy dynamic quantization functions, along with their CUDA/HIP kernels. Code relying on these APIs will need to be updated to use the new blockwise or other supported quantization methods.
  • Removed legacy sparse functionality (`spmm_coo`, `spmm_coo_very_sparse`) and dropped the cusparse/hipsparse dependencies. Users of these functions must migrate to alternative sparse matrix multiplication implementations.

Migration Steps

  1. Update PyTorch to at least version 2.4.
  2. Update code that uses the `research` module, non-blockwise (`block_wise=False`) optimizers, or legacy dynamic quantization functions to use supported alternatives.
  3. Migrate from `spmm_coo` and `spmm_coo_very_sparse` to alternative sparse matrix multiplication implementations.
  4. Ensure 4-bit weights passed to `matmul_4bit` are in the `[out_features, in_features]` orientation, not transposed `[in_features, out_features]`.
  5. On macOS 26+, install the `kernels` package for optimal MPS backend performance; otherwise, a naive fallback will be used.

✨ New Features

  • New fused 4-bit GEMM kernels for inference on CUDA, offering up to 4x speedup for small-to-medium batch sizes across Turing through Blackwell GPUs.
  • Fused 4-bit SIMT GEMM on ROCm for small inference batches, validated on gfx1100, gfx1201, and gfx1151.
  • AMD ROCm support is now stable with improved stability and performance.
  • ROCm wheels for Windows are now published for ROCm 7.2 and 7.14.
  • Expanded ROCm GPU target coverage to include RDNA2 consumer GPUs, additional RDNA3 and RDNA3.5 GPUs, and CDNA1 (gfx908) data center GPUs.
  • Substantially improved Apple Silicon (MPS) backend with optimized Metal kernels, enabling all 4-bit and LLM.int8() configurations to work.
  • Faster CPU operations on x86-64 and ARM64 due to improved blockwise quantization and dequantization, with significant gains on fp16 and CPUs without AVX-512.
  • Reduced host-side CPU overhead, particularly for CUDA/ROCm backends.
  • Windows on ARM CPU support with NEON-optimized kernels.
  • New optimizer support on CPU for 32-bit and 8-bit blockwise optimizers.
  • Intel XPU gained paged optimizer support and support for LARS and LAMB optimizers.
  • Fixed Lion optimizer to use decoupled weight decay.
  • Added CUDA 13.2 build wheels.
  • Added a oneAPI 2026 build for Intel XPU, packaging both 2025 and 2026 variants.

🐛 Bug Fixes

  • Fixed `gemv_4bit` bf16 correctness on Intel Arc A-series (Alchemist) GPUs.
  • Fixed QuantState and dict conversions.
  • Fixed Params4bit attribute access for FSDP state_dict traversal.
  • Fixed GlobalOptimManager.override_config not propagating to optimizer.
  • Fixed 4-bit quantization for weight matrices not divisible by blocksize.
  • Fixed ROCm compile-time warp size to use runtime query in host code.
  • Fixed ROCm blocksize 32 4-bit quantization and GEMV kernels on AMD CDNA.
  • Fixed ROCm/CUDA library lookup version tag string.
  • Fixed Lion to use decoupled weight decay.
  • Other optimizer fixes.

Affected Symbols

research moduleblock_wise=False optimizerslegacy dynamic quantization functionsspmm_coospmm_coo_very_sparseigemmbatched_igemmcheck_matmulmatmul_4bitMPS backendgemv_4bitQuantStateParams4bitGlobalOptimManagerLion optimizer

⚡ Deprecations

  • `igemm`, `batched_igemm`, and `check_matmul` are deprecated and now emit warnings. Users should migrate to newer, recommended functions.
  • Passing 4-bit weights quantized in transposed `[in_features, out_features]` orientation to `matmul_4bit` now emits a `DeprecationWarning`. Support for this orientation is likely to be removed in the future. Users should ensure weights are in the `[out_features, in_features]` orientation.