Change8

0.48.1

📦 bitsandbytesView on GitHub →
🐛 2 fixes🔧 1 symbols

Summary

Version 0.48.1 addresses a critical regression in LLM.int8() affecting inference with pre-quantized checkpoints and fixes an issue with 8bit parameter device movement.

🐛 Bug Fixes

  • Fixed a regression introduced in 0.48.0 that caused poor inference results with pre-quantized checkpoints in HF transformers when using LLM.int8().
  • Fixed regression in 8bit parameter device movement.

🔧 Affected Symbols

LLM.int8()