0.48.1
📦 bitsandbytesView on GitHub →
🐛 2 fixes🔧 1 symbols
Summary
Version 0.48.1 addresses a critical regression in LLM.int8() affecting inference with pre-quantized checkpoints and fixes an issue with 8bit parameter device movement.
🐛 Bug Fixes
- Fixed a regression introduced in 0.48.0 that caused poor inference results with pre-quantized checkpoints in HF transformers when using LLM.int8().
- Fixed regression in 8bit parameter device movement.
🔧 Affected Symbols
LLM.int8()