v5.14.1
📦 transformersView on GitHub →
🐛 4 fixes🔧 6 symbols
Summary
This patch release addresses several issues related to Inkling model integration, including problems with assisted generation and prefill with specific caching mechanisms. It also includes fixes for deepgemm on multiple devices and an update to FP8 kernels.
🐛 Bug Fixes
- Fixed an issue with sdpa prefill when using position_bias, particularly relevant for Inkling models.
- Resolved an issue affecting assisted decoding for models that use EncoderDecoderCache, including OlmoHybrid.
- Addressed an issue with deepgemm on multiple devices.
- Bumped kernels version for FP8 support.