●Change8

v5.15.0

Breaking Changes
📦 transformersView on GitHub →
⚠ 4 breaking✨ 4 features🐛 26 fixes🔧 26 symbols

Summary

This release introduces several new models including Meta Muse Glimmer, GraniteMoeSWA, A.X-K1/K2, and Cosmos3 Edge. It also includes significant updates to attention mechanisms, vision processing, and generation capabilities, alongside important breaking changes in kernel opt-in and cache cropping.

⚠️ Breaking Changes

  • Kernels are now opt-in rather than mandatory for linear attention models (Mamba, GDN, Conv-only, etc.). Users who relied on automatic kernel selection must explicitly enable kernels to maintain previous behavior.
  • The cache cropping API now only accepts negative values (relative offsets) instead of absolute sizes. Users calling crop methods directly must update their code to pass negative values accordingly.
  • T5 and its model family (MT5, LongT5, etc.) now support SDPA and other attention backends via `ALL_ATTENTION_FUNCTIONS`. The default attention implementation may change, and users relying on the previous eager-only path should explicitly set `attn_implementation="eager"` if needed.
  • Several small private helper functions (e.g., `_is_url`, `_build_image_tokens`) have been removed from multimodal processor files. Users or downstream libraries that imported these private functions directly must remove or replace those references.

Migration Steps

  1. For linear attention models (Mamba, GDN, Conv-only, etc.), explicitly enable kernels if automatic selection was previously relied upon.
  2. Update cache cropping calls to use negative values (relative offsets) instead of absolute sizes.
  3. If using T5 or its family (MT5, LongT5, etc.) and relying on the previous eager-only attention path, explicitly set `attn_implementation="eager"`.
  4. Remove or replace direct imports of private helper functions like `_is_url` and `_build_image_tokens` from multimodal processor files.

✨ New Features

  • Added Meta Muse Glimmer, a new multimodal model for agentic use cases, distilled from Muse to 30B parameters.
  • Added support for Granite-swa and Granitemoe-swa models.
  • Added support for A.X-K1 and A.X-K2 models.
  • Added support for Cosmos3 Edge model.

🐛 Bug Fixes

  • Fixed multi-head latent attention (MLA) cache compression.
  • Refactored linear attention models for better maintainability.
  • Allowed metal-flash-sdpa for OpenAIPrivacyFilter on MPS.
  • Made Gemma 4's heterogeneous attention config explicit using `per_layer_config`.
  • Added paged attention tests support for XPU.
  • Moved `value` padding into the attention interfaces that need it.
  • Simplified function dispatch for linear attention.
  • Optimized flash attention max sequence length computation in vision attention.
  • Fixed `BlockMask` crash in CTRL flex-attention generation.
  • Automatically switched attention implementation to flash.
  • Fixed SDPA prefill with position_bias.
  • Modularized qwen-format vision processors.
  • Updated daily CI Docker image to torch 2.13.0 / CUDA 13.0.
  • Aligned image feature dtype in kosmos2 and kosmos2_5 embedding merge.
  • Speeds up image preprocessing for vision-language models.
  • Fixed vision position-embedding init width fallback in Phi4Multimodal.
  • Fixed Hunyuan-VL PIL image resize parity with reference preprocessing.
  • Fixed image-text-to-text stop_sequence handling.
  • Refactored image loading in tests to use load_test_image helper.
  • Aligned OlmoHybrid to use a native cache in generate.
  • Stopped setting the static cache as an attribute to save memory.
  • Added support for batched Qwen2.5/3-Omni audio generation.
  • Allowed sliding window cache layers to be roll-backed for speculative decoding.
  • Fixed shape mismatch in KyutaiSpeechToText `generate()` last window.
  • Fixed typo in `MusicgenForCausalLM.generate()`.
  • Fixed assisted decoding for models with EncoderDecoder cache & OlmoHybrid.

Affected Symbols

MambaGDNConv-onlyT5MT5LongT5ALL_ATTENTION_FUNCTIONSattn_implementation_is_url_build_image_tokensMulti-Head Latent Attention (MLA)Flash AttentionCTRL flex-attentionSDPAGemma 4Qwen-format vision processorskosmos2kosmos2_5Phi4MultimodalHunyuan-VLimage-text-to-text pipelineQwen2.5/3-OmniKyutaiSpeechToTextMusicgenForCausalLMEncoderDecoder cacheOlmoHybrid