Transformers

This release focuses on fixing AutoTokenizer enforcement, optimizing MoE performance with batched implementations, and significantly improving model loading speeds via meta device initialization.

This release introduces major breaking changes including 'auto' as the default dtype, 50GB shard sizes for saving, and mandatory **kwargs in forward methods. It also adds support for new models (FastVLM, Lasr, PaddleOCR-VL) and a new dynamic weight loader for quantization.

Transformers v5 introduces a major overhaul of the library, featuring a new dynamic weight loading API and a unified tokenizer backend system to simplify internals and improve performance.

This emergency patch fixes a critical bug affecting model loading when local_files_only is set to True and addresses a typo from a previous patch.

This patch release focuses on fixing Mistral tokenizer mappings and Tekken pattern matching, while also correcting a decorator error in the device memory utility.

This patch release focuses on fixing dependency parsing issues with Optax and Poetry, alongside stability improvements for FSDP and Python 3.9 support.

This release introduces support for several next-generation model architectures, including the high-efficiency Qwen3-Next and Qwen3-VL series, the privacy-focused VaultGemma, and the high-speed Longcat Flash MoE.

This release focuses on bug fixes for Jetmoe and Emu3 models, addresses a getter regression, and improves multi-processing performance for processors.

This release introduces a preview of the Vault-Gemma model, a 1B parameter decoder-only model trained with sequence-level differential privacy.

This patch release primarily fixes the new 'dtype' argument in pipelines and addresses several model-specific bugs including Llama4 accuracy and SamAttention attribute errors.

This release introduces a preview of the EmbeddingGemma model, a highly efficient 308M parameter multilingual embedding model optimized for on-device RAG and mobile use cases.

This release introduces several major vision and multimodal models including Dino v3, SAM 2, and Ovis 2, alongside a significant refactor of the caching system to optimize memory for sliding window attention.

This patch release corrects a technical error in the previous release process to properly apply the fix for issue #40197.

Patch release 4.55.3 focuses on stability improvements for FlashAttention-2 on Ascend NPU, FSDP sharding fixes, and critical bug fixes for GPT-OSS and Mamba models.

Patch release 4.55.2 fixes a critical regression in Flash Attention 2 (FA2) generations caused by a missing utility import in version 4.55.1.

Patch release 4.55.1 focuses on stabilizing the MXFP4 quantization for GPT-OSS models and resolving device-related bugs across several multimodal models like Idefics and SmolVLM.

This release introduces GLM-4.5V, a high-performance multimodal reasoning model based on GLM-4.5-Air, featuring advanced capabilities in image, video, and GUI analysis.

OpenAI released GPT OSS, an open-source (Apache 2.0) MoE model family in 21B and 117B sizes featuring 4-bit MXFP4 quantization and Flash Attention 3 support. These models are optimized for reasoning and agentic tasks, compatible with the new Responses API and standard transformers workflows.

A maintenance patch release focused on fixing regressions in cache inheritance, device placement, and distributed training (TP/device-mesh) across various model architectures like ModernBERT, GPT2, and Mamba.

This release focuses on reducing library bloat and increasing speed through refactored Llama models, megablocks kernel integration, and native distributed training. It also introduces several new model architectures including Ernie 4.5, Voxtral, DeepSeek-V2, and LFM2.

This preview release introduces Baidu's Ernie 4.5 model family to Transformers, including a 0.3B dense model and MoE variants (21B and 300B).

A small patch release (v4.53.3) that refactors OpenTelemetry integration by removing explicit provider setter calls.

This release introduces a preview of the ModernBERT Decoder, a causal language model variant of the ModernBERT architecture designed for autoregressive generation and sequence classification.

This patch release focuses on critical bug fixes for GLM-4.1V and GLM-4V models, resolves hardware-specific issues on Ascend NPU, and deprecates the sliding window feature.

This patch release focuses on bug fixes for Vision Language Models (VLMs) like Qwen2-VL and SmolVLM, alongside introducing packed tensor format support for various attention backends.

Release v4.53.0 introduces several major model architectures including Gemma 3n, Dia TTS, Kyutai STT, and the massive 456B MiniMax model. The update focuses heavily on multimodal capabilities, efficient parameter usage, and long-context support.

This release introduces a preview of the Kyutai-STT model architecture, featuring 1B and 2.6B parameter checkpoints for high-accuracy speech-to-text transcription.

This release introduces a preview of the V-JEPA 2 model, a state-of-the-art self-supervised video encoder for motion understanding and robot manipulation tasks.

This release introduces a preview of the ColQwen2 model, a visual-based document retrieval system that leverages the Qwen2-VL backbone for late interaction similarity scoring.

This patch release focuses on bug fixes for Vision Language Models (Qwen-VL, PaliGemma), attention scaling corrections for OPT, and compatibility improvements for older PyTorch versions.

This patch release fixes issues related to torch distributed initialization and protects ParallelInterface imports to ensure stability in distributed environments.

This patch release re-introduces 3D parallel training support while fixing a device map override bug and improving import error clarity.

This release introduces several major multimodal and specialized models, including the Qwen2.5-Omni streaming model, the high-precision SAM-HQ segmenter, and the D-FINE real-time object detector.

This release introduces the Conversational Speech Model (CSM), an open-source contextual text-to-speech model capable of generating natural speech from multi-turn dialogue context.

This release introduces the GraniteMoeHybrid model architecture, a hybrid design combining state space layers and Mixture-of-Experts (MoE) attention, available as a stable preview ahead of the v4.52.0 minor release.

This release introduces a preview of the D-FINE model, a high-performance real-time object detector featuring fine-grained distribution refinement for superior localization accuracy.

This release introduces a preview of SAM-HQ, an enhancement to the Segment Anything Model that provides higher quality segmentation masks with minimal additional parameters.

This preview release introduces the BitNet model architecture to the transformers library, enabling high-performance 1-bit LLM inference.

This release introduces LlamaGuard 4 and Llama Prompt Guard 2, providing multimodal safety moderation for text and images. It is available as a preview tag prior to the official v4.52.0 minor release.

This release introduces Qwen2.5-Omni, an end-to-end multimodal model capable of perceiving text, images, audio, and video while generating synchronized text and speech responses.

This preview release introduces support for the InternVL 2.5 and 3 family of multimodal models, featuring a native multimodal pre-training paradigm and state-of-the-art performance on visual-linguistic tasks.

This release introduces a preview of the Janus and Janus-Pro models, a unified multimodal framework capable of both visual understanding and text-to-image generation by decoupling visual encoding pathways.

This release introduces a preview of TimesFM, a decoder-only foundation model for time-series forecasting, available as a specialized tag on top of transformers v4.51.3.

This release introduces a preview of the MLCD vision model, a foundational visual model optimized for multimodal LLMs like LLaVA, developed by DeepGlint-AI.

This patch release introduces support for the GLM-4 model and includes several fixes for PyTorch version compatibility, specifically regarding FlexAttention.

A minor patch release focusing on Llama4 model corrections and the introduction of Attention Quantization with FBGemm and Tensor Parallelism.

This patch release focuses on stabilizing Llama 4 support and fixing compatibility issues with torch 2.6.0, DeepSpeed, and weight initialization.

This release introduces support for Llama 4, Phi4-Multimodal, DeepSeek-v3, and Qwen3 architectures, alongside a major documentation overhaul and modularization of speech models.

This release introduces support for the DeepSeek-V3 (DeepSeek-R1) model, featuring MLA and DeepSeekMoE architectures, available via a specific git tag on top of version 4.50.3.

This patch release fixes bugs related to beam search output cropping, BLIP-2 floating-point precision mismatches, and PixtralProcessor configuration.

A patch release focusing on backend stability, specifically fixing image processing for Gemma3 and Qwen2-VL, and updating torch version validation.

A patch release addressing minor bugs in hub kernels, remote code, and specific model implementations like Chameleon and PyTorch deformable attention.

Release v4.50.0 introduces a new model-based release strategy and adds support for several major vision-language models including Gemma 3, Aya Vision, Mistral 3.1, and SigLIP-2.

This release introduces Mistral 3 (Mistral Small 3.1) to the Transformers library, a 24B parameter model featuring 128k context length and advanced vision-language capabilities.

This release introduces Google's Gemma 3 multimodal models to the transformers library, featuring a SigLIP vision encoder and Gemma 2 language decoder with support for high-resolution image cropping and multi-image inference.

This release introduces Aya Vision 8B and 32B, multilingual multimodal models combining SigLIP-2 vision encoders with Cohere language models, available via a specialized transformers release tag.

This release introduces SigLIP-2, a new family of multilingual vision-language encoders featuring improved semantic understanding and support for native aspect ratio image processing via the NaFlex variant.

This release introduces SmolVLM-2, a lightweight vision-language model based on Idefics3 and SmolLM2 that supports multi-image and video processing.

This release introduces several new models including Helium, Qwen2.5-VL, and Zamba2, alongside a new CLI chat feature and standardized fast image processors.

This patch release primarily addresses Python 3.9 compatibility issues, fixes device failures in the RoPE module, and resolves generation bugs for PaliGemma2.

This patch release primarily restores Python 3.9 compatibility and fixes regressions related to DBRX model loading and HybridCache mask slicing.

Patch release v4.48.1 fixes a typo in Phi model attention bias, resolves a logic error in gradient accumulation loss, and patches Moonshine's generate wrapper.

This release introduces several major model architectures including ModernBERT, Aria (MoE), and Bamba (Mamba-2), while adding a TimmWrapper to integrate timm library models directly into the Transformers ecosystem.